The comparison between Method-II and Method-III

To illustrate the effect of vertical disparity used in HNN, Method-II and Method-III are applied to a realistic image pair which is shown in Fig 5.7 and with the feature points in the ROI. The limitation for the number of the feature points is set as [20, 30] and the threshold T is set as 30. To deal with the stereo matching problem of Fig. 5.7, the parameters of Method-II and Method-III also have to be determined by the proposed genetic algorithm. The processes of the genetic algorithm applied to Method-II and Method-III are shown in Fig. 5.8 and Fig. 5.9, respectively. Since the number of matched pairs in target is 12, the best fitness value is also 12. When the stop condition of the genetic algorithm is satisfied, the chromosome with the maximum fitness value in the population is chosen to determine the parameters of HNN so that the parameters of Method-II are set as [11.51, 26.24, 8.89, 1.16] and the parameters of Method-III are set as [17.97, 24.02, 6.37, 2.2]. Table 5.4 is the corresponding result when Method-II applied to Fig. 5.7 and reaches its stable state.

The pair number 11 in Table 5.4 is an error matched pair, but this pair does not occur in the Table 5.5 which is the result of the Method-III applied to Fig. 5.7 since the vertical disparity of point number 18 in left image and point number 13 in right image is large. From the Table 5.5, the result is one-to-one and correct matching that verifies the Method-III can be used to improve the performance by reducing the matched pair with the large vertical disparity. Besides, to show the performance of the parameters, the Method-III with the parameters [17.97, 24.02, 6.37, 2.2] is also applied to other realistic stereo image pairs shown in Fig. 5.8. The testing results are the results using the parameters [17.97, 24.02, 6.37, 2.2] and the training results are the results using another parameters which are obtained by applying the proposed genetic algorithm to each image pair. The error term represents how many matched pairs are not identical

to the target, and the percentage of the correct matching by using the Method-III with the parameters [17.97, 24.02, 6.37, 2.2] is 91.24%.

1 2

Fig. 5.7 The realistic image pair with the feature points in the ROI, (a) left image with 21 feature points, (b) right image with 20 feature points.

0 50 100 150 200 250 300 350 400 450 500

Fig. 5.8 The result of genetic algorithm corresponding to Method-II.

5 10 15 20 25 30 35 40 45 50 55 60

Fig. 5.9 The result of genetic algorithm corresponding to Method-III.

Correct/Error/Target

Fig. 5.10 The left images of the realistic image pairs in the experiment.

Table 5.4 The result of the Method-II applied to Fig. 5.7.

Table 5.5 The result of the Method-III applied to Fig. 5.7.

Pair

5.2.3 A simple application to obtain the relative distance between objects

A simple application to detect objects and find the relative distance between the detected objects is also proposed in the following experiment. The first step is to establish a scene as the background shown in Fig. 5.11, and the disparities of matched pairs in the background images will be derived by applying the Method-III and are marked in Fig. 5.12. Secondly, if there are two objects appear in the scene, such as Fig.

5.13 in which two toy cars appear, the disparities of the two objects can be acquired shown in Fig. 5.14 by applying the Method-III to Fig. 5.13 and deleting the feature points of background. After that the relative distance information can be obtained by comparing the disparities of background and two objects. The feature point with large disparity is more closed to the two cameras than the feature point with small disparity.

From the Fig. 5.14, the feature points of the left toy car have the larger disparities so that the left car is more closed to the two cameras than the right toy car. In addition, to compare the Fig. 5.12 and Fig. 5.14, the information that two objects are in front of the box in the center of the image is also obtained.

Fig. 5.11 The background image pair with the feature points in the ROI, (a) left image with 21 feature points, (b) right image with 20 feature points.

22

Fig. 5.12 The feature points with disparities in the background image.

1 2

Fig. 5.13 The feature points of the two objects which appear in the ROI of image pair.

38 39 35

3530

27 29 28 29

Fig. 5.14 The feature points of the two objects and with their disparities.

Chapter 6

Conclusions and Future Works

An intelligent algorithm to detect the disparity of feature point by solving the stereo matching problem is proposed in this thesis. The algorithm consists of feature extraction, stereo matching process and a searching parameters algorithm. The feature extraction is implemented by the Harris corner detector, which extracts the corners of objects as the feature points. The stereo matching problem is formulated as a minimization of a neural network’s energy function which represents the constraints on the solution. The neural network is a 2D Hopfield neural network with connection weights between neurons to minimize the energy function. Since there are four parameters in the energy function have to determined, a real-coded genetic algorithm is applied to automatic search those parameters which make the performance of the stereo matching process better. The contribution of this thesis is to design the energy function, and the experimental results show that using this energy function provides good matching results. Besides, the execution time of the test image pairs on a PC with AMD 1.92 GHz CPU is estimated between 2 and 3 seconds. Finally, after solving the stereo matching problem, the disparity of the matching points can be obtain.

Although the proposed ISMB have some achievements of stereo matching, it does not yet provide a full performance and several improvements to the ISBM may be possible. To enhance the performance, two works are considered in the future works: (1) using the analog Hopfield neural network to avoid the local minimum of the energy function. (2) to extract the edges as complementary features.

Reference

[1] K. Nishihara and T. Poggio, “Stereo vision for robotics,” Proc. Robotics Research, First Int. Symp. , pp. 489–505, 1984.

[2] D. J. Kriegman, E. Triendl and T. O. Binford, “Stereo vision and navigation in buildings for mobile robots,” IEEE Trans. Robotics and Automation, vol. 5, No. 6, 1989.

[3] D. Murray and J. J. Little, “Using Real-Time Stereo Vision for Mobile Robot Navigation,” Autonomous Robots, vol. 8, No. 2, pp. 161–171, 2000.

[4] M.Bertozzi and A. Broggi, “GOLD: A parallel realtime stereo vision system for genetic obstacle and lane detection, “ IEEE Trans. Image Processing, vol. 7, pp.

62-81, Jan. 1998.

[5] L. Zhao, and C. E., Thorpe, “Stereo and Neural Based Pedestrian Detection,”

IEEE Trans. Intelligent Transportation Systems, Vol. 1, No. 3, pp. 148–154, Sep.

2000.

[6] W. Y. Yau and H. Wang, “Fast Relative Depth Computation for an Active Stereo Vision System,” Real-Time Imaging, vol. 5, No. 3, pp.189−202, June, 1999.

[7] G. Medioni and W. H. Tsai, “Segment based stereo matching,” Computer Vision Graphics Image Process, Vol. 31, pp. 2−18, 1985.

[8] Y. Shirai, Three−Dimensional Computer Vision, Springer−Verlag, New York, 1987.

[9] J. J. Lee, J. C. Shim and Y. H. Ha, “Stereo correspondence using the Hopfield

neural network of a new energy function,” Pattern Recognition vol. 27, No. 11, pp. 1513−1522, Nov. 1994.

[10] W. E. L. Grimson, “Computational experiments with a feature based stereo algorithm,” IEEE Trans. Pattern Anal. Mach. Intell. , vol. PAMI−7, pp. 17−34, 1985.

[11] G. Medioni and R. Nevatia, “Segment based stereo matching,” Comput. Vision Image Processing, vol. 31, pp. 2−18, 1985.

[12] T. Ozanian, “Approaches for stereo matching – A review,” Modeling Identification Control, vol. 16, pp. 65-94, 1995.

[13] G. Pajares and J. M. Cruz, “Local stereovision matching through the ADALINE neural network,” Pattern Recognition Letters, vol. 22, pp. 1457−1437, 2001.

[14] J. Hopfield, and D. W. Tank, “Neural computation of decisions in optimization problems,” Biological Cybernetics, Vol. 52, pp. 141−152, 1985.

[15] G. Pajares, J. M. Cruz and J. Aranda, “Relaxation by Hopfield network in stereo image matching,” Pattern Recognition, vol. 31, No. 5, pp. 561−574, 1998.

[16] Y. Ruichek, “Multilevel- and Neural-Network-Based Stereo-Matching Method for Real-Time Obstacle Detection Using Linear Cameras,” IEEE Trans.

Intelligent Transportation Systems, Vol. 06, No. 1, pp. 54 – 62 , Mar. 2005.

[17] T. H. Sun, “Improving stereo matching quality with scanline-based asynchronous Hopfield neural networks,” Journal of the Chinese Institute of industrial

Engineers, Vol. 24, No. 1, pp. 50−59, 2007.

[18] N. M. Nasrabidi and C. Y. Choo, “Hopfield network for stereovision correspondence,” IEEE Trans. Neural Network, Vol. 03, No. 1, pp. 123−135, Jan.

1992.

[19] K. Achour and L. Mahiddine, “Hopfield Neural Network Based Stereo Matching Algorithm,” Journal of Mathematical Imaging and Vision, Vol. 16, No. 1, pp.

17−29, Jan. 2002.

[20] C. Harris and M. Stephens, “A combined corner and edge detector,” Fourth Alvey Vision Conference, Manchester, UK, pp. 147–151, 1988.

[21] C. T. Lin and C. S. George Lee, Neural Fuzzy Systems A Neural−Fuzzy Synergism to Intelligent Systems, Prentice−Hall Inc, 1994.

[22] S. S. Yu and W. H. Tsai, “Relaxation by the Hopfield neural network,” Pattern Recognition, Vol. 25, No. 2, pp. 197−209, 1992.

[23] M. Srinivas and M. Patnaik, “Genetic algorithms: a survey,” IEEE Trans.

Computer, Vol. 27, No. 6, pp. 17−26, 1994.

[24] A. Blanco, M. Delgado and M. C. Pegalajar, “A real-coded genetic algorithm for training recurrent neural networks,” Neural Networks, Vol. 14, No. 1, pp. 93-105, 2001.