Chapter 6 Artificial Landmarks Detection in Images Using Space Line Detection
6.4 Proposed Method for Detecting and Localizing Traffic cones
6.4.2 Experimental results of traffic cone detection
(6.19)
which is equivalent to
A K B (6.20)
where
B a1 (5.21)
Finally, we can illustrate the LY line which goes through the corner point and is perpendicular to the ground by the Equation (6.20).
6.4.2 Experimental results of traffic cone detection
Some experimental results of detecting the traffic cone using the proposed method are given in this section. An input omni-image with a traffic cone is shown in Figure 6.14. After conducting the feature extraction and Canny edge detection processes, we obtain an edge-point image as shown in Figure 6.15. The final result of traffic cone detection is shown in Figure 6.16(a) and the relative position of the traffic cone with respect to the vehicle is shown in Figure 6.16(b).
Figure 6.14 The omni-image with a traffic cone.
Figure 6.15 The result of traffic cone segmentation using the Canny edge detector.
(a)
(b)
Figure 6.16 The result of traffic cone detection and the obtained position of the traffic cone. (a) The result image of extracting the LX, LY, and LZ lines of the traffic cone (b) The relative position of the traffic cone with respect to the vehicle position.
Chapter 7
Experimental Results and Discussions
7.1 Experimental Results
In this section, we will show some experimental results of the proposed vehicle navigation system for use as a machine guide dog in the learning and navigation processes. The experimental environment was an outdoor sidewalk in National Chiao Tung University as shown in Figure 7.1(a). We illustrate the outdoor environment including a gray sidewalk, a red curb line, and some landmarks as shown in Figure 7.1(b). The portion to the right of the red curb line is part of an around-campus road.
Lawn corner Signboard
Traffic cone Fixed Obstacle
Sidewal Road k
Tree trunk
Stop line on road
(a) (b)
Figure 7.1 The experimental environment. (a) A side view. (b) Illustration of the environment.
In the learning process, a trainer guided the vehicle by the use of a learning
arriving at appropriate locations on the sidewalk, the vehicle was commanded to learn the positions of specific landmarks like signboard, tree trunk, stop line, …, etc. In addition, the position of the fixed obstacle was recorded manually by localizing its position on the omni-image as shown in Figure 7.3. At the end of the learning process, the trainer obtained a navigation map with a navigation path and other environment landmarks as illustrated in Figure 7.4.
Figure 7.2 The Learning interface of the proposed vehicle system.
In the navigation process, the vehicle started from the same position just like in the learning process and navigated along the recorded navigation path nodes with the curb line following technique. Then, the vehicle detected many types of landmarks and localized its position. Some results of landmark detection are shown in Figure 7.5.
By conducting curb detection, the vehicle kept its path parallel to the curb. A result of curb detection is given in Figure 7.6. Besides, after detecting the fixed obstacle in the navigation path, the vehicle adopted the obstacle avoidance procedure to avoid it as shown in Figures 7.7. Finally, the vehicle reached the appointed terminal node successfully, and the path map with a record of each vehicle position in the navigation
process is illustrated in Figure 7.8.
(a)
(b)
Figure 7.3 Learning of a fixed obstacle. (a) The position of fixed obstacle on the omni-image (Lime-colored points clicked by the trainer). (b) Computed fixed obstacle positions in the real world.
Blind navigation node Start / Terminal node
Curb-following navigation node
Vehicle localization node
Tree trunk landmark node
Lawn corner landmark node
Fixed obstacle node Traffic cone landmark node
Stop line landmark node
(a) (b)
(c)
(d) (e)
Figure 7.5 Some results of landmark detection. (a) A tree trunk detection result with LY line drawn in red. (b) A traffic cone detection result with LX, LY, and LZ line drawn in dark blue, lime, and navy blue, respectively. (c) A lawn corner detection result with two boundary lines drawn in navy blue.
(d) The result of stop line with three boundary lines drawn in yellow and navy blue, respectively. (e) A signboard detection result with LY line drawn in red and lime, respectively.
Figure 7.4 Illustration of the learned navigation path.
Figure 7.6 The result of curb line detection.
(a) (b)
(c) (d)
Figure 7.7 The vehicle reads the fixed obstacle position from the navigation path and change the path to avoid it. (a)~(d) show the process of fixed obstacle avoidance.
In Table 7.1, we show the errors in percentage between the actual position of the landmarks and the estimated positions of the landmarks of 8 times of navigations
landmark position is 7.52%. These small error percentages show that the precision of the proposed system is satisfactory for real applications.
Figure 7.8 The recorded path map in the navigation process. (Blue points represent the vehicle path and other points with different color represent different localized landmark positions in different detections).
7.2 Discussions
By analyzing the experimental results of the vehicle navigation, we find some problems. Firstly, for sidewalk curb detection, we detect the curb with a specific surface in the campus of National Chiao Tung University. More kinds of curb lines with different colors should be learned for the line following technique. Also, the light reflection caused by the plastic camera enclosure created in the omni-image also causes ill effects in image analysis. A possible solution is to learn these specific
we may spend much time to detect the LX line and localize it. A possible solution is to implement an embedded system to speed up the calculation. Finally, more experiments in different environments should also be conducted to test our system more thoroughly.
Table 7.1 Precision of estimated landmark positions and their error percentages.
navigation
No. Real position Estimated position
Chapter 8
Conclusions and Suggestions for Future Works
8.1 Conclusions
Construction of a machine guide dog using a two-mirror omni-camera and an autonomous vehicle has been proposed in this study. To implement such as a system, several methods have been proposed.
At first, by the pano-mapping technique proposed by Jeng and Tsai [25], we calibrate the two-mirror omni-camera used in this study by recording the relationship between the image pixels and the real-world azimuth and elevation angles. Next, by the use of a learning interface designed in this study, a trainer can guide the vehicle to navigate on a sidewalk and construct a navigation path conveniently including the path nodes, alone-path landmarks, and relevant guidance parameters.
Next, two new space line detection techniques based on the pano-mapping technique have been proposed. Each space line, which when projected on an omni-image becomes a conic-section curve, is detected by the use of analytic formulas and the Hough transform technique. In addition, for the three types of space line which exists in landmarks like the tree trunk, the lawn corner, the signboard, the stop line on roads, and the traffic cone, we can further compute its position directly using omni-images according to the pano-mapping technique.
Also, several landmark detection techniques have been proposed for conducting vehicle navigation. Firstly, a curb line detection technique has been proposed for use to guide the vehicle on a safe path as well as to adjust the odometer reading of the
vehicle orientation. Next, some natural and artificial landmark detection techniques have been proposed as well. The three types of space lines found in these landmarks using the techniques can be used to localize the vehicle in the navigation process.
Furthermore, to conduct the landmark detection works more effectively in the outdoor environment, techniques for dynamic threshold adjustments have also been proposed, which can be used to handle different lighting conditions.
Good landmark detection results and successful navigation sessions on a sidewalks in the National Chiao Tung university campus show the feasibility of the proposed methods.
8.2 Suggestions for Future Works
According to our experience obtained in this study, several suggestions and related interesting issues worth further investigations in the future are stated in the following:
(1) it seems necessary to develop some techniques to detect moving objects, like pedestrians walking on the sidewalk or people riding bikes;
(2) it is a challenge to detect natural landmarks which have no obvious color information to conduct vehicle navigation in more complicated outdoor environments;
(3) it is desired to design a new camera system which has a smaller size for more convenient uses by the blind people;
(4) it is a challenge to develop additional techniques to guide the vehicle to pass crossroads, like recognizing traffic signals and following zebra crossings, etc.;
(5) it may be necessary to add the capability of warning the user via sound in danger
(6) It is interesting to combine other facilities like range finders to implement the system for more complicated applications.
(7) It is desired to utilize properties of trigonometric functions to reduce the range of the Hough space to speed up the computation time.
References
[1] J. Borenstein and I Ulrich, “The GuideCane － a computerized travel aid for the Active guidance of blind pedestrians,” Proceedings of IEEE International Conference on Robotics and Automation, pp. 1283–1288, Albuquerque, NM, USA, Apr. 1997.
[2] V. Ivanchenko, J. Coughlan, W. Gerrey, and H. Shen, “Computer vision-based clear path guidance for blind wheelchair users” Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility, pp.
291–292, New York, NY, USA, 2008.
[3] Hideo Mori, Shinji Kotani, and Noriaki Kiyohiro, "A robotic travel aid
“HITOMI”," Proceedings of the IEEE/RSJ/GI International Conference onIntelligent Robots and Systems '94, vol. 3, pp. 1716–1723, Munich, Bavaria, Germany, 1991.
[4] Y. Z. Hsieh and M. C. Su, “A stereo-vision-based aid system for the blind,” M. S.
Thesis, Department of Computer Science and Information Engineering, National Central University, Jhongli, Taoyuan, Taiwan, June 2006.
[5] S. Wenqin, J. Wei, and C. Jian, “A machine vision based navigation system for the blind,” IEEE International Conference on Computer Science and Automation Engineering, vol. 3, pp. 81-85, Shanghai, People's Republic of China, July 2011.
[6] S. Willis and S. Helal, “RFID information grid for blind navigation and wayfinding,” Proceedings of the 9th IEEE International Symposium on Wearable Computers, pp. 34–37, Washington, DC, USA, Oct. 2005.
[7] L. Ran, S. Helal, and S. Moore, “Drishti: an integrated indoor/outdoor blind navigation system and service,” Proceedings of 2nd IEEE Annual Conference on
2004.
[8] M. F. Chen and W. H. Tsai, "Automatic learning and guidance for indoor autonomous vehicle navigation by ultrasonic signal analysis and fuzzy control techniques," Proceedings of 2009 Workshop on Image Processing, Computer Graphics, and Multimedia Technologies, National Computer Symposium, pp.
473–482, Taipei, Taiwan, Nov. 2009.
[9] E. Abbot and D. Powell, "Land-vehicle navigation using GPS," Proceedings of the IEEE, vol. 87, no. 1, pp. 145–162, Jan. 1999.
[10] Sami Atiya and Gregory D. Hager, “Real-time vision-based robot localization,”
IEEE Transactions on Robotics and Automation, vol. 9, no. 6, pp. 785–800, Dec.
1993.
[11] M. C. Chen, “Vision-based security patrolling in indoor environments using autonomous vehicles,” Proceedings of 2005 Conference on Computer Vision, Graphics and Image Processing, pp. 811–818, Taipei, Taiwan, Republic of China.
[12] K. L. Chiang and W. H. Tsai. “Vision-based autonomous vehicle guidance in indoor environments using odometer and house corner location information,”
Proceedings of 2006 IEEE International Conference on Intelligent Information Hiding and Multimedia Signal Processing (IIHMSP-2006), pp. 415–418, Pasadena, California, USA, Dec. 18-20, 2006.
[13] M. Agrawal and K. Konolige, “Real-time localization in outdoor environments using stereo vision and inexpensive GPS,” Proceedings of 18th International Conference on Pattern Recognition, Hong Kong, People's Republic of China, pp.
1063–1068, vol. 3, Aug, 2006.
[14] S. Y. Tsai and W. H. Tsai, "Simple automatic path learning for autonomous
Proceedings of 2008 International Computer Symposium, vol. 2, pp. 207-212, Taipei, Taiwan, Dec. 2008.
[15] D. L´opez, K. Sj¨o, C. Paul, and P. Jensfelt, “Hybrid laser and vision based object search and localization,” Proceedings of IEEE International Conference on Robotics and Automation, pp.26362643, Pasadena, CA, USA, May 19-23, 2008.
[16] W. Lui and R. Jarvis, “Eye-Full Tower: A GPU-based variable multibaseline omnidirectional stereovision system with automatic baseline selection for outdoor mobile robot navigation,” Robotics and Autonomous Systems, pp.
747-761, vol. 58, no. 6, Apr. 2010.
[17] H. Fu, Z. Cao, and X. Cao, “Embedded omni-vision navigator based on
multi-object tracking,” Machine Vision and Applications, pp. 349-358, vol. 22, no. 2, 2011.
[18] D. Ishizuka, A. Yamashita, R. Kawanishi, T. Kaneko, and H. Asama,
“Self-localizaion of mobile robot equipped with omnidirectional camera using image matching and 3D-2D edge matching,” Proceedings of IEEE International Conference on Computer Vision Workshops, Barcelona, Spain, pp. 272-279, Nov.
6-13, 2011.
[19] C. J. Wu and W. H. Tsai, “Location estimation for indoor autonomous vehicle navigation by omni-directional vision using circular landmarks on ceilings,”
Robotics and Autonomous Systems, vol. 57, no. 5, pp. 546-555, May 2009.
[20] Taiwan Guide Dog Association, “About Taiwan Guide Dogs Information,”
Available online:
http://www.guidedog.org.tw/.
[22] J. K. Huang, “Autonomous vehicle navigation by two-mirror omni-directional imaging and ultrasonic sensing techniques,” M. S. Thesis, Department of Computer and Information Science, National Chiao Tung University, Hsinchu, Taiwan, June 2010.
[23] Y. H. Chou and W. H. Tsai, “Guidance of a vision-based autonomous vehicle on sidewalks for use as a machine guide dog,” Proceeding of 2011 Conference on Computer Vision, Graphic, and Image Processing, Chiayi, Taiwan, 2011.
[24] S. W. Jeng and W. H. Tsai, "Using pano-mapping tables to unwarping of omni-images into panoramic and perspective-view Images," Proceeding of IET Image Processing, vol. 1, no. 2, pp. 149–155, June 2007.
[25] W. H. Tsai, “Moment-preserving thresholding: a new approach,” Computer Vision, Graphics, and Image Processing, vol. 29, no. 3, pp. 377-393, 1985.
[26] C. J. Wu and W. H. Tsai, "An omni-vision based localization method for automatic helicopter landing assistance on standard helipads," Proceedings of 2nd International Conference on Computer and Automation Engineering, vol. 3, pp. 327–332, Singapore, 2010.