Chapter 8 Conclusions and Suggestions for Future Research
8.2 Suggestions for Future Research
The following topics may be investigated in the future:
1. use of vanishing point information formed by long parallel lines;
2. use of house beam information formed by mutually parallel lines;
3. use of parallel line information in an image acquired by a two-mirror omni-camera;
4. use of two perpendicular lines in an image acquired by a longitudinally-coxial omni-camera pair;
5. use of new-typed omni-cameras in various applications for more precise and effective image acquisition and analysis;
6. aplying applications of the proposed methods to other types of vehicles, such
162
as airplane, UAV, surveillance car, toy car, etc.;
7. exploitations of other applications of autonomous vehicles, like automatic car parking and driving, security patrolling, intelligent transportation, airplane landing, outer space planet exploration, etc.
163
References
[1] M. Betke and L. Gurvits, “Mobile robot localization using landmarks,” IEEE Transactions on Robotics and Automation, vol. 13, no.2, pp. 251-263, Apr. 1997.
[2] Y. Yagi, Y. Nishizawa, and M. Yachida, “Map-based navigation for a mobile robot with omni-image sensor copis,” IEEE Transactions on Robotics and Automation, vol. 11, no. 5, pp. 634-648, Oct. 1995.
[3] J. Gaspar, N. Winters, and J. Santos-Victor, “Vision-based navigation and environmental representations with an omni-camera,” IEEE Transactions on Robotics and Automation, vol. 16, no. 6, Dec. 2000.
[4] E. Menegatti, T. Maeda and H. Ishiguro, “Image-based memory for robot navigation using properties of the omni-images,” Robotics and Autonomous Systems, vol. 47, pp. 251-267, July 2004.
[5] H. Koyasu, J. Miura, and Y. Shirai, “Recognizing moving obstacles for robot navigation using real-time omni-directional stereo vision,” Journal of Robotics and Mechatronnics, vol. 14, no. 2, pp. 147-156, June 2002.
[6] C. Cauchois, E. Brassart, B. Marhic, and C. Drocourt, “An absolute localization method using a synthetic panoramic image base,” Proceedings of IEEE Workshop on Omnidirectional Vision, Copenhagen, Denmark, June 2002, pp.
128-135.
[7] C. Becker, J. Salas, K. Tokusei, and J.-C. Latombe, “Reliable navigation using landmarks,” IEEE Robotics and Automation Conference, 1995, pp. 401 – 406.
[8] Y. Ogawa, J. H. Lee, S. Mori, A. Takagi, C. Kasuga and H. Hashimoto, “The positioning system using the digital mark pattern-the method of measurement of a horizontal distance,” Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, pp.731-741, 1999.
[9] S. J. Ahn, W. Rauh, and M. Recknagel, “Circular coded landmark for optical 3D-measurement and robot vision,” Proceedings of International Conference on Intelligent Robots and Systems, pp.1128-1133, 1999.
164
[10] S. Kim and S.Y. Oh, “Slam in indoor environments using omni-directional vertical and horizontal line features,” Journal of Intelligent and Robotic Systems, vol. 51, no. 1. (15 January 2008), pp. 31-43.
[11] J. Kannala and S. Brandt, “A generic camera calibration method for fish-eye lenses,” Proceedings of the 17th International Conference on Pattern Recognition; Cambridge, U.K, vol. 1, pp. 10-13, Aug. 23-26, 2004.
[12] S. Shah and J. K. Aggarwal, “Intrinsic parameter calibration procedure for a (high-distortion) fish-eye lens camera with distortion model and. accuracy estimation,” Pattern Recognition, vol. 29, no. 11, pp. 1775-1788, Nov. 1996 [13] Liu, Y. C., K. Y. Lin, and Y. S. Chen, “Bird’s-eye view vision system for vehicle
surrounding monitoring,” Proceedings of Conference on Robot Vision, Berlin, Germany, Feb. 20, 2008, pp. 207-218.
[14] S. W. Jeng. “A study on camera calibration and image transformation techniques and their applications,” Ph. D. Dissertation, Institute of Information Science and Engineering, National Chiao Tung University, Hsinchu, Taiwan, Republic of China, June 2007.
[15] Zhihui, et al., “Catadioptric omni-directional stereo vision and its applications in moving objects detection,” in Computer Vision, Z. Xiong (ed.), Vienna, Austria, Nov. 2008
[16] H. Koyasu, J. Miura and Y. Shirai, “Real-time omnidirectional stereo for obstacle detection and tracking in dynamic environments,” Proceedings of 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 1, pp.
31-36, Maui, Hawaii, U. S. A., Oct. 29-Nov. 03, 2001.
[17] C. T. Ho and L. H. Chen, “A high-speed algorithm for elliptical object detection,” IEEE Transactions on Image Processing, Vol. 5, No. 3, pp. 547-550, Mar. 1996.
[18] L. Ma, Y. Chen, and K. L. Moore, “Flexible camera calibration using a new analytical radial undistortion formula with application to mobile robot localization,” Proceedings of the IEEE International Symposium on Intelligent Control, pp. 799-804, 2004.
[19] Z. Zhang, “A flexible new technique for camera calibration,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.22, no.11, pages 1330-1334, 2000.
[20] I. Shimshoni, “On mobile robot localization from landmark bearings,” IEEE
165
Transactions on Robotics & Automation, vol. 18, no. 6, December 2002.
[21] A. Merke, S. Welker, and M. Riedmiller, “Line based robot localization under natural light conditions,” Proceedings of European Conference on Artificial Intelligence (ECAI) 2004-Workshop on Agents in Dynamic & Real-Time Environments, Valencia, Spain, 2004.
[22] M. Dissanayake, P. Newman, M., S. Clark, H. Durrant-Whyte, and M. Csorba,
“A solution to the simultaneous localization and map building (SLAM) problem,” IEEE Transactions on Robotics & Automation, vol. 17, no. 3, pp.
229-241,2001..
[23] H. L. Chou and W. H. Tsai, "A new approach to robot location by house corners,"
Pattern Recognition, vol. 19, No. 6, 1986, pp. 439-451.
[24] S. Y. Chen and W. H. Tsai, "Determination of robot locations by common object shapes," IEEE Transactions on Robotics & Automation, vol. 7, no. 1, pp.
149-156, 1991.
[25] O. Parlaktuna, T. Bilir, and A. Yazici, “Localization of a mobile robot using natural landmarks and sensor fusion,” Proceedings of 4th International Conference on Electrical & Electronics Eng. (ELECO 2005), Bursa,Turkey, pp.394-397, Dec. , 2005.
[26] S. Y. Park, S. C. Jung, Y. S. Song, and H. J. Kim, “jo4,” Proceedings of 2008 IAPR Workshop on Cognitive Information Processing, Santorini, Greece, June 9-10, 2008.
[27] J. P. Barreto and H. Araujo, “Fitting conics to paracatadioptric projections of lines,” Computer Vision & Image Understanding, vol. 101, 151–165, 2006.
[28] R. I. Hartley, “Camera calibration using line correspondences,” Proceedings of DARPA Image. Understanding Workshop, pp. 361-366, 1993.
[29] A. Ramisa, A. Tapus, R. L. de Mantaras, R. Toledo, “Mobile robot localization using panoramic vision and combinations of feature region detectors,”
Proceedings of IEEE International Conference on Robotics & Automation, Pasadena, CA, USA, May 19-23, 2008, pp. 538-543.
[30] B. Siemiątkowska1 and R. Chojecki, “Mobile robot localization based on omnicamera,” Proceedings of 5th IFAC/EURON Symposium on Intelligent Autonomous Vehicles, Lisboa, Portugal, July 5-7, 2004.
[31] D. Cobzas and H. Zhang, “Mobile robot localization using planar patches and a stereo panoramic model,” Proceedings of Vision Interface Annual Conference,
166
Ottawa, Canada, pp. 94- 99.
[32] J. P. Barreto and H. Araujo, “Geometric properties of central catadioptric line images and their application in calibration,” IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 27, no. 8, Aug. 2005.
[33] X. Ying and H. Zha, “Geometric interpretations of the relation between the image of the Absolute Conic and Sphere Images,” IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 28, no. 12, Dec. 2006.
[34] T. Mashita, Y. Iwai, and M. Yachida, “Calibration method for misaligned catadioptric camera,” IEICE Transactions on Information & Systems, vol. E89-D, no. 7, pp. 1984-1993, July 2006.
[35] H Ukida, N Yamato, Y Tanimoto, T Sano, and H Yamamoto, “Omni-directional 3D Measurement by Hyperbolic Mirror Cameras and Pattern Projection,”
Proceedings of 2008 IEEE Conference on Instrumentation & Measurement Technology, Victoria, BC, Canada, May 12-15, 2008, pp. 365-370.
[36] S. W. Jeng and W. H. Tsai, "Using pano-mapping tables to unwarping of omni-images into panoramic and perspective-view Images," IET Image Processing, vol. 1, No. 2, pp. 149-155, June 2007.
[37] R. O. Duda and P. E. Hart, “Use of the Hough transformation to detect lines and curves in pictures,” Communication ACM, vol. 15, no. 1, 1972 pp. 11-15.
[38] Z. F. Yang and W. H. Tsai, “Using parallel line information for vision-based landmark location estimation and an application to automatic helicopter landing,” Robotics and Computer-Integrated Manufacturing, vol. 14, pp.
297–306, 1998.
[39] S. Saripalli, J. F. Montgomery, and G. S. Sukhatme, “Visually-guided landing of an unmanned aerial vehicle,” IEEE Transactions on Robotics and Automation, vol. 19, no. 3, pp 371-381. June 2003.
[40] R. O. V. Shakernia, C. S. Sharp, Y. Ma, and S. Sastry, “Multiple view motion estimation and control for landing an unmanned aerial vehicle,” Proceedings of 2002 IEEE International Conference on Robotics and Automation, Washington, DC, U. S. A., pp. 2793 - 2798, May 2002.
[41] P. Garcia-Pardo, G. Sukhatme, and J. Montgomery, "Towards vision-based safe landing for an autonomous helicopter", Robotics and Autonomous Systems, vol.
38, no. 1, pp. 19-29, 2001.
[42] L. Mejias, P. Campoy, S. Saripalli, and G. S. Sukhatme, “A visual servoing
167
approach for tracking features in urban areas using an autonomous helicopter,”
Proceedings of 2006 IEEE International Conference on Robotics and Automation, Orlando, Florida, U. S. A., pp. 2503-2508, May 2006.
[43] A. C. Tsai, P. W. Gibbens and R. H. Stone, “Visual position estimation for automatic landing of a tail-sitter vertical takeoff and landing unmanned air vehicle,” Mechatronics and Machine Vision in Practice, J. Billingsley and R.
Bradbeer (eds.) Springer-Verlag, New York, New York, U. S. A., pp. 181-191, 2008.
[44] O.A. Yakimenko, I. I, Kaminer, W. J. Lentz and P. A. Ghyzel, “Unmanned aircraft navigation for shipboard landing using infrared vision,” IEEE Transactions on Aerospace and Electronic Systems, vol. 38, pp. 1181-1200, 2002.
[45] J. M. Hespanha, O. A.Yakimenko, I. I. Kaminer, and A. M. Pascoal, ”Linear parametrically varying systems with brief instabilities: an application to vision/inertial navigation,” IEEE Transactions on Aerospace and Electronic Systems, vol. 40, pp. 889-902, 2004.
[46] S. Hrabar and G. S. Sukhatme “Omnidirectional vision for an autonomous helicopter,” Proceedings of IEEE International Conference on Robotics and Automation, Taipei, Taiwan, Sept. 14-19, 2003, pp. 3602-3609.
[47] C. Demonceaux, P. Vasseur, and C. Pegard, “Omnidirectional vision on uav for attitude computation,” Proceedings of 2006 IEEE International Conference on Robotics and Automation, Orlando, Florida, U. S. A. , pp. 2842 – 2847, May 2006.
[48] J. C. Bazin, I. Kweon, C. Demonceaux, and P. Vasseur, “UAV attitude estimation by vanishing points in catadioptric images,” Proceedings of 2008 IEEE International Conference on Robotics and Automation, Pasadena, California, U.
S. A. , pp. 2842 - 2847, May 2008.
[49] C. J. Wu and W. H. Tsai, “Location estimation for indoor autonomous vehicle navigation by omni-directional vision using circular landmarks on ceilings,”
Robotics and Autonomous Systems, vol. 57, no. 5, pp. 546-555, May 2009.
[50] H. H. Lin and J. H. Lin “A Visual Positioning System for Vehicle or Mobile Robot Navigation,” IEICE Transactions on Information and Systems, vol. E89-D, no. 7, 2006, pp. 2109-2116
[51] K. Sakurai, S. Kyo and S. Okazaki “Overtaking Vehicle Detection Method and
168
Its Implementation Using IMAPCAR Highly Parallel Image Processor,” IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences, vol. E91-D, no. 7, 2008, pp. 1899-1905.
[52] C. C. Lai and W. H. Tsai “Estimation of Moving Vehicle Locations Using Wheel Shape Information in Single 2-D Lateral Vehicle Images by 3-D Computer Vision Techniques. Robot,” Robotics and Computer-Integrated Manufacturing, vol. 15, 1999, pp. 111-120.
[53] Y. Cao, A. Renfrew and P. Cook “Vehicle Motion Analysis Based on a Monocular Vision System,” Proc. RTIC 2008 Conference, Manchester, UK, 2008, pp. 1-6.
[54] C. J. Wu and W. H. Tsai, Location Estimation for Indoor Autonomous Vehicle Navigation by Omni-Directional Vision Using Circular Landmarks on Ceilings, Robotics and Autonomous Systems, vol. 57, no. 5, 2009, pp. 546-555.
[55] O. A. Aider, P. Hoppenot, and E. Colle, “A model-based method for indoor mobile robot localization using monocular vision and straight-line correspondences,” Robotics and Autonomous Systems, vol. 52, pp. 229-246, Aug.
2005.
[56] E. E. Hemayed, “A survey of camera self-calibration,” Proceedings of IEEE Conference on Advanced Video and Signal Based Surveillance, pp. 351- 357, Miami, Florida, USA, July 21-22, 2003.
[57] Z. F. Yang and W. H. Tsai, “Viewing corridors as right parallelepipeds for vision-based vehicle localization,” IEEE Transactions on Industrial Electronics, vol. 46, no. 3, pp. 653-661, June 1999.
[58] C. H. Ku and W. H. Tsai, “Obstacle avoidance in person following for vision-based autonomous land vehicle guidance using vehicle location estimation and quadratic pattern classifier,” IEEE Transactions on Industrial Electronics, vol. 48, no. 1, pp. 205-215, Feb. 2001.
[59] T. Takeshita, T. Tomizawa and A. Ohya, “A house cleaning robot system – path indication and position estimation using ceiling camera,” Proceedings of International Joint Conference on SICE-ICASE, pp. 2653-2656, Busan, Korea, Oct. 18-21, 2006.
[60] Y. T. Wang and W. H. Tsai, “Indoor security patrolling with intruding person detection and following capabilities by vision-based autonomous vehicle navigation,” Proceedings of 2006 International Computer Symposium (ICS 2006)
169
- Workshop on Image Processing, Computer Graphics, and Multimedia Technologies, Taipei, Taiwan, Dec. 4-6, 2006.
[61] H. C. Chen and W. H. Tsai, “Optimal security patrolling by multiple vision-based autonomous vehicles with omni-monitoring from the ceiling,”
Proceedings of 2008 International Computer Symposium, Taipei, Taiwan, Nov.
13-15, 2008.
[62] J. Gomes, L. Darsa, B. Costa, and L. Velho, Warping and Morphing of Graphical Objects, Morgan Kaufmann Publishers, Inc., San Francisco, CA, USA, 1999.
[63] Z. Xiong, W. Chen and M. Zhang, “Catadioptric omni-directional stereo vision and its applications in moving objects detection,” in Computer Vision, Z. Xiong (ed.), Vienna, Austria, Nov. 2008
[64] S. Baker and S. K. Nayar, “A theory of single-viewpoint catadioptric image formation,” International Journal of Computer Vision, vol. 35, no. 2, pp.
175-196, 1999.
[65] G. Tarak and M. M. Trivedi, “Parametric ego-motion estimation for vehicle surround analysis using omni-camera,” Machine Vision and Applications, vol. 16, no. 2, pp. 85-95, 2005.
[66] M. Senoh, F. Kozawa, and M. Yamada, “Development of shape measurement system using an omnidirectional sensor and light sectioning method with laser beam scanning for Hume pipes,” Optical Engineering, vol. 45, no. 6, June 2006.
[67] T. Boult, “DOVE: Dolphin omni-directional video equipment,” Proceedings of IASTED International Conference on Robotics and Automation, Honolulu, Hawaii, USA, Aug. 14-16, 2000.
[68] S. K. Nayar, “Catadioptric omni-camera,” Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, San-Juan, Puerto Rico, June 1997, pp.
482-488.
[69] Y. Onoe, N. Yokoya, K. Yamazawa, and H. Takemura, “Visual surveillance and monitoring system using an omni-directional video camera,” Proceedings of 14th International Conference on Pattern Recognition, Brisbane, Australia, vol. 1, pp.
588-592, Aug. 16-20, 1998.
[70] X. H. Ying and Z. Y. Hu, “Catadioptric camera calibration using geometric invariants,” IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.
26, no. 10, pp. 1260-1271, Oct. 2004.
170
[71] S. W. Jeng and W. H. Tsai, "Analytic image unwarping by a systematic calibration method for omni-cameras with hyperbolic-shaped mirrors," Image and Vision Computing, vol. 26, no. 5, pp. 690-701, May 2008.
[72] S. W. Jeng and W. H. Tsai, “Construction of perspective and panoramic images from omni-images taken from hypercatadioptric cameras for visual surveillance,” Proceedings of 2004 IEEE International Conference on Networking, Sensing, and Control, Taipei, Taiwan, pp. 204-209, March 21-23, 2004.
[73] K. Fukunaga, Introduction to Statistical Pattern Recognition (2nd edition).
Academic Press, New York, NY, USA, 1990.
[74] R. Swaminathan, M.D. Grossberg, S.K. Nayar, “Caustics of catadioptric cameras,” Proceedings of Eighth IEEE International Conference on Computer Vision, Vancouver, BC, Canada, vol. 2, July 7–14, 2001, pp. 2–9.
[75] J. Gomes, L. Darsa, B. Costa, and L. Velho. Warping and Morphing of Graphical Objects. Morgan Kauffmann Publishers, San Francisco, CA, USA, 1999.
171
Publication List
Journal Papers:
(1) C. J. Wu and W. H. Tsai (2009/05). "Location Estimation for Indoor Autonomous Vehicle Navigation by Omni-Directional Vision Using Circular Landmarks on Ceilings," Robotics and Autonomous Systems, Vol. 57, No. 5, pp. 546-555. (SCI, EI)
(2) C. J. Wu and W. H. Tsai "Unwarping of Images Taken by Misaligned
Omni-cameras without Camera Calibration by Curved Quadrilateral Morphing Using Quadratic Pattern Classifiers, " Optical Engineering (accepted and to appear) (SCI, EI).
(3) C. J. Wu and W. H. Tsai "A Systematic Approach to Indoor Vision-Based Robot Localization Using Corner Features in Omni Images," submitted to the IEEE Transactions on Industrial Electronics. (SCI, EI)
(4) C. J. Wu and W. H. Tsai " An Omni-vision Based Self-localization Method for Automatic Helicopter Landing on Standard Helipads," submitted to the IEEE Transactions on Aerospace and Electronic Systems. (SCI, EI)
(5) C. J. Wu and W. H. Tsai " A Space-Mapping Method for Object Location Estimation Adaptive to Camera Setup Changes for Vision-based Au-tomation Applications" submitted to IEEE Transactions on. Circuits and Systems for Video Technology (SCI, EI).
(6) C. J. Wu and W. H. Tsai "Omni-vision Based Localization of Lateral Vehicles for Car Driving Assistance," submitted to Robotics and Computer-Integrated
Manufacturing (SCI, EI).
(7) C. J. Wu and W. H. Tsai "Automatic Airplane Landing by Omni-vision Based Self-localization,” submitted to IEEE Transactions on. Circuits and Systems for Video Technology (SCI, EI)
Patents
(8) C. J. Wu, S. Y. Tsai, and W. H. Tsai. "Automatic Ultrasonic And
Computer-Vision Navigation Device And Method Using The Same," USA patent, No.12/365190. (pending)
(9) C. J. Wu and W. H. Tsai(2007). “A Landmark-Based Vehicle Location System
172
And Its Technique,” Republic of China Patent, No. I 274971, 2007 (granted).
(10) K. F. Chien, C. J. Wu and W. H. Tsai. "A Video Surveillance System And Its Method Based On Data Hiding And Video Encoding," Republic of China Patent, No. 200837661. (pending)
(11) H. Y. Chen, M. J. Han, C. J. Wu, H. H. Lin, C. J. Kang, Y. Y. Yang, K. T, Song, W.
H. Tsai, and J. H. Chuang. "A Mobile System of Image Acquisition And Its Controlling Method," Republic of China Patent, No. 096127495. (pending).
(12) C. J. Wu, S. Y. Tsai, and W. H. Tsai. ”A Method for Mobile Robot Navigation System with Simple Learning Procedures by Ultrasonic Sensing And Computer Vision Techniques,” Republic of China Patent, No. 97131096. (pending).
Conference Papers
(13) C. J. Wu and W. H. Tsai "A novel method for lateral vehicle localization by omni-cameras for car driving assistance," Proceedings of 13th International Conference on Knowledge-Based and Intelligent Information & Engineering Systems (KES2009), Santiago, Chile, September 28-30, 2009.
(14) C. J. Wu and W. H. Tsai "Adaptation of Space-Mapping Methods for Object Location Estimation to Camera Setup Changes --- A New Study," Proceedings of 13th International Conference on Knowledge-Based and Intelligent Information
& Engineering Systems (KES2009), Santiago, Chile, September 28-30, 2009.
(15) C. J. Wu and W. H. Tsai. “Location estimation for indoor autonomous vehicle navigation by omni-directional vision using circular landmarks on ceilings,”
Proceedings of 2005 Conference on Computer Vision, Graphics and Image Processing, Taipei, Taiwan, Republic of China, 21-23 Aug.,2005
Technology transfers and patent authorizations
(16) C. J. Wu, Y. T. Wang, W. H. Tsai, et al., Contract No. NCTU-05A032, Micro star international, Taiwan, April 2006.
(17) C. J. Wu, S. Y. Tsai, W. H. Tsai, et al., Contract No.NCTU-08A018, Micro star international, Taiwan, April 2008.
(18) C. J. Wu, S. Y. Tsai, W. H. Tsai, et al., Contract No. NCTU-08A051, Industrial technology research institute of Taiwan, Nov. 2008.
173
Vita
Chih-Jen Wuwas born in Taichung, Taiwan, R.O.C., in 1978. He received the B.S. degree and the M. S. degree in engineering science from National Cheng Kung University, Tainan, Taiwan, in 2000 and 2002, respectively, and the Ph.D. degree in the Institute of Computer Science and Engineering, College of Computer Science from National Chiao Tung University in 2009.
Mr. Wu worked as a research assistant in the Laboratory of System Integration in National Cheng Kung University from August 2000 to July 2002, and as a research engineer in the Computer Vision Laboratory in National Chiao Tung University since August 2002 till now. His current research interests include computer vision, robotics, pattern recognition, and their applications.