Discussions

From our experiments and the results, we can see that the goal of utilizing a pair of two-camera omni-directional imaging devices equipped on the video surveillance vehicle roof to perform video surveillance of nearby cars has been achieved.

However, the proposed system still has some problems. In this study, we adopt the method of optical flow analysis to compute the motion vectors of the road surface appearing in consecutive omni-images and estimate the moving direction of the video

surveillance vehicle using these motion vectors. As a result, moving objects, such as moving cars or a group of people, in the image will result in undesired motion vectors, leading to erroneous detection results of the moving direction of the vehicle itself. A possible solution is to add extra functions for detecting these unusual motion vectors and ignoring them.

Moreover, to conduct the video surveillance work at the outdoor space, the sun light is an important factor to consider. The unsuitable adjustment of the camera parameters and the shadow produced by the sun light will affect the result of the car detection experiments. A possible solution for this problem is to record typical climate conditions and the corresponding suitable camera parameters and threshold values for car shape segmentation and other image processing works. In this way, the system can be chosen appropriate data set for each climate condition at the time of nearby–car detection. Of course, it is always desired to have fully automatic method for car detection for all climate conditions.

In this study, the detected car in the experiments is a saloon car. As a result, to increase the accuracy of detecting car position, the assumption of car height is 80 cm.

If the detected car is not a saloon car, we have to adjust the height parameter to make the car mask match the detected region for estimating car position.

Chapter 8

Conclusions and Suggestions for Future Works

8.1 Conclusions

In this study, a video surveillance system utilizing a pair of two-camera omni-directional imaging devices equipped on the video surveillance vehicle roof to monitor the surrounding environment has been proposed. With the advantage of mobility of the video surveillance vehicle and the wide FOV of the omni-camera system, several methods have been proposed for various purposes of driving condition monitoring and nearby car detection, as summarized in the following.

(1) A method for speeding up generation of perspective-view images for vehicle surrounding environment monitoring has been proposed, which, with the help of a perspective mapping table, can generate perspective-view images in realtime.

(2) A method for analyzing the omni-images of surrounding environments for car-driving assistance has been proposed, which, by optical flow analysis, provides the driver of the video surveillance vehicle a relevant perspective-view image of blind spots during car turning.

(3) A method for off-line inspection of the driving history has been proposed, by which all the sequential omni-images of the driving history can be recorded

online and displayed off-line in the form of a perspective-view image sequence with the viewing direction determined by mouse clicks.

(4) A method for monitoring a nearby static car around a static video surveillance vehicle has been proposed, which eliminates the ground region in acquired omni-images and detect the nearby car shape in the image by region growing and morphological techniques and displays a top-view surround map with the detected car included for inspection of its relative position.

(5) A method of monitoring of a nearby static or moving car with a moving video surveillance vehicle has been proposed, which uses motion vectors produced by optical flow analysis as well as color information of segmented objects to detect the nearby car shape, and uses a rectangular-shaped car model to match the detected car for estimating the car position and generating the surround map.

The experimental results shown in the previous chapters have revealed the feasibility of the proposed system

8.2 Suggestions for Future Works

According to the experience obtained this study, in the following we make suggestions of some interesting issues, which are worth further investigation in the future.

1. Increasing the speed of computation to achieve vehicle detection in realtime, e.g., by parallel computing.

2. Developing the capability of detecting and tracking multiple nearby vehicles in the surrounding environment.

3. Developing more applications of car-driving assistance using the omni-camera system, e.g., analysis of the driving behavior.

4. Adding the capability of detecting passing-by persons with a moving video surveillance vehicle.

5. Enhancing the image analysis capability to detect more information of the nearby car, e.g., the size of the vehicle.

6. Only using an omni-camera to compute the 3D data of the detected car.

References

[1] C. Micheloni, G.L. Foresti, C. Piciarelli and L. Cinque, “An autonomous vehicle for video surveillance of indoor environments,” IEEE Transactions on Vehicular Technology, vol. 56, no. 2, March 2007.

[2] G.L. Foresti, C. Micheloni and L. Snidaro, “Event classification for automatic visual-based surveillance of parking lots,” Proceedings of 17th International Conference on Pattern Recognition, vol. 3, pp. 314–317, 2004.

[3] M. Bramberger, R. P. Pflugfelder, A. Maier, B. Rinner, B. Strobl and H. Schwabach, “A smart camera for traffic surveillance,” Proceedings of 1st Workshop on Intelligent Solutions in Embedded Systems, pp. 153–164, Vienna, Austria, 2003.

[4] Y. Onoe, N. Yokoya, K Yamazawa, and H. Takemura, “Visual surveillance and monitoring system using an omnidirectional video camera,” Proc. 1998 Int’l Conf. Pattern Recog, vol. 1, pp. 588–592, Brisbane, Australia, Aug. 16-20, 1998.

[5] T. Mituyosi, Y. Yagi, and M. Yachida, “Real-time human feature acquisition and human tracking by omnidirectional image sensor,” Proc. IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, pp.

258-263, Tokyo, Japan, July 30-Aug. 1, 2003.

[6] L. He, C. Luo, F. Zhu, Y. Hao, J. Ou and J. Zhou, “Depth map regeneration via improved graph cuts using a novel omnidirectional stereo sensor,”Proceedings of 11th IEEE International Conference on Computer Vision (ICCV2007), pp. 1-8, Oct. 14-21, Rio de Janeiro, Brazil, 2007.

[7] H. Koyasu, J. Miura, and Y. Shirai, “Realtime omnidirectional stereo for obstacle detection and tracking in dynamic environments,” Proceedings of 2001

IEEE/RSJ Internatonal Conference on Intelligent Robots and Systems, pp. 31-36, Maui, Hawaii, USA, Oct./Nov, 2001.

[8] S. W. Jeng and W. H. Tsai, “Using pano-mapping tables for unwarping of omni-images into panoramic and perspective-view images,” Journal of IET Image Processing, Vol. 1, No. 2, pp. 149-155, June 2007.

[9] T. Ehlgen and T. Pajdla, “Maneuvering aid for large vehicle using omnidirectional cameras,” IEEE Workshop on Applications of Computer Vision, pp. 17–17, Austin, Texas, US, Feb. 2007.

[10] T. Gandhi and M.M. Trivedi, “Motion analysis for event detection and tracking with a mobile omni-directional camera,” ACM Multimedia Systems Journal, Special Issue on Video Surveillance, vol. 10, no. 2, pp. 131–143, 2004.

[11] N. Murakami, A. Ito, Jeffrey D. Will , Michael Steffen, K. Inoue, K. Kita, S.

Miyaura, “Development of a teleoperation system for agricultural vehicles,”

Computers and Electronics in Agriculture, vol. 63, pp. 81-88, Aug. 2008.

[12] R. Aufrère, J. Gowdy, C. Mertz, C. Thorpe, C.C. and T.Y. Wang, “Perception for collision avoidance and autonomous driving,” Mechatronics, Vol. 13, pp.

1149-1163 ,2003.

[13] Hughes, C., Glavin, M., Jones, E.and Denny, P. “Wide-angle camera technology for automotive applications: a review,” IEEE Transactions on Intelligent Transportation Systems, pp. 19–31, Mar. 2009.

[14] C. Hughes, M. Glavin1, E. Jones1 and P. Denny, “Wide-angle camera technology for automotive applications: a review,” IEEE Transactions on Intelligent Transportation Systems, vol. 3, no. 1, pp. 19–31, 2009.

[15] B. D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” Proc. 7th nternational Joint Conference onArtificial

Intelligence, Vancouver, Canada, pp. 674–679, 1981.

[16] J. Kim and Y. Suga, “An omnidirectional vision-based moving obstacle in mobile robot,” International Journal of Control, Automation, and Systems, vol. 5, no. 6, pp. 663-673, Dec. 2007.

[17] S. Gupte, O. Masoud, R.F. K. Martin, and N.P. Papanikolopoulos, “Detection and classification of vehicles,” IEEE Transactions on Intelligent Transportation Systems, vol. 3, no. 1, Mar. 2002.

[18] R. Cucchiara, M. Piccardi, and P. Mello, “Image analysis and rule-based reasoning for a traffic monitoring system,” IEEE Transactions on Intelligent Transportation Systems,vol. 1, no. 2, June 2000.

[19] L. W. Tsai, J. W. Hsieh and K. C. Fan, “Vehicle detection using normalized color and edge map,” IEEE Transactions on Image Processing, vol. 16, no. 3, Mar.

2007.

[20] P. H. Yuan, K. F. Yang and W. H. Tsai, “Security monitoring around a video surveillance car with a pair of two-camera omni-directional imaging devices,”

Proceedings of 2010 Workshop on Image Processing, Computer Graphics, and Multimedia Technologies, International Computer Symposium, pp. 325-330, Tainan, Taiwan, Dec. 2010.

[21] C. J. Wu, “New localization and image adjustment techniques using omni-cameras for autonomous vehicle applications,” Ph. D. Dissertation, Institute of Computer Science and Engineering, National Chiao Tung University, Hsinchu, Taiwan, Republic of China, July 2009.

[22] W. H. Tsai, “Moment-preserving thresholding: a new approach,” Computer Vision, Graphics, and Image Processing, vol. 29, no. 3, pp. 377-393, 1985.