Compare with the similar work on GPU [2], there are many improvements on our architecture. In the visually-impaired aid system, we can not equip a GPU because the power consumption is too large. The volume is another issue, the platform of GPU is too huge to be portable. Except for power and area, the self-evident advantages on ASIC compare to GPU. Our ar-chitecture is more high-speed and saving of memory. Tab. 5.2 shows the comparison between the similar work [2] and our architecture. On the mem-ory usage, we reduce SRAM from 13900K to 32K bytes. Fig. 5.5 shows the comparison of average processing time. In this figure, our architecture is faster than the compared work.
Our proposed work utilized TSMC 65nm technology. The detail specifi-cation shown in Tab. 5.3. The chip is operated under 200MHz with 56mW in power consumption. Through our methodologies, 2000 frame with 640∗480
0 10000 20000 30000 40000 50000 60000 70000
1 ms 2 ms 3 ms 4 ms 5 ms 6 ms
The compared work on GPU Out proposed method
Figure 5.5: The comparison of processing time to the similar work [2].
resolution per second in general case and 28 frames per second in worst case.
Our proposed architecture shows the high efficiency processing performance in frame rate and memory usage.
Table 5.2: The comparison of segmentation algorithm between the current implementation and our proposed method.
Abramov.’s [2] Proposed Work Platform NVIDIA GeForce GTX295 GPU ASIC with TSMC 65nm Processing Speed 2000 pixels per ms 9600 pixels per ms
Memory Usage 13900K Bytes 32K Bytes
Power consumption 300W 56mW
57
Table 5.3: Chip summary.
Item Specification Gates/SRAM 453K Gates / 32KB
Operating Freq. 200MHz
Power Consumption Average 52.5mW
Input Image 640*480
Frame Rate(General Case) 2000fps
Frame Rate(Worst Case) 28fps
Co mp uta tio na l B loc k
Figure 5.6: The memory usage of the computational block. There are 8 256∗128 SRAMs in our architecture. The access of each computational block only need one clock.
Chapter 6 Conclusion
In the thesis, we design a portable vision-based visually impaired aid sys-tem. In our system, we proposed three robust algorithm for the traffic analysis and navigation. The chip implementation is also a main contri-bution to largely reduce the computational time. The first algorithm is intelligent depth-based obstacle detection. This algorithm aims to assist the visually-impaired in detecting obstacles with distance information for safety. With analysis of the depth map, segmentation and noise elimination are adopted to distinguish different objects according to the related depth information. Obstacle extraction mechanism is proposed to capture obsta-cles by various object proprieties revealing in the depth map. The proposed system can also be applied to emerging vision-based mobile applications, such as robots, intelligent vehicle navigation, and dynamic surveillance sys-tems. Experimental results demonstrate the proposed system achieves high accuracy. In the indoor environment, the average detection rate is above 96.1%. Even in the outdoor environment or in complete darkness, 93.7%
detection rate is achieved on average.
The second part is the depth characteristic analysis. These three func-tions in this part are road detection, wall detection and stair detection.
Depth analysis has become a popular field on computer vision recently be-59
cause of the accuracy and robustness. We proposed three powerful methods for the visually-impaired aid system. Regardless of color variation, the pro-posed algorithm still work with high performance. The experimental results could also prove the robustness. In the road detection algorithm, The de-tection rate is above 93% and the false alarm rate is 3.25%. In the wall detection algorithm, there is about 94.05% detection rate and 2.3% false alarm rate. In the stair detection, our proposed algorithm achieves 96.05%
detection rate and 0.8% false alarm rate.
The third part is accurate positioning system based on street view recog-nition. Vision-based technique is employed for dynamically recognizing shop or building signs on the GPS map. Two mechanisms including view-angle invariant distance estimation and path refinement are proposed for robust and accurate position estimation. Through the combination of visual recog-nition technique and GPS scale data, the real user location can be accurately inferred. Experimental results demonstrate that the proposed system is reli-able and feasible. Compared with 20m error of position estimation provided by the GPS, our system only has 0.97m error estimation.
In the final part, the critical path, segmentation and morphology, in our system are picked out to discuss and design. we proposed a new architec-ture and chip implementation with TSMC 65nm technology. The chip is operated under 200MHz with 56mW in power consumption. Through our methodologies, 2000 frame with 640∗480 resolution per second in general case and 28 frames per second in worst case. Our proposed architecture shows the high efficiency processing performance in frame rate and memory usage.
Totally speaking, we design a portable vision-based visually impaired aid system for the visually impaired. Previous mobility tools such as white canes and guide dogs for the blind still have limited usability. It is incon-venient in crowded regions and some social situations. Several commercial
61 products such as electronic canes are limited to the functionality in visual information extraction in the surrounding environment. Compared to tradi-tional ETAs, vision-based navigation tools are smarter to detect preceding obstacles as well as extract visual information of the world in real-time.
However, state-of-the-arts of vision-based navigation systems are too bulky and still reveal many limitations those restrict the reliability and usabil-ity in practice. To address these problems, we aim to develop an integrated vision-based navigation system to assist blind persons from their basic needs to advanced requirements in lives.
Bibliography
[1] Z. Hu, F. Lamosa, and K. Uchimura, “A complete u-v-disparity study for stereovision based 3d driving environment analysis,” in 3-D Digital Imaging and Modeling, 2005. 3DIM 2005. Fifth International Confer-ence on, pp. 204 – 211, june 2005.
[2] A. Abramov, T. Kulvicius, F. Worgotter, and B. Dellen, “Real-time image segmentation on a gpu,” in Facing the Multicore-Challenge (R. Keller, D. Kramer, and J.-P. Weiss, eds.), vol. 6310 of Lecture Notes in Computer Science, pp. 131–142, Springer Berlin / Heidelberg, 2011.
[3] J. Borenstein and I. Ulrich, “The guidecane-a computerized travel aid for the active guidance of blind pedestrians,” in Robotics and Au-tomation, 1997. Proceedings., 1997 IEEE International Conference on, vol. 2, pp. 1283 –1288 vol.2, apr 1997.
[4] D. Bolgiano and J. Meeks, E., “A laser cane for the blind,” vol. 3, p. 268, june 1967.
[5] A. G. J. P. Graffgna M. Guzzo G. Costa and O. Nasisi, “Mobility and orientation aid for blind persons using artifcial vision,” june 2007.
[6] D. Dakopoulos and N. Bourbakis, “Wearable obstacle avoidance elec-tronic travel aids for blind: A survey,” vol. 40, pp. 25–35, 2010.
[7] T. Ifukube, T. Sasaki, and C. Peng, “A blind mobility aid modeled after echolocation of bats,” vol. 38, pp. 461 –465, may 1991.
63
[8] J. Zelek, R. Audette, J. Balthazaar, and C. Dunk, “A stereo-vision system for the visually impaired,” 2000.
[9] M. Adjouadi, “A man-machine vision interface for sensing the environ-ment.,” vol. 29, pp. 57–76, 1992.
[10] O. Arif, W. Daley, P. Vela, J. Teizer, and J. Stewart, “Visual track-ing and segmentation ustrack-ing time-of-flight sensor,” in Image Processtrack-ing (ICIP), 2010 17th IEEE International Conference on, pp. 2241 –2244, sept. 2010.
[11] S. Meers and K. Ward, “Face recognition using a time-of-flight camera,”
in Computer Graphics, Imaging and Visualization, 2009. CGIV ’09.
Sixth International Conference on, pp. 377 –382, aug. 2009.
[12] P. Gemeiner, P. Jojic, and M. Vincze, “Selecting good corners for struc-ture and motion recovery using a time-of-flight camera,” in Intelligent Robots and Systems, 2009. IROS 2009. IEEE/RSJ International Con-ference on, pp. 5711 –5716, oct. 2009.
[13] L. Chen, B. long Guo, and W. Sun, “Obstacle detection system for visually impaired people based on stereo vision,” in Genetic and Evo-lutionary Computing (ICGEC), 2010 Fourth International Conference on, pp. 723 –726, dec. 2010.
[14] Z. Yankun, C. Hong, and N. Weyrich, “A single camera based rear obstacle detection system,” in Intelligent Vehicles Symposium (IV), 2011 IEEE, pp. 485 –490, june 2011.
[15] C. Pantilie, S. Bota, I. Haller, and S. Nedevschi, “Real-time obstacle detection using dense stereo vision and dense optical flow,” in Intel-ligent Computer Communication and Processing (ICCP), 2010 IEEE International Conference on, pp. 191 –196, aug. 2010.
65 [16] P. Jeong and S. Nedevschi, “Obstacle detection based on the hybrid road plane under the weak calibration conditions,” in Intelligent Vehi-cles Symposium, 2008 IEEE, pp. 446 –451, june 2008.
[17] R. Adams and L. Bischof, “Seeded region growing,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 16, pp. 641 –647, jun 1994.
[18] H. A. Q. Yu and H. Wang, “A stereovision method for obstacle de-tection and tracking in non-flat urban environments,” Pattern Anal-ysis and Machine Intelligence, IEEE Transactions on, vol. 19, no. 2, pp. 141 –157, 2005.
[19] P. Lombardi, M. Zanin, and S. Messelodi, “Unified stereovision for ground, road, and obstacle detection,” in Intelligent Vehicles Sympo-sium, 2005. Proceedings. IEEE, pp. 783 – 788, june 2005.
[20] X. Li, X. Yao, Y. Murphey, R. Karlsen, and G. Gerhart, “A real-time vehicle detection and tracking system in outdoor traffic scenes,” in Pattern Recognition, 2004. ICPR 2004. Proceedings of the 17th Inter-national Conference on, vol. 2, pp. 761 – 764 Vol.2, aug. 2004.
[21] J. McCall and M. Trivedi, “Video-based lane estimation and track-ing for driver assistance: survey, system, and evaluation,” Intelligent Transportation Systems, IEEE Transactions on, vol. 7, pp. 20 –37, march 2006.
[22] J. Zhao, J. Katupitiya, and J. Ward, “Global correlation based ground plane estimation using v-disparity image,” in Robotics and Automation, 2007 IEEE International Conference on, pp. 529 –534, april 2007.
[23] N. Soquet, D. Aubert, and N. Hautiere, “Road segmentation supervised by an extended v-disparity algorithm for autonomous navigation,” in Intelligent Vehicles Symposium, 2007 IEEE, pp. 160 –165, june 2007.
[24] T. Gandhi and M. Trivedi, “Pedestrian protection systems: Issues, sur-vey, and challenges,” Intelligent Transportation Systems, IEEE Trans-actions on, vol. 8, pp. 413 –430, sept. 2007.
[25] N. Suganuma, M. Shimoyama, and N. Fujiwara, “Obstacle detection using virtual disparity image for non-flat road,” in Intelligent Vehicles Symposium, 2008 IEEE, pp. 596 –601, june 2008.
[26] N. Suganuma, M. Shimoyama, and N. Fujiwara, “Obstacle map gener-ation using virtual disparity image for non-flat road,” in SICE Annual Conference, 2008, pp. 1920 –1925, aug. 2008.
[27] C.-H. Lee, Y.-C. Lim, S. Kwon, and J.-H. Lee, “Obstacle localization with a binarized v-disparity map using local maximum frequency values in stereo vision,” in Signals, Circuits and Systems, 2008. SCS 2008. 2nd International Conference on, pp. 1 –4, nov. 2008.
[28] Z. Kim, “Robust lane detection and tracking in challenging scenar-ios,” Intelligent Transportation Systems, IEEE Transactions on, vol. 9, pp. 16 –26, march 2008.
[29] J. Alvarez, T. Gevers, and A. Lopez, “Vision-based road detection using road models,” in Image Processing (ICIP), 2009 16th IEEE In-ternational Conference on, pp. 2073 –2076, nov. 2009.
[30] C. Guo and S. Mita, “Drivable road region detection based on homog-raphy estimation with road appearance and driving state models,” in Autonomous Robots and Agents, 2009. ICARA 2009. 4th International Conference on, pp. 204 –209, feb. 2009.
[31] C. Guo, S. Mita, and D. McAllester, “Stereovision-based road bound-ary detection for intelligent vehicles in challenging scenarios,” in Intel-ligent Robots and Systems, 2009. IROS 2009. IEEE/RSJ International Conference on, pp. 1723 –1728, oct. 2009.
67 [32] S. Zhou, J. Gong, G. Xiong, H. Chen, and K. Iagnemma, “Road detec-tion using support vector machine based on online learning and eval-uation,” in Intelligent Vehicles Symposium (IV), 2010 IEEE, pp. 256 –261, june 2010.
[33] J. Alvarez and A. L? andopez, “Road detection based on illuminant in-variance,” Intelligent Transportation Systems, IEEE Transactions on, vol. 12, pp. 184 –193, march 2011.
[34] H. Moradi, J. Choi, E. Kim, and S. Lee, “A real-time wall detection method for indoor environments,” in Intelligent Robots and Systems, 2006 IEEE/RSJ International Conference on, pp. 4551 –4557, oct.
2006.
[35] A. A. E. Abhinav Gupta and M. Hebert, “Blocks world revisited: Image understanding using qualitative geometry and mechanics,” in ECCV, vol. 6314/2010, pp. 842–496, 2010.
[36] D. Hoiem, A. Efros, and M. Hebert, “Geometric context from a single image,” in Computer Vision, 2005. ICCV 2005. Tenth IEEE Interna-tional Conference on, vol. 1, pp. 654 – 661 Vol. 1, oct. 2005.
[37] J. Hesch, G. Mariottini, and S. Roumeliotis, “Descending-stair detec-tion, approach, and traversal with an autonomous tracked vehicle,” in Intelligent Robots and Systems (IROS), 2010 IEEE/RSJ International Conference on, pp. 5525 –5531, oct. 2010.
[38] A. Albert, M. Suppa, and W. Gerth, “Detection of stair dimensions for the path planning of a bipedal robot,” in Advanced Intelligent Mecha-tronics, 2001. Proceedings. 2001 IEEE/ASME International Confer-ence on, vol. 2, pp. 1291 –1296 vol.2, 2001.
[39] Y. Cong, X. Li, J. Liu, and Y. Tang, “A stairway detection algo-rithm based on vision for ugv stair climbing,” in Networking, Sensing
and Control, 2008. ICNSC 2008. IEEE International Conference on, pp. 1806 –1811, april 2008.
[40] S. Yamaguchi and T. Tanaka, “Gps standard positioning using kalman filter,” in SICE-ICASE, 2006. International Joint Conference, pp. 1351 –1354, oct. 2006.
[41] A. B. Y. Hel, R. Martinl, “Approximate iterative least squares algo-rithms for gps positioning,” in Signal Processing and Information Tech-nology (ISSPIT), 2010 IEEE International Symposium on, pp. 231 – 236, dec. 2010.
[42] M. Mosavi, “Frequency domain modeling of gps positioning errors,” in Signal Processing, 2006 8th International Conference on, vol. 4, nov.
2006.
[43] M. Dawood, C. Cappelle, M. El Najjar, M. Khalil, and D. Pomorski,
“Vehicle geo-localization based on imm-ukf data fusion using a gps receiver, a video camera and a 3d city model,” in Intelligent Vehicles Symposium (IV), 2011 IEEE, pp. 510 –515, june 2011.
[44] M. S. A. Hakeem., R. Vezzani and R. Cucchiara, “Estimating geospatial trajectory of a moving camera,” in Pattern Recognition, 2006. ICPR 2006. 18th International Conference on, vol. 2, pp. 82 –87, 0-0 2006.
[45] F. Lin, B. Chen, and T. Lee, “Vision aided motion estimation for unmanned helicopters in gps denied environments,” in Cybernetics and Intelligent Systems (CIS), 2010 IEEE Conference on, pp. 64 –69, june 2010.
[46] D. Lowe., “Distinctive image features from scale-invariant keypoints,”
in International Journal of Computer Vision, vol. 60, pp. 91–110, 2004.
69 [47] S. Wess, K. Althoff, and G. Derwand, “Using k-d trees to improve the retrieval step in case-based reasoning,” in Topics in Case-Based Reasoning (S. Wess, K.-D. Althoff, and M. Richter, eds.), vol. 837 of Lecture Notes in Computer Science, pp. 167–181, Springer Berlin / Heidelberg, 1994.
[48] M. Fischler and R. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” in Communications of the ACM, vol. 24, pp. 381–395, June 1981.
[49] D. Drop, Z. Latala, J. Kasperek, R. Patury, P. Rajda, J. Sadowski, L. Szydlowski, and L. Wojnar, “Hardware accelerated watershed based echocardiographic image segmentation,” in Computers in Cardiology, 2005, pp. 793 –796, sept. 2005.
[50] K.-T. Hu, Y.-T. Pai, S.-J. Ruan, and E. Naroska, “A hardware-efficient color segmentation algorithm for face detection,” in Circuits and Sys-tems (APCCAS), 2010 IEEE Asia Pacific Conference on, pp. 688 –691, dec. 2010.