Most complete mapping systems require three considerations, which are the spatial alignment of consecutive data frames to achieve localization task, the detection of loop closures and the globally consistent alignment of all data frames [1: Henry et al. 2012].
This thesis implements the 3D mapping system considering the spatial alignment
without the loop detection and global consistency. Since the feature-based localization method is processing frame-by-frame, the accumulating drift of all data frames is large when the camera moves for a long distance and therefore the endpoints of a loop cannot be aligned together. This is the main problems of the proposed 3D model reconstruction system of this thesis that should be solved in the future. Besides, since the geometry of binocular stereo camera is fixed, the physical relationship between left and right images can be another constraint to make the localization result more accurate as proposed in [18: Kitt et al. 2010]. Moreover, the proposed 3D model reconstruction system does not
consider how to model and update the mapping data. In [1: Henry et al. 2012], each point in 3D model are transformed to the “surface element (Surfel)” data structure with the proposed update strategy. With Surfel mapping model and update strategy, not only the visualization result is improved by using surface representation, but also make the update task more easily.
On the other hand, for the proposed stereo refinement method, hole region to be filled is selected by a fix threshold currently. However, due to the properties of camera projection, the size of a certain hole changes according to the distance to the camera coordinate. Therefore, a dynamic range of the filling hole selection mechanism based on the measurement distance to the camera coordinate is the future work to improve the method.
For the proposed object detection and tracking system, three aspects can be improved and extended. First of all, this thesis use visibility-based occupancy grid to detect object. However, the advantage of Bayesian occupancy filter (BOF) framework does not implement currently in thesis. With BOF framework, a static global map can be constructed by several frame data, and then moving object can be filtered out by comparing the local u-disparity occupancy grid map to the global occupancy grid map.
The same concept implemented in Cartesian space using laser range finder has been proposed in [34: Wolf et al. 2004]. Secondly, as the experiment results mentioned in Subsection 6.3.2, the Kalman filter with constant velocity motion model is not quite accurate when an object moves along a circular path. To overcome this problem, extended Kalman filter (EKF) with nonlinear dynamic model might be a solution. Third, the proposed object tracking system has not been integrated into the proposed 3D environment reconstruction system. Combining the localization method in the first topic mentioned in Section 4.1 and the u-disparity occupancy grid with BOF framework to handle the dynamic environment is the next work of this thesis.
References
[1: Henry et al. 2012]
Peter Henry, Michael Krainin, Evan Herbst, Xiaofeng Ren, Dieter Fox, “RGB-D Mapping: Using Kinect-style Depth Cameras for Dense 3D Modeling of Indoor Environments,” International Journal of Robotics Research, vol. 31, no. 5, pp.
647-663, April 2012.
[2: Marcincin et al. 2012]
J.Novak-Marcincin, J. Torok, J. Barna, M. Janak, L. Novakova-Marcincinova and V. Fecova, “Realization of 3D Models for Virtual Reality by Use of Advanced Scanning Methods,” in Proceedings of IEEE International Conference on Cognitive Infocommunications, pp. 787-790, December 2-5, 2012.
[3: Park et al. 2012]
DongRyeol Park, Joon-Kee Cho and Yeon-Ho Kim, “A Visual Guidance System for Minimal Invasive Surgery Using 3D Ultrasonic and Stereo Endoscopic Images,”
in Proceedings of IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics, Roma, Italy, pp. 872-877, June 24-27, 2012.
[4: Noonan et al. 2009]
David P. Noonan, Peter Mountney, Daniel S. Elson, Ara Darzi and Guang-Zhong
Yang, “A Stereoscopic Fibroscope for Camera Motion and 3D Depth Recovery during Minimally Invasive Surgery,” in Proceedings of IEEE Conference on Robotics and Automation, Kobe, Japan, pp. 4463-4468, May 12-17, 2009.
[5: Zeisl et al. 2012]
Bernhard Zeisl, Kevin Koser and Marc Pollefeys, “Viewpoint Invariant Matching via Developable Surfaces,” in Proceedings of the 12th International Conference on Computer Vision, pp. 62-71, 2012.
[6: Suarez et al. 2012]
Jesus Suarez and Robin R. Murphy, “Using the Kinect for Search and Rescue Robotics,” in Proceedings of the 2012 IEEE International Symposium on Safety, Security and Rescue Robotics (SSRR), pp. 1-2, November 5-8, 2012.
[7: Hu et al. 2012]
Gibson Hu, Shoudong Huang, Liang Zhao, Alen Alepijevic and Gamimi Dissanayake, “A robust RGB-D SLAM algorithm,” in Proceedings of IEEE International Conference on Intelligent Robots and Systems, Vilamoura, pp.
1714-1719, 7-12 October, 2012.
[8: Murray et al. 2005]
Don Murray and James J. Little, “Patchlets: Representing Stereo Vision Data with
Applications of Computer Vision, Breckenridge, CO, vol. 1, pp. 192-199, 5-7 January, 2005.
[9: Bsel et al. 1992]
Paul J. Bsel and Neil D. McKay, “A Method for Registration of 3D Shapes,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no.2, pp.
239-256, February 1992.
[10: Turk et al. 1994]
Greg Turk and Marc Levoy, “Zippered Polygon Meshes from Range Images,” in Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques, New York, USA, pp. 311-318, July 24-29, 1994.
[11: Chen et al. 1991]
Yang Chen and Gerard Medioni, “Object Modeling by Registration of Multiple Range Images,” in Proceedings of IEEE International Conference on Robotics and Automation, Sacramento, CA, vol. 3, pp. 2724-2729, April 9-11, 1991.
[12: Johnson et al. 1997]
Andrew Edie Johnson and Sing Bing Kang, “Registration and Integration of Textured 3D Data,” in Proceedings of IEEE International Conference on Recent Advances in 3-D Digital Imaging and Modeling, Ottawa, Canada, pp. 234-241, May 12-15, 1997.
[13: Men et al. 2011]
Hao Men, Biruk Gebre and Kishore Pochiraju, “Color Point Cloud Registration with 4D ICP Algorithm,” in Proceedings of IEEE International Conference on Robotics and Automation (ICRA), Shanghai, pp. 1511-1516, May 9-13, 2011.
[14: Makadia et al. 2006]
Ameesh Makadia, Alexander Patterson and Kostas Daniilidis, “Fully Automatic Registration of 3D Point Clouds,” in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp.
1297-1304, June 17-22, 2006.
[15: Arun et al. 1987]
K. S. Arun, T. S. Hung and S. D. Blostein, “Least-squares Fitting of Two 3-D point Sets,” IEEE Transaction on Pattern Analysis and Machine Intelligence, vol. 9, no. 5, pp. 698-700, September 1987.
[16: Scaramuzza et al. 2011]
Davide Scaramuzza and Friedrich Fraundorfer, “Visual Odometry [Tutorial],”
IEEE Robotics and Automation Magazine, vol. 18, no. 4, pp. 80-92, December 2011.
[17: Nister et al. 2004]
Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol.1, pp. 652-659, June 27-July 2, 2004.
[18: Kitt et al. 2010]
B.Kitt, A. Geiger and H. Lategahn, “Visual Odometry Based on Stereo Image Sequences with RANSAC-based Outliers Rejection Scheme,“ in Proceedings of IEEE Intelligent Vehicles Symposium, San Diego, USA, pp. 486-492, June 21-24, 2010.
[19: Jachalsky et al. 2010]
Jorn Jachalsky, Markus Schlosser and Dirk Gandolph, “Confidence Evaluation for Robust, Fast-Converging Disparity Map Refinement,” in Proceedings of IEEE International Conference on Multimedia and Expo (ICME), Suntec City, pp.
1399-1040, July 19-23, 2010.
[20: Lowe 2004]
David G. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,”
International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110, January 2004.
[21: Dalal et al. 2005]
Navneet Dalal and Bill Triggs, “Histograms of Oriented Gradients for Human Detection,” in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA, vol. 1, pp. 886-893, June 25,
2005.
[22: Saravanakumar et al. 2010]
S. Saravanakumar, A.Vadivel and C.G Saneem Ahmed, “Multiple Human Object Tracking using Background Subtraction and Shadow Removal Techniques,“ in Proceedings of IEEE International Conference on Signal and Image Processing (ICSIP), Chennai, pp. 79-84, December 15-17, 2010.
[23: Lee et al. 2003]
Dar-Shyang Lee, Jonathan J. Hull and Berna Erol, “A Bayesian Framework for Gaussian Mixture Background Modeling,” in Proceedings of IEEE International Conference on Image Processing , vol. 3, pp. 973-976, September 14-17, 2003.
[24: Barnich et al. 2011]
Olivier Barnich and Marc Van Droogenbroeck, “ViBe: A Universal Background Subtraction Algorithm for Video Sequences,” IEEE Transactions on Image Processing, vol. 20, no. 6, pp. 1709-1724, June 2011.
[25: Enzweiler et al. 2009]
Markus Enzweiler and Dariu M. Gavrila, “Monocular Pedestrian Detection:
Survey and Experiments,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no.12, pp. 2179-2195, December 2009.
Feng Tang, Michael Harville, Hai Tao and Ian N. Robinson, “Fusion of Local Appearance with Stereo Depth for Object Tracking,” in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Anchorage, AK, pp. 1-8, 23-28 June, 2008.
[27: Labayrade et al. 2002]
Raphael Labayrade, Didier Aubert and Jean-Philippe Tarel, “Real Time Obstacle Detection in Stereovision on Non Flat Road Geometry Through “V-disparity”
Representation,” in Proceedings of IEEE Intelligent Vehicle Symposium, vol.2, pp.
646-651, June 17-21, 2002.
[28: Hu et al. 2005]
Zhencheng Hu, Francisco Lamosa and Keiichi Uchimura, “A Complete U-V-Disparity Study for Stereovision Based 3-D Driving Environment Analysis,”
in Proceedings of IEEE International Conference on 3-D Digital Imaging and Modeling, pp.204-211, June 13-16, 2005.
[29: Perrollaz et al. 2012]
Mathias Perrollaz, John-David Yoder, Amaury Negre, Anne Spalanzani and Christian Laugier, “A Visibility-Based Approaching for Occupancy Grid Computation in Disparity Space,” IEEE Transactions on Intelligent Transportation Systems, vol. 13, no. 3, pp. 1383-1393, September 2012.
[30: Perrollaz et al. 2010]
Mathias Perrollaz, Anne Spalanzani and Didier Aubert, “Probabilistic Representation of the Uncertainty of Stereo-Vision and Application to Obstacle Detection,” in Proceedings of IEEE Intelligent Vehicles Symposium, San Diego, USA, pp. 313-318, June 21-24, 2010.
[31: Oniga et al. 2010]
Florin Oniga and Sergiu Nedevschi, “Processing Dense Stereo Data Using Elevation Maps: Road Surface, Traffic Isle, and Obstacle Detection,” IEEE Transactions on Vehicular Technology, vol. 59, no. 3, pp. 1172-1182, March 2010.
[32: Viola et al. 2003]
Paul Viola, Michael J. Jones and Daniel Snow, “Detecting Pedestrians Using Patterns of Motion and Appearance,” in Proceedings of IEEE International Conference on Computer Vision (ICCV), Nice, France, vol. 2, pp. 734-741, October 13-16, 2003.
[33: Enzweiler et al. 2008]
M. Enzweiler, P. Kanter and M. Gavrila, “Monocular Pedestrian Recognition Using Motion Parallax,” in Proceedings of IEEE Intelligent Vehicles Symposium, Eindhoven, Netherlands, pp. 792-797, June 4-6, 2008.
Denis Wolf and Gaurav S. Sukhatme, “Online Simultaneous Localization and Mapping in Dynamic Environments,” in Proceedings of the IEEE International Conference on Robotics and Automation, New Orleans, LA, USA, vol. 2, pp.
1301-1307, April 26-May 1, 2004.
[35: Danescu et al. 2012]
Radu Danescu, Cosmin Pantilie, Florin Oniga, and Sergiu Nedevschi, “Particle Grid Tracking System Stereovision Based Obstacle Perception in Driving Environments,” IEEE Transactions on Intelligent Transportation Systems Magazine, vol. 4, no. 1, pp. 6-20, January 26, 2012.
[36: Barth et al. 2009]
Alexander Barth and Uwe Franke, “Estimating the Driving State of Oncoming Vehicles From a Moving Platform Using Stereo Vision,” IEEE Transactions on Intelligent Transportation Systems, vol. 10, no. 4, pp. 560-571 , December 2009.
[37: Nedevschi et al. 2007]
Sergiu Nedevschi, Corneliu Tomiuc and Silviu Bota, “Stereo-Based Pedestrian Detection for Collision Avoidance Applications,” IEEE Transactions on Intelligent Transportation System, vol. 10, no. 3, pp. 380-391, September 2009.
[38: Krotosky et al. 2007]
Stephen J. Krotosky and M.M. Trivedi, “On Color-, Infrared-, and
Multimodal-Stereo Approaches to Pedestrian Detection”, IEEE Transactions on Intelligent Transportation Systems, vol. 8, no. 4, pp.619-629, December 2007.
[39: Li et al. 2009]
Liyuan Li, Jerry Kah Eng Hoe, Shuicheng Yan and Xinguo Yu, “ML-Fusion based Multi-Model Human Detection and Tracking for Robust Human-Robot Interfaces,” in Proceedings of IEEE International Workshop on Applications of Computer Vision (WACV), Snowbird, UT, pp. 1-8, December 7-8, 2009.
[40: Zitnick et al. 2002]
C. Lawrence Zitnick and Takeo Kanade, “A Cooperative Algorithm for Stereo Matching and Occlusion Detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 7, pp. 675-684, July 2000.
[41: Comaniciu et al. 2003]
Dorin Comaniciu, Visvanathan Ramesh and Peter Meer, “Kernel-Based Object Tracking,” IEEE Transaction on Pattern Analysis and Machine Intelligence, vol. 25, no. 5, pp. 564-577, May 2003.
[42: Steder et al. 2011]
Bastian Steder, Radu Bogdan Rusu, Kurt Konolige and Wolfram Burgard, “Point Feature Extraction on 3D Range Scans Taking into Account Object Boundaries,” in
(ICRA), Shanghai, pp. 2601-2608, May 9-13, 2011.
Websites
[43: SIFT Keypoint Detector from David Lowe 2013]
SIFT Keypoint Detector. (2005, July). In David Lowe Personal Page. Retrieved April 3, 2013, from http://www.cs.ubc.ca/~lowe/keypoints/
[44: Kalman Filter from website 2013]
Kalman Filter Toolbox for MATLAB. (2004, June 7). In UBC. Retrieved June 6, 2013, from http://www.cs.ubc.ca/~murphyk/Software/Kalman/kalman.html
[45: HSL and HSV from wiki 2013]
HSL and HSV. (2013, June 6). In Wikipedia. Retrieved June 6, 2013, from http://en.wikipedia.org/wiki/HSL_and_HSV
[46: Connected-Component Labeling from wiki 2013]
Connected-Component Labeling. (2013, June 6). In Wikipedia. Retrieve June 6, 2013, from http://en.wikipedia.org/wiki/Connected-component_labeling
[47: Random Sample Consensus from wiki 2013]
RANSAC. (2013, May 13). In Wikipedia. Retrieved May 3, 2013, from http://en.wikipedia.org/wiki/RANSAC
[48: Interpolation from wiki 2013]
Interpolation. (2013, May 31). In Wikipedia. Retrieved May 31, 2013, from
http://en.wikipedia.org/wiki/Interpolation
[49: Accuracy For Stereo Vision from PointGrey 2010]
Article 63: How is depth determined from a disparity image? (2010, July 19). In PointGrey Official Knowledge Base. Retrieved May 31, 2013, from http://www.ptgrey.com/support/kb/index.asp?a=4&q=85
[50: UTE120 Combo ExpressCard from Uptech 2013]
UTE120 Combo ExpressCard. (2013, July). In Uptech Website. Retrieved July 30, 2013, from http://www.uptech.tw/product_detail.php?prod_id=488
[51: BumbleBee2 Product Datasheet from PointGrey 2013]
BumbleBee2 Documents- Product Datasheet. (2012, June). In PointGrey Official
Website. Retrieved July 12, 2013, from
http://www.ptgrey.com/products/bumblebee2/bumblebee2_xb3_datasheet.pdf
[52: URG-04LX-UG01 from Hokuyo]
Hokuyo URG-04LX-UG01 Documents- Product Datasheet. (2013, July 30). In Hokuyo Official Website. Retrieved July 30, 2013, from
http://www.hokuyo-aut.jp/02sensor/07scanner/urg_04lx_ug01.html [53: SICK LMS100 from SICK]
SICK LMS100 Datasheet. (2013, July 30). In SICK Official Website. Retrieve July