In this thesis, we present how we obtain efficient finger motion estimation from noisy depth image. We introduced gesture recognition as a feature space lookup problem for finger motion estimation. Using a simple feature allowed us to generate a discriminative feature space using normalized depth images without overfitting, and enabled real-time performance (200fps). Generating spatial and temporal candidates helped us to obtain further accuracy under noisy input data. Finally, by introducing a reliable distance appearance for candidates voting, we further increased stability and therefore generated realistic finger motion.
As future work, we plan to integrate multi-resolution and synthetic dataset for more accurate estimation under occlusion. Furthermore, employing interacting hand gesture may help us to generate more interesting motions.
32
References
[AAS04] V. Athitsos, J. Alon, S. Sclaroff, and G. Kollios. BoostMap: A method for efficient approximate similarity rankings. In Proc. Computer Vision and Pattern Recognition (CVPR), vol. 2, 268–275, 2004.
[AS03] V. Athitsos and S. Sclaroff. Estimating 3D hand pose from a cluttered image.
In Proc. Computer Vision and Pattern Recognition (CVPR), vol. 2, 432–439, 2003.
[AT04] A. Agarwal and B. Triggs. 3D human pose from silhouettes by relevance vector regression. In Proc. Computer Vision and Pattern Recognition (CVPR), 2004.
[ATC05] D. Anguelov, B. Taskar, V. Chatalbashev, D. Koller, D. Gupta, and A. Ng.
Discriminative learning of markov random fields for segmentation of 3D scan data. In Proc. Computer Vision and Pattern Recognition (CVPR), 2005.
[BEK09] A. Bleiweiss, E. Eilat, and G. Kutliroff. Markerless motion capture using a single depth sensor. In SIGGRAPH ASIA Sketches, 2009.
[BM98] C. Bregler and J. Malik. Tracking people with twists and exponential maps.
In Proc. Computer Vision and Pattern Recognition (CVPR), 1998.
[BM09] L. Bourdev and J. Malik. Poselets: Body part detectors trained using 3D human pose annotations. In Proc. International Conference on Computer Vision (ICCV), 2009.
[BMB11] A. Baak, M. M¨uller, G. Bharaj, H. Seidel, and C. Theobalt. A Data-Driven Approach for Real-Time Full Body Pose Reconstruction from a Depth Camera. In Proc. International Conference on Computer Vision (ICCV), 2011.
33
[DDH06] G. Dewaele, F. Devernay, R. Horaud, and F. Forbes. The alignment between 3-d data and articulated shapes with bending surfaces. In Proc. European Conference on Computer Vision (ECCV), 578–591, 2006.
[FH05] P. Felzenszwalb and D. Huttenlocher. Pictorial structures for object recognition. International Journal of Computer Vision (IJCV), 61(1):55–79, Jan. 2005.
[GPT10] V. Ganapathi, C. Plagemann, S. Thrun, and D. Koller. Real time motion capture using a single time-of-flight camera. In Computer Vision and Pattern Recognition (CVPR), 2010.
[GSK11] R. Girshicky, J. Shottony, P. Kohliy, A. Criminisiy, and A. Fitzgibbony.
Efficient Regression of General-Activity Human Poses from Depth Images.
In Proc. International Conference on Computer Vision (ICCV), 2011.
[GWK05] D. Grest, J. Woetzel, and R. Koch. Nonlinear body pose estimation from depth images. In Proc. Deutsche Arbeitsgemeinschaft für Mustererkennung (DAGM), 2005
[HRM12] L. Hoyet, K. Ryall, R. McDonnell, and C. O’Sullivan. Sleight of Hand:
Perception of Finger Motion from Reduced Marker Sets. In Proc. Interactive 3D Graphic (I3D), 2012.
[IF01] S. Ioffe and D. Forsyth. Probabilistic methods for finding people. International Journal of Computer Vision (IJCV), 43(1):45–68, 2001.
[KBK10] A. Kolb, E. Barth, R. Koch, and R. Larsen. Time-of-flight sensors in computer graphics. In proc. Computer Graphics Forum (CGF), 29(1):141–
159, 2010.
[KHS10] E. Kalogerakis, A. Hertzmann, and K. Singh. Learning 3D mesh segmentation and labeling. ACM Trans. Graphics, 29(3), 2010.
34
[MM03] G. Mori and J. Malik. Estimating human body configurations using shape context matching. In Proc. International Conference on Computer Vision (ICCV), 2003.
[NFC07] R. Navaratnam, A.W. Fitzgibbon, and R. Cipolla. The joint manifold model for semi-supervised multi-valued regression. In Proc. International Conference on Computer Vision (ICCV), 2007.
[OKA12] I. Oikonomidis, N. Kyriazis, and A. Argyros. Tracking the Articulated Motion of Two Strongly Interacting Hands. In Proc. Computer Vision and Pattern Recognition (CVPR), 2012.
[OS08] R. Okada and B. Stenger. A single camera motion capture system for human-computer interaction. The Institute of Electronics, Information and Communication Engineers (IEICE), E91-D:1855–1862, 2008.
[PGK10] C. Plagemann, V. Ganapathi, D. Koller, and S. Thrun. Real-time identification and localization of body parts from depth images. In Proc.
International Conference on Robotics and Automation (ICRA), 2010.
[PY06] J. Park and Y. Yoon. LED-glove based interactions in multi-modal displays for teleconferencing. In International Conference on Artificial Reality and Telexistence (ICAT), 2006.
[RF03] D. Ramanan and D. Forsyth. Finding and tracking people from the bottom up.
In Proc. Computer Vision and Pattern Recognition (CVPR), 2003.
[RRR08] G. Rogez, J. Rihan, S. Ramalingam, C. Orrite, and P. Torr. Randomized trees for human pose detection. In Proc. Computer Vision and Pattern Recognition (CVPR), 2008.
[RS00] R. Rosales and S. Sclaroff. Inferring body pose without tracking body parts.
In Computer Vision and Pattern Recognition (CVPR), pages 721–727, 2000.
35
[RSH05] L. Ren, G. Shakhnarovich, J. Hodgins, H. Pfister, and P. Viola. Learning silhouette features for control of human motion. ACM Transactions on Graphics 24, 4, 1303–1331, 2005.
[RYZ11] Z. Ren, J. Yuan, and Z. Zhang. Robust Hand Gesture Recognition Based on Finger-Earth Mover’s Distance with a Commodity Depth Camera. In Multimedia, 2011.
[SBR04] L. Sigal, S. Bhatia, S. Roth, M. Black, and M. Isard. Tracking loose-limbed people. In Proc. Computer Vision and Pattern Recognition (CVPR), 2004.
[SFC11] J. Shotton, A. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio, R. Moore, A. (CVCG) at Computer Vision and Pattern Recognition (CVPR), 2010.
[SMF04] E. Sudderth, M. Mandel, W. Freeman, and A. Willsky. Distributed occlusion reasoning for tracking with nonparametric belief propagation. In Neural Information Processing Systems (NIPS), 2004.
[STT06] B. Stenger, A. Thayananthan, P. Torr, and R. Cipolla. Model-based hand tracking using a hierarchical bayesian filter. IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI), 28, 9, 1372–1384, 2006.
[SVD03] G. Shakhnarovich, P. Viola, and T. Darrell. Fast pose estimation with parameter-sensitive hashing. In Proc. International Conference on Computer Vision (ICCV), 750–757, 2003.
[TU08] Z. Tu. Auto-context and its application to high-level vision tasks. In Proc.
Computer Vision and Pattern Recognition (CVPR), 2008.
36
[UD08] R. Urtasun and T. Darrell. Local probabilistic regression for activity-independent human pose inference. In Proc. Computer Vision and Pattern Recognition (CVPR), 2008.
[WP09] R. Wang and J. Popovi´c. Real-time hand-tracking with a color glove. ACM Trans. Graphics, 2009.
[WPP11] R. Wang, S. Paris, and J. Popovi´c. 6D Hands: Markerless Hand Tracking for Computer Aided Design. In User Interface Software and Technology (UIST), 2011.
[ZF07] Y. Zhu and K. Fujimura. Constrained optimization for human pose estimation from depth sequences. In Proc. Asian Conference on Computer Vision (ACCV), 2007.
[KINECT] Microsoft Kinect for Windows
http://www.microsoft.com/en-us/kinectforwindows/