I-Shou University Institutional Repository:Item 987654321/12909
49
0
0
全文
(2) Region Flow Detection for Hand Gesture Recognition. Student Advisor Co-advisor. Sheng-En Haung Dr. Jyh-Horng Jeng Dr. Yih-Lon Lin. A Thesis Submitted to Department of Information Engineering I-Shou University In Partial Fulfillment of the Requirements For the Master Degree in Information Engineering June, 2011 Kaohsiung, Taiwan, Republic of China.
(3)
(4) UNO. JUBEAT. MAN lab. !. lab 31 lab. !. lab 0.
(5) D.C.. lab. !. lab. lab. !.
(6) ................................................................................................................................ I ........................................................................................................................... II ..........................................................................................................................III ...............................................................................................................................V ABSTRACT ................................................................................................................ VI 1. ......................................................................................1. 2. ..........................................................................................................3 2.1. .............................................................................................3. 2.2. .................................................................................................7. 3. ........................................................................................................10 3.1 ROI .................................................................................................................10 3.2 ROI 3.3. 4. 5. ................................................................................12 ...........................................................................................17 ........................................................................................22. 4.1. ...........................................................................................................24. 4.2. ...........................................................................................................25. 4.3. ...........................................................................................................28. 4.4. ...............................................................................................32 ............................................................................................36 ......................................................................................................................37.
(7) 2.1. .............................................................................................4. 2.2. .................................................................................................5. 2.3. .................................................................................................................6. α. 2.4. .....................................................................................................7. 2.5. .................................................................................................................9. 3.1. ...................................................................................................11. 3.2. .......................................................................................................11. 3.3. .......................................................................................................12. 3.4. .......................................................................................13. 3.5. .......................................................................................13. 3.6. .......................................................................................14. 3.7. .......................................................................................14. 3.8. ...................................................................................15. 3.9. ...................................................................................16. 3.10. .................................................................................16. 3.11. .................................................................................17. 3.12. .................................................................................................................18. 3.13. .........................................................................................19. 3.14. .........................................................................................19. 3.15. .........................................................................................20. 3.16. .........................................................................................20. 3.17. .........................................................................21. 3.18. .........................................................................21. 4.1. ...........................................................................................23. 4.2. ...................................................................................................24. 4.3. ROI. ...................................................................................................24. 4.4. ...................................................................................26. 4.5. ...................................................................27.
(8) 4.6. ...................................................................................................30. 4.7. ...................................................................................................33. 4.8. ...............................................................................................................34. 4.9. .......................................................................................................35.
(9) 4.1 Type I and Type II errors................................................................................................22 4.2. ...................................................................................25. 4.3. ...................................................................................25. α. 4.4. ...................................................................................26. 4.5. ROI. -. ..................................................................27. 4.6. ROI. -. ......................................................27. 4.7. ROI. -. α. ..............................................28. 4.8. ROI. -. α. ..............................................28. 4.9. ...................................................................................29. 4.10. .....................................................................................29. 4.11. .....................................................................................29. 4.12 y = 200. ........................................................................30. 4.13 y = 50. ..........................................................................31. 4.14 x = 50. ..........................................................................31. 4.15 x = 260. ........................................................................32.
(10) ….
(11) ABSTRACT In recent years, moving object recognition and detection have attracted many scholars study. They use methods such as gesture recognition, human motion recognition, color detection, and so on. At present, they used color detection in gesture recognition. The disadvantage of this method is the need of large computation. If the problem of race color is considered, the difficulty on recognition will increase. For example: yellow, white, and black people. In this study, we use the residual of subtracting adjacent images as the data of source dynamic detection. This method operates on grayscale images and also suitable for real-time system. The purpose of this study is to correctly recognize the hand gestures in the complex background. This thesis introduces the traditional frame difference method, the motion history image, and the detection in the region of interest in which flow statistics data and density variation data are analyzed on the detection lines and detection region.. Keyword. Motion History Image, Gesture Recognition, Flow Detection, Region of Interest. VI.
(12) 1 [1] [2-4]. [5]. [6-7]. [8-9]. …. …. (1). (2). (background substraction). (flesh colored). (3). (optical flow). (4). (edges). (5). (frame difference). 1.
(13) (motion history image, MHI) [10-12]. (region of interest, ROI)[13]. (spatio-temporal slice, STS)[14-16]. (density-based spatial clustering of applications with noise, DBSCAN)[17-18] [19]. 2.
(14) 2. 2.1 RGB YCbCr Y. (2.1). Y = 0.299 × R + 0.587 × G + 0.114 × B. f t−1 ( x, y ). (2.2). f t ( x, y ). (2.1). Ft ( x, y ). frame. frame. Ft ( x, y ) = f t ( x, y ) − f t −1 ( x, y ). (2.2). Tg Tg. 0. Tg. 255. 0. Tg. 255. (2.3). 255, if Ft ( x, y ) > Tg f b ( x, y ) = 0, else. (2.3). 2.1 ;. 3.
(15) (a) 10th frame. (b) 11th frame. (c) Tg = 1. (d) Tg = 10. (e) Tg = 50. (f) Tg = 100 2.1 2.2(b). 2.2(d). 2.2(f). 4.
(16) (a) 102th frame. (b). (c) 332th frame. (d). (e) 203th frame. (f) 2.2. 2.3 2.3(d) 2.3(f). 5. 2.3(b).
(17) (a) 111th frame. (b). (c) 144th frame. (d). (e) 316th frame. (f) 2.3. 6.
(18) 2.2 (motion history image, MHI). Ft ( x, y ) f t ( x, y ). f t−1 ( x, y ). M t ( x, y ). α. Ft ( x, y ). M t ( x, y ). (2.4). M t ( x, y ) = Ft ( x, y ) + M t −1 ( x, y ) − α. α. (2.4). α M. 255. α. 0. α. 255. α. ; 255 2.4. 0. α. 0. 2.4(c) (f) 100. 2.4(b). 130. (a) 100th frame. (b) 130th frame. 7. 0. 2.4(a).
(19) (c) α = 0. (d) α = 5. (e) α = 50. (f) α = 100. (e) α = 150. (f) α = 200. α. 2.4 2.5. 60th frame. 2.5. 32. 15. 8. 268th frame.
(20) nth frame. 20. 50. 55. 60. 268. 2.5. 9.
(21) 3. 3.1 ROI ROI(region of (spatio-temporal slice, STS). interest, ROI). (horizontal slice) slice). (vertical. (diagonal slice) (pan). (tilt) (dissolve). (zoom). (static). (cut) ;. 10.
(22) t−n. (a). (b). t. (c). 3.1 3.1. 3.1(a). A. B. A 3.1(b). A. B B. 3.1(c). 3.2 3.2. 120 × 20. 80 × 30. 3.2. 3.3. 3.2. 11. t+n.
(23) 3.3. 3.2 ROI. (3.1) (3.3). 1, if Ft ( x, y ) > Tg B ( x, y ) = 0, else P = ∑ B( x, y ). (3.1) (3.2). x , y∈R. Dt =. P A. (3.3). Ft ( x, y ) P. ( x, y ). Tg. B( x, y ). R. A. 3.4. ROI. 3.7. 12. Dt.
(24) 3.4. 3.4. 3.5. 3.5 ROI ; 13.
(25) 3.6. 3.6. 3.7. 14.
(26) 3.7. (3.4). Vt = Dt − Dt −1 Dt −1. (3.4). t −1. Dt. Vt. 3.8 3.11. n. 3.8. 15.
(27) 3.9. 3.8. 3.11. 3.10. 16.
(28) 3.11. ;. 3.3 (level of confidence, LOC). (σ − 2 µ , σ + 2 µ ). 95.4% (σ − 2 µ , σ + 2 µ ). ; 3.12. 17. 95.4%.
(29) 3.12. (3.5) (3.6) x2. Ft1 = ∑ f (i, y ), if i = x1 y2. Ft2 = ∑ f ( x, i ), if i = y1. TD1 > Dt1 > TD2. and TV1 > Vt2 > TV2. (3.5). TD3 > Dt2 > TD4. and TV3 > Vt2 > TV4. (3.6). 18.
(30) Ft1. Ft2. TD1~ 4. TV1~ 4. Dt1. 3.13. Vt1. i. 3.16. Tg. 350 300 250 200 150 100 50 0 80-89. 90-99. 100-109. 110-119. 120-129. 130-139. 140-149. 150-159. 110-119. 120-129. 130-139. 140-149. 150-159. 3.13. 350 300 250 200 150 100 50 0 80-89. 90-99. 100-109. 3.14. 19.
(31) 350 300 250 200 150 100 50 0 120-129 130-139 140-149 150-159 160-169 170-179 180-189 190-199 200-209 210-219 220-229 230-239. 3.15. 350 300 250 200 150 100 50 0 120-129 130-139 140-149 150-159 160-169 170-179 180-189 190-199 200-209 210-219 220-229 230-239. 3.16 3.17 3.18. 3.17 3.18. 20.
(32) 80 70 60 50 40 30 20 10 0 1. 101. 201. 301. 401. 501. frame 3.17. 80 70 60 50 40 30 20 10 0 1. 101. 201. 301. frame 3.18. 21. 401. 501. 601.
(33) 4 4.1 Type I errors. FN. Type II errors. (4.1) (4.2) 15. Fp. 450. frame. 300. 15. frame. ’. F-score. Accuracy. (4.3) (4.4). 4.1 Type I and Type II errors. 4.1. Tp. Fp. FN. TN. FP. Tp ; TN FN. (4.1). (4.4). P. (Precision) (F-score). R. (Recall). F. Ac. (Accuracy) P=. R=. Tp. (4.1). T p + Fp. Tp. (4.2). T p + FN. F = 2⋅. P⋅R P+R. (4.3). 22.
(34) Ac =. TP + TN TP + TN + FP + FN. (4.4) 360 × 240. Tg P4. x. y. P1. P3 4.1. x. P2 P0. y. Line1 P1. P3. P2. P0. P4. Line2. 4.1. L1. L2 (4.5). (4.6). L1 : 2 x + 3 y − 720 = 0. (4.5). L2 : 2 x − 3 y = 0. (4.6). 23.
(35) 4.1 Tg. 4.2. 4.3. 4.2. ft. 4.3. f t −1. 4.2. ft. f t −1. 4.3. ROI. 24.
(36) 4.2 ROI Tg. 32 64 96 128. TP 14 15 15 13. TN. FN 12 10 9 2. 1 2 3 10. FP 3 3 3 5. Ac. F. 50% 57% 60% 76%. 0.65 0.70 0.71 0.79. TN 11 11 10 12. TP 14 13 12 8. FN. FP 2 2 2 0. 3 4 6 10. Ac. F. 83% 80% 73% 67%. 0.85 0.81 0.75 0.61. Ac. F. 55% 45% 45% 30%. 0.71 0.60 0.52 0.00. 4.3 ROI Tg. TN. TP. 32 64 96 128. 6 5 5 0. 0 0 1 2. FN 7 7 6 5. FP 7 8 8 13. Ac. F. 30% 25% 30% 10%. 0.48 0.40 0.41 0.00. TN. TP 11 8 6 0. FN 7 6 4 1. 0 1 3 6. FP 2 5 7 13. Tg ;. Tg ROI. Tg. ; Tg. 32. Tg. 32. 4.2 α TV1 TV2 4.4. TD1 TD2. Tg Tg. 25. 32. α. 4.4.
(37) ft. f t −1. 4.4. α. 4.4. α. TN. TP. 8 16 32 64 96 128. 3 7 9 11 11 11. 0 0 0 0 0 0. Ac. R. P. F. 15% 35% 45% 55% 55% 55%. 0.30 0.50 0.56 0.61 0.61 0.61. 0.23 0.54 0.70 0.85 0.85 0.85. 0.26 0.52 0.62 0.71 0.71 0.71. FP 7 7 7 7 7 7. α. 4.4 Tg. FN. 10 6 4 2 2 2. 96. 4.5. 32. 4.6. Tg. 32. α. 96 4.5. 26.
(38) ft. f t −1. 4.5 4.5 TD1 TD2. 0.1 0.05 0.01 0.01 0.005 0.015 0.003 0.005. TP. 0.5 0.5 0.3 0.25 0.2 0.2 0.2 0.1. ROI TN. 4.6 TV1 TV2 0.01 0.005 0.003 0.005 0.005 0.005. 0.2 0.2 0.2 0.1 0.05 0.04. FN 1 1 2 3 4 4 2 7. 9 10 11 11 11 10 11 1. 7 10 9 10 10 10. TN. FP 6 6 5 4 3 3 5 0. ROI. TP. R. P. F. 50% 55% 65% 70% 75% 70% 65% 40%. 0.6 0.71 0.69 0.73 0.79 0.78 0.69 0.08. 0.70 0.77 0.85 0.85 0.85 0.77 0.85 1.00. 0.64 0.74 0.76 0.79 0.81 0.77 0.76 0.14. Ac. R. P. F. 45% 60% 50% 60% 65% 70%. 0.58 0.67 0.60 0.67 0.71 0.77. 0.54 0.77 0.70 0.77 0.77 0.77. 0.56 0.71 0.64 0.71 0.74 0.77. FN. 2 2 1 2 3 4. 4 3 2 2 2 3 2 12. Ac. FP 5 5 6 5 4 3 27. 6 3 4 3 3 3.
(39) 4.5. 0.005 0.2. TD1 TD2. α. 4.7. 4.5 4.7. α. TN. TP. 8 16 32 64 96 128. ROI. 3 4 8 9 11 12 4.6. α. -. FN 2 2 3 4 4 2. FP 5 5 4 3 3 5. 10 9 5 4 2 1. TV1 TV2. Ac. R. P. F. 25% 30% 55% 65% 75% 70%. 0.38 0.45 0.67 0.75 0.79 0.71. 0.23 0.31 0.62 0.70 0.85 0.92. 0.29 0.36 0.64 0.72 0.81 0.80. R. P. F. 0.63 0.50 0.60 0.90 0.89 0.75. 0.39 0.39 0.46 0.70 0.61 0.46. 0.005 0.04. α. 4.8. 4.5 4.8. α. TN. TP. 8 16 32 64 96 128. ROI. 5 5 6 9 8 6. α. -. FN 4 2 3 6 6 5. Ac. FP 3 5 4 1 1 2. 8 8 7 4 5 7. 45% 35% 45% 75% 70% 55%. 4.3. 10. 4.9. 4.11. 28. 0.48 0.43 0.52 0.78 0.72 0.57.
(40) 4.9 80 ~89 55 109. y. 90 ~99 140 153. 100 ~109 206 104. 110 ~119 75 50. 120 ~129 11 0. 130 ~139 0 0. 140 ~149 0 0. 150 ~159 0 0. 4.10. x. y = 30 y = 40 y = 50. 120 ~129. 130 ~139. 140 ~149. 150 ~159. 160 ~169. 170 ~179. 180 ~189. 190 ~199. 200 ~209. 210 ~219. 220 ~229. 230 ~239. 0 0 0. 0 0 0. 0 0 0. 0 0 0. 16 28 76. 95 100 141. 136 125 243. 49 17 16. 0 0 0. 0 28 12. 0 9 123. 0 0 12. 4.11. x. y = 190 y = 200 y = 210. 120 ~129. 130 ~139. 140 ~149. 150 ~159. 160 ~169. 170 ~179. 180 ~189. 190 ~199. 200 ~209. 210 ~219. 220 ~229. 230 ~239. 0 0 0. 4 0 0. 37 23 8. 185 42 32. 110 46 10. 109 52 5. 94 10 0. 11 0 0. 0 0 0. 0 0 1. 141 2 3. 307 0 0. 4.9 4.9 4.10. 4.11 ;. 4.6. 29.
(41) 4.6. frame. 15. 3. Frame x = 120 ~ 129 x = 130 ~ 139 x = 140 ~ 149 x = 150 ~ 159 x = 160 ~ 169 x = 170 ~ 179 x = 180 ~ 189 x = 190 ~ 199 x = 200 ~ 209 x = 210 ~ 219 x = 220 ~ 229 x = 230 ~ 239. 4.12. 4.15. 4.12. 4.15. 15. 0 0 0 0 0 0 0 0 0 0 0 0 0 0. 0 0 10 16 13 0 0 0 0 0 0 0 3 39. 0 0 10 16 13 0 0 0 0 0 1 0 1 1. 0 0 10 16 13 2 5 0 0 0 1 0 2 7. 30. frame. 30. 5. frame. 4.12 y = 200 15 90 105 165 ~75 ~150 ~210. 15. frame. 225 ~255. 270 ~315. 330 ~345. 360 ~390. 405 ~480. 495. 0 0 10 18 18 15 7 0 0 0 1 0 4 22. 0 0 10 18 19 20 7 0 0 0 2 0 3 7. 0 0 10 18 25 30 10 0 0 0 2 0 3 19. 0 0 10 18 26 30 10 0 0 0 2 0 1 1. 0 0 10 18 32 33 10 0 0 0 2 0 2 9. 0 0 26 46 49 52 10 0 0 0 2 0 4 80.
(42) Frame x = 120 ~ 129 x = 130 ~ 139 x = 140 ~ 149 x = 150 ~ 159 x = 160 ~ 169 x = 170 ~ 179 x = 180 ~ 189 x = 190 ~ 199 x = 200 ~ 209 x = 210 ~ 219 x = 220 ~ 229 x = 230 ~ 239. 30 ~90 0 0 0 0 0 0 0 0 0 0 0 0 0 0. Frame y y y y y y y y. = 80 ~ 89 = 90 ~ 99 = 100 ~ 109 = 110 ~ 119 = 120 ~ 129 = 130 ~ 139 = 140 ~ 149 = 150 ~ 159. 4.13 y = 50 120 150 180. 0 0 0 0 29 27 10 2 0 11 0 0 5 79. 0 0 0 0 29 32 17 2 0 11 9 0 3 21. 0 0 0 0 29 32 28 2 0 11 14 0 2 16. 4.14 x = 50 15 90 ~75 0 0 3 0 0 7 3 0 0 0 0 0 0 0 0 0 0 3 0 13. 210. 240. 270. 300. 330. 360. 390. 420. 0 0 0 0 29 32 59 8 0 11 35 5 4 53. 0 0 0 0 53 70 98 8 0 11 51 5 4 117. 0 0 0 0 53 70 99 8 0 11 52 5 2 2. 0 0 0 0 53 70 114 8 0 11 65 5 2 28. 0 0 0 0 68 102 161 14 0 12 81 5 6 117. 0 0 0 0 68 102 170 14 0 12 86 5 2 14. 0 0 0 0 68 124 191 14 0 12 99 5 3 56. 0 0 0 0 76 141 222 14 0 12 109 5 4 66. 105 ~150 0 22 40 16 0 0 0 0 3 65. 31. 165 ~210 14 49 81 38 11 0 0 0 5 115. 225 ~255 37 83 109 38 11 0 0 0 3 85. 270 ~315 51 109 146 52 11 0 0 0 4 91. 330 ~345 51 116 157 71 11 0 0 0 3 37. 360 ~390 55 140 206 75 11 0 0 0 4 77.
(43) y = 80 ~ 89 y = 90 ~ 99 y = 100 ~ 109 y = 110 ~ 119 y = 120 ~ 129 y = 130 ~ 139 y = 140 ~ 149 y = 150 ~ 159. 4.15 x = 260 Frame 30 ~120 0 0 0 0 0 0 0 0 0 0. 150 ~240 51 22 20 0 0 0 0 0 3 93. 270 ~330 51 51 47 36 0 0 0 0 3 92. 360. 79 87 76 36 0 0 0 0 3 93. 4.4 4.7. ROI. 32. 390 ~420 87 88 76 36 0 0 0 0 2 9. 450 ~480 109 153 104 50 0 0 0 0 4 129.
(44) f t −1. ft. No. Yes. 4.7. 4.8 32. 4.8. Tg. 4.8(b) Tg. 4.8(d) 4.8(f). 33.
(45) (a) 111th frame. (b). (c) 144th frame. (d). (e) 316th frame. (f) 4.8. 4.9. 34.
(46) (a) 111th frame. (b). (c) 144th frame. (d). (e) 316th frame. (f) 4.9. 4.9. Tg. 4.9(b) 4.9(d) 4.9(f). 35.
(47) 5. ;. 36.
(48) [1]. Horn, B. K. P. and B. G. Schunck, "Determining optical flow," Artificial Intelligence, vol. 17, no. 1, pp. 185-203, 1981. 2010. [2] [3] 2009 [4] 2009 [5] 2006 [6]. Paul Viola and Michael Jones, “Rapid Object Detection using a Boosted Cascade of Simple Features,” IEEE Conference on Computer Vision and Pattern Recognition, 2001.. [7]. Simon Baker and Iain Matthews, “Lucas-Kanade 20 Years On: A Unifying Framework,” International Journal of Computer Vision, vol. 56, no. 3, pp. 221-255, 2004.. [8] 2009 [9] 2002 [10] James Davis, “Hierarchical Motion History Images for Recognizing Human Motion,” Proceedings of IEEE Workshop on Detection and Recognition of Events in Video, pp. 39-46, 2001. [11] M. Valstar, M. Pantic, and I. Patras, “Motion History for Facial Action Detection in Video,” Proceedings of IEEE International Conference on Systems, Man and Cybernetics, pp. 635-640, 2004. [12] D. Weinland, R. Ronfard, and E. Boyer, “Motion History Volumes for Free Viewpoint Action Recognition,” Proceedings of IEEE International Workshop on modeling People and Human Interaction, 2005. [13] 2006. 37.
(49) [14] A. Saoudi and H. Essafi, “Spatio-temporal video slice edges analysis for shot transition detection and classification,” World Academy of Science, Engineering and Technology, vol. 28, 2007. [15] Aissa Saoudi and Hassane Essafi,. Spatio-Temporal Video Slice Edges Analysis for. Shot Transition Detection and Classification,. International journal of signal. processing, vol. 4, no. 1, 2007. [16] 2004 [17] Martin Ester, Hans-Peter Kriegel, Jörg Sander and Xiaowei Xu, “A density-based algorithm for discovering clusters in large spatial database with noise,” In: International Conference on Knowledge Discovery in Databases and Data Mining (KDD-96), AAAI Press, Portland, Oregon, pp. 226-231, 1996. [18] 2007 [19] 2003. 38.
(50)
相關文件
structure for motion: automatic recovery of camera motion and scene structure from two or more images.. It is a self calibration technique and called automatic camera tracking
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp... Annealed
• A delta-gamma hedge is a delta hedge that maintains zero portfolio gamma, or gamma neutrality.. • To meet this extra condition, one more security needs to be
[3] Haosong Gou, Hyo-cheol Jeong, and Younghwan Yoo, “A Bit collision detection based Query Tree protocol for anti-collision in RFID system,” Proceedings of the IEEE
Godsill, “Detection of abrupt spectral changes using support vector machines: an application to audio signal segmentation,” Proceedings of the IEEE International Conference
In the processing following action recognition, this paper proposes a human behavior description model to describe events occurring to human and exerts as decision and
D.Wilcox, “A hidden Markov model framework for video segmentation using audio and image features,” in Proceedings of the 1998 IEEE Internation Conference on Acoustics, Speech,
[16] Goto, M., “A Robust Predominant-F0 Estimation Method for Real-time Detection of Melody and Bass Lines in CD Recordings,” Proceedings of the 2000 IEEE International Conference
