Chapter 5 System Architecture and Experimental Results
5.4 Experimental Results of the Human Behavior Analysis
To evaluate the performance of the system in the recognition of human behaviors and postures, we test 30 video sequences containing 2 kinds of abnormal behaviors and 500 testing data for each posture.Table.2 shows the recognition rates of postures.
We calculate the ratio of success frame number to the total testing frame number.
From the results, we can find that the performance for the sit-pose is the lowest. Since sit-pose is a posture relatively similar to the stand-pose, it is misrecognized as stand-pose. The stand pose shows a lower performance than the lie-pose. This is because the stand-pose is extracted in moving situation and therefore would involve more complicated background situations. On the other hand the lie-pose is extracted on static situation, on a fixed background. Consequently, it presents a higher recognition rate. Table.3 shows the behavior recognition results. The false positive means that a test incorrectly gives a positive result. The true positive means that a test correctly gives a positive result. From Table.3, we know that our method correctly detect the 2 abnormal behavior but wrongly detect 4 video sequence as abnormal behaviors. The reason of wrong detection is caused by the error recognition of posture.
Table.2 The recognition rates for three postures
Behavior Type Success Frame/Total Frame
Sit pose 74%
Stand pose 83%
Lie pose 90%
Table.3 The recognition results for abnormal behavior Number of
Abnormal video
Number of true positive
Number of false positive
Abnormal behavior 2 2 4
Chapter 6
Conclusion and Future work
In this thesis, we presented a system for object-based video tracking and human abnormal behavior analysis on surveillance videos. We adopted a simple but effective shadow elimination algorithm to eliminate shadow in object segmentation. From the experimental results, we know that the shadow threshold deciding the shadow elimination result. Besides, we also designed two matching algorithms, using color and shape information of object combine with a score function, to track objects.
Especially, the multiple objects matching algorithm successfully detect the occlusion and split objects. Besides, we designed a finite state machine to analyze human abnormal behavior and obtain a satisfactory result. Based on this system structure, we implemented a tracking system and analyzing human abnormal behavior.
To improve the performance and the robustness of system, some enhancements can be done in the future:
(i) Dynamic finding a threshold while eliminating shadow. We use a pre-decided threshold in the shadow elimination module. In the future, we wish to design an algorithm to find the threshold dynamically.
(ii) Finding a more reliable algorithm in multiple objects matching algorithm.
In the multiple objects matching algorithm, we use a greedy approach to find the matching. We wish to find a better method to solve this problem.
(iii) Extracting more kinds of posture and behavior from video. In our system, we only extract sit, stand, lie three postures and only analyze the faint and falling down two abnormal behavior. In future, we wish to extract more postures and design a complex final state machine to analyze more
abnormal behavior.
(iv) Summarization and abstraction of human behavior in the video. The object-based abstractions are very valuable and useful. The system can be further extended for the content retrieval and management. To achieve this, we can use the MPEG-7 descriptors to describe the contents with the detected abnormal behaviors and generated abstractions. And thus we can manage a database for surveillance and monitoring videos and important contents can be retrieved efficiently. We believe that the extraction of content will become more and more important and one day such a kind of systems will be widely adopted in the future.
Reference
[1] Y. Tsaig and A. AverBuch, “Automatic Segmentation of Moving Objects in Video Sequence: A Region Labeling Approach,” IEEE Transactions on Circuits and Systems for Video Technology, Vol.12, NO. 7, pp.597–612, 2002.
[2] J.C. Choi, S.W. Lee and S.D. Kim, “Spatio-Temporal Video Segmentation Using a Joint Similarity Measure,” IEEE Transactions on Circuits and Systems for Video Technology, Vol.7, NO. 2, pp. 279–286, 1997.
[3] D. Wang, “Unsupervised Video Segmentation Based on Watersheds and Temporal Tracking,” IEEE Transactions on Circuits and Systems for Video Technology, Vol.8, NO. 5, pp. 539–546, 1998.
[4] H.T. Nguyen, M. Worring and A. Dev, “Detection of Moving Objects in Video Using a Robust Similarity Measure,” IEEE Transactions on Image Processing, Vol.9, NO. 1, pp.137–141, 2000.
[5] T. Aach, A. Kaup and R. Mester, “Statistical Model-Based Change Detection in Moving Video,” Signal Processing, Vol.31, NO. 2, pp.203–217, 1993.
[6] A. Neri, S. Colonnese, G. Russo and P. Talone, “Automatic moving object and background separation,” Signal Processing, Vol.66, pp.219–232, 1998.
[7] D.D. Giusto, F. Massidda and C. Perra, “A Fast Algorithm for Video Segmentation and Object Tracking,” The 14th International Conference on Digital Signal Processing, Vol.2, pp.697–700, Cagliari Univ., Italy, 2002.
[8] S.Y. Chien, S.Y. Ma and L.G. Chen, “Efficient Moving Object Segmentation Algorithm Using Background Registration Technique,” IEEE Transactions on Circuits and Systems for Video Technology, Vol.12, NO. 7, pp. 577–586, 2002.
[9] E.P. Ong, B.J. Tye, W.S. Lin and M. Etoh, “An Efficient Video Object Segmentation Scheme,” Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, Vol.4, pp.IV-3361–IV-3364, Singapore, 2002.
[10] J.H. Pan, C.W. Lin, C. Gu and M.T. Sun, “A Robust Video Object Segmentation Scheme with Prestored Background Information,” IEEE International Symposium on Circuits and Systems, Vol.3, pp.803 – 806, Seattle, WA, USA, 2002.
[11] R. Cucchiara, C. Grana, M. Piccardi and A. Prati, “Detecting Moving Objects, Ghosts and Shadows in Video Streams,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.25, NO. 10, pp.1337–1342, 2003.
[12] S.Y. Chien, Y.W. Huang, B.Y. Hsieh, S.Y. Ma and L.G. Chen” Fast video segmentation algorithm with shadow cancellation, global motion compensation, and adaptive threshold techniques” Multimedia, IEEE Transactions on Digital Object Identifier,Vol.6, NO. 5,pp.732–748, 2004.
[13] M. Dahmane and J. Meunier, “Real-time video surveillance with self-organizing
maps” Computer and Robot Vision Proceedings. The 2nd Canadian Conference, pp.136–143, May 2005.
[14] A. Leone, C. Distante; N. Ancona, E. Stella and P. Siciliano, ” Texture analysis for shadow removing in video-surveillance systems” Systems, Man and Cybernetics, 2004 IEEE International Conference, Vol. 7, pp.6325–6330, Oct 2004.
[15]W.M. Lu and Y.P. Tan, ”A vision-based approach to early detection of drowning incidents in swimming pools” Circuits and Systems for Video Technology, IEEE Transactions on Digital Object Identifier, Vol. 14, Issue 2, pp.159 - 178, Feb. 2004.
[16] F. Oberti, S. Calcagno, M. Zara, and C.S. Regazzoni, “Robust Tracking of Human and Vehicles in Cluttered Scenes With Occlusions,” Proceedings of International Conference on Image Processing, Vol.3, pp.629 – 632, Genova, Italy, 2002.
[17] C.G. Kim and J.N. Hwang, “Fast and Automatic Segmentation and Tracking for Content-Based Application,” IEEE Transactions on Circuits and Systems for Video Technology, Vol.12, NO. 2, pp.122–129, 2002.
[18] Y.W. Chen, D.Y. Chen and S.Y. Lee, “Moving Object Tracking for Video Surveillance in Compressed Videos,” The 7th International Conference on Internet and Multimedia Applications and Systems, pp.695–698, 2003.
[19] Y.K. Jung, K.W. Lee and Y.S. Ho, “Content-Based Event Retrieval Using Semantic Scene Interpretation for Automated Traffic Surveillance,” IEEE Transactions on Intelligent Transportation Systems, Vol.2, NO. 3, 2001.
[20] Y. Huang, T.S. Huang and H. Niemann, “Segmentation-Based Object Tracking Using Image Warping and Kalman Filtering,” Proceedings of International Conference on Image Processing, Vol.3, pp.601–604, Urbana, IL, USA, 2002.
[21] G.Z. Cao, J.P. Jiang and J.Q. Chen, “An improved object tracking algorithm based on Image Correlation,” IEEE International Symposium on Industrial Electronics, Vol.1, pp.598–601, Hanzhou, China, 2003.
[22] G.L. Foresti, ” Object recognition and tracking for remote video surveillance” Circuits and Systems for Video Technology, IEEE Transactions, Vol. 9, Issue 7, pp.1045–1062, Oct. 1999.
[23] T. Wada and T. Matsuyama, “Multiobject Behavior Recognition by Event Driven Selective Attention Method” IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol.22, NO.8, Aug 2000.
[24] I. Haritaoglu, D. Harwood and L.S. David, “W4Real-Time Surveillance of People and Their Activities.” IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol.24,NO.8, Aug 2000.
[25] R. Polana and R. Nelson, “Recognizing Activities,”Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 815–818, 1994.
[26] B.A. Jezekiel, Z.Q. Wang, W. Pandif and S. Rajaram. “Human Activity Recognition Using Multidimensional Indexing.” IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol.24, NO.8, Aug 2002.
[27] C.D. Liu, P.C. Chuug and Y.N. Chung “Human Home Behavior Interpretation from Video Streams” Networking, Sensing and Control, 2004 IEEE International Conference, Vol. 1, pp.192–197 , Mar. 2004.
[28] P. Peursum, S. Venkatesh, G.A.W. West and H.H. Bui, ” Object labelling from human action recognition” Pervasive Computing and Communications. Proceedings of the First IEEE International Conference, pp.399–406, Mar. 2003.
[29] Y. Chen and D.H. Ballard,” Learning to recognize human action sequences” Development and Learning, 2002. Proceedings. The 2nd International Conference, pp.28–33, Jun. 2002.
[30] J.W. Davis and A. Tyagi, ” A reliable-inference framework for recognition of human actions” Proceedings. IEEE Conference on Advanced Video and Signal Based Surveillance, pp.169–176, Jul. 2003.
[31] R. Cucchiara, C. Grana, A. Prati and R. Vezzani, ” Probabilistic posture classification for Human-behavior analysis” Systems, Man and Cybernetics, Part A, IEEE Transactions, Vol. 35, Issue 1, pp.42–54, Jan. 2005.
[32] J. Yamato, J. Ohya and K. Ishii, ” Recognizing human action in time-sequential images using hidden Markov model” Computer Vision and Pattern Recognition, 1992.
Proceedings CVPR '92., 1992 IEEE Computer Society Conference, pp.379–385, Jun.
1992.
[33] P. Sangho and J.K. Aggarwal, ”Semantic-level Understanding of Human Actions and Interactions using Event Hierarchy” Computer Vision and Pattern Recognition Workshop, pp.12–12, Jun. 2004.
[34] S. Muller-Schneiders, T. Jager, H.S. Loos, W. Niem and R. Bosch, ” Performance Evaluation of a Real Time Video Surveillance System” Visual Surveillance and Performance Evaluation of Tracking and Surveillance, 2nd Joint IEEE International Workshop, pp.137–144, Oct. 2005.
[35] X.D. Liu and G.D. Su, ” A new network-based intelligent surveillance system”
Signal Processing Proceedings, WCCC-ICSP 2000. 5th International Conference, Vol.
2, pp.1187–1192, Aug. 2000.
[36] M. Valera and S.A. Velastin, ” Real-time architecture for a large distributed surveillance system” Intelligent Distributed Surveillance Systems,pp.41–45, Feb.
2004.
[37] L. Fuentes and S.A. Velastin, ” Advanced Surveillance: From Tracking To Event Detection” Latin America Transactions, IEEE (Revista IEEE America Latina), Vol.
2, Issue 3, pp.1–1, Sept.2004.
[38] M. Valera and S.A. Velastin, ” Intelligent distributed surveillance systems: a review” Vision, Image and Signal Processing, IEE Proceedings, Vol. 152, Issue 2, pp.192–204, Apr. 2005.
[39] X. Desurmont, A. Bastide, C. Chaudy, C. Parisot, J.F. Delaigle and B. Macq, ” Image analysis architectures and techniques for intelligent surveillance systems”
Vision, Image and Signal Processing, IEE Proceedings, Vol. 152, Issue 2, pp.224–231, Apr. 2005.
[40] Y.T. Chien, Y.S. Huang, S.W. Jeng, Y.H. Tasi and H.X. Zhao, ” A real-time security surveillance system for personal authentication” Security Technology, Proceedings. IEEE 37th Annual 2003 International Carnahan Conference, pp.190–195, Oct. 2003.
[41] C. Kim and J.N. Hwang, ” Object-based video abstraction for video surveillance ystems” Circuits and Systems for Video Technology, IEEE Transactions, Vol.
12, Issue 12, pp.1128 – 1138, Dec.2002.
[42] S. Morita, K. Yamazawa and N. Yokoya, ” Networked video surveillance using multiple omnidirectional cameras” Computational Intelligence in Robotics and Automation, IEEE International Symposium, Vol. 3, pp.1245–1250, Jul.2003.
[43] Q.Z. Wu, H.Y. Chang and K.C. Fan, “Motion Detection Based on Two-Piece Linear Approximation for Cumulative Histograms of Ratio Image in Intelligent Transportation Systems,” Proceedings of IEEE International Conference on Networking, Sensing & Control, Vol.1, pp.309–314, 2004.
[44] J.B. Kim, H.S. Park, M.H. Park and H.J. Kim, “Unsupervised Moving Object Segmentation and Recognition Using Clustering and A Neural Network,”
International Joint Conference on Neural Networks, Vol.2, pp. 1240–1245, Taegu , South Korea, 2002.