Ubiquitous International Volume 3, Number 3, July 2012
Feature-Based Vehicle Flow Analysis and Measurement for a
Real-Time Traffic Surveillance System
Deng-Yuan Huang1, Chao-Ho Chen2, Wu-Chih Hu3, Shu-Chung Yi4 and Yu-Feng Lin5
1
Department of Electrical Engineering Dayeh University
Ag. Loukas, 65404, Kavala, Greece
168 University Rd., Dacun, Changhua 515, Taiwan (R.O.C.) [email protected]
2
Department of Electronic Engineering National Kaohsiung University of Applied Sciences, 415 Chien Kung Rd., Kaohsiung 807,Taiwan (R.O.C.)
thouho; [email protected]
3Department of Computer Science and Information Engineering, National Penghu University of Science and Technology 300 Liu-Ho Rd., Makung, Penghu 880, Taiwan (R.O.C.)
Department of Computer Science and Information Engineering, National Changhua University of Education
Changhua City, 500, Taiwan (R.O.C.)
Received January 2012; revised June 2012
Abstract. A feature-based vehicle flow analysis and measurement system is proposed for a
real-time traffic surveillance system. The system includes moving object segmentation, background updating, feature extraction, and vehicle tracking and classification. Moving object segmentation is firstused to extract the contour of vehicles. By analyzing the contours of vehicles and their corresponding minimal bounding box, salient discriminative features of vehicles are obtained. The tracking of moving targets is then achieved by comparing extracted features and by measuring the minimum distance between two consecutive images. To increase the accuracy of vehicle classification, the temporal correlation of moving objects tracked between video frames is taken into consideration. In addition, the velocity of each vehicle and the vehicle flow through the field of vision are calculated by analyzing the features of vehicles. Experimental results show that classification rates of 96.4% and 92.7% for cars and bikes, respectively, can be achieved using the feature of aspect ratio. The bikes here refer to motorcycles, scooters, or bicycles. The average accuracy of vehicle flow measurement of 96.9% is obtained, indicating the feasibility of the proposed method.
Keywords:Vehicle counting, Vehicle detection, Object segmentation, Background subtraction, Motion detection, Feature extraction.
1. Introduction. the past decades, traffic Congestion has become a major concern in a modern society due to the increase of vehicles.Solutions include widening roads and banning cars from central business districts. However, traffic congestion remains a serious problem due to the limitations of road construction and expansion. Numerous studies have thus been conducted on intelligent transportation systems (ITSs), which integrate microelectronics, AI, robotics, sensing, communication, and control. ITSs are considered to have the most potential for solving the problems of traffic congestion, where real-time machine vision that can augment drivers visual capabilities is the most attractive technology in ITSs.
In general, traffic-flow monitoring systems can be divided into two categories. One is the traditional embedded system with a voltage return circuit. This type of system is reliable but the cost of implementation is relatively high. In addition, roads need to be dug up for installation and maintenance, which greatly influence traffic mobility. The other is a hanging type system that uses sensors such as cameras, radar, and infrared sensors. Camera-based systems are most attractive due to their lower cost of installation and maintenance compared to those of other systems. In this paper, we propose a camera-based vehicle flow analysis and measurement system that can be used for real-time traffic control and monitoring, which are important for ITSs.
The fundamental task of most video processing algorithms is to extract the regions of interest (ROIs) of moving objects. Video images are segmented into moving objects (or foregrounds) and the remaining parts (or backgrounds) to facilitate subsequent tracking procedures. Motion segmentation in video sequences focuses on detecting regions corresponding to moving objects such as cars and bikes; it can be categorized as: (1) background subtraction [1]-[4], (2) tem-poral differencing [5]-[7], (3) optical flow [8], and (4) block motion estimation [9]. Background subtraction can extract the most feature pixels but it is extremely sensitive to lighting changes. Temporal differencing is suitable to dynamic environments but its performance to extract rel-evant feature data is not satisfied. Optical flow can detect moving objects in the presence of camera motion but its computational complexity is very high. Block motion estimation can reduce the computational complexity of the optical flow method but the accuracy may be also decreased.
Motion detection in many tracking systems involves background subtraction. Lighting varia-tions in a scene can cause serious problems formany background subtraction methods. Ridderet al. [1][3] modeled each pixel in a background with a Kalman filter to allow their system to adapt to lighting changes in a scene. In their method, the background estimator performs well for human body tracking in real-world scenes with illumination changes due to daylight or mov-ing clouds or both. Although their method uses a pixel-wise automatic threshold, it recovers slowly and cannot handle bimodal backgrounds well. Generally, there are two types of Gaussian method for background subtraction: a single Gaussian and a mixture of Gaussians [2]. A mix-ture of Gaussians is more suitable than a single Gaussian for dealing with slow lighting changes, slow moving objects, and camera noise. Basically, a mixture of Gaussians can represent the color distribution of each pixel to handle variations due to factors such as lighting, shadows, and camera noise. By updating parameters, their system can track people and cars in outdoor environments. Moreover, another application for ship detection by background subtraction was presented by Hu et al. [4].
Based on frame differencing, moving targets can be extracted from a real-time video stream using the pixel-wise difference between consecutive frames. This approach can classify humans, vehicles, and background clusters [5]-[7]. Once classified, targets are tracked bya combination of temporal differencing and template matching. The improved approach makes the segmentation of moving objects more compact by using two difference images obtained from three consecutive frames in video sequences, where the two difference images are processed using a logical AND operation. For obtaining the better segmentation result, another scheme utilizes both methods of adaptive background subtraction and temporal differencing for moving objects detection.
Garlic and Loncaric [8] used optical flow to extract the feature vectors in a video sequence. The feature vectors extracted are further clustered using a K-means clustering algorithm to determine the characteristic image regions, which allows the detection of a moving object in video images. Chen et al. [9] proposed a multipath search with flatted-hexagon pattern for block motion estimation to achieve adjustable speed and accuracy in the block matching algorithm. This method can greatly reduce the computational complexity of optical flow even though it may slightly decrease the performance of optical flow.
To detect cars and bikes in traffic surveillance sequences, background initialization, foreground detection, and background updating are three essential procedures. Object tracking methods can be classified as region-based tracking [10], active contour-based tracking [11][12], feature-based tracking [13]-[15], and model-feature-based tracking [16][17]. Among these methods, feature-feature-based
tracking is most widely used due to its robustness; even with partial occlusion, some of the features of moving objects are still visible, and the approach can adapt to varying illumination, such as daylight, twilight, or night-time conditions. In general, the features of a moving object can be classified as: (1) global feature-based, such as the center of gravity, color, and area [13]; (2) local feature-based, such as segments and vertices [14]; and (3) dependence graph-based [15], such as structure changes between features. However, since tracking performance highly depends on the selection of features, the problem of grouping, i.e., what set of features belongs to a moving object, is introduced. In this paper, several salient features are presented for classifying cars and bikes in video images, and counting and analysis processes are proposed.
Morris and Trivedi [18] proposed a real-time highway monitoring system called VECTOR for tracking and classifying vehicles from live video streams. In their method, the front-end ofthe system includes object detection by adaptive background subtraction, Kalman filter-based tracking, and vehicle classification with tracking details. During object detection, blob features of an object are extracted, such as area, bounding ellipse, and image moments, and transformed using linear discriminant analysis (LDA). The reduced feature set is then applied to classify vehicles into 8 main classes using a weighted K nearest neighboring (wKNN) method. Efforts have been made to improve the performance of the VECTOR system by adding real-time sit-uational awareness to highway monitoring for high-level activity and behavior analysis [19]. A path prediction module was proposed to detect abnormal trajectories and to predict future in-tent of drivers, where the spatio-temporal motion characteristics of motion paths are encoded by a hidden Markovmodel (HMM).
In recent years, traffic monitoring systems have been presented to automatically extract im-portant traffic parameters using only cameras. These systems often combine tracking, object classification, traffic parameter extraction, and event detection. For these systems, visual track-ing is important for measurtrack-ing vehicle flow at intersections, which can be used to improve urban mobility efficiency. Moreover, vehicle path trajectories are statistically predicted for behavior analysis in a driver assistance system (DAS). In [20], a framework for tracking and categorizing multiple vehicles using Markov chain Monte-Carlo particle filters (MCMC PF) was proposed to classify moving objects into motorcycles, cars, light trucks, and heavy trucks. The method has also been applied to the tracking of pedestrians [21].
Lai et al. [22] developed a traffic surveillance system that includes three phases: vehicle region extraction, vehicle tracking, and classification. In their method, the background subtrac-tion method is used to segment foreground regions from highway scenes. Geometric features and a shadow-removal algorithm are used to remove false regions and to improve segmenta-tion accuracy, respectively. The graph-based tracking method is often adopted to determinethe correspondence between vehicles detected at different instances.Two measures, suchas aspect ratio and compactness, are usedto performthe classification of vehicles. More recently, a sto-chastic approach called particle filtering [23], which is also known as the sequential Monte Carlo method, has been widely used in video object tracking; it relaxes the Gaussian assumption of vehicle motion due to the nonlinearity of the state transition.
Traffic vehicle counting is important in traffic scene analysis. Wakabayashi and Aoki [24] proposed a traffic flow measurement system that uses a pair of stereo slit cameras. However, they did not extract vehicle type and velocity, which are important parameters for traffic surveillance. Therefore, the present study proposes a traffic surveillance system that can determine vehicle type and velocity besides vehicle counting.
The rest of this paper is organized as follows. In Section 2, we outline the proposed method for vehicle flow counting, which includes vehicle segmentation, background updating, feature extraction, vehicle tracking, and vehicle classification. Section 3 describes the experimental results and their evaluations. Finally, the conclusion is given in Section 4.
2. The Proposed Method. A flowchart of the proposed method for vehicle flow analysis and measurement is shown in Fig. 1. Moving objects are first segmented from a sequence of video images using the schemes of motion detection and background updating. Motion detection is used to analyze the temporal correlation of moving objectsin successive frames. A frame
difference mask and a background subtraction mask are utilized to acquire the initial object mask on which the problem of stationary objects in backgrounds can be solved. Moreover, boundary refinement is introduced to reduce the influence of shadows and the problem of residual background. Each segmented object, denoting a vehicle, is then bounded by a rectangle; the height, width, and area of the rectangle are regarded as important features of that vehicle. Based on these salient features, each vehicle can befurther classified as a car or a bike.
Figure 1. Flowchart of the proposed method for vehicle flow analysisand measurement
2.1. Vehicle segmentation. Video cameras for a traffic surveillance system are often mounted at major roads in a modern city. Since the background in videos is almost stationary, back-ground subtraction [1]-[4] is thus suitable for detecting moving objects such as pedestrians or vehicles. Since the performance of vehicle segmentation highly depends on the extraction of a reliable background, the reference mask is established using background subtraction and then background updating to prevent the background information from containing those of moving targets. In this paper, we used the temporal information of the mean and standard deviation of gray level distribution in consecutive frames to model the background for each point, and updat-ing is adaptively proceeded, by which the gray level providupdat-inga maximum occurrence probability is then assigned to that of the absolute background for that point. As to the detailed information of how to build this model, the readers can refer to our previously published paper [25].
Once background mask is established, foreground objects such as cars and bikes can be readily extracted by the proposed multi-object segmentation algorithm, as illustrated in Fig.2. In the proposed method, we first scanned the binary image from left to right and row by row, as shown in Fig. 2(a), to obtain two larger regions with moving objects. Next, we scanned these two regions from top to bottom and column by column, as shown in Fig. 2(b), to obtained four smaller regions having moving objects. Finally, we scanned these four regions again from left to right and row by row, as shown in Fig. 2(c), to obtain the minimum bounding box of each moving object. The resulting segmentation of each moving object is shown in Fig. 2(d). Figure 3 shows the resulting segmentation of cars and bikes under different traffic situations by the proposed approach.
2.2. Vehicle feature extraction. Numerous features can be extracted from a moving target, such as texture, color, and shape. These features can be roughly classified as spatial features or temporal features. Spatial features are often used to discriminate objects at a given time, and temporal features are used for recognizing an object at different points in time. To recognize targets, it is necessary to obtain specific features that can discriminate between various moving objects.
Figure 2. Illustrations of the proposed multi-object segmentation algorithm
Figure 3. Resulting segmentation of cars and bikes under different traffic situations
For a moving vehicle, features such as perimeter and area may vary with timedue to the use of a fixed video camera. To reduce the effect of feature variation, an analysis of the boundingbox
4. Conclusion. This paper presents a feature-based method of vehicle analysis and counting for bidirectional roads in a real-time traffic surveillance system. Vehicle features, such as the aspect ratio, area ratio, and compactness, are extracted from a video sequence and used to categorize vehicles as cars or bikes. A simple and efficient tracking algorithm is introduced to robustly track vehicles. Classification rates of 96.4% and 92.7% for cars and bikes, respectively, are achieved using the feature of aspect ratio. The counting accuracy on an average of 96.9% of cars and bikes was obtained by the proposed method of vehicle analysis and counting. Furthermore, the estimation error of vehicle velocity is less than 5 km/h.
Acknowledgment. This work is partially supported by HuperLabortories Co. and National
Science Council under Grant NSC96-2622-E-151-016-CC3, Taiwan, R.O.C.
REFERENCES
[1] C. Ridder, O. Munkelt, and H. Kirchner, Adaptive background estimation and foreground detection using kalman-filter, Proc. of International Conference on Recent Advances in Mechatronics, pp. 193-199, 1995. [2] C. Stauffer, and W. Grimson, Adaptive background mixture models for real-time tracking, Proc. of IEEE
Conference on Computer Vision and Pattern Recognition, pp. 246-252, 1999.
[3] T. Y. Chen, C. H. Chen, D. J. Wang, and Y. C. Chiou, Real-time video object segmentation algorithm based on change detectionand background updating, Journal of Innovative Computing, Information and Control, vol. 5, no.7, pp. 1797-1810, 2009.
[4] W. C. Hu, C. Y. Yang, and D. Y. Huang, Robust real-time ship detection and tracking for visual surveillance of cage aquaculture, Journal of Visual Communication and Image Representation, vol. 22, no. 6, pp. 543-556, 2011.
[5] A. J. Lipton, H. Fujiyoshi, and R. Patil, Moving target classification and tracking from real-time video, Proc.
of the 4th IEEE Workshop on Applications of Computer Vision, pp. 8-14, 1998.
[6] C. H. Chen, T. Y. Chen, D. J. Wang, and T. J. Chen, A cost-effective people-counter for a crowd of moving people based on two-stage segmentation, Journal of Information Hiding and Multimedia Signal Processing, vol. 3, no. 1, pp. 12-25, 2012.
[7] C. Y. Yang, C. H. Chen, W. C. Hu and S. S. Su, Reliable moving vehicle detection based on the filtering of swinging tree leaves and raindrops, Journal of Visual Communications & Image Representation, vol. 23, no. 4, pp. 648-664, 2012.
[8] S. Galic, and S. Loncaric, Spatio-temporal image segmentation using optical flow and clustering algorithm
Proc. of the First International Workshop on Image and Signal Processing and Analysis, pp. 63-68, 2000.
[9] C. H. Chen, T. Y. Chen, D. J. Wang, and Y. F. Li, Multipath flatted-hexagon search for block motion estimation, Journal of Information Hiding and Multimedia Signal Processing, vol. 1, no. 2, pp. 110-131, 2010. [10] S. McKenna, S. Jabri, Z. Duric, A. Rosenfeld, and H. Wechsler, Tracking groups of people, Computer Vision
and Image Understanding, vol. 80, no. 1, pp. 42-56, 2000.
[11] A. Mohan, C. Papageorgiou, and T. Poggio, Example-based object detection in images by components, IEEE
Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 4, pp. 349-361, 2001.
[12] A. Galata, N. Johnson, and D. Hogg, Learning variable-length markov models of behavior, Journal of
Com-puter Vision and Image Understanding, vol. 81, no. 3, pp. 398-413, 2001.
[13] B. Schiele, Model-free tracking of cars and people based on color regions, Proc. of IEEE International
Workshop on Performance Evaluation of Tracking and Surveillance, pp. 61-71, 2000.
[14] B. Coifman, D. Beymer, P. McLauchlan, and J. Malik, A real-time computer vision system for vehicle tracking and traffic surveillance, Journal of Transportation Research Part C: Emerging Technologies, vol. 6, pp. 271-288, 1998.
[15] T. J. Fan, G. Medioni, and G. Nevatia, Recognizing 3-D objects using surface descriptions, IEEE Trans.
Pattern Analysis and Machine Intelligence, vol. 11, no. 11, pp. 1140V1157, 1989.
[16] W. Hu, T. Tan, L. Wang, and S. Maybank, A survey on visual surveillance of object motion and behavio,
IEEE Trans. Systems, Man, and CyberneticsXPart C: Applications And Reviews, vol. 34, no. 3, pp. 334-352,
2004.
[17] W. F. Gardner,and D. T. Lawton, Interactive model-based vehicle tracking, IEEE Trans. Pattern Analysis
and Machine Intelligence, vol. 18, no. 11, pp. 1115V1121, 1996.
[18] B. T. Morris, and M. M. Trivedi, Real-time video based highway traffic measurement and performance monitoring, Proc. of the IEEE Conference on Intelligent Transportation System, pp. 59V64, 2007.
[19] B. T. Morris, and M. M. Trivedi, Learning, modeling, and classification of vehicle track patterns from live video, IEEE Trans. Intelligent Transportation Systems, vol. 9, no. 3, pp. 267-276, 2008.
[20] F. Bardet, T. Chateau, and D. Ramadasan, Unifying real-time multi-vehicle tracking and categorization,
[21] Y. Goyat, T. Chateau, L. Malaterre, and L. Trassoudaine, Vehicle trajectories evaluation by static video sensors, Proc. of the 9th InternationalConference on Intelligent Transportation Systems Conference, pp. 864-869, 2006.
[22] J. C. Lai, S. S. Huang, and C. C. Tseng, Image-based vehicle tracking and classification on the highway,
Proc. of International Conference on Green Circuits and Systems, pp. 666-670, 2010.
[23] J. Zhu, Y. Lao, and Y. F. Zheng, Object tracking in structured environments for video surveillance applica-tions, IEEE Trans. Circuits and Systems for Video Technology, vol. 20, no. 2, pp. 223-234, 2010.
[24] Y. Wakabayashi, and M. Aoki, Traffic flow measurement using stereo slit camera, Proc. of the 8th
Interna-tional IEEE Conference on Intelligent Transportation Systems, pp.198-203, 2005.
[25] D. Y. Huang, C. H. Chen, W. C. Hu, and S. S. Su, Reliable moving vehicle detection based on the filtering of swinging tree leaves and raindrops, Journal of Visual Communication and Image Representation, vol. 23, no. 4, pp. 648-664, 2012.
[26] L. Bo and Z. Heqin, Using object classification to improve urban traffic monitoring system, Proc. IEEE
International Conference on Neural Networks and Signal Processing, pp. 1155-1159, 2003.
[27] B. L. Tseng, C. Y. Lin, and J. R. Smith, Real-time video surveillance for traffic monitoring using virtual line analysis, Proc. IEEE International Conference on Multimedia and Expo, vol. 2, pp. 541-544, 2002.
[28] M. Lei, D. Lefloch, P. Gouton, and K. Madani, A video-based real-time vehicle counting system using adaptive background method, Proc. IEEE International Conference on Signal Image Technology and Internet Based
Systems, pp. 523-528, 2008.
[29] C. Pornpanomchai, T. Liamsanguan, and V. Vannakosit, Vehicle detection and counting from a video frame,
Proc. IEEE International Conference on Wavelet Analysis and Pattern Recognition, pp. 356-361, 2008.
[30] H. S. Mohana, M. Ashwathakumar, and G. Shivakumar, Vehicle detection and counting by using real time traffic flux through differential technique and performance evaluation, Proc. IEEE International Conference
