Objects Extraction

Foreground segmentation is the first step of objects extraction and it’s to detect regions corresponding to moving objects such as vehicles and pedestrians. The modules of objects tracking and behavior analysis only need to focus on those regions of moving objects. There are three conventional approaches for foreground segmentation outlined in the following:

1) Optical flow. Optical-flow-based motion segmentation uses characteristics of the flow vectors of moving objects over time to detect moving regions in an image sequence. This method is often applied to 3D-reconstruction research [1] or activity analysis work [2].

Optical-flow-based methods also can be used to discriminate between different moving groups, e.g., the optical flow of background resulting from camera motion is different with

optical flow resulting from moving objects. There are some algorithms proposed to assist in solving equations of optical flow. Differential technique, region-based matching, energy-based method and phase-based method are main methods, which are used for optical flow framework. Barron [3] presented those approaches of optical flow and evaluated the performances and measurement accuracy. Meyer et al. [2] computed the displacement vector field to initialize a contour based on the tracking algorithm for the extraction of articulated objects. But the optical flow method is computationally complex and very sensitive to noise, and cannot be applied to video streams in real time without the specialized hardware.

2) Temporal differencing. This method takes the differences between two or three consecutive frames in an image sequence to extract moving regions. Comparing with optical flow method, the temporal differencing is less computation and easy to implement with real-time tracking systems. Besides, the temporal differencing is adaptive to dynamic environments. But it is poor in extracting all the relevant pixels, e.g., there may be holes left inside moving objects. Some research used three-frame differencing instead of the two-frame process. Lipton et al. [4] used temporal differencing method to detect moving objects in real video streams.

3) Background subtraction. Background subtraction-based method is an easy and popular method for motion segmentation, especially under those situations with a relatively static background. It detects moving regions by taking the difference between the current image and the reference background image in a pixel-by-pixel sequence. It did a good job to extract complete and clear objects region. But it’s sensitive to changes in dynamic environment derived from lighting and extraneous factors etc. Hence, a good background model is indispensable to reduce the influence of these changes. Haritaoglu et al. [5] built a statistical model by representing each pixel with three values: its minimum and maximum intensity values, and the maximum intensity difference between consecutive frames observed

during the training period. These three values are updated periodically.

Besides those basic methods described above, there are other approaches or combined methods for foreground segmentation. Elgammal et al. [6], [22] presented nonparametric kernel density estimation techniques as a tool for constructing statistical representations for the scene background and foreground regions in video surveillance. Its model achieved sensitive detection of moving targets against cluttered background. Kamijo [7] proposed a spatio-temporal markov random field model for segmentation of spatio-temporal images.

Kato et al. [8] used a hidden Markov model/Markov random field (HMM/MRF)-based segmentation method that is capable of classifying each small region of an image into three different categories: vehicles, shadows of vehicles, and backgrounds. The method provided a way to model the shadows of moving objects as well as background and foreground regions.

As mentioned previously, active construction and updating of background are important to object tracking system. Therefore, it’s a key process to recover and update background images from a continuous image sequences automatically. Unfavorable conditions, such as illumination variance, shadows and shaking branches, bring many difficulties to this acquirement and updating of background images. There are many algorithms proposed for resolving these problems. Median filtering on each pixel with thresholding based on hysteresis was used by [9] for building a background model. Friedman et al. [10] used a mixture of three Gaussians for each pixel to represent the foreground, background, and shadows with an incremental version of EM (expectation maximization) method. Ridder et al.

[11] modeled each pixel value with a Kalman filter to cope with illumination variance.

Stauffer et al. [12] presented a theoretic framework for updating background with a process in which a mixed Gaussian model and the online estimation were used. McKenna et al. [13] used an adaptive background model with color and gradient information to reduce the influences of shadows and unreliable color cues. Cucchiara et al. [14] based the background subtraction

method and combined statistical assumptions with the object level knowledge of moving objects to update the background model and deal with the shadow. They also used optical flow method to improve object segmentation. Li et al. [15] proposed a Bayesian framework that incorporated spectral, spatial, and temporal features to characterize the background appearance. Under this framework, a novel learning method was used to adapt to both gradual and sudden background changes.

In our system, we proposed foreground segmentation framework based on background subtraction and temporal differencing. We also introduced an adaptive background updating algorithm using the statistic index. It’s effective to cope with the gradual and sudden changes of the environment.