Application to Traffic Flow Estimation

For investigating traffic congestion problems, the background segmentation module has been integrated into an ITMS for extracting traffic parameters. Our research team have established a real-time web video streaming system to monitor the traffic in Hsinchu Science Park [69]. The system provides an H.263 video stream to an ITMS. H.263 is suitable for digital video transmission over networks with a high compression and decompression ratio [70]. However, due to image compression and decompression, the degraded image is neither stable nor consistent for image processing of image-based applications. This phenomenon introduces extra challenges to image processing design.

In this study, video stream provided by the imaging system is employed to measure the traffic flow at a multi-lane entrance of the Science Park. The video stream in CIF format (352 x 288 pixels) is transmitted to the computer in the lab at a rate of 7 frames per second through ADSL. The system architecture of the real-time traffic monitoring system is shown in Fig. 3-6.

The image system consists of three parts: image processing module, vehicle detection module, and traffic parameter extraction module. The image processing module adopts an ActiveX component to decompress the actual image from the video stream and converts the image into gray-level format. Real-life images are employed to construct a background image. Based on

Fig. 3-6. Block diagram of the image-based traffic parameter extraction system.

46

the constructed background image, a binary image of the moving vehicle is obtained through foreground segmentation. The detection module employed a detection window that behaves like a loop detector to count the number of vehicles in a multi-lane road [71]. The detection window is able to check if there is a vehicle entering or leaving the window from the binary image. It simultaneously detects multiple vehicles from the binary image. Traffic parameter extraction module calculates the traffic flow data and provides information to the ITS center.

Fig. 3-7 illustrates a display of the system results. The upper left part of the figure shows a real-time image and the lower right part shows the background image created from the image sequence. The upper right part depicts the binary image of moving vehicles as well as the processed results of the detection window. The detected vehicles count and the traffic flow data are displayed on the lower left portion of the figure. The estimated traffic flow is calculated through a measure of vehicle count using the equation below:

traffic flow =

dur car

t

N , (3.10)

Fig. 3-7. The display of traffic flow estimation.

47

where

N is the detected count and

_car

t is the time duration. A video clip of experimental

_dur results can be found at: http://isci.cn.nctu.edu.tw/video/JCTai/Traffic_flow.wmv.

3.4 Summary

An algorithm of a group-based histogram is proposed to build the Gaussian background model of each pixel from traffic image sequences. This algorithm features improved robustness against transient-stop of foreground objects and sensing noise. Furthermore, the method features low computational load, and thus, meets the real-time requirements in many practical applications. The proposed method can be extended to construct a color background image to further increase the robustness during intensity analysis.

48

Chapter 4

Cast-Shadow Detection in Traffic Image

4.1 Introduction

Cast-shadow suppression is an essential prerequisite for image-based traffic monitoring.

Shadows of moving objects often cause serious errors in image analysis due to misclassification of shadows or moving objects. It is necessary to design a shadow suppression method to improve the accuracy of image analysis. Shadow detection can be divided into two types: shape-based approaches [33]-[34] and spectrum-based approaches [37]-[39]. In the shape-based method, sophisticated models are constructed to identify the shadow according to the object and its surrounding illumination conditions. The accuracy of shadow detection depends on the knowledge of environment conditions. Moreover, it can not meet the real-time requirement because the computation load of the spatial analysis is heavy.

in contrast, spectrum-based approaches exploit color space information to find the shadow.

The model is built by strenuously analyzing the shadows in image frames. Generation of the model is difficult and inefficient. Additionally, conversion between different color spaces requires much computation time and degrades the real-time performance. Based on the Lambertian assumption, RGB ratios between lit pixels and shadowed pixels can be treated as a constant in image sequences. This information leads us to the development of a RGB ratio model to detect shadow pixels in traffic imagery. The proposed approach does not require many image sequences to construct the model. Instead, the model can be easily built using a

49

shadow region in a single image frame. To increase the accuracy of shadow detection, two types of spatial analysis are proposed to verify the actual shadow pixels.

The following sections will focus on how the ITMS with shadow suppression is developed. Section 4.2 focuses on shadow detection. A comparison with existing methods is presented in Section 4.3. Section 4.4 gives our concluding remarks.

4.2 Cast-Shadow Detection in Traffic Image

The block diagram of the proposed shadow suppression method is presented in Fig. 5-1.

The complete system consists of four modules: a background estimation module (Chapter 3), a background removal module (Chapter 3), a shadow detection module, and a shadow verification post-procession module. The shadow detection module uses an RGB spectral ratio model to identify shadows. The shadow verification module employs spatial analysis schemes to check whether a shadow pixel is true or not. Accordingly, moving objects and the cast-shadows can be separated.

4.2.1 Spectral Ratio Shadow Detection

A traffic scene is illuminated by a faraway point source (the sun) and a diffused source (the sky). The cast shadow is caused by sunlight occlusion in the scene. The distance between objects and their cast shadows is negligible in traffic scenes, compared with the distance between the light source and the objects. Thus, this type of the cast shadow is mostly an umbra or a strong shadow [35]. Shadow regions are darker than the background and their color spectrum also differs from that of the background. Since the RGB components of each pixel of the roadway differ from one another, it is impossible to use a unique model to detect the shadow of each pixel by merely using RGB color space. Therefore, color space conversion or normalization is employed to find the model. To resolve the generation of a

50

Background Estimation

Spatial Analysis Image

Sequences

Vehicle Blob for Image Analysis RGB ratio Gaussian

Shadow Model Foreground

Segmentation

Fig. 4-1. The block diagram of the proposed shadow detection method.

shadow model, we hypothesize that the RGB ratio between lit regions and shadow regions is constant for all pixels of a traffic scene in image sequences. This hypothesis facilitates the construction of a unique model for shadow detection in an image frame. The model-built procedure can be effectively achieved by using a shadow region in the image, not from image sequences. Detailed reasoning of the hypothesis is presented in Appendix C.

According to the hypothesis, in each RGB channel, the color ratio between a lit pixel and a shadow pixel can be treated as a constant for each pixel in the traffic imagery. As a result, a pixel is classified as a shadow pixel if its RGB component satisfies

lit

shadow

R

R

=

α

and

G

_shadow =

β G

_lit and

B

_shadow

= γ B

_lit , (4.1) where (

R

_lit,

G

_lit,

B

_lit) is the RGB value of a lit pixel, (

R

_shadow,

G

_shadow,

B

_shadow) is the RGB value of a shadow pixel and α^,β^,γ is less than 1.

Based on (4.1), a shadow-region-based statistical nonparametric approach is developed to construct the ratio model for shadow detection of all pixels in the image frame. Gaussian models are exploited to represent the constant RGB-color ratios between a lit pixel and a

51

shadow pixel in this approach. The unique ratio model can be found by analyzing shadow samples taken from the shadow region in an image frame. To cope with the variation of the ratio, we selected the Gaussian distribution inside ±1.5σ (88.6%.) as a threshold. Thus, a shadow pixel can be determined as follows:

 the RGB standard deviation ratio of pixels when shadowed in the background image.

Figures 4-2(a)-(d) illustrate an example of the RGB Gaussian shadow models. In Fig.

4-2(a), 100 samples of shadow pixels in the image frame are selected to build the Gaussian

(a) (b)

(c) (d)

Fig. 4-2. The Gaussian models of RGB ratio of recorded samples and shadow-region data.

2 1 3

52

RGB ratio model, as depicted by small white dots in the shadow region of the figure. To validate the Gaussian model found from the shadow region, we tested shadow data at three points (indicated by large white dots in Fig. 4-2(a)). Furthermore, all the recorded data are also combined and used to verify the hypothesis of constant color ratio. The Gaussian RGB ratio models of recorded data (labeled by 1, 2, 3 and total) and shadow-region data (labeled by region) are depicted in Figs. 4-2(b)-(d), respectively. The mean and the standard deviation of the RGB ratio of each sample are presented in Table 4-1. In our design, a point is regarded as a shadow point if its RGB ratio satisfies (4.2). The results computed from the four groups of samples (labeled by 1, 2, 3 and total) demonstrate that the accuracy is higher than 86%, as shown in Table 4-1. It demonstrates the effectiveness of the hypothesis and therefore, one can use the shadow-region-based RGB ratio model to determine the shadow in image sequences.

Figure 4-3 illustrates an example of shadow detection. By using (4.2) and the Gaussian

Table 4-1The Gaussian Models Of RGB Ratio of Recorded Data and Shadow-Region Data

Item Mean Standard

deviation Accuracy R ratio of region 0.443 0.076 86.64%

R ratio of 1 0.416 0.067 88.16%

R ratio of 2 0.406 0.055 91.58%

R ratio of 3 0.440 0.064 92.16%

R ratio of total samples 0.420 0.063 91.15%

G ratio of region 0.437 0.085 86.64%

G ratio of 1 0.413 0.060 96.39%

G ratio of 2 0.416 0.055 96.70%

G ratio of 3 0.440 0.055 97.65%

G ratio of total samples 0.423 0.057 97.25%

B ratio of region 0.463 0.099 86.64%

B ratio of 1 0.447 0.065 97.47%

B ratio of 2 0.452 0.053 99.41%

B ratio of 3 0.477 0.054 99.30%

B ratio of total samples 0.459 0.058 99.02%

53

model, one can detect the shadow, as shown in Fig. 4-3(a). Figure 4-3(b) shows the detected moving region. The result of statistical nonparametric shadow suppression is shown in Fig.

4-3(c). One observes in Fig. 4-3(c) that some shadow pixels are not recognized as expected.

This is mainly due to uncertainties in the image sensing. We handle this insufficiency and improve the performance of shadow suppression by adding a spatial analysis post-processing step.

4.2.2 Spatial Analysis for Shadow Verification

Two types of shadow detection error may occur, namely, shadow detection failure and object detection failure. The first type of error occurs if a shadow pixel has ratios outside the detection range of the shadow model, causing the shadow to not be recognized. As shown in Fig. 4-3(c), some shadow pixels are not recognized. The second type of failure detection occurs if the ratios of an object pixel lie inside the detection range of the shadow model. This occurs especially when the ratios are higher than the mean of Gaussian ratio model and still inside the detection range; there are almost no shadow pixels in this region, as shown in

(a) (b)

(c) (d)

Fig. 4-3. Explanation of shadow suppression steps. (a) Original image. (b) Moving object segmentation result of background removal. (c) Shadow segmentation result of spectral ratio shadow detection. Detected shadow is indicated by white area. (d) Segmentation result of shadow suppression after spatial analysis.

Shadow detection failure

Object detection

failure

54

sample plots of Figs. 4-2(b)-(d). For instance, in Fig. 4-3(c), partial pixels of a vehicle are misclassified as shadow pixels. To improve the accuracy of shadow detection, a post-processing of spatial analyses is added for shadow confirmation. The spatial analyses are performed to confirm the true shadows and the true objects according to their geometric properties.

4.2.3 Size Discrimination of Moving Object Candidates

In the process of shadow detection, shadow candidates sometimes break actual shadow into small isolated blobs. Generally the sizes of these small shadow blobs are smaller than the detected moving vehicles in image sequences. These small blobs will not be considered moving object candidates. Thus, one can discriminate the small blobs of shadow from the big blobs of moving objects by using the size information. In our design, all blobs of moving object candidates are grouped into different regions using a connected components labeling algorithm. The regions that have small sizes are recognized as shadow regions.

4.2.4 Border Discrimination of Moving Cast Shadow Candidates

The moving blobs segmented by background removal consist of shadow pixels and object pixels. In practice, the true shadow pixels cluster in the fringes of the blobs. If a part of the detected vehicle is misclassified as a shadow, most of boundaries of this region will be located inside the candidate foreground, as shown in Figs. 4-3(b)-(c). If the shadow candidate is a true shadow, more than a half of the boundary is adjacent to the boundary of foreground candidates. Thus, one can use the boundary information of a shadow-candidate region to confirm whether the shadow is a true shadow or not [40]. In our design, the boundaries of the foreground candidate are segmented by Sobel edge detection. Next, each distinct candidate shadow region is determined by using a connected components labeling algorithm. Sobel edge detection is also used to find the edge of each distinct shadow region [64]. The number

55

Nf of boundary shadow pixels that are adjacent to the boundary of a foreground-candidate region and the number N_s of all boundary shadow pixels are computed. The shadow is considered a true shadow if the ratio

s f

N

N is greater than 50%. The confirmation result of

spatial analysis is shown in Fig. 4-3(d). It indicates that the accuracy of shadow detection is greatly improved in comparison to the original.

4.3 Comparison Results

For traffic monitoring and surveillance applications, shadow suppression prevents misclassification and erroneous counting of moving vehicles. The goal of shadow suppression is to minimize the false negatives (

FN , the shadow pixels misclassified as

_S background/foreground) and the false positives (

FP , the background/foreground pixels

_S misclassified as shadow pixels). In order to systematically evaluate the performance of the proposed method, we adopted two metrics, namely the shadow detection rate η and the object detection rate ξ [36] for quantitative comparison:

S

where

TP (resp.

_S

TP ) is the number of shadow (resp. foreground) pixels correctly

_F identified, and

FN (resp.

_S

FN ) is the number of shadow (resp. foreground) pixels falsely

_F identified. A comparison with existing methods has been carried out to validate the performance of the proposed algorithm. A statistical nonparametric (SNP) approach [72] and a deterministic nonmodel-based (DNM) approach [38] were selected for comparison. The SNP

56

approach treated object colors as a reflectance model from the Lambertian hypothesis. The work used the normalized distortion of the brightness

α

_i^′ and the distortion of the chrominance

C D

_i′, computed from the difference between the background color of a pixel and its value in the current image, to classify a pixel in four categories:

Foreground :

C D

_i′>

τ

_CD or

α

_i′<

τ

_αlo else

The DNM approach works in the HSV color space. Shadow detection is determined according to the following equation:

 frame k) respectively.

To achieve an objective comparison, the model for the background image is updated in advance for all algorithms in the test. Figure 4-4 shows the test results of different methods (Proposed, DNM [38] and SNP [72]) from a benchmark sequences Highway-I [36]. First, we check the computation time of each algorithm for efficiency comparison, as shown in Table 4-2. The proposed algorithm requires the least computation time. It is reduced to 3.6%-21%

of the other two methods because it merely utilizes division operation to obtain the shadow (4.5)

57

(a) (b)

(c) (d)

Fig. 4-4. Comparison results of shadow suppression method (red pixels represent the moving vehicle;

the blue pixels represent the attached shadow). (a) Original image. (b) The proposed method. (c) The SNP method. (d) The DNM method.

Table 4-2 Computation Time for Shadow Spectral Analysis

Algorithm Proposed SNP DNM

Time (msec) 3.4 93 15.6

Specification:

Image size= 352x240,

CPU type= Intel Pentium 4 2.4GHz, RAM=448MB.

information. The SNP algorithm takes the longest time because of its complex normalization, which consists of square, square root, addition, and division operations. Second, we examine the shadow detection rate and the object detection rate of each algorithm. The ground truth for each frame is necessary for calculating the quantitative metrics of (4.3) and (4.4). We manually and accurately classified the pixels into foreground, background, and shadow

58

regions in the image sequences. Figures 4-5 and 4-6 show the comparison results of shadow detection rate and object detection rate between the proposed method, SNP, and DNM. The mean values of accuracy corresponding to the plots in Figs. 4-5 and 4-6 are reported in Table 4-3. The results evaluated by Prati et al. [36] (by analyzing tens of frames for each video sequence representative of different situations) are also listed in Table 4-3. The experimental results demonstrate that the proposed method generally provides more reliable object detection results compared with other state-of-the-art methods. The results obtained from algorithms with or without spatial analysis are also listed in the table for checking the effectiveness of spatial analysis. One observes that as expected, spatial analysis improves the performance of shadow detection. For instance, the shadow detection rate of the DNM algorithm is increased from 57% to 80%. The merit of spatial analysis is that it can combine with other existing shadow suppression methods for further improvement of performance.

The proposed method outperforms the other two methods in shadow suppression and moving object detection because it uses the ratio model, which is constructed from only a single image frame. The video clip of the experimental results can be found at:

http://isci.cn.nctu.edu.tw/video/JCTai/shadow1.wmv.

Fig. 4-5. Comparison result of shadow detection rate between the proposed method, DNM, and SNP.

59

Fig. 4-6. Comparison result of object detection rate between the proposed method, DNM, and SNP.

Table 4-3 The Accuracy of Detection Results.

Method Shadow

detection rate^*1 Object detection

rate^*1 Shadow

detection rate^*2 Object detection rate^*2 Proposed &Spa. 77.5% 72.2% 76.86% 80.52%

Proposed 62.9% 60.7% 71.97% 70.14%

SNP 72.6% 49.9% 74.48%

(81.59%^*3)

59.39%

(63.76^*4)

DNM 66.4% 56.07% 67.94%

(69.72%^*3)

68.57%, (76.93%^*4)

*1: The mean accuracy of detection results obtained by analyzing 300 frames for each video sequence.

*2: The mean accuracy of detection results obtained by analyzing tens of frames for each video sequence representative of different situations.

*3: Results from [36].

*4: Results from [36], the false positives belonging to the background were not considered in the computation of the object detection rate.

4.4 Summary

In this chapter, a shadow-region-based statistical nonparametric method has been developed to construct a ratio model for shadow detection of all pixels in an image frame.

60

Based on the Lambertian assumption, RGB ratios between lit pixels and shadowed pixels can be treated as a constant in image sequences. This assumption leads us to the development of a novel ratio model for detecting shadow pixels in traffic imagery. The proposed approach does not require many image sequences to construct the model. Instead, the model can be easily built using a shadow region in a single image frame. To increase the accuracy of shadow detection, two types of spatial analysis are proposed to verify the actual shadow pixels.

61

Chapter 5

Vehicle Detection and Tracking for Traffic Parameter Estimation

5.1 Introduction

Recently, image tracking has become an important technique for traffic monitoring and surveillance applications. Many algorithms based on image tracking have been developed for real-time traffic parameter estimation. After the moving vehicles have been segmented, the process of traffic parameter estimation consists of three main stages: vehicle detection, vehicle tracking, and traffic parameter estimation. Most existing algorithms used a special-design region lying in each lane to detect and track entering vehicles. The main drawback of these methods is that only similarly sized vehicles passing through a particular region of the road can be detected and tracked. If the size of a vehicle differs from the predefined one or if the vehicle does not pass through the particular region, it will not be detected and tracked by these existing algorithms.

In this chapter, we propose an image-based traffic monitoring system that automatically detects and tracks multiple different-sized vehicles that travel in any portion of a multi-lane road. We adopted active contour models to represent vehicle contours in an image frame. A method based on image measurement is developed to predict initial positions and sizes of vehicles for image contour generation. This method features simultaneous detection of

在文檔中 交通參數估測系統之攝影機參數校正與影像追蹤 (頁 59-0)