Foreground Extraction

variation of brightness distortion of block in position (i,j) by computing the root mean square of the distortion, and the variation of color distortion of block in position (i,j) is

N

The two variation values will be factors of threshold to decide foregrounds.

To avoid the influences like light and shadow in individual frame, we take multiple background frames and computed mean contrast values of every block in every frame. It may take much time to do the computation, but this can be finished in advance.

All the information will be compared with current frames for foreground extraction and motion object identification later.

2.2 Foreground Extraction

2.2.1 Method 1

We also compute the values of mean, standard deviation and contrast of all blocks in current frames.

Afterwards, we compare the contrasts of blocks in the same position between background model and current frame. If the difference of contrasts between two blocks exceeds the threshold we set previously (the value will be assigned empirically), the block will be called “foreground block”, otherwise it is a background block.

Drawbacks of the above method would produce include the following. When a pure-colored block (ex. all the colors of pixels in the block are white) changes to another pure-colored block, the standard deviations of them are both zero so that both their contrasts are zero. It means that the difference of contrast is zero that the block wouldn’t identify as a foreground block.

2.2.2 Method 2

To compensate the problems, another approach from [9] was applied (Huang and Chen proposed a modified version in [19]). We also modified the method from pixel to block level. In figure 2.1, Ei represents the color vector of ith block in background model, its contents include its mean RGB values. And Ii is the same block but from current frame instead. Next, αEi is the projection of Ii on Ei, we compute the length of difference between Ii and αEi, that is, | Ii-αEi |, this will be an index for us to estimate whether a block is foreground block or not (| Ii-αEi | is called the color distortion of i in [9]). The decision is simple, if the value is greater than threshold, the block will be a foreground block.

If a color vector is part of the other vector or vice versa, it indicates the distance | Ii-αEi | is 0, with above definition the block won’t be a foreground block. For example, v₁ = (50,50,50) and v₂ = (200,200,200). In RGB color space v1 is a color nearly gray, and v2 is nearly white. It reveals a controversy that the two colors are similar or not, and we choose to comply with the original

definition. There are some disadvantages of this mechanism discovered in experiments, we modified the algorithm to reduce failures in foreground extraction.

Fig. 2.1：Illustration of the distance of two color vectors in RGB space. Ii and Ei are the two vectors, αEi the projection of Ii on Ei, |Ii-αEi | is the distance.

We changed the definition of color distortion of a block in current frame from | Ii-αEi | to equation (2.3), and brightness distortion is equation (2.2), in the end we rescale _i,j to be

j

The rule of block classification is

,

After identifying all foreground blocks in current frame, we merge these blocks to “foreground regions”. A foreground region is a connected component of foreground blocks. The definition of connection is that if two foreground blocks are 8-neighbors, they are connected, otherwise they are not. Because of the effect of noise, we set a threshold for foreground regions, if the size of a region didn’t exceed the threshold, the region will be treated as noise instead of foreground, even if exactly the region contains motion objects. To express the foreground region easily, the region will be represented by a rectangle whose length and width are multiples of length and width of a block, respectively. So in a foreground region there are both foreground and background blocks, even the later locate in a region, they won’t be analyzed in following processes.

Figure 2.2 showed the performance of foreground extraction, figure 2.2a is one of background frame, which is similar to background model. We can see some people and cars in figure 2.2b, which represent motion objects. The blocks in figure 2.2c are foreground blocks, they cover these objects and parts of a window of the building in rear of motion objects. We combined these blocks in figure 2.2d, four objects are surrounded by large blocks, but because of the threshold of size, the window in top-right side of the image is ignored.

(a) (b)

(c) (d)

Fig. 2.2：Foreground blocks and regions (a) Background (b) Some motion objects appeared (c) Drawing foreground blocks (d) Foreground regions.

Sudden change of light is another problem of background subtraction, in indoor scene the light source is usually from electronic lights, which can’t always offer rays with stable intensity or sometimes are influenced by objects near them. In outdoor scene, the light source is usually sunlight, whose intensity can be affected by cloud and change with time. In a cloudy day, we can figure out there is not enough light source to take sharp pictures.

Some examples of light switch are displayed in figure 2.3, the sources of images are from PETS2001 dataset 3, we snapshot some frames of the video. Figure 2.3a is the first frame in the video, the background is almost identical to figure 2.2. A apparent light change happened in figure 2.3b, the intensity of light source increased, it’s believed that the reason is moving cloud. In figure

2.3c and figure 2.4d there are also obvious light changes, the former is darker than figure 2.3b and the later is similar to figure 2.3b. It’s interesting that in the video with length about 5 minutes, there are various light switches between frames. Tracking process in the video will be a great challenge duo to the various intensities of light source. With the kind of video we have to build background model carefully to adapt light problems.

(a) (b)

(c) (d)

Fig. 2.3：Changes of light. (a) Frame 0 (b) Frame 885 (c) Frame 3050 (d) Frame 4245.