Background Extractor (BE)

BE is constructed as a single-layer Codebook model in [2, 3]. But, different form [2, 3]

that need a long period to construct the background, we only use one frame to construct the background of BE. The algorithm of BE is designed for color imagery and each pixel of BE is modeled as a single-layer Codebook consist of n codewords. Each codeword is includes two elements, a RGB vector and a six-tuple . Where and are the minimum and maximum brightness of each codeword , respectively, is the occurred frequency, is the longest time interval that the codeword is NOT recurred, and and are the first and last access times, that the codeword has occurred, respectively. The algorithm of BE construction is shown in Algorithm 1. We use the first frame of video sequence to construct the BE. In the construction procedure, each pixel is constructed as a Codebook based on the region that include a set of pixels in a neighborhood around the current pixel, as shown in Fig. 5, where the current pixel is referred to as X. The length of Codebook would be dependent on the pixel location. In

Fig. 5 (a), as the current pixel X is inside the image and the neighborhoods A-H are adopted. Fig. 5 (b)-(e) and (f)-(i) represent the other cases of current pixel X’s position when it is located at the corners or the border of an image. Because we use single frame to construct the BE, the minimum and maximum brightness of each codeword , and , are set to be the same. Because BE only use a single frame to construct model, BE is faster than [2, 3]. The spatial information contributed by neighborhoods of current pixel, which is reflected in the simplified single-layer Codebook model and useful to overcome the problem of dynamic background.

After the construction of BE, we classify the incoming pixel to either background or foreground in the pixel classification stage. The incoming pixel is classified into background if it satisfies two conditions, the color distortion and the range of brightness.

In the condition of color distortion, the matched codeword with RGB vector is Fig. 5 The chart of connected neighborhoods around the current pixel X. (a) the current pixel is inside the image. (b)-(e) the current pixel is at the corner of the image.(f)-(i) the current pixel is at the border of the image.

found from its Codebook base on the following color distortion measure: respectively. If the current pixel simultaneously satisfies the color distortion and brightness, it is classified as background. The current pixel will be updated into BE if it is classified as background in the pixel classification stage. We propose two policy of updating: regular update and two-way propagation, which in order to combine the temporal information and maintaining the spatial consistency for BE construction. The algorithm of the update policies of BE is shown in Algorithm 2.

Information of temporal is added into the regular update policy as that in [2, 3].

The RGB vector is computed by averaging the value of red, green, and blue colors with the current pixel and the parameters in are all updated. In addition, the two-way propagation policy is used to combine the color information around the current pixel means that the current pixel propagates the color information to a neighbor and a neighbor propagates the color information to the current pixel mutually.

In the first steps of the two-way propagation, we randomly choice a neighbor from 8 connected neighborhoods around the current pixel. In order to find the matched codeword from that neighbor, we base on two criteria: the color distortion and the range of brightness that described in previous section. If we found the matched codeword, we update the color information into the matched codeword according to the equation (3).

Otherwise, we create a new codeword add to the chosen neighbor and then we propagate the color information of the current pixel which is classified as background to the codeword by the following setting:

,

(4) Fig. 6 The two-way propagation policy. (a) the first-propagation direction. (b) the second-propagation direction.

On the other step, we randomly choice a codeword from the 8 connected neighborhoods around the current pixel. If there is a matched codeword, we updated the matched codeword base on the equation (3). Otherwise, we create a new codeword for the current pixel and assign the value according to the equation (4). An example of two-way propagation is shown in Fig. 6. Fig. 6 (a) shows the first step which propagates the color information from the current pixel X to a randomly chosen neighbor pixel C, as indicated by the blue line. Fig. 6 (b) shows the second step of the two-way propagation. The regular update policy is useful to maintain BE and capture the foreground accurately. Furthermore, the two-way propagation policy considers the space information which can efficiently solve the problems of dynamic background.

Because endless increase the codewords, makes the size of Codebook became too large and that increased the time of matching. Therefore, we rejected the unsatisfied codewords which have the longest interval according to the following equation:

, (5)

where is the number of frames which is set to be 500 in our method. The filter policy would make the Codebook became empty when the threshold is too small or the codeword is not recurred for a longest interval. Therefore, we proposed the reconstruction policy to solve the problem, if the Codebook became empty, the reconstruction policy is triggered according to the construction of BE that described in Section 3.2.2.