Irregular Condition Detection

iMin = p q i

R Min R p q , the shift vector v that produces the minimum _i correlation value for region i represents the local motion vector of this region, i.e.,

( , ), for ( , )

i i iMin

v = p q R p q =R . (2.7)

2.2.3. Irregular Condition Detection

By analyzing the curves of correlation values corresponding to image sequences with various conditions, it is found that the curve of correlation values is related to the reliability of motion detection. Figs. 2.4 and 2.5 show the various correlation curves corresponding to different video images with different conditions. Fig. 2.4 is a video image captured by a hand-held camcorder of a daily life scene. Fig. 2.5 is a video image captured by an in-car video camera of an outside scene. It is found that the irregular conditions detected in daily life scenes can be detected in in-car applications as well. Fig. 2.4 (a) and 2.5 (a) show a normal condition where the peak of correlation is obvious in each region. In Fig. 2.4 (b) and 2.5 (b), the curves look like a valley. This means only one dimension of correlation data is reliable and it lacks of feature in the other direction. Fig 2.4 (b) is short of features in the x direction.

On the contrary, Fig 2.5 (b) is short of features in the y direction. Fig. 2.4 (c) and Fig. 2.5 (c) show examples with repeated patterns. Fig. 2.4 (c) is an image of an office partition filled with grid-hole patterns, especially on region 2 and 4, which causes multiple peaks on these regions. Fig. 2.5 (c) is an image of a brick wall with a fence in the bottom area, and it also causes multiple peaks in the correlation curves, especially within region 1 due to the pure bricks that are repeated in this area. Fig. 2.4 (d) represents the condition of a moving object; a bear moves from the left side to the right side of the image sequence. It causes double peaks in the curve of region 2 and the value of R_iMin is larger than the value of a normal image, such as Fig. 2.4 (a). Fig. 2.5 (d) represents an image sequence of a motorcycle moving from the right side to the left. It causes double peaks within region 1 of the curve and the value of RiMin is larger than that of a normal image as well. The example shown in Fig. 2.4 (e)

contains a white wall on the right side of the image. Obviously, it is very hard to distinguish the peak from the correlation curve in region 4. Fig. 2.5 (e) contains a large low-contrast area (blue sky) on the upper left corner of the image. It is also hard to distinguish the peak in region 1.

Although the curve of correlation values is related to the reliability of motion detection, it is still too complex to directly use these curves to evaluate the reliability of motion detection. Therefore, we propose a strategy that combines the minimum projections of correlation curve in x and y directions (minimum projections) and the inverse triangle method to detect the irregular conditions from each region. The mathematical expression of minimum projections can be written as:

_ min( ) min ( , ) in x and y directions in region i , respectively. Fig. 2.6 shows the examples of minimum projections of the correlation curve in x and y directions. Fig. 2.6(a) is the minimum projection of Fig. 2.5(a), that is regular and the determination of motion vector in each region is clear and consistent. Fig. 2.6(b) is the minimum projection of Fig. 2.5(b), which shows a lack of features in y direction (horizontal). The values of the minimum projection of correlation curve in y direction are within a small range and erratic with multiple peaks so that the determination of the minimum value is very hard.

In order to determine the reliability of motion vector easily, the feature extraction of reliability is performed by the proposed inverse triangle method through the minimum projections in x and y directions to obtain the reliability indices. Fig. 2.7 shows the illustration of the inverse triangle method. In the first step, we find T_i_ min that represents the global minimum of the minimum projection curve in region i and can be calculated by Eq. (2.9).

In the second step, we calculate S and _xi S by Eq. (2.10), where _yi offset is the altitude of the inverse triangle. n and _xi n are defined as the numbers of _yi S and _xi S , _yi respectively (see Eq. (2.11)). d and _xi d are defined as the distances of two vertexes of the _yi base of the inverse triangle obtained by Eq. (2.12). The cost level of x and y directions are calculated by Eq. (2.13). The higher cost level means a lower confidence level. Since the condition of multiple peaks seriously degrades and affects the determination of reliability, the

reliability. The example shown in Fig. 2.7 is a curve with twin peaks which will get the penalty of d_xi −n_xi. In the third step, we determine the confidence indices of x and _i y in _i region i through a threshold denoted as TH . The lower cost level represents a higher reliability. In the final step, summing up the counts of reliable motion components of x and y in the four regions as Eq. (10), we get Num x and ( )_i Num y( ), 1 ~ 4 _i i= .

Step 3. Set the threshold,TH , for determining the reliability indices

If _ cost x_i < TH Then (2.14)

End if

Step 4. Calculate the numbers ofx and_i y in four regions _i ( ) sum of ( is reliable)

( ) sum of (y is reliable)

i i

i i

Num x x

Num y

⎧ =

⎨ =

⎩ (2.15)

1 ~ 4 i=

(a)

(b)

(c)

(d)

(e)

Fig.2.4. Various correlation curves that correspond to image sequences with different conditions.(I) (a) A normal condition (b) Features lacking in the vertical direction (book) (c) Repeated patterns (office) (d) Moving object (bear) (e) Large low-contrast area (white wall) Video images captured by a camcorder in daily life scene.

(a)

(b)

(c)

(d)

(e)

Fig.2.5. Various correlation curves that correspond to image sequences with different conditions (III). (a) A normal condition (b) Features lacking in the horizontal direction (gate) (c) Repeated patterns (brick) (e) Moving object (motorcycle) (f) Large low-contrast area (sky) Video images captured by an in-car video camera during outside scene.

-50 0 50

Fig. 2.6. Examples of minimum projections of correlation curve from x and y directions in four regions. (a) Regular image sequence. (b) Ill-conditioned image sequence.

dxi

nxi

offset _ min Ti

Fig. 2.7. Illustration of the proposed inverse triangle method