Vehicle Turning Decision

C. Estimation of motion vectors by optical flows

The optical flow analysis method may be used to estimate the motion vectors of objects, surfaces, and edges caused by the relative motions between two consecutive omni-images. Assume that the displacements of concerned objects in the image are small and the illumination is stable. Under such conditions, the motion vectors between two consecutive image frames which are taken at times t and t + dt can be estimated by the optical flow analysis method in the following way. If the image

intensity is continuous and can be differentiated, the image intensity at time instant t is constrained by

I(x, y, t) = I(x + dx, y + dy, t + dt), (6.1)

where the function I is the image intensity, x and y specify the location of the point in the image, and t is the sampling time.The image constraint at I(x + dx, y + dy, t + dt) in Equation (6.1) can be expressed as a truncated Taylor series in the following way:

( , , ) ( , , ) I I I

By Equations (6.1) and (6.2), it follows that:

Ix(p)Vx + Iy(p)Vy = −It(p) (6.3) uniquely using the data of the single point p. However, the Lucas-Kanade method [19]

may be adopted here to solve the problem, which divides an image into small regions and assumes that the displacements of the image content within a small neighborhood of the concerned point p are small and approximately constant. Accordingly, we may set a window around point p with n pixels, p1, p2, ..., pn inside the window. Then, the local image motion vector (V_x, V_y) at p with image coordinates (x, y) must satisfy the following equations according to Eq. (6.3):

1 1 1

Eqs. (6.4) can be expressed in a matrix form:

Accordingly, the motion vectors of consecutively acquired images using Eq. (6.6) can be estimated above.

D. Transformation of motion vectors

To estimate the moving direction of the video surveillance vehicle, the motion vectors produced by the optical flow analysis method are used. However, the images captured with the omni-cameras are distorted. As a result, before computing the direction angle of these motion vectors, the transformation of the motion vectors from the omni-image plane to the world coordinate system as shown in Fig. 6.1(b) is necessary. In Section 4.4.1, we discussed how we transform the omni-image coordinates into the world coordinates. The configuration of such a transformation of the motion vector of a real-world point on the ground is shown in Fig. 6.1(a). Assume that we transform the image coordinates into the world coordinates (X, Y), and transform the next image coordinates into the world coordinates (X′, Y′). Then, the directional angle A_i of the motion vector V_i with respect to the X-axis can be computed by

1

where the value i ranges from 0 to the number of motion vectors.

By Equation (6.7), all the motion vectors produced by optical flow analysis can be transformed into the WCS and their directional angles can be computed for analyzing the moving direction of the video surveillance vehicle as discussed next.

β illustration of the camera system and the motion vector. (b) The ray tracing of a scene point P on the ground projected on the hyperbolical-shaped mirror.

E. Elimination of Outliers

When we drive the video surveillance vehicle, the omni-cameras will often shake due to rough road conditions. Therefore, the shake of the omni-cameras might cause creations of short-length noise motion vectors. In order to eliminate such noise to increase the accuracy of the vehicle moving direction estimation result, only motion vectors with lengths be larger than a pre-selected threshold value TH are selected for vehicle direction estimation.

To eliminate the outliers of motion vectors resulting from noise, each directional angle of the remaining motion vectors is regarded as a feature and the standard deviation value is computed accordingly, as shown in Fig. 4.5. More specifically, let the angles of these motion vectors be denoted as Ai, and let the total number of motion vectors be denoted as n. Then, the mean value A of these motion vectors may be vector data can be calculated as follows:

2 TH], an outlier will be checked and discarded. After all the outliers are eliminated, we compute the mean value of the remaining data as the desired directional angle of the moving direction of the video surveillance vehicle.

F. Estimation of the Moving Direction of the Vehicle

The moving directions of the video surveillance vehicle may be categorized into three classes  turning to the right, turning to the left, and moving forward. And the ranges of the directional angles of the three classes are determined by our experimental experiences. They are listed in Table 6.1, which may be used to classify the results of the directional angles derived by Eq. (6.8) into the three vehicle moving directions.

Figure 6.2 A distribution chart of the direction angles of motion vectors.

Table 6.1 The range of the angles of the three vehicle moving directions.

State Degree

Moving forward 261°~ 279°

Turn to the left 180°~260°

Turn to the right 280°~ 360°

Besides, a finite state machine (FSM) can be used to determine the moving direction of the video surveillance vehicle [17]. The finite state machine adopted for use in this study is composed of six states, which can be categorized into three classes:

(1) turning to the right; (2) turn to the left; and (3) moving forward. As illustrated in Figure 6.3, the input to the FSM is taken to be “1”; else, to be “0.” Here, the input “1”

is used to specify the same turning direction at the current state, and the input “0” to specify a different turning direction at the current state.

Figure 6.3 A graph of finite state machine proposed to determine the moving direction [17].

6.3 Organization of Guidance Map for