Details of design

In this section, we introduce the design procedure of the proposed two-mirror omni-camera in detail. An illustration of its structure is shown in Figure 2.2. The upper mirror is bigger, named Mirror 1, and the lower is smaller, named Mirror 2. They are both made to be of the hyperbolic shape. To construct the desired two-mirror omni-camera, we took two major steps as described in the following.

Step 1: Decision of the size and the position of the mirrors.

To construct the desired two-mirror omni-camera, the first step is to decide the light injection and reflection positions on the two mirrors in the imaging process. An assumption is made in advance, that is, the radius which is calculated as the distance from "the spot where the light rays are reflected from the Mirror 2" to O_ci is half of the radius which is calculated as the distance from "the spot where the light rays are reflected from the Mirror 1" to O_ci.

Based on the assumption mentioned above, a simulated omni-image is shown in Figure 3.5, where the red part is the region which is reflected by Mirror 2, and the blue part is the region which is reflected by Mirror 1. The radius of the blue region is two times to the radius of the red region.

Before deriving the formulas, we define some terms or notations appearing in Figure 3.6 which is simplified from Figure 2.2 as follows:

(1) x₁ is the radius of Mirror 1;

(2) x2 is the radius of Mirror 2;

(3) d₁ and d₂ are the distance from Mirror 1 to the lens center, O_c and the distance from Mirror 2 to Oc, respectively;

(4) f is the focal length of the perspective camera;

(5) w is the width of the CMOS sensor;

(6) b₁, c₁ are the parameters of Mirror 1; and (7) b2, c2 are the parameters of Mirror 2.

Figure 3.5 The desired omni-image.

In Figure 3.6, the blue lines are drawn to represent the light rays which go into the center of Mirror 1 with an incidence angle of 0^o and are reflected onto the image plane (the CMOS sensor). The red lines are similarly interpreted which go into the center of Mirror 2. Then, focusing on the marked area in Figure 3.6, we get two pairs of similar triangles, (△AO₁O_c, △DO_ciO_c) and (△BO₂O_c, △CO_ciO_c). By the property of similar triangles, we can describe the relationship between them by the following equations:

ci

Figure 3.6 A simplified modal from Figure 3.4.

Step 2: Calculation of the parameters of the mirrors.

The second step is to calculate the parameters of the hyperbolic shape of the mirrors. As shown in Figure 3.4, the relation between the CCS and the ICS of an omni-camera system, as derived by Wu and Tsai [7], may be described by

2 2

where r  u²  v² in the ICS, α is the elevation angle of the mirror, and β is the depression angle of O_c.

Let the omni-camera have the largest FOV, and the incidence angle be 0^o, which specifying a light ray going from a point P(X, Y, Z) into the mirror center and is reflected to the image plane. As illustrated in Figure 3.7 which is a simplified version of Figure 3.4, from △O1ROc we can obtain the following equations: length of the mirrors. In addition, we can obtain the following equation:

2 2

based on Equation (3.6) with 0.

Equation (3.11) may be transformed, by multiplying the denominator by (b²－c²)cosθ, to be:

(b²＋c²)sinθ－2bc＝0, (3.12)

or equivalently, to be

b²sinθ－2cb＋c²sinθ＝0 (3.13)

from which we can get the parameter b by the following equations:

2 2 2

Combining Equation (3.10) and Equation (3.14), we get

2 2

which can be simplified in the following way:

2 2

Because d is the distance from a focus pointof the hyperbolic shape (the mirror center) to the other focus point O_c (the lens center) as shown in Figure 3.6, we can obtain the parameter cof the mirror by

c＝0.5 × d. (3.17)

By equations (3.9) and (3.17), we can solve Equation (3.16) to get

1

Now we can use the general analytic equations, Equations (3.17) and (3.18), to compute the parameters of the mirrors as long as we have the values of x and d.

To calculate the parameters of Mirror 2, we have to make a second assumption, that is, the radius x₂ is a pre-selected value. Accordingly, we can obtain the distance d₂ in Equations (3.5), and then use Equations (3.17) and (3.18) to calculate the parameters b₂ and c₂. And the parameter a₂ may be computed by Equation (3.2).

Finally, we want to get the parameters of Mirror 1. It is mentioned in [12] that a longer baseline (shown as O O_{1 2} in Figure 3.6, whose width is equal to d1  d2) of the two mirrors will yield a larger disparity (the difference between the elevateon angles) of the two mirrors in the image. If the baseline is too smaller, the range data will not be accurate. Therefore, we assume the baseline to be of an appropriate length, denoted say as lb as shown in Table 3.1, so that we can get d1 ＝ d2 ＋ lb and so the value of x1 by Equation (3.4). Then, we can get the parameters b₁ and c₁ of Mirror 1 by using the general equations of (3.17) and (3.18), and the parameter a1 may be computed accordingly by Equation (3.2).

The calculated parameters of the proposed two-mirror omni-camera are shown in Table 3.1 and some simulation images of the camera are shown in Figure 3.8. We use Equation (3.3) to create the simulation images with a focal length of 6 mm and the parameter values listed in Table 3.1. As illustrated in Figure 3.8, the top-left image is the perspective image; the top-right image and the bottom-right image are the simulation images taken by Mirror 1 and Mirror 2, respectively; and the bottom-left image is the simulation image which is composed of the top-right and the bottom-right

one.

Table 3.1 The parameters of the proposed two-mirror omni-camera obtained from the design process.

a b Baseline l_b

Mirror 1 11.46cm 9.68cm

Mirror 2 2.41cm 4.38cm 20cm

Figure 3.7 The light ray is reflected from the biggest radius from the mirror.

3.2.3 3D data acquisition

In this section, we will introduce the principle of the computation of the 3D range data and the derivation of it in the proposed system. In Figure 3.9(a), the point P in the CCS goes through each center of the two mirrors, and α₁ and α₂ are the elevation angles in each mirror, respectively. A triangle △O1O2P formed by two light rays (the blue line and the red line) is shown in Figure 3.9(b). The distance d is between the point P and O1, and the baseline length of the two mirrors is the distance h between O₁ and O₂. We can derive the following equations by the law of sines based on the geometry shown in Figure 3.9(b):

2 1 2

Figure 3.9 Computation of 3D range data using the two-mirror omni-camera. (a) The ray tracing of a point P in the imaging device. (b) A triangle in detail (part of (a)).

Based on Figure 3.9, a front-facing placement of the two-mirror omni-camera values of (X, Y, Z) by the following equations:

X ＝ d × cosα1 × sinθ, Y ＝ d × cosα₁ × cosθ,

Z ＝ d × sinα1. (3.15)

In the proposed system, the optical axis instead is not arranged to be parallel to the ground in order to expand the overlapping area of the both mirrors, and the reason is mentioned in detail in Chapter 2. Figure 3.11 is a side view of Figure 3.10 and is shown to illustrate the slanted-up placement of the two-mirror omni-camera in the proposed system. Specifically, let the optical axis of the camera of the proposed camera system be elevated by the angle of γ, and the position of P(X, Y, Z) be translated to P′(X′, Y′, Z′) in the new coordinate system. Then, according to the trigonometry principle, we can use a rotation matrix R to represent the relationship between P and P′ as follows for computing the new coordinates (X′, Y′, Z′):



As a summary, we have introduced in this chapter the design principle of the proposed two-mirror omni-camera and derived some equations we used for the related imaging processing and 3D data acquisition processes. We will describe the proposed method to calibrate the designed two-mirror omni-camera in the next chapter.

O₁

Figure 3.10 The frontal placement of the two-mirror omni-camera.

Figure 3.11 A side view of Figure 3.8.

Chapter 4

Calibrations of Vehicle Odometer and Proposed Two-mirror

Omni-camera 4.1 Introduction

In this chapter, we describe how we calibrate the two major devices in the proposed system, namely, (1) the odometer which is used to record the position and the direction of the autonomous vehicle, and (2) the proposed two-mirror omni-camera which is used to “see” the situation in environments. We will show the calibration method to correct the vehicle odometer in Section 4.2 and the calibration of the proposed two-mirror omni-camera in Section 4.3.

4.2 Calibration of Vehicle Odometer

4.2.1 Idea of proposed odometer calibration method

Before using any machine, a calibration process for it is usually necessary. When the autonomous vehicle is moving, the information about its direction and position is provided by the odometer in the vehicle. In a short experiment which we conducted for calibration of the odometer, we controlled the autonomous vehicle manually, and recorded the real position reached by the vehicle as well as the odometer readings at the reached spot. We found that the odometer values are not the same as the real position, and that the mechanical error increases as the vehicle’s move distance increases.

4.2.2 Problem definition

In this study, we have designed a careful experiment to test the mechanical error.

We used the tiles on the floor in our laboratory to assist locating the autonomous vehicle in the experiment. Figure 4.1 shows the floor of the laboratory and each tile on it has a square shape with size 30.5cm×30.5cm. The equipments we used in this experiment are sticky tapes, a measuring tape, two laser levelers, and the autonomous vehicle.

The steps of the experiment are described as follows and shown in Figure 4.2.

1. Paste sticky tapes forming a shape of L on the floor by extending the straight lines between several tiles.

2. Use the laser levelers to adjust the heading of the autonomous vehicle to be parallel to a line of the L-shape and let the vehicle lie on the L-shape.

3. Command the vehicle to move forward on the L-shape with a distance of N tiles, where N ＝ 3, 6, 9, 12.

4. Stop the vehicle after it moves for a distance of N tiles.

We found the position that the vehicle arrived at is no more on the L-shape and that the moving path is deviated to the left. We then measured the amount of deviation, fit the resulting deviations of several vehicle movements as a curve of the second order, and use the curve to calibrate the odometer reading of the vehicle in subsequent navigations. More details will be described subsequently. The curve fitting technique we use is described in the next section.

4.2.3 Proposed curve fitting for mechanical error correction

(a) (b)

Figure 4.1 The floor of Computer Vision Laboratory in National Chiao Tung University. (a) Using the lines on the floor of the laboratory to assist the experiment. (b) Pasting the tape to form an L-shape on the floor.

Figure 4.2 An illustration of the experiment.

The curve fitting scheme we used is described in the following.

1. Generate a curve of a kth-degree polynomial function:

y = a ＋ a x ＋… ＋ ax^k, where i = 1, 2, …, n. (4.1)

2. Use the least-square-error technique to fit the measured deviation data (x₁, y₁),

3. To get the coefficients of Equation (4.1), set the partial derivatives of A with respect to a0 through ak equal to zero to get

4. Transform Equation (4.1) into a matrix form XA = Y as follows:

0 1

5. Multiply both sides of (4.5) above by a transpose of X to obtain the following equation:

1

which is a the matrix form of Equation (4.4).

6. Rewrite Equations (4.5) through (4.7) in the following ways:

XA = Y, (4.8)

X^TXA = X^TY, (4.9)

leading to the following solution for the problem:

A ＝ (X ^TX)^-1X^TY. (4.10)

4.2.4 Proposed calibration method

To obtain the data of deviation distances, we conducted the steps of the experiment mentioned in Section 4.2.2 and used the curve fitting technique to build a calibration model of the odometer described by Equation (4.1) in which the coefficients a0 through ak may be obtained by Equation (4.10). A list of the deviation data of our experiment are shown in Table 4.1 and the curve fitting result described by Equation (4.1) through Equation (4.10) is shown in Figure 4.3. The final calibration model is y = 0.00004x² + 0.00096x  0.4.

The overall odometer calibration process is described as an algorithm in the following.

Algorithm 4.1. Building a calibration model of the odometer.

Input: An incremental number N = 1.

Output: A calibration model of the odometer described by Equation (4.1) through (4.10).

Steps:

Step 1. Paste sticky tapes to form an L-shape on the ground by extending the straight lines between the ground tiles.

Step 2. Use the laser levelers to adjust the heading of the autonomous vehicle to be parallel to a line of the L-shape and let the vehicle lie on the shape.

Step 3. Command the vehicle to move forward on the L-shape with a distance of N tiles, stop the vehicle after it moves for a distance of N tiles, and measure the deviation distance between the vehicle and the L-shape, for all N = 1, 2, …, 17.

Step 4. Repeat Step3 for three times.

Step 5. Use the curve fitting technique to build a calibration model of the odometer.

Figure 4.3 The calibration model of the odometer built by the curve fitting technique.

4.3 Calibration of Designed Two-mirror

Omni-camera

4.3.1 Problem definition

The distance information is important for the autonomous vehicle while the vehicle is navigating in the environment. In the proposed method, the autonomous vehicle can detect the guide line and navigate along it. The autonomous vehicle can avoid the obstacles in the environment. As we know that the images that the camera captured are projected from the 3D world coordinate system or Global coordinate system (GCS) onto the 2D image coordinate system (ICS), it is necessary to conduct a calibration work for the camera to establish a relationship between the 3D world coordinates and the 2D image coordinates.

Table 4.1 The data of the experiment of building the calibration model of the odometer.

Move Distance (cm) Distance of Deviation (cm)

4.3.2 Idea of proposed calibration method

As mentioned in Figure 3.4, the light ray of a point P(X, Y, Z) in the CCS first goes onto the mirror center O_m, then is reflected to the lens center O_c by the hyperbolic-shaped mirror surface, and finally is projected on the image plane to form the image point q(u, v) in the ICS. The elevation angle  shown in Figure 3.4 is formed

by the incoming light ray which goes from P to O_m. The idea here is to design a model to describe the relationship between the elevation angle and the point q in the ICS. The proposed calibration method is described in detail in the next section.

4.3.3 Proposed calibration process

In [7], Wu and Tsai mentioned two misalignments in an omni-camera system. The first is the so-called axial-directional misalignment and the other is the so-called lateral-directional misalignment. In an axial-directional misalignment, the optical axis of the camera does not go through the mirror center; instead, it is parallel to the mirror axis. The omni-image center is shifted a distance from the lens center in this phenomenon. In a lateral-directional misalignment, the mirror axis is not parallel to the optical axis of the camera. This misalignment of the mirror axis not only destructs the SVP property but also the rotational invariance property in omni-images.

A prototype of the proposed two-mirror omni-camera is shown in Figure 4.4 which is a modified version of Figure 3.4. To avoid the two misalignment problems mentioned previously, two properties are necessary to be satisfied: (1) the optical axis of the camera is maintained to be perpendicular to the two mirrors; and (2) the optical axis of the camera is aligned with the mirror axis O O_{1 2}. There are three purposes we have to achieve to calibrate the proposed two-mirror omni-camera as follows.

Purpose 1 － making the optical axis of the camera perpendicular to the two mirror bases with the lens center and the mirror center both aligned with the vertical axis.

Some equipments as shown in Figure 4.5 are prepared in advance for works to achieve the goal of “making the optical axis of the camera perpendicular to the two mirror bases with the lens center and the mirror center both aligned with the vertical

axis,” including:

Figure 4.4 An illustration of the proposed two-mirror omni-camera.

(1) a horizontal leveler; (2) a planar mirror M; (3) a blueprint of Mirror 1, C1; (4) a transparent plastic enclosure; and (5) a perspective camera. Mirror 1 was produced according to the design shown by the blueprint of Figure 4.5(b). The red circles on the outer ring of the base of the enclosure in the figure are fixed points, and the enclosure is affixed with steel nails punched through these fixed points. The enclosure is used to cover the mirrors in the camera system to prevent rain and unexpected touch from the environment.

Referring to Figure 4.5, we define several notations in advance. The black circle C2 in Figure 4.5(b) is the image of the camera lens reflected by the planar mirror M. CC1

is the center of C₁ which is the crossing point by the two dotted lines in Figure 4.5, and CC2 is the center of C2 which is detected by a binary thresholding technique.

1. Use the horizontal leveler to make sure that the top cover of the enclosure is horizontal to the base of the bigger mirror;

2. Embed the perspective camera in the top cover of the enclosure and take images, while the camera is not fixed in it yet (see the illustration shown in Figure 4.4);

3. Overlap CC₂ on CC₁ by moving the perspective camera to make O O_{1 c} perpendicular to the base of Mirror 1 in Figure 4.4.

4. Fix the perspective camera now.

Purpose 2 － making the smaller mirror base parallel to that of the bigger one and their axes aligned.

In the last step, we make the perspective camera parallel to Mirror 1, the optical axis of the camera perpendicular to Mirror 1, and the lens center and the center of Mirror 1 to align with the vertical axis, as shown in Figure 4.5(a). In this step, we want to make the base of Mirror 2 parallel to that of Mirror 1. If this can be achieved, there is no alignment problem about the axes of the two cameras and Mirror 1, Mirror 2 and the lens are parallel to each other.

We use the rotational invariance property of the omni-image in this step and illustrate it with Figure 4.6. The property says: if the two mirrors, Mirror 1 and Mirror 2, are parallel, two image points q1 and q2 in the ICS, which are projected by both mirrors from a single scene point P in the GCS are on an identical radial line in the image (the blue line and the red line in the image plane shown in Figure 4.6).

Based on this property, we attached several pieces of paper strips on the enclosure to test if the base of Mirror 2 parallel to that of Mirror 1. If not, the edge lines of these paper strips will not be in the same radial directions (i.e., will not align respectively) in the omni-image, as shown in Figure 4.7(a). The contrary case with the edge lines

aligned is shown in Figure 4.7(b).

(a)

C

₁

M

C2

CC1 CC2

The blueprint of Mirror 1 The fixed points

(b)

Figure 4.5 The first step of the proposed calibration method. (a) Make sure the enclosure is parallel to the Mirror 1. (b) Make sure the lens center is match on the mirror center.

Figure 4.6 The property, rotation invariance of the omni-camera.

Purpose 3 － Recording the relationship between the elevation angle and the image coordinates.

An idea of a mapping-based approach proposed by Jeng and Tsai [22] is adopted for the purpose of establishing the relationship between the elevation angle and the image coordinates for use in this study. This idea does not conduct camera calibration to estimate camera intrinsic and extrinsic parameters, but creates a so-called

An idea of a mapping-based approach proposed by Jeng and Tsai [22] is adopted for the purpose of establishing the relationship between the elevation angle and the image coordinates for use in this study. This idea does not conduct camera calibration to estimate camera intrinsic and extrinsic parameters, but creates a so-called

在文檔中 利用雙鏡面環場影像攝影和超音波感測技術作戶外自動車學習與導航之研究 (頁 42-0)