Review of Adopted Pano-mapping Method for Omni-image and

In order to calibrate the omni-camera, the pano-mapping method proposed by Jeng and Tsai [5] is adopted in this study. First of all, we record the relationship between the pixel in the omni-image and the elevation and azimuth angles of the corresponding world-space point with respect to the focal center of the mirror. The coordinates of the world-space points in the omni-images, called landmark points, are measured manually with respect to a selected origin in the world space. To facilitate selecting the landmark point pairs, a user interface is provided, as shown in Fig. 4.12.

And the more landmark points are selected, the more accurate the table is.

(a) (b)

(c) (d)

Figure 4.11 Detection of the landmark by ellipse shape fitting. (a) The image before region growing. (b) The image after region growing (c) Generating an ellipse for every region. (d) Deciding the best-fit ellipse shape, where the blue shape is the best-fit ellipse, and the green shape is an erroneous ellipse.

Figure 4.12 interface to for user to select the landmark points.

Due to the nonlinear shape of the hyperbolical-shaped mirror, the radial-directional mapping must be represented as a non-linear function fr, called radial stretching function. As shown in Figure 4.13, each elevation angle corresponds to a radial distance. In this study, fr is approximated by the following 5th-degree polynomial equation:

1 2 3 4 5

0 1 2 3 4 5

r( )

r = f ρ =a +a ×ρ +a ×ρ +a ×ρ +a ×ρ +a ×ρ . (4.25) More specifically, each elevation angle ρ of a scene spot P at world coordinates (X, Y, Z) corresponds to the radius r of the corresponding point p in the omni-image.

Moreover, Jeng and Tsai [5] proposed an algorithm to compute the desired coefficients a₀ through a₅,by which we can find out the relationship between the radius r and the elevation angle ρ by the use of a non-linear function f_r. However, the mirror is not perfect with rotational symmetry in the entire angle range from 0^o to 360^o. So we divided the 360^o range of azimuth angles of the mirror equally into six parts, each with 60^o, and then applied the above process to obtain six radial stretching functions fr1 through fr6 for the six parts with each fri described by the coefficients a0i

through a_5i with i = 1, 2, …, 6.

Figure 4.13 Mapping between a radius distance r and elevation angle ρ.

The procedure of constructing the pano-mapping table with the radial stretching function for each of the six azimuth angle ranges is described here. The pano-mapping table is a 2-dimensional table with its horizontal and vertical axes specifying the azimuth angle θ and the elevation angle ρ, respectively. An illustration of the pano-mapping table is shown in Fig. 4.14, and an example of the pano-mapping table is shown in Table 4.2.

Each entry E_ij with indices (i, j) in the pano-mapping table corresponds to an azimuth-elevation angle pair (θi, ρj). The azimuth-elevation pair represents an infinite set of points on a light ray with the azimuth angle θi and the elevation angle ρj with respect to the focal center in the WCS. We divide the range 2π of the azimuth angles into M intervals and the range of the elevation angles between two pre-selected limits, ρs and ρe, into N intervals. Due to the property of rotational invariance of omni-imaging, the azimuth angle φ of the scene point P in the WCS with respect to the X-axis is identical to the azimuth angle θ of the corresponding point p in the image with respect to the u-axis. Thai is, we have θ = φ.

(a) (b)

Figure 4.14 Illustration of mapping between the azimuth-elevation angle pair of the omni-image and the horizontal and vertical axes of the pano-mapping table, respectively.

Finally, the six sets of coefficients can be estimated for the six radial stretching functions, and the corresponding pano-mapping table can be filled with the corresponding image coordinates.

Table 4.2 An example of the pano-mapping table.

θ1 θ2 θ3 θ4 … θM

After creating the pano-mapping table, we can now describe the adopted method to compute 3D information from a two-camera omni-directional imaging device. As shown in Figure 4.15(a), the point P projected on each hyperbolical-shaped mirror forms a pair of corresponding points in the upper omni-image and the lower omni-image captured with a two-camera omni-imaging device. The elevation angles of point P on the hyperbolical-shaped mirrors are defined as α1 and α2, respectively.

Also, the center of the upper hyperbolical-shaped mirror is assumed to be the origin of the world coordinates (0, 0, 0). It is desired now to compute the stereo depth data of

point P in terms of the two elevation angles α₁ and α₂.

To obtain 3D information of a scene point P(x, y, z), finding two elevation angles α₁ and α₂ by looking up a pano-mapping table is required. As shown in Figure 4.15(b), the distance d between the point P and the upper mirror center c1 is computed by the triangulation principle shown in Figure 4.15(a) using the equation below:

2 1 2

where the parameter b is the baseline of the stereo imaging device. The equation of (4.27) may be reduced to be the following equation by trigonometry:

2

Figure 4.15 Computation of 3D information using the two-camera omni-directional imaging device. (a) The ray tracing of a scene point P in the imaging device with a hyperbolical-shaped mirror. (b) A triangle in detail (part of (a)).

As a result, the horizontal distance dw and the vertical distance Z may be computed as follows:

1 image coordinates (u, v) in the image coordinate system (ICS). Then, we can use point I to calculate the azimuth angle θ. A triangulation which is illustrated in Figure 4.16, includes an azimuth angle θ between the X-axis and point I. As a result, the azimuth angle θ can be computed by the following equation:

1 1 property of omni-imaging, the azimuth angle of a point in the ICS is the same as that of the corresponding point in the WCS. We can calculate the parameters x and y by the distance dw and the azimuth angle θ in the WCS as follows:

1 2

Figure 4.16 System configuration of upper omni-camera with a hyperbolical-shaped mirror.