Feature Extraction

Two kinds of landmark points are then extracted subsequently from the binary dot-matrix pattern image with aids from the original pattern images. The first is the center of each white dot in the image, which we mention as a dot center for simplicity in the sequel. The second kind of extracted landmark point is the intersection point of a screen border line with the line formed by a row or a column of the dot centers in the binary dot-matrix pattern image, which we call an extended dot center in the sequel. For contrast,

Fig. 5.3 A TDM image created by images in Figs. 5.2(a) and (b).

and extended one. These extracted landmark points can be used as anchor points for calibration. They are marked by crosses “+” in Fig. 5.4(f). We also see from the results shown in Fig. 5.4 that the proposed method is quite effective in dealing with non-uniform lighting appearing in the acquired images.

We now describe how we extract these two types of landmark points in more detail.

There are three major steps in the extraction process: (a) extraction of the screen border; (b) extraction of the original dot centers; and (c) extraction of the extended dot centers.

(a) Extraction of screen border

In order to extract the screen border, we establish a geometric model for the border shape and use an edge matching technique to fit the model, thus obtaining a polygon for use as the shape of the screen border. The main purpose is to have a more precise

Fig. 5.4 An example of image analysis of dot-matrix pattern image. (a) White-rectangle pattern image; (b) Black-rectangle pattern image. (c) Dot-matrix pattern image. (d) TDM. (e) Binary image of (c). (f) Extracted feature points (with marks “+”).

(a) (b) (c)

(d) (e) (f)

rectangular screen in an acquired image caused by imperfect optical geometry of the camera lens, which is mostly barrel or pincushion distortion, we model the screen region in the image as an eight-sided polygon with eight vertexes and eight line segments, denoted by Pi and Li, respectively, where i = 0, 1, …, 7, as shown in Fig. 5.5.

We then design a procedure to extract the vertexes from the TDM image precisely.

First, two mutually-perpendicular lines are set at the center of the TDM image, as shown in Fig. 5.6. A 2×6 scan window as shown in Fig. 5.6(b) is moved from the TDM image center along the vertical line to its two ends to extract the vertexes P₁ and P₅ on the upper and lower border lines, respectively. P3 and P7 on the right and left border lines, respectively, are extracted similarly. Extraction of the vertexes is based on the concept of edge detection using the values of the 12 pixels in each scan window.

Specifically, referring to Fig. 5.6(b), let the average grayscale value of the group of the upper six pixels in the window be denoted as G₁, and let that of the lower six pixels as G₂. Then, either of the vertexes of P1 and P5 is detected by searching the maximum of the edge values |G₁ − G2| along the previously-mentioned vertical line. A pixel with the maximum

P

0

P

₁

P

2

P

3

P

4

P

5

P

6

P

7

L

0

L

₁

L

2

L

3

L

₄

L

₅

L

6

L

7

Fig. 5.5 Polygonal model of a screen region.

With P1, P3, P5, and P7 extracted, we can extract the other vertexes and then the polygon sides accordingly. We only describe how we extract P₂ and the polygon sides L₁ and L2 here. The others are extracted in similar ways. Referring to Fig. 5.7, first we define a rectangular search window using the coordinates of vertexes P₁ and P₃, using the coordinates (x1, y1) of P1 and the coordinates (x3, y3) of P3 in the following way. The center

P

a of the rectangular search window is taken to be (x3, y1), and the width and the height of the window are taken to be |x3 − x1|×2 and |y3 − y1|×2, respectively.An illustration of the window formation is shown in Fig. 5.7(a).

Fig. 5.6 Extracting the vertex P

1

from the TDM.

G

₁

G

₂

(a) (b)

P

₁

P

3

P

5

P

₇

P

₁

P

₃

P

₂

' L

1

'

L

2

'

G

1

G

2

E

i = G1 - G2

Best fit

Search window

P

2

segments for L1 and L2 in the search window. For each pixel P2

' in the window, we

construct first two candidate line segments L1

' and L

2

' by connecting P

2

' to P

1 and P3, respectively. We then place a fixed number of 6×2 or 2×6 scan windows (like those mentioned previously for detecting P1 and P3) on the candidate lines L1

' and L

2

' and

compute the edge value |G₁ − G2| for each of the scan windows. See Fig. 5.7(b) for an illustration. All such edge values are summed up to get a border fitting measure E(P2

') for P

₂

', which we call the edge energy of P

₂

'. Then the pixel P

₂

'

_max with the maximum edge energy in the rectangular search window is chosen as the desired vertex P2. And the desired polygon sides L₁ and L₂ are just the line segments connecting P₂

'

_max to P₁ and to P₃, respectively.

(b) Dot center extraction

The TDM and the dot-matrix pattern images are used next to extract the dot centers in the dot-matrix pattern image. The details are described in the following.

(b.1) Labeling connected components in binary dot-matrix pattern image

First, the grayscale image G_g of the dot-matrix pattern (for example, see Fig.5.4(c)) is thresholded into a binary dot-matrix pattern image Gb by Eq. (5.2). Fig. 5.4(e) illustrates an example of the result. Then, a connected-component analysis algorithm is applied to Gb

to extract the connected components in G_b. Some constraints on the component area, the ratio of the component width to the height, etc. are used to filter out unwanted noise. The resulting connected components are white dots, denoted as CCi, i = 0, 1, …, n − 1, in the sequel.

(b.2) Finding a cross-shaped group of five white dots near image center

Then we try to find a group of five white dots which form a five-dot cross shape near

In the same way, the nearest dot obtained by searching downward from C0 is denoted as C4. We also do leftward and rightward searches similarly and the results are denoted as C1 and

C

₂, respectively. See Fig. 5.8 for an illustration.

Next, we want to construct a cross-matrix pattern (CMP) from the binary dot-matrix pattern image G_b. The CMP has N×N crosses, with each cross “+” being located at a dot center. For this purpose, we first constructed a rough CMP with their crosses evenly distributed horizontally and vertically with equal distances. An example constructed from Fig. 5.8 can be seen in Fig. 5.9. The horizontal distance Wg and the vertical one Hg

between every two crosses are computed respectively as the averages of the halves of the width and the height of the five-dot cross shape obtained previously, i.e., W_g and H_g are computed as

W

g = (|C1 − C0| + |C2 − C0|)/2; Hg = (|C3 − C0| + |C4 − C0|)/2, (5.3) where | ⋅ | represents the distance between two white dots.

(b.3) Fine tuning of positions of crosses in rough CMP to obtain desired CMP Fig. 5.8 A 5-dot cross shape near the center of binary dot-matrix pattern image.

C

1

C

0

C

2

C

3

C

4

The crosses in the rough CMP are not located at the real centers of the white dots.

See Fig. 5.9 for this phenomenon. But they can be used as the start positions for finding more accurate dot center locations. For this purpose, the rough CMP is first superimposed on the dot-matrix pattern image G_g with the center of the five-dot cross shape overlapping on the center of Gg, or equivalently, on the center of CC0. Then, we use the deformable template matching (DTM) method [43][44] to do the fine tuning work of finding more accurate dot centers.

A deformable template DT for circular region detection as shown in Fig. 5.10(b) with the parameters of its center (x_i, y_i) and radius r_i is used to detect each white dot in the dot-matrix image Gg. The detection is started from the position of the cross of a white dot

CC

j in the rough CMP. And a search range, denoted as SR, is defined for the search of a best-match of the template DT with a desired dot. The width and the height of SR are defined to be the values of Wg and Hg defined in Eq. (5.3), respectively. SR is centered at a position in G_g corresponding to the center of a cross in G_b. See Fig. 5.10(a) for an illustration. At every search point (k, l) defined by a position (xk yk), denoted as SPk, in the search range SR for a radius rl of the template DT, the edge values of the eight search windows around the circle of the DT are computed and summed up as the edge energy of

Fig. 5.9 Rough CPM superimposed on the binary dot matrix pattern image.

found after all search points are tried, and the corresponding SPmax at location (xmax, ymax) is taken to be the desired center position for the white dot CCj. A cross “+” is then drawn at (x_max, y_max). We do the same process for all the other dots, and this completes the construction of the desired CMP. Fig. 5.4(f) shows an example of the final results of applying this process.

(c) Extraction of extended dot centers on screen border

So far, we have obtained a CMP which includes accurate position information of the original dot centers, and an approximating polygon of the screen border. We need further the positions of the extended dot centers on the screen border. These positions can be calculated from the geometric information mentioned above. We describe the details below.

Referring to Fig. 5.11, an extended dot center point Pi on the left-hand side of the screen border can be extracted by utilizing the position information of the first and the second dots CCi1 and CCi2 on the ith row of the dot matrix. First, we obtain the line Li

which connects the centers of CC_i1 and CC_i2 and extend it to the left-hand side. If the line

Fig. 5.10 Deformable template matching. (a) SR and CC

j

. (b) Circular template with 8

scan windows.

(xi, yi) ri

W_g

H_g

CC

_j

SR

(a) (b)

dot center point Pi. We do similar works to obtain all the other extended dot centers. The details are omitted. In Fig. 5.4(f), the points marked with crosses “+” on the screen border are the extraction results.

在文檔中 攝影機校準及影像轉換技術與其應用之研究 (頁 113-121)