• 沒有找到結果。

soil grass

4.3 Object (spatial) Detection

The baseball field is characterized by a well-defined layout of specific colors as described in Fig. 4-4. Furthermore, important lines and the bases are in white color, and auditorium (AT) is of high texture and no dominant color as shown in Fig. 4-4(b).

L-AT R-AT

L-AT R-AT

(a) Full view of real baseball field (b) Illustration of baseball field Figure 4-4 The field objects and features.

Each object will be elaborated as follows.

(1) Back auditorium (AT):

The top area which contains high texture and no dominant colors is considered as the auditorium, as the black area above the white horizontal line in Fig. 4-5(a).

(2) Left auditorium (L-AT) and right auditorium (R-AT):

The left area and right area which contains high texture and no dominant colors is considered as the left auditorium and right auditorium, as the left black area and the right black area marked with the white vertical line in Fig. 4-5 (b) and Fig. 4-5 (c).

(a) (b) (c)

(a) (b) (c)

Fig. 4-5 Illustration of (a) back auditorium (b) left auditorium (c) right auditorium

(3) Left line (LL) and right line (RL) :

A Ransac algorithm, which finds the line parameter of line segments [12], is applied to the line pixels and then finds the left or right line. The line pixel is high intensity pixel greater than threshold σl excluding pixels in large white area and auditorium area. Either two pixels at a horizontal distance of ±τ pixels or at a vertical distance of

±τ pixels must be darker than σd, where σd << σl. Fig. 4-6 shows the concept of excluding white area. The parameter τ should be set to approximately the double court line width. As illustrated in Fig 4-6, each square represents one pixel and the central one drawn in gray is a candidate pixel. Assuming that white lines are typically no wider than τ pixels (τ = 6 in our system), we check the four pixels, marked ‘V’ and

‘H’, at a distance of τ pixel away from the candidate pixel on the four directions. The central candidate pixel is identified as a white line pixel only if both pixels marked

‘H’ or both pixels marked ‘V’ are with lower brightness than the candidate pixel.

Fig. 4-6 Line pixel detection excluding large white area.

This process prevents that white pixels are extracted in large white areas including auditorium area or white uniforms. Fig. 4-7 is an example of line pixel detection.

Fig. 4-7 (b) shows that the intensity of pixels higher than a threshold of I component in HSI color space and Fig. 4-7 (c) shows that the remaining high intensity pixels after line pixel detection. In Fig. 4-7 (c), the high intensity pixels in pitch mound and

auditorium area is vanished.

Fig. 4-7 The result of retained high intensity pixel after line pixel detection algorithm: (a) original data (b) high intensity pixel before line pixel detection (c) high intensity data after line pixel detection

After line pixel detection, the Ransac algorithm is applied to find line parameter as shown in Fig. 4-8. In Fig 4-8(a), the black point represents the high intensity pixel after the line pixel detection. In Fig 4-8 (b), two points are randomly chosen and the red line is the line passing through the two points. In a specific area (the distance between the point and the line is less than 2, τ = 4), the high intensity pixel will be accumulated denoted a score in this time. The action will repeat until the number of remaining high intensity pixel is less than N or the pre-defined iterative number is reached. Finally, the highest score and the line parameter will be stored as the proposed line in Fig 4-8 (c) and Fig. 4-8 (d).

(c) (a) (b)

Find line parameter and score=14

2.Random pick two point and count the score in specific area.

3.After several iterations,

find the highest score. 4.Terminate when line pixel number is less σ.

Find line parameter and score=14

find line parameter

2.Random pick two point and count the score in specific area.

3.After several iterations,

find the highest score. 4.Terminate when line pixel number is less σ.

Find line parameter and score=14

find line parameter

2.Random pick two point and count the score in specific area.

3.After several iterations,

find the highest score. 4.Terminate when line pixel number is less σ.

(a) (b)

(c) (d)

Find line parameter and score=14

find line parameter

2.Random pick two point and count the score in specific area.

3.After several iterations,

find the highest score. 4.Terminate when line pixel number is less σ.

Find line parameter and score=14

find line parameter

2.Random pick two point and count the score in specific area.

3.After several iterations,

find the highest score. 4.Terminate when line pixel number is less σ.

Find line parameter and score=14

find line parameter

2.Random pick two point and count the score in specific area.

3.After several iterations,

find the highest score. 4.Terminate when line pixel number is less σ.

Find line parameter and score=14

find line parameter

2.Random pick two point and count the score in specific area.

3.After several iterations,

find the highest score. 4.Terminate when line pixel number is less σ.

Find line parameter and score=14

find line parameter

2.Random pick two point and count the score in specific area.

3.After several iterations,

find the highest score. 4.Terminate when line pixel number is less σ.

Find line parameter and score=14

find line parameter

2.Random pick two point and count the score in specific area.

3.After several iterations,

find the highest score. 4.Terminate when line pixel number is less σ.

Find line parameter and score=14

find line parameter

2.Random pick two point and count the score in specific area.

3.After several iterations,

find the highest score. 4.Terminate when line pixel number is less σ.

(a) (b)

(c) (d)

Fig. 4-8 Ransac algorithm for finding line parameter.

(4) Pitch mound (PM):

An ellipse soil region surrounded by a grass region would be recognized as pitcher’s mound as shown in Fig. 4-9. Bounding box is applied to ellipse detection.

The procedure is described as follows. (1) stop at a brown pixel, (2) find the upper, lower, left, and right bound stopped at the first green pixel in upper, lower, left, and right direction, (3) count the brown pixel percentage in the bounding box, and (4) the percentage in specific range will be considered a ellipse. Some illegal ellipse could fit the proportion of soil in bounding box. The illegal ellipse should be deleted to improve the frame classification and baseball event classification. A threshold of the brown pixel difference will be set in diagonal lines, upper and lower region, and right and left region as shown in Fig. 4-10. In Fig. 4-10, only Fig. 4-10(c) is a correct ellipse.

(5) First base (1B) and third base (3B):

The square region located on right line, if detected, in soil region would be identified as first base as shown in Fig. 4-9. Similarly, the square region located on left line, if detected, in soil region would be identified as third base.

(6) second base (2B):

In a soil region, a white square region on neither field line would be identified as second base as shown in Fig. 4-9.

(7) home base (HB):

Home base is located on the region of the intersection between left line and right line as shown in Fig. 4-9.

2B

1B

RL HB

LL

PM 2B

1B

RL HB

LL

PM

Fig. 4-9 Shows the objects of 1B, 2B, HB, LL, RL, and PM.

(a) (b) (c)

(a) (b) (c)

Fig. 4-10 Deletion of illegal ellipse 4.4 Frame Classification

Classification is divided into two orientations in this approach, one is frame type classification by using rule table as listed in Table 4-1 modified from [4], and another is baseball event classification by using HMM. The classification of frame types can assist in realizing the shot transition or for other purposes. Sixteen frame types are defined and classified based on the position or percentage of some objects and features as described in section 4-3 in rule table. Sixteen typical region types is: IL (infield left), IC (infield center), IR (infield), B1 (first base), B2 (second base), B3 (third base), OL (outfield left), OC (outfield center), OR (outfield right), PS (play in

soil), PG (play in grass), AD (audience), RAD (right audience), LAD (left audience), CU (close-up), and TB (touch base), as shown in Fig. 4-11.

RAD: right audience

LAD: left audience RAD: right audience TB: touch base CU: close-up

LAD: left audience TB: touch base CU: close-up

Fig. 4-11: Sixteen typical frame types

The rules of frame type classification are listed in Table 4-1 modified from [4].

The symbols of first column are our sixteen defined frame type (IR, IC…) as shown in Fig. 4-11. Wf is the frame width, the function P(Area) return the percentage of the area Area in a frame, X(Obj) returns the x-coordinate of the center of the field object Obj, and W(Obj) returns true if the object Obj exists. Each field frame is classified into one of sixteen frame types using rule table (Table 4-1 modified from [4]). For example: a field frame would be classified as B1 frame type if the frame meets the following conditions: The percentage of AT is no more than 10%, the object of PM does not exist, the object of RL and 1B must exist, the percentage of soil is more than 30%.

IR:

{ ( ) ( ) ( ) }

unknown others

Table 4-1 Rules of frame type classification modified from [4].

Each frame is first recognized by the distribution of dominant color and white

pixels (intensity data). After object detection and looking up the rule table, we can know the detected objects and features in each frame and shot transition represented as an annotated string as shown in Fig. 4-12. The content of the sample field shot in Fig.4-12 says that the ball is first batted into the left infield. Then, the shortstop picks up the ball and throws it to the first baseman. The batting process can be appropriately abstracted by the output string: IL (infield left) Æ PS (player in soil) Æ B1 (first base). In order to filter out the misclassifications of frame types within a shot, some regular rules are applied. For example, the shot transition IC Æ IR (or PS) Æ B1 appears frequently in the baseball event of ground out. When the current frame type is not IR or PS, but the previous frame type is IC and the next frame type is B1, the system will change the current type from incorrect type to IR. The regular rule is a set created previously from observation.

… … …

… …

Frame (play region type) classification

IL IL PS B1

LL,RL,PM,B2 Grass>60%,soil<30%

LL,PM

Grass>60%,soil>30% Grass<60%,soil>30% RL,B1 Grass<60%,soil>30%

… … …

… …

Frame (play region type) classification

IL IL PS B1

LL,RL,PM,B2 Grass>60%,soil<30%

LL,PM

Grass>60%,soil>30% Grass<60%,soil>30% RL,B1 Grass<60%,soil>30%

Fig. 4-12 Illustration of the annotated string of ground out example after frame classification

相關文件