Baseball Event Classification

The proposed baseball event classification system has been tested on Major League Baseball of broadcast baseball video. The categories to be recognized are twelve types of baseball events: single, double, pop up, fly out, foul out, ground out, two-base out, right foul ball, left foul ball, double play, home run, and home base out as described in Table 4-2. The video source was gotten from three different Major League Baseball games of broadcast video and digitized into 352×240 pixel resolution. The experimental result of baseball event type classification is in the

Table 5-3 and Table 5-4. Table 5-3 shows the precision and recall of clips manually clipped from broadcast videos. Table 5-4 shows the precision and recall of clips automatically clipped from broadcast video.

Event type Total Correct false Precision (%)

Recall (%)

Single 25 20 1 95.2 80.0

double 8 2 1 66.7 25.0

Pop up 7 7 2 77.8 100.0

Fly out 22 21 4 84.0 95.5

Foul out 1 1 0 100.0 100.0

Ground out 29 27 3 90.0 93.1

Two-base out 4 4 0 100.0 100.0

Right foul ball 12 12 0 100.0 100.0

Left foul ball 6 6 2 75.0 100.0

Double play 1 1 2 33.3 100

Home run 6 5 1 85.7 83.3

Home base out 1 1 0 100.0 100.0

Total 122 106 87.3

Table 5-3 Recognition of baseball events manually clipping Event type Total Correct false Precision

(%)

Recall (%)

Single 25 20 3 87.0 80.0

double 8 2 1 66.7 25.0

Pop up 7 7 2 77.8 100.0

Fly out 22 19 5 86.4 79.2

Foul out 1 1 0 100.0 100.0

Ground out 29 27 4 87.1 93.1

Two-base out 4 2 0 100.0 50.0

Right foul ball 12 12 0 100.0 100.0

Left foul ball 6 6 2 75.0 100.0

Double play 1 1 2 33.3 100.0

Home run 6 5 1 85.7 83.3

Home base out 1 1 0 100.0 100.0

Total 122 102 84.4

Table 5-4 Recognition of baseball events automatically clipping

Both the precision and recall are about 80% except for the precision of double, double play and the recall of double, two-base out. The low recall rate of baseball event double and two-base out might result from the missed detection of field object 2B. The low precision rate of baseball event double might be that the transitions of double and Home run are similar if the batter hits the ball to the audience wall.

The low precision rate of baseball event double play might be that the transitions of double play and ground out are similar if the batter hits the ball around the second base as shown in Fig. 5-1. Fig. 5-2 shows the miss detection of right foul ball and home run due to the similar shot transition. Fig. 5-3 shows some ambiguities in nature of baseball events such as ground out and left foul ball even if those baseball events are judged by people. Fig. 5-4 shows miss detection between single and ground out because the player in first base does not catch the ball and we do not detect the ball object.

The miss detection of highlights can be classified into four reasons: (1) similar shot transition, (2) miss object detection, (3) detected objects are not enough, and (4) ambiguity in nature. These could be improved by detecting the object of ball and players, or add additional information such as scoreboard information. Overall, we still achieve good performance.

(a)

(b) (a)

(b)

Fig. 5-1 Comparison between (a) ground out and (b) double play

(a)

(b) (a)

(b)

Fig. 5-2 Comparison between (a) right foul ball and (b) home run

(a)

(b) (a)

(b)

Fig. 5-3 Ambiguity of (a) left foul ball (b) replay of left foul ball

Fig. 5-4 Ambiguity of ground out and single 5.3 Other Discussions

In the line detection in section 4.3, lines are detected by Ransac algorithm rather than Hough transform. Hough transform can detect most of lines in a frame, but the time complexity is too complicated. Ransac algorithm can find “apparent” lines. The time complexity depends on the number of line pixels. In baseball game, the most important lines are left line and right line. The left line or right line in each frame of baseball game is marked with white color. The number of lines is not too many and

the lines are apparent in baseball field. According to the observation, we adopt Ransac algorithm to do line detection. The algorithm is applied in other sports such as tennis [12].

Second base detection in section 4.3, some conditions will be viewed as second base as shown in Fig. 5-5.

Fig. 5-5 Ambiguity in second base

We should set a threshold of region size of second base, but it is very difficult to set the boundary because the size of second base varies in each frame due to the zooming of the camera. It is also the main reason of poor detection in B2 frame type classification.

Determining the type of Markov model is also an important issue in our system.

The original Hidden Markov model is shown in Fig. 5-6. At the same time, the complicated model can be simplified to left-to-right Hidden Markov model as shown in Fig. 5-7 and the model is applied extensively in speech recognition. In baseball game, baseball events are usually composed of several shots and each baseball event has specific shot transition. Take four baseball events in Fig. 2-4 [5] as an example.

Some phenomena can be exploited such as the baseball event of Home run: The state of Audience View and Running Closeup are bi-directed. So, original Hidden Markov Model is adopted in our system for baseball event classification.

1

2 3

1

2 3

Fig. 5-6 3-state original Hidden Markov Model

1 2 3

1 2 3

Fig. 5-7 3-state left-to-right Hidden Markov Model

All discussions above, we know the current problems in the proposed system. The detail of how to improve the problems will be described in chapter 6.

Chapter 6

Conclusion and Future Work

We can achieve good performance of baseball event classification due to the high precision in object detection and varieties of frame types. High precision in object detection makes the precision of frame type classification higher and increases varieties of frame types and baseball event types. High precision in frame type classification of R-AT frame type and L-AT frame type makes the precisions of foul ball and foul out baseball event types higher. High precision in object detection of bases and lines, frame type classification of infield types and outfield types makes the precisions of ground out and fly out baseball event types higher than those in previous work. All points above, some viewpoints are concluded in our system: (1) the frame classification achieves a high precision which is about 90%, (2) the hitting baseball event types are detected more than those in previous works, and (3) the precisions in classification of fly out, foul ball, foul out, and ground out are better than those in previous works.

However, some problems as shown in Fig. 5-1, Fig. 5-2, Fig. 5-3, and Fig. 5-4 in section 5-2 can be improved by adding some intelligent information such as scoreboard information. Take double play and ground out as an example. We can recognize the two different baseball events from the #out information in scoreboard even if both shot transitions are similar. Furthermore, we hope that the baseball event type classification in proposed system is robust, but the current system just adapts to MLB video sources. All problems above, our future works are as follows. (1) Add scoreboard information to solve some ambiguities of baseball events, (2) Add player or ball tracking to raise the correctness rate of baseball event classification, and (3) Increase varieties of baseball video sources such as Nippon Professional Baseball (NPB) as training data to make the proposed system more robust.

Bibliography

[1] M. Kumano, Y. Ariki, K. Tsukada, S. Hamaguchi, and H. Kiyose, “Automatic Extraction of PC Scenes Based on Feature Mining for a Real Time Delivery System of Baseball Highlight Scenes,” IEEE international Conference on Multimedia and Expo, vol. 1, pp. 277-280, 2004.

[2] H. S. Chen, H. T. Chen, W. J. Tsai, S. Y. Lee, and J. Y. Yu, ”Pitch-by-Pitch Extraction from Single View Baseball Video Sequences,” IEEE International Conference on Multimedia and Expo, pp. 1423-1426, 2007.

[3] W. Hua, M. Han, and Y. Gong, “Baseball scene classification using multimedia features,” IEEE International Conference on Multimedia and Expo, vol. 1, pp.

821-824, 2002.

[4] H. T. Chen, M. H. Hsiao, H. S. Chen, W. J. Tsai, and S. Y. Lee, “A baseball exploration system using spatial pattern recognition,” Proc. IEEE International Symposium on Circuits and Systems, pp.3522-3525, May 2008.

[5] P. C. , M. H., and Y. Gong,“Extract Highlight From Baseball Game Video With Hidden Markov Models,” International Conference on Image Processing, vol. 1, pp.609-612, 2002.

[6] T. Mochizuki, M. Tadenuma and N. Yagi, “Baseball Video Indexing Using Patternization Of scenes and Hidden Markov Model,” IEEE International Conference on Image Processing, vol. 3, pp. III -1212-15, 2005.

[7] N. H. Bach, K. Shinoda and S. Furui, “Robust Highlight Extraction Using Multi-stream Hidden Markov Model For Baseball Video,” IEEE International Conference on Image Processing, vol. 3, pp. III- 173-6, 2005.

[8] M. Fleischman, B. Roy and D. Roy, “Temporal Feature Induction for Baseball Highlight Classification,” Proc. ACM Multimedia Conference, Augsburg, Germany, pp. 333-336, 2007.

[9] M. H. Hung, C. H. Hsieh and C. M. Kuo, “Rule-based Event Detection of Broadcast Baseball Videos Using Mid-level Cues,” in Proceedings of IEEE International Conference on Innovative Computing Information and Control, pp.

240-244, 2007.

[10] C. H. Liang, W. T. Chu, J. H. Kuo, J. L. Wu, and W. H. Cheng, “Baseball event detection using game-specific feature sets and rules,” in Proceedings of IEEE

International Symposium on Circuits and Systems, vol. 4, pp.3829-3832, 2005.

[11] M. H. Hung and C. H. Hsieh, “Event Detection of Broadcast Baseball Videos,”

IEEE Transactions on Circuits and Systems for Video Technology, vol. 18, no.12, pp.1713-1726, 2008.

[12] D. Farin, J. Han and H. N. Peter, “Fast Camera Calibration for the Analysis of Sport Sequences,” IEEE International Conference on Multimedia & Expo, pp.

482–485, 2005.

[13] L. R. Rabiner and B. H. Juang, “An introduction to hidden Markov models,”

IEEE Signal Processing Magazine, vol. 3, no. 1, pp4-16, 1986.

[14] G.D. Forney, “The Viterbi algorithm,” Proceedings of the IEEE, vol. 61, no. 3, pp. 263-278, 1973.

[15] B. H. Juang and L. R. Rabiner, “The segmental K-means algorithm for estimating the parameters of hidden Markov models,” IEEE Transaction of Acoustics Speech and Signal Processing, vol. 38, no. 9, pp. 1639-1641, Sept.

1990.

[16] B. H. Juang and L. R. Rabiner, “A probabilistic distance measure between HMMs,” AT&T Technical Journal, vol. 64, no. 2, pp. 391-408, 1985.

[17] X. D. Huang, Y. Ariki, and M. A. Jack, “Hidden Markov Models for Speech Recognition,” Edingurgh Univ. Press, 1990.

[18] L. R. Rabiner and B. H. Juang, “A Tutorial on Hidden Markov Models and selected Applications in Speech Recognition,” Proceedings of the IEEE, pp.

257-286, 1989.

[19] H. T. Chen, M. C. Tien, Y. W. Chen, W. J. Tsai and S. Y. Lee, “Physics-based ball tracking and 3D trajectory reconstruction with applications to shooting location estimation in basketball video,” Journal of Visual Communication and Image Representation, vol. 20, no. 3, pp. 204-216, 2009.

[20] Hua-Tsung Chen, Hsuan-Sheng Chen, Ming-Ho Hsiao, Wen-Jiin Tsai and Suh-Yin Lee, “A Trajectory-Based Ball Tracking Framework with Visual Enrichment for Broadcast Baseball Videos”, Journal of Information Science and Engineering (JISE), vol. 24, no. 1, pp.143-157, Jan 2008.