HMM Learning for Each Baseball Event

… …

…

Frame (play region type) classification

IL IL PS B1

LL,RL,PM,B2 Grass>60%,soil<30%

LL,PM

Grass>60%,soil>30% Grass<60%,soil>30% RL,B1 Grass<60%,soil>30%

… … …

… …

…

Frame (play region type) classification

IL IL PS B1

LL,RL,PM,B2 Grass>60%,soil<30%

LL,PM

Grass>60%,soil>30% Grass<60%,soil>30% RL,B1 Grass<60%,soil>30%

Fig. 4-12 Illustration of the annotated string of ground out example after frame classification

4.5 HMM Learning for Each Baseball Event

One HMM is created for each baseball event for recognizing time-sequential observed symbols. In our proposed method, twelve baseball events listed in Table 4-2

are defined so that there are twelve HMMs. Given a set of training data from each type of baseball event, we want to estimate the model parameters λ = (A, B, π) that best describe that each baseball event. First of all, Segmental K-means algorithm is used to create an initial HMM parameter λ and then Baum-Welch algorithm as described in section 3-3 is applied to re-estimate each HMM parameters ^{λ =}

(

)

of baseball event.

Single Right foul ball Double Left foul ball

Pop up Foul out

Fly out Double play Ground out Home run Two-base out Home base out Table 4-2 List of twelve baseball events

In our proposed method, two features such as grass and soil, and ten objects as shown in Fig. 4-4(b) are used as observations represented as a 1×12 vector to record whether the object appears or not. To apply HMM to time-sequential video, the extracted features represented as a vector sequence must be transformed into a symbol sequence by rule table as listed in Table 4-1 for later baseball event recognition. This is a well known technique, called vector quantization [17]. For vector quantization, codewords g_j∈Rⁿ represents an observation vector in the feature R space. ⁿ Codewordg is assigned to symbol_j v . Consequently, the size of code book equals _j the number of HMM output symbols. Sixteen shots as shown in Fig. 4-11 are viewed as hidden states.

Conventional implementation issues in HMM include (1) number of states, (2)

initialization, and (3) distribution of observation at each state. The first problem of determining the number of states is determined empirically and differs from each baseball event. The second problem can be approached by random initialization or using Segmental K-mean algorithm as described in section 3-3. Finally, the last problem can be solved by trying several models such as Gaussian model and choose the best one. In our approach, we choose Gaussian distribution. The following is the detailed description of each essential element.

State S: The number of states is selected empirically depending on different baseball event and each hidden state represents a shot type.

Observation O: the symbol mapped from rule table.

Observation distribution matrix B: use K-means algorithm and choose the Gaussian distribution at each state [15].

Transition probability matrix A: the state transition probability, which can be learned by Segmental K-means algorithm.

Initial state probability matrix π: the probability of occurrence of the first state, which is initialized by Segmental K-means algorithm after determining the number of states.

After determining the number of states and setting the initial tuple λ, to maximize the probability of the observation sequence given the model, we can use the Baum-Welch algorithm as described in section 3-3 to re-estimate the HMM parameterλ . The initial probability, transition probability, output symbol distribution can be re-estimated by Eq. (30) (31) (32) and then replace initial tupleλwithλ. A detail procedure is shown in Table 4-3:

Input: a set of observed symbol sequences (mapped by rule table)O¹O²...O^w, and number of states are determined as input parameter.

Initialization: use Segmental K-means algorithm to compute initial λ and compute score: Score =

_∑

(

)

1

|λ Repeat {

For each observed sequence O { ^w

Using the given λ to calculate the following variable:

( )

αt at each time t, state i using forward algorithm by Eq. (12) (13)

( )

βt at each time t, state i using backward algorithm by Eq. (16) (17)

( )

γt at each time t, state i by Eq. (24)

( )

εt at each time t, state transition from i to j by Eq. (26) }

Calculate

∑ ( )

= T

t t i

1

γ ,

∑

( )

= 1 1 T

t t i

γ ,

∑

( )

= 1 1 T ,

t

t i j

ε by Eq. (27) (28) (29)

Re-estimate ^{λ =}

(

)

_∑

(

)

1

|λ If Score’ < Score

Jump from Repeat loop Else {

Score = Score’

λ=λ }

}

Table 4-3 HMM for baseball event learning 4.6 Baseball Event Recognition

The idea behind using the HMMs is to construct a model for each of the baseball event that we want to recognize. HMMs give a state based representation for each highlight. After training each baseball event model, we calculate the probability

(

)

P O|λ of a given unknown symbol sequence O for each highlight modelλ_i. We can then recognize the baseball event as being the one by the highest probable baseball event HMM.

Chapter 5

Experimental Result and Discussion

To test the performance of baseball event classification, we implement a system capable of recognizing twelve different types of baseball events. The test contains two parts: (1) frame type recognition (2) baseball event type recognition. The test data source can be divided into two groups, one is manual clips, and another is auto-segmented clips. In the first group, all clips are hand cut. In the second group, an ending point of clip is determined by detected close-up or specific shot. All video sources are Major League Baseball (MLB). 120 baseball clips from three different MLB video sources as training data and 122 baseball clips from two different MLB video sources as test data. The experimental result is shown in the following sections.