Chapter 3 Scene Change Detection of Basketball Video and Its Application in
3.1 Scene Change Detection Using GOP-Based Method
In order to analyze tactics in the basketball video, we have to detect scene change and cut the video into clips. After that, we classify clips into three kinds of shot and choose the full-court view shots to do further processing.
Most of existed approaches detect scene change frame by frame. However, the scene change does not occur on each frame, hence it is not necessary to do frame-wise scene change detection. We use a GOP-based method to improve the efficiency of scene change detection. The format of MPEG-Ⅱ includes a GOP layer. As Fig. 3-2 shows, a GOP structure contains the header and an intra-frame coding frame (I-frame) accompanies series of frames in two types including predictive coding frame (P-frame), and bi-directionally predictive coding frame (B-frame).
GOP Header I B B P B B P B B P B B GOP Header I B B
Fig. 3-2 Structure of GOP.
The GOP-based scene change detection approach has two steps[26]. The workflow of this approach is shown in Fig. 3-3. In the first step (Inter-GOP scene
change detection), the possible occurrence of scene change is checked GOP by GOP instead of frame by frame. If a GOP is detected having possible scene change, go to the second step. In the second step (Intra-GOP scene change detection), we check whether the scene change exits and find the actual frame where the scene change occurs within the GOP. The detailed process of the two steps is described in Section 3.1.1 and 3.1.2, respectively.
Fig. 3-3 The workflow of the scene change detection method.
3.1.1 Inter-GOP scene change detection
For each I frame, divide it into k sub-regions. Sum the DC values in each sub-region. The image feature of a GOP { | 1,... }
1 ,
, DC i k
SumDC
g Ni
j i j
i
g = =
=
∑
=
, where i is the index of sub-region in I-frame, N is the total number of DC values in the i i th sub-region, and DCi,jis the j th DC value of sub-region i . The Distance
Inter-GOP scene change detection Calculate the difference
in each GOP-pair
Intra-GOP scene change detection Find out the actual scene change
frame within the GOP Does the difference exceed the threshold?
Yes
No Step 1.
Step 2.
between two GOPs g and g+1 is represented asD(g,g+1), and the value of D(g,g+1) is computed as follows:
marki =1 if |SumDCg,i −SumDCg+1,i |>threshold_subregion marki =0 otherwise
∑
==
+ k
i
marki
g g D
1
) 1 , (
WhenD(g,g+1)≤threshold_GOP, which means the successive GOPs are similar, we say no scene change occurs. WhenD(g,g+1)>threshold_GOP which means GOP g and GOP g+1 are dissimilar, we assume that a possible scene change occurs in GOP g+1. However, large difference may be caused by the camera motion and object moving rather than the real scene change. To solve this problem, Intra-GOP scene change detection is proposed.
3.1.2 Intra-GOP scene change detection
The Fast Pure Motion Vector Approach[27] is used for efficient scene change detection within a GOP. This approach only uses motion vectors of B-frames to detect scene change since B-frames are motion-compensated with respect to referential frames. If a B-frame is most similar to previous referential frame, most of the motion vector will refer to forward direction. If a B-frame is most similar to back referential frame, most of the motion vector will refer to backward direction. Two notations are defined below.
Rb: The ratio of number of backward motion vectors over the number of forward motion vectors.
Rf : The ratio of number of forward motion vectors over the number of backward motion vectors.
B I B B B P B B
…… ……
More Backward Less Forward
Peak Rb Scene change Case1: Scene change occurs on I-frame or P-frame.
Case2: Scene change occurs on the first B-frame between two successive reference frames.
Case3: Scene change occurs on the second or later B-frame between two successive
reference frames.
We discuss the three cases and infer the rule to find the actual scene change frame.
Fig. 3-4 Scene change occurs on I-frame or P-frame.
Case 1 is shown in Fig. 3-4. If scene change occurs on I-frame or P-frame, the first previous B-frame will be similar to its previous referential frame. Therefore, most of the motion vectors of the first previous B-frame refer to the forward referential frame, and Rf of the first previous B-frame will be very large and exceed the threshold_ Rf .
Fig. 3-5 Scene change occurs on the first B-frame.
Scene change Less Backward More Forward
Peak Rf
B I B B B P B B
…… ……
Scene change
Less Backward More Forward
Peak Rf Peak Rb
More Backward Less Forward
B I B B B P B B
…… ……
Case 2 is shown in Fig. 3-5. If Scene change occurs on the first B-frame between two successive reference frames, the B-frame itself will be similar to its back referential frame. Therefore, most of the motion vectors of this B-frame refer to the backward referential frame, and Rbof the first B-frame will be very large and exceed the threshold_Rb.
Fig. 3-6 Scene change occurs on the second or later B-frame.
Case 3 is shown in Fig. 3-6. If scene change occurs on the second or later B-frame between two successive reference frames, the B-frame itself will be similar to its back referential frame and the first preceding B-frame will be similar to previous referential frame. Therefore, most of the motion vectors of the first preceding B-frame refer to its forward referential frame, and the second B-frame mostly refer to the backward referential frame; i.e. Rf of the first B-frame andRbof the second B-frame will be very large.
After examining values of Rb and Rf on B-frames, scene changes could be detected in GOP whileRb or Rf on B-frame exceeds the predefined threshold.
Some noise such as camera or object moving which leads to possible scene change in the first step can be removed because such kinds of frame are usually not similar to both its previous and back referential frame, and its values of Rb and Rf on
Detect Dominant Color Region
Reset Frame Buffer and Set Weights
Update Statistics
Compute Color Statistics
Construct
Add to Frame Buffer
Fusion of
Detection Result Set weight=1.0
New Frame
Segmented Image Primary Color Space
Initial Statistics
Yes N frames
In Buffer Enough
Dominant Color Pixels
Inconsistent Segmented
Mask Yes
Yes Local Statistics
Control Color Space