Efficient and effective wipe detection in MPEG compressed video based on the macroblock information

Efficient and Effective Wipe Detection in MPEG Compressed Video Based on the Macroblock

Information

Soo-Chang Pei, and Yu-Zuong Chou

Department of Electrical Engineering National Taiwan University Taipei, Taiwan 10617, R. 0. C

ABSTRACT

In this paper, we propose an effective and efficient wipe detection method based on the macroblock information of the MPEG compressed video. Two types of wipe transitions are taken into consideration. By analyzing the prediction directions of B frames, which are revealed in the MB types, the scene change region of each frame can be found. Once the accumulation of the scene change regions covers most area of the frame, the sequence will be considered a wipe transition sequence. Besides, uncommon intra-coded MBs of the B frame can also be applied as an indicator of the wipe transition. A very simple analysis based on small amount of MB type information is sufficient to achieve wipe detection directly on MPEG compressed video.

1. INTRODUCTION

In order to facilitate the video retrieval and browsing mechanism, video analysis becomes a very important issue to act as the initial step. The scene change detection method, as the fundamental function of video analysis, arouses a lot of interests and many related algorithms have been proposed. Generally, scene changes are divided into two types, abrupt scene change and gradual scene change; each requires different detection method. Compared with abrupt scene changes, gradual scene changes not only require more complex detection method but also probably produce disturbance on abrupt scene change detection. As to gradual scene change, two mostly encountered transition types are dissolve and wipe transitions. The wipe sequence detection method, relative to dissolve sequence, is less discussed and concerned papers are also fewer. Some of related approaches perform the wipe detection based on uncompressed video[ 11 [2]. Although good performance is obtained, high computation load is required to handle the different shapes, directions and patterns of the wipe effects. Wu, Wolf and Liu[3] have proposed a wipe detection based on the DC sequence of MPEG compressed video. But it is only capable to detect horizontal and vertical wipes. We have also proposed a scene analysis algorithm [7], which

includes abrupt scene change, fast panning and dissolve sequence detection, based on MB type information and satisfactory simulation results are obtained. Directly on the MPEG compressed video, the different characteristics of prediction directions revealed in the MB type information corresponding to different time locations of scene changes can be used for the abrupt scene change detection. Here, we want to extend this concept to wipe detection. Taking the advantage of frame-based accuracy of abrupt detection, the wipe detection can be achieved by accumulating the scene change regions over a period of time. For another type of wipe sequence in which the next scene is moving onto the current scene, recording the number of intra-coded MBs in B and P frames can perform the wipe detection effectively. Compared with other proposed methods, this novel method benefits from easy extraction and simple analysis of MB type information. In addition, this method works no matter where the initial location and in what direction the next scene is wiped onto the current scene. Satisfactory simulation results are obtained in our experiments.

2. PROPOSED WIPE DETECTION METHOD In this paper, two types of wipe transitions illustrated in Fig. 1 are discussed and experimented. In the type A wipe sequence, the current scene is gradual wiped by co- located area of the next scene. As to type B wipe sequence, the next scene gradually moves onto the current scene. It is worthwhile to note that the direction, the initial position and even the shape of the wipe effect can be arbitrary in real application. We first introduce the abrupt scene change detection method on which the wipe detection method performs based. Owing to two anchor frames belonging to two different shots, the prediction directions of B frames can be applied to indicate the scene change as shown in Fig.3. The concept of MBGxy shown in Fig. 2 can help to find the geometric position where the change occurs. The property of frame-based detection accuracy and the capability to locate the scene change region facilitate the wipe detection. Fig. 4 describes this idea. By analyzing the MB type information of B frames, the scene change region of each frame is found and used

953 0-7803-6297-7/00/$10.00 0 2000 IEEE

to track the wipe transition. According to the accumulation of scene change regions, the ratio of the region wiped by the next scene to total area can be calculated. Once the ratio exceeds a pre-defined threshold, and the scene change occurring in any single frame are smaller than Ts, designed to avoid the false alarm from the abrupt scene change, a wipe transition sequence is found. In practical application, the wipe detection algorithm is activated once the number N,p of Intra-coded

MBs in the P frame exceeds the threshold

T p

or an I frame is encountered. The activation status will be released when N,p drops under the threshold Top. The ratio of changed region accumulation during the period of the activation status to all area of the frame is used for wipe detection. The activation mechanism helps to eliminate the trouble of setting the threshold of the length of frames to monitor. Once the ratio exceeds the pre-defined threshold Top, this sequence of frames will be regarded as a wipe sequence. Only simple analysis on MB type information is sufficient for the type A wipe detection.

For the type B wipe transition, different speeds, at which the next scene to move onto the current scene, should require different detection methods. If the speed is within the tracking ability of the motion estimation mechanism, that is, the search window, the detection algorithm is the same as the type A wipe detection since the Fig. 4 is still valid. However, fast moving speed will make neither forward nor backward prediction of B frames adequate and also make the type A wipe detection ineffective. Fortunately, the lack of adequate frames for prediction results in uncommon Intra-coded MBs of B frames. This phenomenon inspires us a new idea of type B wipe detection. The number of Intra-coded MBs can be used as the indicator and compared with a pre-defined threshold To,, to detect the type B wipe transition. In

practice our method keeps on monitoring the number of Intra-coded MBs of P frame to decide whether the activation status has to be activated. Once the activation status is encountered, the algorithm starts to detect and accumulate scene change regions and simultaneously monitor the number of intra-coded MBs of B frames. As the activation status is released, the ratio of changed regions and the number of the intra-coded MBs of B frames will be calculated and used for type A and type B wipe detection respectively.

Fig 1: (A) Type A wipe transition (B) Type B wipe transition

954

0

MB tn i o s n e change trams

changed HB

Fig 2: An illustration of concept of MBGxy for locating the geometric location of changed MBs.

nsl-.nlMnS~

(S)

Fig 3: The prediction patterns of B frames for abrupt scene change detection.

PPI B B PSI

Fig 4: An illustration of wipe type A detection by accumulating the scene change regions.

.

3. EXPERIMENTAL RESULTS

In order to examine the proposed method, a series of sequences are edited for experiments. Each frame of these sequences is SIF (352x240) in size and coded using TM5 (M=3, N=15, 4Mbps). For the type A wipe detection, the sequence shown in Fig. 5 is experimented and the scene change detection and accumulation results corresponding to each frames are shown in Fig.6 and Fig. 7, where each white dot represents a scene changed MB. Simulation results clearly show that the method can track the wipe transition and successfully detect it. For the type B wipe detection, Fig. 8 shows how the Tib and Tip are applied to detect the type B wipe sequence from frame 16 to 30.

Fig 5: The test wipe sequence edited from news.

Fig 6: The detection results of scene change regions from the sequence shown in Fig 5, where each white dot

represents a scene change MB.

Fig 7: The accumulated scene change regions for type A wipe detection

Fig 8: The Intra-coded MB of B and P frames for type B wipe detection

4. DISCUSSION 4.1. The influence of bit rates

The bit rates of MPEG video streams can vary from one to several hundred Mbps. As a result, the influence of

bit rates should be taken into account to verify the validity of the proposed method. Refer to the rate control mechanism of TM5, the rate is adjusted by changing the quantization scale and irrelevant to macroblock type decision. The macroblock type is determined by the spatial complexity of residue information. Therefore, the influence of the variation of bit rates is insignificant and, in other words, our method can perform well on the video streams of different bit rates. An experiment based on different bit rates has been done in [7] and simulation results also support this point.

4.2. The different GOP structure

In the MPEG standard, the encoder is not defined sot that different GOP structure could be used for MPEG

coding. As a result, different type of prediction patterns of B frames should be adopted for different GOP structure. The algorithm of this paper is based on the N=15, M=3 GOP structure. We have also presented the modification of prediction patterns for different GOP structure[7]. Simulation results also show these modified patterns work well.

Proposed

4.3. The performance measurement

In order to measure the performance of this proposed method, we edit a two-minute sequence containing 15 wipe transitions from news and sports video sequences. The simulation results listed in Table 1 show that the proposed method performs well in both detection and false alarm rate. It is worthwhile to note that three directions of the wipe effects, which include vertical, horizontal and diagonal directions, are simulated.

Detected Miss False Alarm

15 0 1

Change Detection on MPEG Coding Parameter Domain” ICIP, 1998, pp. 888 -892

[6] C.W. Ngo, T. C. Pong and R.T.Chin, “Camera break

detection by partitioning of 2D spatio-temporal images in MPEG Domain”, Multimedia Computing and Systems, 1999. IEEE International Conference on Volume: 1 , 1999 , Page(s): 750 -755

[7] Soo-Chang Pei and Yu-Zuong Chou, “Efficient

MPEG Compressed Video Analysis Using Macroblock Type Information” to appear in IEEE Trans. on Multimedia, Vol. 1, No. 4, December 1999, pp.321-333.

I I I I I

5. CONCLUSIONS

An effective and efficient wipe detection algorithm is proposed based on the MB type information. This method benefits from the easy extraction of MB type information, simple analysis and independence from the direction and initial location of the wipe effect. Last, we have to emphasize that this method is valid only when B frames are used in the MPEG coding process. As a result, the future work to integrate other methods using motion vector or DCT coefficient is necessary to cope with the sequence containing only I and P frames.

5. REFERENCE

[

13

Adnan M. Alattar, “Wipe scene change detector for use with video compression algorithms and MPEG-

7”, IEEE Trans. on Consumer Electronics, Volume: 44 1 , Feb. 1998 , Page(s): 43 -51

[2] W.A.C. Fernando, C.N. Canagarajah, D.

R. Bull, ”Wipe Scene Change Detection in Video Sequence”, ICASP, 1999, pp.294 -298.

[3] Min Wu, Wayne Wolf and Bede Liu, “An Algorithm

For Wipe Detection”, ICIP, 1998, pp. 893-897.

[4]

Stuart J. Golin, “New metric to detect wipes and

other gradual transitions in video”, SPIE, Vol. 3653,

[5] Masaru Sugano, Yasuyuki Nakajima, Hiromasa

Yanagihara and Akio Yoneyama, “A Fast Scene 1998, pp. 1464- 1474