An Interactive MPEG-1 Browsing System

(1)

AN INTERACTIVE MPEG-1 BROWSING SYSTEM

Jiun-Jie Huang,’ Chun-Hung Lin,’ Ja-Ling WU,’,~ Jiun Shiu,’ Yuh-Ming Huang,’ and Chiu-Ting Hsu’ ’Communication and Multimedia Lab., Department of Computer Science and Information Engineering

National Taiwan University, Taipei, 107, Taiwan, R.O.C.

’Department of Information Engineering, National Chi-Nan University, Puli, 545, Taiwan, R.O.C.

ABSTRACT

-

In this paper, a personal computer (PC) based interactive i”EG-1 browsing system is Proposed. The can get t h e general information Of

Although more and more MPEG video programs are available, it is possible that the quality of the presented video programs is too rough t o be accepted by the picky spectators. The video quality of the encoded bitstreams video bitstreams, such as t h e encoding structure a n d

t h e header parameters of each encoding layer via this browsing system. T h e users can also get t h e detailed information of each decoding step. W i t h t h e aid of t h e proposed browsing system, users will have t h e ability t o analyze any M P E G - 1 bitstream easily. In addition, this system also provides t h e capability t o encode M P E G - 1 video bitstreams and t o adjust t h e encoding parameters interactively when t h e quality of t h e encoded video is not good enough. T h e browsing system can also be used as an useful tool during t h e development stage of an M P E G encoding or decoding system.

1 INTRODUCTION

Nowadays, in the multimedia era, MPEG is the widespread industrial standard of video compression [1]-[3], and gradually prevailed in our daily life. The MPEG codec and the related products have be- come a popular topic. During the developing process of these products, various kinds of bugs might occur. It is very difficult t o find where the bugs reside sometimes. A convenient and powerful debugging tool is therefore required for shortening the developing time. In the proposed browsing system, users can get the header information and the encoding structure of each layer to analyze the MPEG-1 coded bitstreams. Moreover, they can get the decoding data of each decoding step to debug their MPEG-1 codec step by step. By this way, users can find and fix the bugs quickly.

depends entirely on the selection of encoding parameters. Therefore, the decision of the encoding parameters plays an important role in MPEG encoding. However, the decision rule is usually unalterable and general for the whole video in the encoding process because most of the available MPEG encoding systems are not subtle enough to adjust the parameters. Therefore, the rule might not fit all the conditions. In the proposed browsing system, users can encode MPEG-1 video by setting different arguments. The users can also adjust the encoding parameters of the macroblocks for each video frame interactively. The visual result of the modified macroblocks can be shown on the screen immediately; therefore, users can choose the appropriate parameters to improve the video quality. The adjustment of parameters compensates the defects of the encoded video sequences caused by inappropriate selection of encoding parameters. The browsing system can also be used as an useful tool to explore the relationship between the encoding parameters and the video quality.

2 MPEG-1 BROWSING SYSTEM

The system diagram of the proposed MPEG-1 browsing system is depicted in Fig. 1. The system consists of five functional units: video e n c o d e r , s t r u c t u r e ana- l y z e r , parameter a n a l y z e r , video p l a y e r , and en- c o d i n g parameter a d j u s t i n g unit.

(2)

Huang, et al.: An Interactive MPEG-1 Browsing System ---c

-

561 Encoding Parameter A d j u s t i n g

I

V i d e o S o u r c e Encoder

1

U n i t J Parameter

Figure 1: The system diagram of the MPEG-1 browsing system.

These functional units, except the encoding parameter adjusting unit, can be independently operated. For example, users can only use the structure analyzer t o analyze the structure of their video files. Therefore, the system can be used for different purposes.

If

users use the MPEG- 1 browsing system as a debugging tool, they can take advantages of the video player, structure analyzer and parameter analyzer to analyze the MPEG-1 bitstreams. For the necessity of debugging, these functional units all support the feature of randorn access.

When users want to generate MPEG-1 video bitstreams, they can use the MPEG encoding unit of the system to do this, and then use the video player of the system t o examine the video quality. If there are some defects in the coded video frames, users can adjust the encoding parameters interactively to refine the video quality.

The features of each functional unit are described as follows.

Video Player

The video player unit can be used t o examine the video quality of MPEG-1 bitstreams and to check which video frame should be response for the problems ap- peared.

Structure Analyzer

The structure analyzer unit shows the structure of the bitstreams as a tree structure and reports the encoding type of each tree node; therefore, users can understand the structure of each encoding layer. These information can be used to check the validness of a given MPEG-1

bitstream. Users can also move the scope from one tree node to another through the embedded structure.

Parameter Analyzer

The parameter analyzer unit shows the header parameters of each layer and dumps the block data for each decoding process. In the debugging process, users can easily locate the decoding/encoding step where the bugs occurred by checking the decoded parameters.

MPEG Encoder

The MPEG encoder unit of this browsing system can encode an MPEG-1 video sequence according t o different parametric arguments.

Encoding Parameter Adjusting

This unit provides users an interactive capability of adjusting the encoding parameters according t o the visual results.

3 ADJUSTMENT

OF

ENCODING

PARAMETERS

The encoding parameter adjusting unit is a very important component in the browsing system. An interactive user-interface should be provided in this unit. Moreover, unnecessary macroblock re-encoding has t o be avoided. The development of the unit is based on the kernel of MPEG encoding and decoding techniques. On the other hand, the properties of MPEG bitstreams are also taken into consideration. Fig. 2 shows the flow chart of the encoding parameter adjusting unit.

I n t e r a c t i v e GU I Encode MPEG

+

B i t s t r e a m Decode Display

Figure 2: The

flow

chart of encoding parameter adjust-

(3)

By the interactive user-interface, users can easily find the macroblocks that need to be modified and understand the encoding parameters of each macroblock. In our implementation, the encoding parameters of each macroblock in the current frame are stored in a database. When users modify the encoding parameters, the associated values in the database are modified ac- cordingly. Meanwhile, the associated macroblocks are re-encoded, re-decoded, and shown on the associated parts of the screen. Users can thus realize the effect of the adjustment.

While the encoding parameters are modified, the associated macroblocks have to be re-encoded. The original bitstream has t o be decoded before the macroblocks are re-encoded, and the corresponding header parameters are replaced by the new ones, such that the original bitstream structure can be maintained. For the macroblocks that don’t need t o be re-encoded, the original data streams can be placed directly on the new bitstream. The block diagram of the macroblock re- encoding is given in Fig.

3 .

I VLC Decode I

macroblock: forward reference flag, backward reference flag, and current frame flag. The current frame flag of a macroblock is set if that macroblock is modified by users. The forward reference flag and the backward reference flag are set if the forward referenced macroblock and the backward referenced macroblock of the current macroblock are modified respectively. A macroblock has to be re-encoded when one of the three flags is set.

4 SYSTEM FUNCTION AND APPLI-

CATIONS

The feature of the proposed browsing system is that the video bitstreams can be browsed at arbitrary positions. Users can check out the information and play video frames from arbitrary positions of the video bitstreams. The system functions of different functional units are detailed in this section.

4.1 Structure Analyzer

The structure of the neighboring areas of the specified byte address can be dumped on the screen via

1

the structure analyzer unit. In the “Structure” menu

item, users can select the structure layer, including “Pack”, “Packet”, “GOP” (Group Of Picture), “Pic- ture”

,

“Slice”

,

“Macroblock”, and “Block” layers. For example, the picture structure of the loaded bitstream near the byte address 141508 is shown on the screen, as in Fig. 4.

Figure 3: T h e block diagram of macroblock re-encoding.

For the sake of motion compensation [4], if the modified frame is an I-Picture or a P-Picture, it might have been referenced by the other frames. Those frames that reference the modified frame have to be re-encoded. The system have to find out how many macroblocks of the referencing frames should be modified. These macroblocks are re-encoded according to the new modified referenced frame.

In order to know whether a macroblock is affected by the referenced frame, three flags are set for each

121476 U

(4)

Huang, et al.: An Interactive MPEG-1 Browsing System

It can be seen from the figure that the starting byte address of each structure item is shown before that item. The byte address specified by users are listed on the upper right corner of the screen, and the structure item that contains the specified1 byte is displayed in reversed color. Users can change the specified address by pressing the arrow keys UP

(‘(,’’)

and

DOWN (‘($”).

The structure layers can be shifted by pressing L E F T (t)

and R I G H T

(+),

or selecting the new layers via the menu items. Thus, the structure analyzer unit provides users a very convenient interface. Users can analyze the layer structure on any specified address of’the bitstreams.

In addition to pressing the arrow keys, users can also press

a hot key to specified the new address

via a dialog

box.

4.2 Parameter Analyzer

The parameter analyzer unit displays the header parameters and other related information of a structure layer which the specified byte address belongs to. Users can select the structure layer in the “Parameter” menu item, as shown in Fig. 5. The associated information will be shown on the screen. The structure layers can also be shifted by pressing

LEFT

and RIGHT.

4 ‘1

J60t E14

28 14

Figure 5: Parameter analyzing.

When the “Block” layer is selected, the contents of the block which contains the specified byte, are shown on the screen in addition t o the encoding parameters. Users can observe the change of the block coefficients

569

in different processing stages as shown in Fig.

6.

This will be very helpful in the development stage of

MPEG

encoding/decoding systems.

File Structure P n m t e r Playback Dump Cnlor E m o d e l f d j l s t ... . . .

I

:; wqqj@p$w..+mw ,<.

.

.. .

*

.... ... . .

Ryte Address : ?55?7 Block P a r a m t m DE. & I . Hex. CCI. I I

Block start address 7 5 m

b l t g a r l t Ion B

black p o s l t l o n in ~ c m b l o c k

I

z

At‘ter IDCT block cocff l d c n t s : 4 -2 4 6 6 8 -3 1 -8 -4 -5 -2 -3 B - 5 - 2 2 8 9 3 -1 1 2 B 3 3 4 a 5 E -3 -1 5 -1 -5 -5 2 -1 -2 1 B -1 -1 3 0 -3 -4 -1 B -5 -7 -5 -1 -5 -6 -4 12 1 -3 9 7 2 -2 -1

Figure

6:

The c o e f i c i e n t s of the

block

a f t e r

IDCT.

The contents of the coefficients can be switched via hot keys. The hot keys are defined as,

<F2>:

the coefficients after Huffman decoding,

<F3>: the coefficients after inverse quantization,

<F4>:

the coefficients after inverse discrete cosine transformation (IDCT)

,

<F5>:

the contents of the referenced block,

<F6>:

the final result of the block coefficients.

4.3 Video Player

There are two kinds of video playing functions provided by this browsing system. They are “video sequence playing” and “video frame displaying”, which can be selected from the “Playback’ menu item. The “video sequence playing” function can play the whole video bitstream on the screen, and the “video frame displaying” function can display the frame that contains the specified byte on the screen.

The video can be played from a specified frame or within a portion of that video. T h a t video portion can be marked by pressing

Ctrl+S

(start) and

Ctrl+E

(end). Users can select the video portion by the structure analyzer. The video player will play the video portion only.

(5)

While the video is played, users can press <space> t o pause the frames, and press another hot key to display next frames one a t a time.

The maximum resolution provided in this system is

640

x 480 pixels, and two kinds of color modes, 256 colors and true colors, are supported. The color modes can be switched via the “Color” menu item.

4.4 Video Compression

When users have t o generate an MPEG-1 bitstream, they might want to set up the encoding parameters themselves. The system provides a dialog box for the input of the encoding parameters. The operation steps of video compression are listed as follows:

1.

2.

3.

4.

5.

Press Alt+E t o select the “Encode” menu item.

Select the “File I/O” item in the “Encode” menu item, then a pop-up dialog box will show up for inputting the basic information, as shown in Fig. 7. Select the “Parameter” item in the “Encode” menu item to modify the encoding parameters, as shown in Fig. 8.

Select the “Motion Estimation” item in the “En- code” menu item, and select one of the motion estimation algorithms.

Select the “Start Encoding” item in the “Encode” menu item t o generate the MPEG-1 bitstream.

Figure 7: The

“I/O Files dialog box.

Figure 8: The “Encoding Parameter “ dialog box.

4.5 Parameter Adjusting

When the quality of some frame is not good enough, users can improve the quality via this parameter adjusting function. The result can be directly shown on the screen for finely adjusting the video quality. The operation steps are listed as follows:

1.

2 .

3.

4.

5.

Press Alf+F to select the “File” menu item, and select the “Open” item to load the MPEG bitstream file.

Specify the byte address of the frame that needed to be fixed.

Press

Alt+A

t o select the “Adjust” menu item, and select the “Source” item to input the filename of the original video source.

Set the color mode t o true color mode.

Select the “Parameter” item in the “Adjust” menu item, then the frame that needed to be fixed will be shown on the screen as in Fig. 9

The square box in the frame indicates the macroblock that is fixed currently. Users can press the arrow keys to move the square box and fix the other macroblocks. The quantization scale and the motion vectors of the current macroblock specified by the square box are shown on the upper left corner of the screen.

There are lots of functional keys defined as follows,

Page Up/Down: increase/decrease the quantization

(6)

Huang, et al.: An Interactive MPEG-1 Browsing System _{57 1}

Figure 9: The video f m m e that is not fixed yet.

Arrow keys: move the square box or change the mo-

tion vectors,

<Tab>:

switch the functions of arrow keys,

< F l > :

change the motion compensation mode t o forward prediction,

<F2>: change the motion compensation mode to backward prediction,

<F3>: change the motZion compensation mode to in- terpolation prediction,

<F4>:

switch the intra/non-intra encoding mode,

<F5>:

display the original frame,

< E x > :

restore the modification of the encoding parameters.

For example, the specified macroblock is blurred in Fig. 10. It is known from the information shown on the upper left corner of the screen that the specified macroblock is originally encoded in intra mode, and the quantization scale is 31.

When the quantization scale is changed t o 1, the visual quality is improved as shown in Fig. 11.

In addition t o changing the quantization scale, the macroblock encoding type can be also modified. The

~~

Figure 10: Thp vadeo frame wath blurred macroblocks

Figure 11: scale.

The result of decreasing the quantization

encoding type is changed from intra mode t o forward prediction mode in Fig. 12.

While comparing with Figs. 10 and 12, one can find

that the quality of the macroblocks encoded in different

types might be different even though their quantization scales are the same. The quality of the macroblocks encoded in non-intra mode is usually better than that of the ones encoded in intra mode. It is suggested that users can change the macroblock encoding type at first.

It

the result is still not acceptable, the quantization scale can then be changed. However, the drawback is that the bit rate will increase a little.

Users can press the <Enter> key when a frame has been fixed. Because of motion compensation, those macroblocks that reference the fixed macroblocks

(7)

Figure 12: The result of changing the macroblock en- coding type.

should be re-encoded. The system will detect all the frames that reference the fixed frame. The frames that needed to be fixed are displayed on the screen for modification. If users don’t want to fixed the referencing frames, they can press

<FQ>,

then the macroblocks that needed to be fixed will be re-encoded by using the original encoding parameters.

5

CONCLUSION

The proposed MPEG-1 browsing system provides all necessary information of MPEG-1 bitstreams. Users can check the validness of MPEG-1 bitstreams and debug their MPEG-1 codec with the aid of these information. The system also provides an interactive envi- ronment for adjusting the encoding parameters. Users can improve the video quality and explore the relationship of the visual result and the encoding parameters. It is believed that the proposed browsing system is a very useful tool for developing MPEG-1 video related applications.