D4 M ODEL

4.2.1 D4 Encoder

Figure 4.1 Encoder Architecture of D4 Model.

17

D4 model splits the AC coefficients in an alternative rotating order. Figure 4.1 shows the overall encoder architecture of the D4 model. After the prediction error is transformed by integer DCT, the rotating splitter is performed.

In Figure 4.1, except the rotating splitter block, which is between DCT and quantization block, all the other parts of the system are basically the same as conventional H.264/AVC encoding loop. There are four data paths output from the rotating splitter block, each path is for one descriptor and contains one-quarter of the original information, that is, one of every four consecutive AC coefficients in zig-zag scanning order is assigned to each descriptor. The detailed assignment algorithm will be introduced latter. After the quantization is performed, the quantized data on the four paths are separately entropy encoded to four bit-streams. The inverse quantization is performed on all four descriptors, and then the four split data paths are merged into a single one for reconstruction.

The rotating splitter performs AC coefficient splitting based on the smallest block type in H.264/AVC: 4x4 block, since the integer DCT is a 4x4 2-dimensional transform, the splitting is based on 4x4 block.

For each inter residual macroblock, 16 4x4 block are processed by rotating splitter in the order depicted in Figure 4.2.

0 1 4 5

2 3 6 7

8 9 12 13

10 11 14 15

Figure 4.2 4x4 Block Processing Order in a Macroblock

18

We label the 15 AC coefficients in a 4x4 block with AC0, AC1, AC2….., AC15.

Then, for each 4x4 block, the rotating splitter split the 15 AC coefficients in a way that alternatively assigned the coefficients to the four descriptors in the zig-zag scanning order: the first coefficient is assigned to the first descriptor and the second is assigned to the second …, etc. Figure 4.3 illustrates the assignment of each AC coefficient, and the number in the block is the descriptor number that the coefficient is assigned.

DC 0 1 2 3 ‧‧‧ 0 1 2

Figure 4.3 AC Coefficient Assignment in a 4x4 Block.

There is a problem in the alternative assignment algorithm, that is, the first descriptor always carries the lowest frequency coefficient in consecutive four coefficients, and the fourth descriptor carries the highest frequency coefficient. The lower frequency coefficients carry more energy, which is the characteristic of DCT, and is more important. As a result, the quality of the four descriptors will not be balanced: the first descriptor has the best quality, while the fourth descriptor has the lowest quality. The quality of each descriptor is not the same, which violates the principle of MDC discussed in chapter 1.

To address the problem, the rotating splitter rotates the coefficient assignment among descriptors, thus generates four types of 4x4 block: A, B, C and D. Figure 4.4 illustrates the four types. The number in each block indicates the descriptor number that the coefficient is assigned. As Figure 4.4 shows, Type A begins by assigning AC0 to descriptor 0; type B assigns AC0 to descriptor 1; type C assigns AC0 to descriptor 2 and type D assigns AC0 to descriptor 3. The four types of 4x4 blocks are equally distributed inside each 16x16 macroblock in order to make the resulting descriptors

Zig-zag order: AC0 AC1 AC2 AC3 AC13 AC14 AC15

19

have balanced quality, as shown in Figure 4.5.

Figure 4.4 Four 4x4 Block Types the Rotating Splitter Generates

Figure 4.5 4x4 Block Type Distribution in a Macroblock

Through this type of assignment, the error concealment described in the next sub-section can be utilized efficiently.

4.2.2 Error Concealment in D4

The decoder is responsible for decoding and merging the received descriptors.

The D4 decoding process of any one of the four descriptors is the same as that of the conventional H.264/AVC decoder, except the error concealment function which will be discussed later.

When two or more descriptors are received, the decoder merges the coefficients before inverse DCT transform of a 4x4 block. This could be done by simply adding the coefficients in the same position from different descriptors.

20

For any lost descriptor, the error concealment is done by utilizing AC coefficient prediction through neighboring 4x4 blocks, since adjacent blocks have spatial correlation with each other. The coefficient prediction can take the advantage of the proposed 4x4 block type distribution shown in Figure 4.5, where since the types of adjacent 4x4 blocks are different, the coefficients in the same zig-zag order of neighboring blocks must belong to different descriptors and have very little chance to lose simultaneously. Therefore, error concealment is efficient through neighboring blocks. Figure 4.6 shows the prediction direction, which indicated by the four arrows.

The lost coefficients in type-A are predicted from type-B block, the lost coefficients in type-B block are predicted from type-D block, and so on.

Figure 4.6 4x4 Block Coefficient Prediction Direction

Figure 4.7 shows an example for error concealment of the lost descriptor 3. As we can see, the position labeled “X” means these coefficients are assigned to descriptor 3 and are lost. The left-top block is type A, and the lost coefficients in this block can be predicted from right-top block of type B, where the coefficients of the corresponding positions are assigned to descriptor 0, which is not lost, thus the three coefficient are copied from type-B block to type-A block. Similarily, the coefficients belonging to descriptor 1 in right-bottom type-D block are used to conceal the coefficients labeled „X‟ in type-B block, and etc.

21

Figure 4.7 Coefficient Prediction and Descriptor 3 is Lost