P ROPOSED A LGORITHM D EVELOPMENT ........................................................................................31 T

To efficiently alleviate the aliasing problems in subsample algorithm to maintain the visual quality under the threshold of 0.5 dB for general video sequences, we propose an adaptive motion estimation algorithm using variable subsample ratios and

the proposed algorithm is based on the observation from Fig3.5 ~ Fig.3.10. The spatial variation in a frame is proportion to edge count (EC) and the temporal variation in a frame is in proportion to moving motion block count (MMBC), meaning that it is in inverse proportion to zero motion block count (ZMBC). Therefore, we use one frame as a processing unit and calculate the ZMBC and EC of P-frames in a tested video sequence. Next, we compare ZMBC and EC with threshold values to determine the suitable subsample ratio for the current frame. We recursively execute those steps above, and we can adaptively apply the suitable subsample ratio to each frame in the video sequence and also achieve the target of saving the motion estimation time.

A flowchart of the proposed algorithm is shown Fig.3.1 and the realization procedure of the adaptive motion estimation algorithm using variable subsample ratios is as follows.

Step 1: Setting initial value Set i=1.

We set the initial value in this proposed algorithm. And the proposed algorithm is ready to start

Step 2: Starting

When starting the proposed algorithm, the ith frame of the video sequence is picked out and goes to Step 3.

Step 3: Determining the current frame whether an I-frame or not

If the current frame is an I-frame, the proposed algorithm executes intra-frame coding to encode the current I-frame, and the current frame goes to Step 5; otherwise, the current frame is a P-frame and then goes to Step 4.

We can recognize the I-frame in the video sequence in this step. We don’t change the intra-predication in the proposed algorithm. Hence, the proposed algorithm

uses the same intra-prediction like H.264/MPEG-4 AVC for the I-frame.

Step 4: Adaptively selecting the suitable subsample ratio to the current P-frame The proposed algorithm compares ZMBC and EC of the P-frame with optimal threshold values to adaptively select a suitable subsample ratio and then uses this selected subsample ratio to execute inter-frame coding for the P-frame and then the current P-frame goes to Step 5. In order to guarantee the visual quality is good enough and can achieve the target accuracy. The priority of the compare order is to compare with 16:16 optimal threshold first, the second is to compare 16:8 optimal threshold, the third is to compare 16:4 optimal threshold and the rest use the 16:2 subsample ratio to encode it. The optimal thresholds are in the Table 3.1 and Table 3.2.

Step 5: Determining the current frame whether a last frame or not

If the current frame is the last frame, the procedure goes to Step 6; otherwise, the procedure sets i=i+1 and the next frame goes to Step 3.

Step 6: Ending

If all frames in the current video sequence are encoded, the proposed algorithm is finished. This video is end and all frames in the video sequence have been coded using the proposed algorithm in the H.264/MPEG-4 AVC [4].

start

Selecting a frame from the testing

video sequence

Is this frame a I-frame

Intra prediction coding

Is this frame the last frame?

Fig.3.1 The flowchart of the proposed algorithm

3.2 Define Two Indices ZMBC in Temporal Domain and EC in Spatial Domain

We define the two indices which are ZMBC (zero motion block count) and EC (edge count). The ZMBC can reflect the video sequence spectrum in temporal domain and the EC can reflect that in spatial domain. In the encode system, we get the ZMBC and EC values before encoding, because we have to decide the subsample ratio before encoding. We use the higher subsample ratio for the larger ZMBC. Because of the

larger ZMBC means the frame has a small temporal variation with the reference frame. So, we choose the higher subsample ratio for it to be encoded, and we will not cause the error propagation problem but save the time of motion estimation simultaneously. On the other hand, we use the lower subsample ratio for the smaller ZMBC. The mechanism about how to choose the subsample ratio according to EC index is like the ZMBC index. We use the higher subsample ratio for the smaller EC.

Because the smaller EC means the frame has a small spatial variation. So, we choose the higher subsample ratio for it to be encoded, and we will not cause the aliasing problem but save the time of motion estimation simultaneously. On the other hand, we use the lower subsample ratio for the larger EC to prevent the aliasing problem in the spatial domain.

The ZMBC is the difference between current frame and reference frame. In the beginning, we divide a frame into several blocks. And every block is size of 16×16 called macro-block (MB). Every current macro-block (CMB) is subtracted by the reference macro-block (RMB) which is at the same position in the reference frame to get ZMBC and we only calculate that at the position (0, 0). The basic operation to get the ZMBC is the sum of absolute difference (SAD). In our experience, we choose the threshold is 800 called ThZMBC. When a MB get the SAD is smaller than ThZMBC, it is very likely to be a stationary MB so we define it is a zero motion block (ZMB). And we calculate how many ZMB in a frame, we can get the ZMBC. The lower bound of ZMBC in a frame is zero, but upper bound is the same number of MB in a frame. For example, in CIF (352×288) resolution the ZMBC upper bound is 396 and D1 (720×480) is 1350.

The other index in the spatial domain is EC which is sum of all edge pixels in a frame. Before calculating the EC, we must to extract the edge first. We want to detect where is edge and where is not, so we apply a popular gradient filter called as high

(

pass filter (HPF) Eq.3.1 to do the edge extraction. HPF can let the higher frequency band pass and filter out the lower frequency band. After the procedure of edge extraction, we can get gradient pixel values in a frame. And use the Eq.3.2 to calculate the ThEC. Then the algorithm uses the ThEC value as a condition to pick the edge pixels produced by Eq.3.3. Finally we sum all the edge pixels in a frame to get the EC value.

)

After we get the two indices ZNBC and EC, we will analyze the relation between quality degradation and the two indices at next section.

3.3 Analyze Visual Quality Degradation with Spatiotemporal