Figure 4.9: The MAD mismatch distribution of the experiment 1 within the window size = 3, 5, 8.
4.2.3 Chain Effect of QP and MAD Prediction
In order to analyze the effect of MAD prediction error on our scheme, we survey the distribution of the MAD prediction mismatch ratio in experiment 1. Figure 4.9 shows the distribution results. In Figures 4.9, we find that the larger window size provides more flexibility in frame selection, but introduces more propagation errors. Because the actual MAD is not available during the frame selection, the prediction error chain effect results in the error propagation problem.
From Figure 4.5, we see that there are two factors that affect the performance for the different window sizes. One is the error propagation. The other is the window size.
Figure 4.10 shows the performance of Foreman sequence for the different window sizes.
In the figure, we find that the performance for the different window size are almost the same in low bit-rate coding. In addition, considering the human perceptional system, it is always better to avoid long runs of skipped frames, which may cause an undesirable visual effect. Based on the above-mentioned observations, no matter what the window size to be set ( we set to five here ), our scheme must at least encode one frame in the window.
4.3 Performance Evaluation
After the window size selection analyses have been studied in details, in this section, we evaluate the performance of our proposed scheme. We use
interpolation-quality-Chapter 4. Experiments and Analyses
Foreman
Bitrate (kbps/s)
60 80 100 120 140 160 180
PSNR-Y (dB)
26 28 30 32 34 36
Window Size = 3 Window Size = 5 Window Size = 8
Figure 4.10: The performance of foreman sequence for the window size=3, 5, 8.
oriented frame skipping (IFS) to select the coded frame and then meet the frame rate which we want. In addition, we know that the fixed frame skipping scheme (FFS) and variable frame skipping scheme (VFS) introduced in the previous section. The three schemes are used as baseline for comparison. The experiment process is as follows:
1. Set QP = 24.
2. Encode the test sequences with FFS, VFS, and IFS approaches at frame rate = 15, 10, 7.5, 6, respectively.
3. Obtain the distortion of the coded frames and the coded bit-rate and then mea-sure the distortion of reconstructed test sequences which include the skipped frame reconstructed by frame repetition or motion-compensated frame interpola-tion. This gives a particular combination of rate (R) and distortion (D), an R-D operating point.
4. Repeat step 2 and add 1 to QP until QP = 36.
5. Select the operating points that give the best rate-distortion performance (i.e.
the lowest distortion for a given rate R) and then these operating points form a convex hull.
6. Use the smallest PSNR as quality constraint and the coded bit-rate as rate con-straint.
Through extensive experiments, we obtain the best R-D performance through the proposed scheme to compare the three schemes. Another objective quality measure-ment, such as SSIM, QM, and VQM, is adopted to evaluate the performance besides
Sec 4.3. Performance Evaluation
adopting the PSNR. The R-D performances of the different sequences are given in Figure 4.11-4.14, respectively. Therefore, from the figures, we can find that:
1. Compare the curves produced in the different quality assessments. As shown in Figure 4.11, we find that the R-D performance in PSNR and SSIM are similar, but on the contrary, there are very different trends between the performance in PSNR and NQM, especially for high motion sequences. On the other hand, all schemes show the similar performance with VQM.
2. Compare Figure 4.11(a) with Figure 4.11(e). We find that a tradeoff between spatial and temporal quality is limited in low bit-rate coding. In addition, by comparing Figure 4.12(a) with 4.12(e), we find that frame interpolation can im-prove more in FFS, VFS, and IFS. The main reason is more frames which are skipped in those schemes.
3. Compare the performances of four schemes in Figure 4.11. We find that the performance of the proposed method is superior to the performances of FFS, VFS, and IFS. The similar results can be found in Figure 4.12. Therefore, a good skipped frame selection is still needed.
Chapter 4. Experiments and Analyses
Salesman ( Frame Replication )
Bitrate (kbits/s)
Salesman ( Frame Replication )
Bitrate (kbits/s)
Salesman ( Frame Replication )
Bitrate (kbits/s)
Salesman ( Frame Replication )
Bitrate (kbits/s)
Salesman ( Frame Interpolation )
Bitrate (kbits/s)
Salesman ( Frame Interpolation )
Bitrate (kbits/s)
Salesman ( Frame Interpolation )
Bitrate (kbits/s)
Figure 4.11: The R-D performance comparison of Salesman sequence reconstructed by frame replication and frame interpolation based on the different objective quality
Sec 4.3. Performance Evaluation
Foreman ( Frame Replication )
Bitrate (kbits/s)
Foreman ( Frame Replication )
Bitrate (kbits/s)
Foreman ( Frame Replication )
Bitrate (kbits/s)
Foreman ( Frame Replication )
Bitrate (kbits/s)
Foreman ( Frame Interpolation )
Bitrate (kbits/s)
Foreman ( Frame Interpolation )
Bitrate (kbits/s)
Foreman ( Frame Interpolation )
Bitrate (kbits/s)
Figure 4.12: The performance comparison of Foreman sequence reconstructed by
Chapter 4. Experiments and Analyses
Hall ( Frame Replication )
Bitrate (kbits/s)
Hall ( Frame Replication )
Bitrate (kbits/s)
Hall ( Frame Replication )
Bitrate (kbits/s)
Hall ( Frame Replication )
Bitrate (kbits/s)
Hall ( Frame Interpolation )
Bitrate (kbits/s)
Hall ( Frame Interpolation )
Bitrate (kbits/s)
Hall ( Frame Interpolation )
Bitrate (kbits/s)
Figure 4.13: The performance comparison of Hall sequence reconstructed by frame replication and frame interpolation based on the different objective quality measure-ment.
Sec 4.3. Performance Evaluation
Football ( Frame Replication )
Bitrate (kbits/s)
120 140 160 180 200 220 240 260
PSNR-Y (dB)
Football ( Frame Replication )
Bitrate (kbits/s)
120 140 160 180 200 220 240 260
SSIM
Football ( Frame Replication )
Bitrate (kbits/s)
120 140 160 180 200 220 240 260
New Quality Metric
Football ( Frame Replication )
Bitrate (kbits/s)
120 140 160 180 200 220 240 260
VQM
Football ( Frame Interpolation )
Bitrate (kbits/s)
120 140 160 180 200 220 240 260
PSNR-Y (dB)
Football ( Frame Interpolation )
Bitrate (kbits/s)
120 140 160 180 200 220 240 260
SSIM
Football ( Frame Interpolation )
Bitrate (kbits/s)
120 140 160 180 200 220 240 260
VQM
Figure 4.14: The performance comparison of Football sequence reconstructed by frame replication and frame interpolation based on the different objective quality
mea-CHAPTER 5
Conclusions
In this thesis, we proposed an adaptive frame-skipping scheme which trades the tem-poral quality for the spatial quality, or vice versa, in order to satisfy both the rate and the quality constraints. Among the frames contained in a time window, our approach attempts to determine which should be encoded and what the values of their quanti-zation parameters are. For each admissible frame-skip pattern, its R-D performance is estimated through a D-Q model and a R-Q model, both specifically take into account the effects of frame skipping. As compared with the regular frame skipping and the other previous work, our scheme reveals a significant improvement in R-D performance, especially in fast-motion sequences. Similar results are also observed with the other objective measures, such as SSIM and VQM.
We plan to further extend our study in several directions: (1) to establish a theo-retical foundation for the empirical D-Q model, (2) to alleviate the error propagation in the prediction of the MAD and QP, and (3) to devise an adaptation scheme for the adjustment of window size.
Bibliography
[1] T. Chiang and Y.-Q. Zhang, “A New Rate Control Scheme Using Quadratic Rate-Distortion Modeling,” IEEE Trans. on Circuits and Systems for Video Technology, vol. 7, pp. 246 — 250, Feb. 1997.
[2] A. Vetro, Y. Wang, and H. Sun, “Rate-Distortion Optimized Video Coding Con-sidering Frameskip,” IEEE Int’l Conference on Image Processing, Oct. 2001.
[3] L. Zhuo, X. G. Wang, Z. Wang, D. D. Feng, and L. Shen, “A Novel Rate-Quality Model based H.264/AVC Frame Layer Rate Control Method,” Int’l Conference on Information Communications Signal Processing (ICICS), Dec. 2007.
[4] H. Song and C.-C. J. Kuo, “Rate Control for Low-Bit-Rate Video via Variable-Encoding Frame Rates,” IEEE Trans. on Circuits and Systems for Video Tech-nology, vol. 11, pp. 512 — 521, Apr. 2001.
[5] T. Y. Kuo, “Variable Frame Skipping Scheme Based on Estimated Quality of Non-coded Frames at Decoder for Real-Time Video Coding,” IEICE Trans. on Information and Systems, vol. E88-D, pp. 2849 — 2856, Dec. 2005.
[6] Y. T. Yang, Y. S. Tung, and J. L. Wu, “Quality Enhancement of Frame Rate Up-Converted Video by Adaptive Frame Skip and Reliable Motion Extration,”
BIBLIOGRAPHY
IEEE Trans. on Circuits and Systems for Video Technology, vol. 17, pp. 1700 — 1713, Dec. 2007.
[7] “H.264/AVC Reference Software JM12.3.” http://iphome.hhi.de/suehring/tml/.
[8] Y. Liu, Z. G. Li, and Y. C. Soh, “Adaptive MAD Predication and Refined R-Q Model for H.264/AVC Rate Control,” IEEE Int’l Conference on Acoustics, Speech and Signal Processing (ICASSP), May 2006.
[9] Z. Wang, H. R. S. A. C. Bovik, and E. P. Simoncelli, “Image Quality Assessment:
From Error Visibility to Structural Similarity,” IEEE Trans. on Image Process, vol. 13, Apr. 2004.
[10] “ITS Video Quality Research.” http://www.its.bldrdoc.gov/n3/video/index.php.
[11] M. Pinson and S. Wolf, “A New Standardized Method for Objectively Measuring Video Quality,” IEEE Trans. on Broadcasting, vol. 50, pp. 312 — 322, September 2004.
[12] R. Feghali, F. Speranza, D. Wang, and A. Vincent, “Video Quality Metric for Bit Rate Control via Joint Adjustment of Quantization and Frame Rate,” IEEE Trans. on Broadcasting, vol. 53, pp. 441 — 446, Sep. 2007.