5. Experiments and Analysis
5.2. Frame Level Rate Control Comparison
5.2.2. Frame Bits Variation
0 50 100 150 200 250 300
0 1 2 3 4 5 6
7x 104 Frame Level Bits Variation Comparison Coastguard.cif 30fps 256kBps
Frame No
Bits
First Order RD Model with SAD RC First Order RD Model with ESAD RC AnnexL Rate Control
First Order MB Level RD Model with SAD RC
Fig 14. Frame bits plots for COASTGUARD sequence
0 50 100 150 200 250 300 0
0.5 1 1.5 2 2.5 3
3.5x 104 Frame Level Bits Variation Comparison Foreman.cif 30fps 256kBps
Frame No
Bits
First Order RD Model with SAD RC First Order RD Model with ESAD RC AnnexL Rate Control
SAD MB Level Rate Control
Fig 15. Frame bits plots for FOREMAN sequence
0 50 100 150 200 250 300
0 2 4 6 8 10
12x 104 Frame Level Bits Variation Comparison Stefan.cif 30fps 768kBps
Bits
First Order RD Model with SAD RC First Order RD Model with ESAD RC AnnexL Rate Control
SAD MB Level Rate Control
Fig 16. Frame bits plots for STEFAN sequence
In Fig 14, for three frame level rate control, the differences among them are limited and Annex L has lower variation of frame bit which uses sliding window to remove outlier data points. In end of FORMAN sequence, frame bits of SAD and ESAD has apparent variation because of repeatedly change between two quantization parameters.
In Fig 16, Annex L has sudden vibration, and generally, three methods has similar result.
In addition, SAD_MBL has promising quality and almost the same number of frame bits. Even though SAD_MBL could match target rate very well, there still has some parts which vary violently and should be scene change segment in which motion estimator could not estimate adequate motion vector to lower down error residual
5.2.3. Buffer Status
0 50 100 150 200 250 300
-0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
Frame Level Buffer Fullness Comparison Coastguard.cif 30fps 256kBps
Frame No
Buffer Usage
First Order RD Model with SAD RC First Order RD Model with ESAD RC AnnexL Rate Control
First Order MB Level RD Model with SAD RC
Fig 17. Buffer status plots for COASTGUARD sequence
0 50 100 150 200 250 300 -0.1
0 0.1 0.2 0.3 0.4 0.5 0.6
Frame Level Buffer Fullness Comparison Foreman.cif 30fps 256kBps
Frame No
Buffer Usage
First Order RD Model with SAD RC First Order RD Model with ESAD RC AnnexL Rate Control
SAD MB Level Rate Control
Fig 18. Buffer status plots for FOREMAN sequence
0 50 100 150 200 250 300
-0.2 0 0.2 0.4 0.6 0.8 1 1.2
Frame Level Buffer Fullness Comparison Stefan.cif 30fps 768kBps
Buffer Usage
First Order RD Model with SAD RC First Order RD Model with ESAD RC AnnexL Rate Control
SAD MB Level Rate Control
Fig 19. Buffer status plots for STEFAN sequence
In Fig 17, Fig 18 and Fig 19, its shows ESAD and SAD has stable bits in decoder buffer, and because Annex L rate control uses the idea which limits the number of buffer bits within some range, its buffer status has more different tendency than SAD and ESAD. ESAD has similar variation as SAD, and at scene change point, it has larger variation.
Equivalently, SAD_MBL has the steadiest buffer status like the plot of frame bits variation because of its good approximation of frame bits.
5.3. Macroblock Level Rate Control Comparison
Five macroblock level rate control algorithms, which include SAD_MBL, ESAD_MBL, and TBRC, are used to test their characteristics and performances.
ESAD_MBL is used to solve the issue on SoC platform mentioned in previous section, TBRC is the approach which is suitable for hardware implementation, and SAD_MBL is used again to give comparison between frame level and macroblock level rate control. Another strategies, TM5 [7] and Rho-domain rate control described in section [18], are also taken into account.
5.3.1. PSNR
0 50 100 150 200 250 300
25 26 27 28 29 30 31 32 33
Macroblock Level PSNR Comparison Coastguard.cif 30fps 256kBps
Frame No
PSNR
First Order MB Level RD Model with SAD First Order MB Level RD Model with ESAD Table-Based Rate Control
Rho-Domain Rate Control Test Model 5
Fig 20. PSNR plots for COASTGUARD sequence
0 50 100 150 200 250 300
26 27 28 29 30 31 32 33 34 35
Macroblock Level PSNR Comparison Foreman.cif 30fps 256kBps
Frame No
PSNR
First Order MB Level RD Model with SAD First Order MB Level RD Model with ESAD Table-Based Rate Control
Rho-Domain Rate Control Test Model 5
Fig 21. PSNR plots for FOREMAN sequence
0 50 100 150 200 250 300 22
24 26 28 30 32 34 36
Macroblock Level PSNR Comparison Stefan.cif 30fps 768kBps
Frame No
PSNR
First Order MB Level RD Model with SAD First Order MB Level RD Model with ESAD Table-Based Rate Control
Rho-Domain Rate Control Test Model 5
Fig 22. PSNR plots for STEFAN sequence
Generally speaking, the two rate control methods, ESAD_MBL and TM5, are not very well. TM5 is designed for MPEG-2 video coding which is adequate for high bitrate encoding and there is no syntax limitation in adaptation of quantization parameter, so this might introduce degradation in quality. ESAD_MBL, which is used to remedy the interrupt service time, bases on the modeling units in previous frame to obtain QP for current frame. The main issue is that the actual information like QP, SAD, and number of coding bits could not be acquired, so the previous frame is used to estimate QP and results in low quality.
The three mechanisms, SAD_MBL, Table-based rate control, and Rho-domain rate control, has similar quality. Rho-domain RC applies impressive mapping between QP and percentage of quantization DCT coefficients and has good performance. SAD_MBL, which uses delicate conception, is close to result of Rho-domain RC. Although TBRC has little degradation on PSNR, it still has result close to Rho-domain RC. In addition, TBRC is adequate for hardware implementation and could be designed with efficient circuit.
5.3.2. Frame Bits Variation
0 50 100 150 200 250 300
0 1 2 3 4 5 6
7x 104 Macroblock Level Bits Variation Comparison Coastguard.cif 30fps 256kBps
Frame No
Bits
First Order MB Level RD Model with SAD First Order MB Level RD Model with ESAD Table-Based Rate Control
Rho-Domain Rate Control Test Model 5
Fig 23. Frame bits plots for COASTGUARD sequence
0 50 100 150 200 250 300 0
0.5 1 1.5 2 2.5 3 3.5 4
4.5x 104 Macroblock Level Bits Variation Comparison Foreman.cif 30fps 256kBps
Frame No
Bits
First Order MB Level RD Model with SAD First Order MB Level RD Model with ESAD Table-Based Rate Control
Rho-Domain Rate Control Test Model 5
Fig 24. Frame bits plots for FOREMAN sequence
0 50 100 150 200 250 300
0 2 4 6 8 10
12x 104 Macroblock Level Bits Variation Comparison Stefan.cif 30fps 768kBps
Frame No
Bits
First Order MB Level RD Model with SAD First Order MB Level RD Model with ESAD Table-Based Rate Control
Rho-Domain Rate Control Test Model 5
Fig 25. Frame bits plots for STEFAN sequence
In the point of view in frame bits variation, ESAD_MBL has apparent amplitude, which implies the capability of QP prediction is not good and could not control number of frame bits very well. The variation of SAD_MBL, TBRC, Rho-domain RC, and TM5 are the frame bits similar to the some fixed value, and this implies they could control the output bitrate very well to match the target rate.
5.3.3. Buffer Status
0 50 100 150 200 250 300
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
Macroblock Level Buffer Fullness Comparison Coastguard.cif 30fps 256kBps
Frame No
Buffer Usage
First Order MB Level RD Model with SAD First Order MB Level RD Model with ESAD Table-Based Rate Control
Rho-Domain Rate Control Test Model 5
Fig 26. Buffer status plots for COASTGUARD sequence
0 50 100 150 200 250 300 0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
Macroblock Level Buffer Fullness Comparison Foreman.cif 30fps 256kBps
Frame No
Buffer Usage
First Order MB Level RD Model with SAD First Order MB Level RD Model with ESAD Table-Based Rate Control
Rho-Domain Rate Control Test Model 5
Fig 27. Buffer status plots for FOREMAN sequence
0 50 100 150 200 250 300
-0.2 0 0.2 0.4 0.6 0.8 1 1.2
Macroblock Level Buffer Fullness Comparison Stefan.cif 30fps 768kBps
Frame No
Buffer Usage
First Order MB Level RD Model with SAD First Order MB Level RD Model with ESAD Table-Based Rate Control
Rho-Domain Rate Control Test Model 5
Fig 28. Buffer status plots for STEFAN sequence
In Fig 26, there is large gap between TM5 and other approaches. This results from the large error of first I-frame and the different design for calculating target frame bit. Other methods use similar updating policies for frame bits, so the difference is small. Such as the plots in frame bits variation, ESAD_MBL has large amplitude compared to other approaches, and SAD_MBL has the most stable variation which is close to some fixed value in FORMAN and COASTGUARD sequences and almost one straight line in STEFAN sequence. Because SAD_MBL distributes the as equivalent number of bits as possible among all macroblocks, it can attain the target number of frame bits more correctly.
5.4. Comparisons Among Sequences
Every algorithm described in previous section is used to test its performance.
All sequences are CIF resolution and encoded at 30 frames per second.
5.4.1. PSNR and Output Bitrate
Following tables show the PSNR and output bitrate using different sequences using different target bitrates.
Sequence
Tab 2. PSNR & Output Bitrate for Frame Level Rate Control
Sequence SAD_MBL ESAD_MBL TBRC RhoRC TM5
Foreman Tab 3. PSNR & Output Bitrate for Macroblock Level Rate Control
According to Tab 2 and Tab 3, use the approach, first order R-D model w/SAD (SAD), as the referenced rate control. It could be observed that on average the PSNR of frame level rate control will be about 02 to 0.7 dB higher than macroblock level rate control. This results from the intrinsic differences between these two differences of rate control mechanisms in quantization and R-D modeling unit.
In frame level rate control algorithms, proposed first order R-D model w/ESAD (ESAD) gives well PSNR and output bitrate compared to first order R-D model w/SAD (SAD) and Annex L rate control. Generally speaking, there is no obvious difference among these three approaches.
In macroblock level rate control algorithms, proposed first order macroblock level R-D model w/ESAD (ESAD_MBL) do not give promising result because of the difficulty in estimating actual number of coding bits. In addition, by means of fine analysis in specific domain, the Rho-domain rate control gives the better performance in most sequences. Table based rate control could get acceptable result in employing a lookup table to get quantization parameters, and in this way it could be implemented into hardware circuit. The result of first order macroblock level R-D model w/SAD (SAD_MBL) is very close to Rho-domain RC because of its excellent ability of controlling frame bits to match target rate.
5.4.2. Mean and Standard Deviation for PSNR Variation
Following tables show the average and standard deviation for PSNR using different sequences at different target bitrates. In the entries of the following tables, the first value is average of PSNR, and the second one is standard deviation of PSNR is.
Sequence
Tab 4. Mean & Standard Deviation for PSNR Variation for Frame Level Rate Control
Sequence Target
SAD_MBL ESAD_MBL TBRC RhoRC TM5
Foreman
Tab 5. Mean & Standard Deviation for PSNR Variation for Macroblock Level Rate Control
To maintain quality as stable as possible is an important thing rate control needs
In Tab 4 of frame level rate control, there is no very apparent difference among these three approaches. Annex L rate control has smallest standard deviation than other two methods for almost all sequences. Proposed first order R-D model w/ESAD (ESAD) has a little larger deviation than first order R-D model because the action of updating parameter for true SAD from motion estimator, but the performance is still acceptable.
In Tab 5 of macroblock level rate control, first order macroblock level R-D model (SAD_MBL) has the average PSNR very close to Rho-domain RC, and so is the standard deviation. The result of table based rate control is a little lower than Rho-domain RC, but it is still acceptable due to the great reduction for hardware implementation. Test Model 5 has not good testing result, and even the worst one among all rate control algorithms.
5.4.3. Mean and Standard Deviation for Bits Variation
Following tables show the average and standard deviation for bits using different sequences at different target bitrates. In the entries of the following tables, the first value is average of frame bits, and the second one is standard deviation of frame bits is.
Tab 6. Mean & Standard Deviation for Frame Bits Variation for Frame Level Rate Control
Sequence Target
SAD_MBL ESAD_MBL TBRC RhoRC TM5
Foreman
Tab 7. Mean & Standard Deviation for Frame Bits Variation for Macroblock Level Rate Control
Low standard deviation of frame bits is also required because large variation of frame will make the overall system unstable, even crash.
In the table of frame level rate control Tab 6, Annex L has lower standard deviation than other two methods. Standard deviation of proposed first order R-D model w/ESAD (ESAD) is similar to first order R-D model w/SAD (SAD), and a little lower than Annex L.
In the table of macroblock level rate control Tab 7, as the same reason mentioned before, first order R-D macroblock level model w/SAD (SAD_MBL) has the lowest standard deviation of frame bits except to STEFAN sequence because of its concentration on controlling frame bits. Table based rate control performs very well in some sequences like FOREMAN and MOBILE, and in others, its performance is similar to Rho-domain RC. The proposed algorithm of first order R-D macroblock level model w/ESAD (ESAD_MBL) is still not as stable as expectation, and some issues are needed to be solved.
6. Conclusion and Future Work
This thesis provides two different solutions to rate control implementation on SoC platforms. For hardware/software co-design approaches, out-of-loop rate control algorithm is proposed to reduce communication overhead between processor cores. In this scheme a simple approximation to SAD without motion estimation is derived and applied to frame-level and macroblock-level rate controls. Further, a novel table-lookup based rate control which is adequate for hardware implementation is proposed for pure hardware video encoder implementation.
6.1. Discussions
Several rate control algorithms are discussed in this paper. Two first order R-D model based algorithms which use true SAD as complexity measure, including frame level (SAD) and macroblock level (SAD_MBL), are used as start points. Then, two out-of-loop rate control algorithms, for frame-level (ESAD) and macroblock-level (ESAD_MBL) RC, are designed for co-design approach for video encoders. ESAD method has competitive performance compared to SAD method and MPEG-4 Annex L rate control. ESAD_MBL method does not performance well compared to SAD_MBL method, Rho-domain RC, and TM5 because of absence of actual coding bits and complexity for each MB.
Moreover, a new rate control (TBRC) is proposed to provide the ability to implement rate control algorithm into hardware circuit directly. There are many advantages to this method. Simply put, it has low complexity and is able to model sophisticated R-D curves that are common in real video sequences. Theoretically, it can do a better job than low-order curve fitting R-D modeling methods that are commonly used in video encoders.
6.2. Future Work
6.2.1. Design Better Measure for Frame-Level or MB-Level Com plexity
In order to estimate true complexity (sum of absolute difference used here), in section 3.2.1 a simple measure, which computes mean and deviation within specific region, is used. Obviously, if better measure could be used to predict true complexity, the performance will be promoted undoubtedly. For example, optical flow-based techniques [22] could be applied, and it may provide better estimation of true complexity.
6.2.2. Use More Sophisticated R-D Model
Proposed algorithms apply first order R-D model to solve issues on SoC platform. Even though it is better to use simple R-D model in MCU, if a sophisticated R-D model is used, the performance can be improved further.
7. Reference
[1] ISO/IEC/JTC1/SC 29/WG 11 N4350, INFORMATION TECHNOLOGY – CODING OF AUDIO-VISUAL OBJECTS – Part 2: Visual.
[2] ITU-T Rec. H.264 / ISO/IEC 11496-10, “Advanced Video Coding”, Final Committee Draft, Document JVT-E022, September 2002
[3] K. Ramchandran, A. Ortega, and M. Vetterli, “Optimal Trellis-based Buffered Compression and Fast Approximations,” IEEE Trans. Image Processing,, Vol. 3, pp.
26-40, Jan. 1994.
[4] K. Ramchandran, A. Ortega, and M. Vetterli, “Bit allocation for dependent quantization with application to multisolution and MPEG video coders,” IEEE Trans. Image
Processing,, Vol. 3, pp. 533-545, Sep. 1994.
[5] J. Choi and D. Park,, “A stable feedback control of the buffer state using the controlled langrange multiplier method,” IEEE Trans. Image Processing,, Vol. 3, pp. 546-558, Sep.
1994.
[6] Wei Ding and Bede Liu, “Rate Control of MPEG Video Coding and Recording by Rate-Quantization Modeling,” IEEE Trans. On Circuit and System for Video Tech., Vol.
6, no. 1, pp. 12-20, Feb. 1996.
[7] ISO/IEC/JTC1/SC29/WG11, “Test Model 5,” JTC1/SC29/WG11 Coding of Moving Pictures and Associated Audio, MPEG 1994
[8] Tihao Chiang and Ya-Qin Zhang, “A New Rate Control Scheme Using Quadratic Rate Distortion Model,” IEEE Trans. On Circuit and System for Video Tech., Vol.7,
pp.246-250, Feb. 1997.
[9] H. M. Hang and J. J. Chen, “Source model for transform video coder and its
application-Part I: Fundamental theory,” IEEE Trans. On Circuit and System for Video Tech., Vol.7, pp.287-298, Apr. 1997.
[10] Liang-Jin Lin and Antonio Ortega, “Bit-Rate Control Using Piecewise Approximated Rate-Distortion Characteristics,” IEEE Trans. On Circuit and System for Video Tech., Vol.8, no.4, pp.446-459, Aug. 1998.
[11] Jordi Ribas-Corbera and Shawmin Lei, “Rate Control in DCT Video Coding for Low-Delay Communications,” IEEE Trans. On Circuit and System for Video Tech.,
IEEE Trans. On Circuit and System for Video Tech., Vol.10, no.7, pp.1154-1158, Oct.
2000.
[13] J. Ribas-Corbera and D. L. Neuhoff, “Optimizing block size in motion-compensated video coding,” J. Electron. Imaging, Vol.7, pp. 155-165, Jan. 1998
[14] A. Gersho and R. M. Gray, Vector Quantization and Signal Compression. Norwell, MA:
Kluwer Academic, 1992, Ch. 6.
[15] E. Murata et al, “Study of relationship between quantizer step size and coded bit amount for video coding,” IEICE Spring Conf., D-190, 1994.
[16] ISO/IEC JTC 1/SC 29/WG 11, 14496-2:2002 Information Technology – Coding of Audio-Visual Objects – Part 2: Visual 3rd Ed., N5546, Pattaya, Thailand, March 2003.
[17] Hwangjun song and C. C. Jay Kuo, “Rate Control for Low-Bit-Rate Video via
Variable-Encoding Frame Rates,” IEEE Trans. On Circuit and System for Video Tech., Vol.11, no.4, pp.512-521, Aug. 2001.
[18] Zhihai He and Sanjit K. Mitra, “Optimum Bit Allocation and Accurate Rate Control for Video Coding via ρ-Domain Source Modeling,” IEEE Trans. On Circuit and System for Video Tech., Vol.12, no.10, pp.840-849, Oct. 2002.
[19] H-C Fang, T-C Wang, Y-W Chang and L-G Chen, “Hardware Oriented Rate Control Algorithm and Implementation for Realtime Video Coding,” ICASSP '03, Vol. 2, pp. 6-10 April 2003.
[20] Byung Cheol Song and Kang Wook Chun, “A Virtual Frame Rate Control Algorithm for Efficient MPEG-2 Video Encoding,” IEEE Trans. On Consumer Electronics, Vol.49, no.2, pp.460-465, May. 2003.
[21] Ashish Jagmohan and Krishna Ratakonda, “MPEG-4 One-Pass VBR Rate Control for Digital Storage,” IEEE Trans. On Circuit and System for Video Tech., Vol.13, no.5, pp.447-452, May. 2003.
[22] Berthold K. P. Horn and Brian G. Schunk, “Determining Optical Flow,” Artificial Intelligence, Vol. 17, no. 1-3, pp. 185-204, 1981.