In our simulation, the proposed algorithm is simulated in H.264/MPEG-4 AVC [4] with software model JM9.2 [36] using AMD 2.0G Hz and the distortion measure is sum of absolute difference (SAD) which is computed for a 16-by-16 macro-block.
We use twelve famous video sequences [37] to be tested and the simulation environment in JM9.2 is shown as in Table 3. From Table 3, the file format of these video sequences is CIF (352 × 288 pixels) and the search range is ±16 in both horizontal and vertical directions for a 16-16 macro-block. The bit-rate control is turned on to maintain a fixed bit rate of 450k bits/s under displaying 30 frames / s. In Chapter 4, we proposed an adaptive subsample ratio decision to pick the suitable subsample ratio and the adaptive subsample ratio threshold decision support six different threshold values between 16:16, 16:8, 16:4 and 16:2, which are shown as in Table 2. To choose the optimal threshold values from Table 2, we simulate these tested video sequences using these subsample ratio decisions respectively in the same simulation condition and then analysis to decide the optimal threshold values from these decisions based on two factors: average quality degradation (∆PSNRY) and average subsample ratio. The PSNRY is defined as Eq.5 where the frame size is N × M, and denote the Y components of original frame and reconstructed frame at (x; y). The ∆PSNRY is defined as Eq.6 and it means the difference of PSNRY which is calculated by a chosen algorithm and PSNRY which is calculated by using full-search block-matching algorithm (FSBM).
(
x y( ) ( ) (
Testing Video Sequences and Simulation Conditions Video
To demonstrate that the proposed algorithm can adaptively select the suitable subsample ratio to each GOP for a tested video sequence, we analysis the average quality degradation of each GOP using Eq.4 for the video sequence “table” and the results is shown as in Fig 5.1. This case is the same with the Fig 3.4 in chapter 3. But the Fig 5.1 adds the distribution of the proposed algorithm to demonstrate the performance of the proposed algorithm. From Fig 5.1, there exists the stronger
temporal variation between third GOP and seventh GOP, the proposed algorithm can adaptively support higher subsample ratio to efficiently reduce the ∆GOP. Besides, the proposed algorithm can adaptively support lower subsample ratio to save power dissipation without affecting the ∆GOP between eleventh GOP and twenty GOP because of the weaker temporal variation.
Fig 5.1: Te average quality degradation of each GOP for the video sequence
“table”
Table 4 shows the simulation results of PSNRY and ∆PSNRY for these tested video sequences using this threshold decision method. Table 5 shows the simulation results of average subsample ratio and overall average subsample ratio for these tested video sequences using this threshold decision method. Because threshold values of Table 2 can be calculated according to the target of average quality degradation of 0.3 dB, the average quality degradation of 0.3 dB is an important index for all tested video sequences. From Table 4 and Table 5, 90%, 85% and 80% statistics of threshold decision method can satisfy all tested video sequences under the average quality
dB. For the video sequences “Dancer” and “Mother Daughter”, their quality degradations in this 70% method are the same and are equal to 0.36 dB. And the quality degradation in this 70% method of video sequence “Foreman” is equal to 0.33 dB. For “Paris”, it is 0.35 dB. And 0.33 dB is for “Weather”. Although the overall average subsample ratio of 65% statistics of threshold decision method is the lowest, the average quality degradation of it exceeds 0.3 dB too much. For example, the average quality degradations of the sequences “Dancer” and “Foreman” are 0.77dB and 0.59 dB. These quality degradations are not acceptable. For the 70% statistics, we can observe the video sequences of fast motion have the maximum acceptable quality degradation for near 0.3 dB. In this quality degradation, the power consumption is the maximum. We can save the power efficiently. For the other threshold values, they can also keep the quality degradation acceptable. But they waste the power to gain the better quality degradation under 0.3 dB. For the low motion video sequences, the algorithm using the threshold value of 70% statistics can select adaptively the minimum power consumption to save power efficiently. The minimum power consumption is the average subsample ratio 16:3. That contains the number of the first P-frame in the GOP using 16:16 subsample ratio and all the rest P-frame using 16:2 subsample ratio. Therefore, in order to minimize the power consumption of motion estimation and maintain the average visual quality about 0.3dB, threshold values of 70% statistics is the optimal choice for adaptively selecting the suitable subsample ratio.And we save 69.6% power consumption and keep quality degradation under 0.36
dB.
Table 4
Analysis of quality degradation using adaptive subsample ratio decision The adaptive subsample ratio threshold decision
90% 85% 80% 75% 70% 65%
The simulation results of average subsample ratio and overall average subsample ratio
Threshold Decision
Dancer 16:15.55 16:15.55 16:15.55 16:14.43 16:11.75 16: 6.91 Foreman 16:14.32 16:13.31 16:12.93 16:10.61 16:10.24 16: 6.06 Fast
Motion
Flower 16:16.00 16:15.10 16:15.10 16:11.98 16: 8.80 16: 5.12 Table 16: 9.50 16: 9.03 16: 7.17 16: 5.32 16: 4.67 16:3.55
location of the proposed algorithm with the optimal threshold value. We can easily observe the relation between the generic subsample ratio algorithm and the proposed algorithm with the optimal threshold value. For Fig 5.2, the quality degradations of these testing sequences using generic subsample ratio algorithms are strong. The maximum quality degradation is 0.93 dB. It happens in “Dancer” sequence using the 16:2 generic subsample ratio. From Fig 5.3, the proposed algorithm can adaptively maintain ∆PSNRY under the threshold of about 0.3 dB and has lower subsample ratio to substantially save power dissipation than the generic subsample ratio algorithm under the same ∆PSNRY for tested video sequences. For Fig 5.3, the quality degradations of these testing sequences using generic subsample ratio algorithms are light. The maximum quality degradation is 0.33 dB and it is acceptable. It happens in
“Weather” and “Paris” sequences using the 16:2 generic subsample ratio. From Fig 5.3, therefore, the proposed algorithm can select the lowest subsample ratio and maintain ∆PSNRY under the threshold of about 0.3 dB. We can determine the performance of the proposed algorithm with different threshold value from Fig 5.2 and Fig 5.3. The optimal threshold value can make the quality degradation of high and low motion video sequence keep near by 0.3 dB. That will save the maximum power consumption. And for low motion video sequence, the selected subsample ratio is the lowest one, 16:2. The power consumption is the minimum, for 16:3. If the other threshold value is used in the proposed algorithm, the location will be away from 0.3 dB and have no the minimum power consumption case.
Table 6
The PSNRY of the proposed algorithm and generic subsample ratio algorithm
Full Search Block Matching
Generic
PSNRY PSNRY PSNRY PSNRY PSNRY PSNRY PSNRY PSNRY PSNRY
Dancer 33.42 33.24 33.09 32.89 32.72 32.56 32.5 32.49 33.06
The ∆PSNRY of the proposed algorithm and generic subsample ratio algorithm
Full Search Block Matching
Generic
∆PSNRY ∆PSNRY ∆PSNRY ∆PSNRY ∆PSNRY ∆PSNRY ∆PSNRY ∆PSNRY
Dancer -0.18 -0.33 -0.53 -0.7 -0.86 -0.92 -0.93 -0.36
Fig 5.2: The results ∆PSNRY of testing sequences “Dancer“, “Foreman“,
“Flower“, “Table“, “Mother Daughter“ and “Weather“ and the proposed algorithm results location
Fig 5.3: The results ∆PSNRY of testing sequences “Children“, “Paris“,
“News“, “Akiyo“, “Silent“ and “Container“ and “Weather“ and the proposed algorithm results location
The subsample algorithm, also called the pixel decimation algorithm, can be, in general, classified into two categories. One is fixed patterns [11]-[15], and the other is adaptive patterns [16] [17]. For the subsample algorithm using fixed patterns [11]-[15], they have to choose the only subsample pattern. In our Experimental Result, it is obvious that the only subsample pattern is not suitable for every video sequence.
Although the subsample algorithm using the fixed pattern make sure the power consumption is low, they can not keep the quality degradation of all video sequence near 0.3 dB. If we want to keep the quality degradation of all video sequence near 0.3 dB using the fixed pattern, we have to choose the subsample ratio of 16:12. Because the worse case is the “Dancer” video sequence shown in Fig 5.2. In order to make the quality degradation of “Dancer” near 0.3 dB, we choose the 16:12 fixed subsample ratio. But it is waste the power consumption to using the 16:12 fixed subsample ratio in the low motion video sequence. Therefore, we have to using the adaptive subsample ratios in all video sequences. In our proposed algorithm with the optimal threshold value, it is achieved the best tradeoff between the quality degradation and the power consumption. It can keep the quality degradation near 0.3 dB, and save the maximum power consumption at the same time.
In order to save more power consumption, we also can combine our algorithm with some fast algorithms, like [4]-[17]. The quality is also near 0.3 dB. We can make the video sequences keep their quality near 0.3 dB. At the same time, the power comsuption can be redued more. We simulate that our algorithm is combined with FME mode [38] in JM9.2. We compare the proposed algorithm in FME mode [38]
with generic subsample ratio algorithms in FME mode [38]. The PSNRY and
∆PSNRY of the proposed algorithm in FME mode [38] and generic subsample ratio algorithm in FME mode [38] are shown in Table 8, Table 9 and Table 10. Fig 5.4 and Fig 5.5 are similar with Fig 5.2 and Fig 5.3 respectively. Fig 5.2 and 5.3 are result of
acceptable. And the power consumption can be got from the average subsample ratio.
We can save the power consumption up to 73.6% in FME mode [38]. For Fig 5.4, the quality degradations of these testing sequences using generic subsample ratio algorithms in FME mode [38] are strong. The maximum quality degradation is 1.05 dB. It happens in “Dancer” sequence using the 16:2 generic subsample ratio in FME mode [38]. From Fig 5.4, the proposed algorithm can adaptively maintain ∆PSNRY under the threshold of about 0.3 dB and has lower subsample ratio to substantially save power dissipation than the generic subsample ratio algorithm under the same
∆PSNRY for tested video sequences. For Fig 5.5, the quality degradations of these testing sequences using generic subsample ratio algorithms in FME mode [38] are light. The maximum quality degradation is 0.3 dB and it is acceptable. It happens in
“Childern” sequence using the 16:2 generic subsample ratio in FME mode [38]. From Fig 5.5, therefore, the proposed algorithm can select the lowest subsample ratio and maintain ∆PSNRY under the threshold of about 0.3 dB. The situation in FME mode [38] is the same in full search mode. Therefore, we can know that our algorithm can be combined with FME [38] and the result is similar with in full search mode. We can save more power consumption using this method, combined with some fast algorithms.
Table 8
The result of the proposed algorithm in FME mode [38]
Full search in FME mode [38]
Proposed Algorithm Method in FME mode [38]
video Sequence
PSNRY PSNRY ∆PSNRY Subsample Rate Average
Dancer 33.48 33.23 -0.25 16:12.35
Overall Average Subsample Rate 16: 4.22
Table 9
The PSNRY of the proposed algorithm and generic subsample ratio algorithm in FME mode [38]
FME Search Block Matching
Generic
PSNRY PSNRY PSNRY PSNRY PSNRY PSNRY PSNRY PSNRY PSNRY
Dancer 33.48 33.31 33.17 33.01 32.85 32.64 32.47 32.43 33.23
-0.17 -0.31 -0.47 -0.63 -0.84 -1.01 -1.05 -0.25
Foreman -0.06 -0.11 -0.17 -0.21 -0.29 -0.45 -0.69 -0.32
Fast Motion
Flower -0.01 -0.03 -0.06 -0.08 -0.15 -0.25 -0.48 -0.29
Table -0.02 -0.03 -0.06 -0.07 -0.11 -0.17 -0.25 -0.23
Mother
Daughter -0.02 -0.04 -0.07 -0.11 -0.15 -0.19 -0.32 -0.24
Weather -0.01 -0.02 -0.05 -0.09 -0.07 -0.13 -0.27 -0.26
Children -0.06 -0.08 -0.11 -0.15 -0.16 -0.23 -0.3 -0.27
Normal Motion
Paris -0.02 -0.04 -0.05 -0.07 -0.1 -0.17 -0.27 -0.25
News -0.02 -0.03 -0.05 -0.07 -0.09 -0.12 -0.13 -0.12
Akiyo -0.01 -0.02 -0.02 -0.03 -0.06 -0.07 -0.12 -0.11
Silent -0.02 -0.04 -0.05 -0.06 -0.06 -0.07 -0.07 -0.06
Slow Motion
Container -0.01 -0.01 -0.01 -0.02 -0.02 -0.03 -0.05 -0.04
Fig 5.4: The results ∆PSNRY of testing sequences “Dancer“, “Foreman“,
“Flower“, “Table“, “Mother Daughter“ and “Weather“ and the proposed algorithm results location in FME mode [38]
Fig 5.5: The results ∆PSNRY of testing sequences “Children“, “Paris“,
“News“, “Akiyo“, “Silent“ and “Container“ and “Weather“ and the proposed algorithm results location in FME mode [38]
We simulate our algorithm in H.264 software model JM 9.2[36] for two situations, full search and FME [38]. We can keep the quality degradation near 0.3 dB and save the maxium power consumption. For full search in JM 9.2, we save 69.6%
power consumption and keep quality degradation under 0.36 dB. For FME in JM 9.2, we save 73.6% power consumption and keep quality degradation under 0.33 dB.
Therefore, the proposed algorithm can steady the video quality in power-saving situation.
and hence dominates main power requirement in video compression. Lots of published papers [4]-[17] have presented efficient algorithms for motion estimation.
But they don’t consider the influence of the video content. Among these fast algorithms [4]-[17], the subsample algorithm [11]-[17] can not only easily combine with other approaches mentioned above but also reduce the number of matching points with flexibly changing subsample ratio. The reason why we choose the adaptive subsample ratios is because we believe that the subsample ratios should be varying with the video content.
An adaptive motion estimation algorithm with variable subsample ratios has been presented. This proposed algorithm can adaptively select the compatible subsample ratio for each current group of picture (GOP). The proposed algorithm is first to analyze the degree of the object-moving between the first P-frame and I-frame for the current GOP and then adaptively selects the suitable subsample ratio to the current GOP according to analysis result. This proposed algorithm has been successful implemented in H.264 with software model JM9.2. An adaptive subsample ratio threshold decision is used to set the compatible threshold values and get the optimal result. The static science is adopted in the adaptive subsample ratio threshold decision. Experimental results has shown that the proposed algorithm can not only adaptively select the suitable subsample ratio to various video sequences but also maintain ∆PSNRY of 0.36 dB at most to save about 69.6% power consumption of motion estimation in a fixed bit rate control on average. The proposed algorithm can
also easily combine with other fast algorithms which reduce the computational complexity of FSBM. For FME in JM 9.2, we save 73.6% power consumption and keep quality degradation under 0.33 dB. Hence the proposed algorithm is suitable for real-time implementation of high quality and power-saving video applications using a powerful CPU.
Mbits/s: Video,” 1993.
[2] ISO/IEC CD 13818-2—ITU-T H.262 (MPEG-2 Video), “Information technology—Generic coding of moving pictures and associated audio information: Video,” 1995.
[3] “Information Technology—Generic Coding of Audio-Visual Objects”Part 2:
Visual, ISO/IEC 14 496-2 (MPEG-4 Video), 1999.
[4] T. Wiegand, G.J. Sullivan, and A. Luthra, “Draft ITU-T Recommendation H.264 and Final Draft International Standard 14496-10 AVC,.VT of ISO/IEC JTC1/SC29/WG11 and ITU-T SG16/Q” 6, Doc. JVT-G050r1, Geneva, Switzerland, May 2003.
[3] J.R. Jain and A.K. Jan, “Displacement measurement and its application in interframe image coding,” IEEE Trrans. Commun., vol.COM-29, pp.1799-1808, Dec. 1981.
[4] M.J. Chen, L.G. Chen, and T.D. Chiueh, “One-Dimensional Full Search Motion Estimation Algorithm For Video Coding,” IEEE Trans. Circuits Syst. Video Technol., vol.4, no.5, pp.504-509, Oct. 1994.
[5] R. Li, B. Zeng, and M.L. Liou, “A New Three-Step Search Algorithm for Block Motion Estimation” IEEE Trans. Circuits Syst. Video Technol., vol.4, no.4, pp.438-442, Aug. 1994.
[6] J.Y. Tham, S. Ranganath, M. Ranganath, and A.A. Kassim, “A Novel Unrestricted Center-Biased Dimond Search Algorithm for Block Motion
Estimation,” IEEE Trans. Circuits Syst. Video Technol., vol.8, no.4, pp.369-377, Aug. 1998.
[7] C. Zhu, X. Lin, and L.P. Chau, “Hexagon-Based Search Pattern for Fast Block Motion Estimation,” IEEE Trans. Circuits Syst. Video Technol., vol.12, no.5, pp.349-355, May 2002.
[8] W. Li and E. Salari, “Successive Elimination Algorithm for Motion Estimation,”
IEEE Trans. Image Processing, vol.4, no.1, pp.105-107, Jan. 1995.
[9] V.L. Do and K.Y. Yun, “A Low-Power VLSI Architecture for Full-Search Block-Matching Motion Estimation,” IEEE Trans. Circuits Syst. Video Technol., vol.8, no.4, pp.393-398, Aug. 1998.
[10] J.H. Luo, C.N. Wang, and T. Chiang, “A Novel All-Binary Motion Estimation (ABME) With Optimized Haredware Architectures,” IEEE Trans. Circuits Syst.
Video Technol., vol.12, no.8, pp.700-712, Aug. 2002.
[11] M. Bierling, “Displacement estimation by hierarchical block matching,” Proc.
SPIE, vol. 1001, pp. 942–951, 1988.
[12] B. Liu and A. Zaccarin, “New fast algorithms for the estimation of block motion vector,” IEEE Trans. Circuits Syst. Video Technol., vol. 3, pp. 148–157, Apr.
1993.
[13] K. T. Choi, S. C. Chan, and T. S. Ng, “A new fast motion estimation algorithm using hexagonal subsampling pattern and multiple candidate search,” in Proc.
ICIP, 1996, pp. 497–500.
[14] C.N. Wang, S.W. Yang, C.M. Liu and T. Chiang, “A Hierarchical Decimation Lattice Based on N-Queen With an Application for Motion Estimation,” IEEE Signal Processing Letters., vol.10, no.8, pp.228-231, Aug. 2003.
[15] C.N. Wang, S.W. Yang, C.M. Liu and T. Chiang, ”A Hierarchical N-Queen Decimation Lattice and Hardware Architecture for Motion Estimation,” IEEE
algorithm for block-motion estimation,” IEEE Trans. Circuits Syst. Video Technol., vol. 10, pp. 1006–1011, Sept. 2000.
[18] A. V. Oppenheim, R. W. Schafer and J. R. Buck, “Discrete-Time Signal Processing”, Prentice Hall
[19] V. Bhaskaran and K. Konstantinides, “Image and Video Compression Standards:
Algorithm and Architectures”, Kluwer Academic.
[20] Iain E. G. Richardson, “H.264 and MPEG-4 Video Compresion: Video Coding for Next-generation Multimedia”, WILEY.
[21] Swee Yeow Yap; McCanny, J.V., “A VLSI architecture for variable block size video motion estimation” Circuits and Systems II: Express Briefs, IEEE Transactions on [see also Circuits and Systems II: Analog and Digital Signal Processing, IEEE Transactions on] Volume 51, Issue 7, July 2004 Page(s):384 - 389
[22] Heiko Schwarz, and Thomas Wiegand, “The Emerging JVT/H.26L Video Coding Standard,” in Proc. of IBC 2002, Amsterdam, NL, September 2002, invited paper.
[23] Kurceren, R.; Karczewicz, M., “Synchronization-predictive coding for video compression: the SP frames design for JVT/H.26L,” Image Processing. 2002.
Proceedings. 2002 International Conference on , Volume: 2 , 2002
[24] P. Pirsch, “VLSI architectures for video compression—A survey,” Proc. IEEE, vol. 83, pp. 220–246, Feb. 1995.
[25] P. M.Kuhn, “Fast MPEG-4 motion estimation: Processor based and flexible VLSI implementations,” J. VLSI Signal Processing Syst. Signal, Image, Video Technol., vol. 23, pp. 67–92, Oct. 1999.
[26] L. de Vos and M. Schobinger, “VLSI architecture for a flexible block matching processor,” IEEE Trans. Circuits Syst. Video Technol., vol. 5, pp. 417–428, Oct.
1995.
[27] J. F. Shen et al., “A novel low-power full-search block-matching motion- estimation design for H.263+,” IEEE Trans. Circuits Syst. Video Technol., no. 7, pp. 890–897, July 2001.
[28] K. M. Yang and L. Wu, “A family of VLSI designs for the motion ompensation block-matching algorith,” IEEE Trans. Circuits Syst., vol. 36, pp. 1317–1325, Oct. 1989.
[29] D. Hilbert, “Üeber die stetige abbildung einer linie aufein flächenstück,”
Mathematische Annalen, vol. 38, pp. 459–460, 1891.
[30] S. Kamata, M. Niimi, and E. Kawaguchi, “A method of an interactive analysis for multi-dimensional images using a Hilbert curve,” The Transactions of the Institute of Electronics, Information and Communication Engineers D-II, vol.
J77-D-II, pp. 1255–1264, July 1994.
[31] R. J. Stevens, A. D. Lehar, and F. H. Preston, “Manipulation and presentation of multidimensional image data using the Peano scan,” IEEE Trans. Pattern Anal.
Mach. Intell., vol. PAMI-5, pp. 520–526, Sept. 1983.
[32] B. Moghaddam, K. J. Hintz, and C. V. Stewart, “Space-filling curves for image compression,” Proc. SPIE Conf. Automatic Object Recognition, vol. 1471, pp.
414–421, 1991.
[33] A. C. Ansari and A. Fineberg, “Image data ordering and compression using Peano scan and LOT,” IEEE Trans. Consumer Electron., vol. 38, pp. 436–445,
estimation,” IEEE Trans., Circuits Syst. Video Technol., vol. 9, pp. 683–691, Aug. 1999.
[36] http://iphome.hhi.de/suehring/tml/download/old jm/JM9.2.zip [37] http://www.m4if.org/resources.php#section26
[38] Zhibo Chen, Peng Zhou and Yun He, “Fast Motion Estimation for JVT,” Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6) 7th Meeting: Pattaya II, Thailand, 7-14 March, 2003
[39] Hsien-Wen Cheng and Lan-Rong Dung, "A Vario-Power ME Architecture Using the Content-Based Subsample Algorithm," IEEE Transactions on Consumer Electronics, Volume 50, Issue 1, pp.349 - 354, Feb 2004.
[40] Hsien-Wen Cheng and Lan-Rong Dung, "EFBLA: A Two-phase matching algorithm for FAST motion estimation," IEE Electronic Letter, Volume 40, Issue 11, pp.660 - 661, May 2004
[41] Hsien-Wen Cheng and Lan-Rong Dung, "A Power-Aware Motion Estimation Architecture Using Content-based Subsampling," Journal of Information Science and Engineering (accept)
[42] Hsien-Wen Cheng and Lan-Rong Dung, "A Content-Based Methodology for Power-Aware Motion Estimation Architecture," IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing (accept)
[43] Hsien-Wen Cheng and Lan-Rong Dung, "EFBLA: A Two-Phase Matching
Algorithm for Fast Motion Estimation," IEEE Pacific-Rim Conference on Multimedia, Dec. 2002.
[44] Hsien-Wen Cheng and Lan-Rong Dung, “A content-based motion estimation
[44] Hsien-Wen Cheng and Lan-Rong Dung, “A content-based motion estimation