Section 5.4 Refined Model and Coding Tool Design
5.4.3 Coding Performance
Test Image Sequences and Platform
Table 5-5 The test sequences and their parameters.
Bit rate Frame rate Number Abbreviation Sequence
(K bps) (fps) of frames PSNR
CR2048 crew 2048 60 600 35
CR1024 crew 1024 60 600 32
SC3072 soccer 3072 60 600 35
107
-IC1536 ice 1536 60 480 38
MB768 mobile 768 30 300 25
FM512 foreman 512 30 300 34
FM1024 foreman 1024 30 300 36
FB1024 football 1024 30 90 35
FG768 flower garden 768 30 250 26
ST1024 Steven 1024 30 300 29
Table 5-5 lists the test image sequences (denoted as the ‘1X’ sequences), their target coding bit rates (which are chosen to produce acceptable image quality), peak signal noise ratio (PSNR), and the other parameters. To test the extreme cases, we enlarge the extent of motion by generating some new sequences consisting of the odd frames of the ‘1X’ sequences (denoted as the ‘2X’
sequences) and one quarter frames of the original (denoted as the ‘4X’ sequences). All the sequences are in the CIF (352X288) resolution. The video coding platform in our experiments is an MPEG-4 (SP@L3) encoder. Only the first frame is coded as I frame, and all the remaining frames are coded as P frames. The motion vector search range is 16, the initial quantization step size is set to 15, and the block size is 16x16. The quantization step is adjusted to achieve the desired bit rate. The frame skip and the block skip (macroblock not coded) modes are not in use.
Performance of Pattern Switching Strategy
Fig. 5-25 and Fig. 5-26 show the performances of ERPS, PHS, APS and DL APS, and Fig.
5-27 and Fig. 5-28 show the performances of GRPS, GPHS, AGPS and DL AGPS, when they are tested on the ‘1X’, ‘2X’ and ‘4X’ sequences under the settings given in Table 5-5. In these figures,
‘ASP’ is the average number of search points per block, and ‘PSNR’ is the average frame PSNR of a sequence.
Regarding the ASP performances of the conventional pattern searches in Fig. 5-25, PHS outperforms ERPS in 1 out of ten 1X sequences, 3 out of ten 2X sequences and 7 out of ten 4X sequences. And the computational complexity of our proposed APS and DL APS are usually
108
-below the lower one of ERPS and PHS. In most cases, DL APS has the lowest computation complexity, and APS takes the second place. On average, APS outperforms EPRS by 4.1%, 8.9%
and 15.6% for the 1X, 2X and 4X sequences, and outperforms PHS by 20.2%, 11.7% and 4.7%.
And DL APS outperforms ERPS by 4.3%, 9.3% and 16.2%, and outperforms PHS by 20.5%, 12.1% and 5.2%. In terms of PSNR in Fig. 5-26, the performances of both APS and DL APS are very close to those of FS in all our test sequences. Specifically, their PSNR performances usually are between those of the constituent pattern searches.
Regarding the ASP performances of the genetic pattern searches in Fig. 5-27, GPHS never outperforms GRPS in 1X sequences. Yet, GPHS outperforms GRPS in 1 out of ten 2X sequences and 2 out of ten 4X sequences. And the computational complexity of our proposed AGPS and DL AGPS are usually below the lower one of GRPS and GPHS. In most cases, DL AGPS has the lowest computation complexity, and AGPS takes the second place. On average, AGPS outperforms GPRS by 0.6%, 2.5% and 4.5% for the 1X, 2X and 4X sequences, and outperforms GPHS by 38.0%, 26.1% and 14.5%. And DL AGPS outperforms GRPS by 0.9%, 2.5% and 4.9%, and outperforms GPHS by 38.3%, 26.1% and 14.9%. In terms of PSNR in Fig. 5-28, the performances of AGPS and DL AGPS are very near to those of FS in all our test sequences.
Likewise, their PSNR performances usually are between those of the constituent pattern searches.
Clearly, the adaptive pattern switching strategy is robust. It does not hurt the low motion variance sequences but effectively reduces the computational complexity on the high motion variance sequences. The proposed algorithms outperform their constituent pattern search algorithms in ASP, and their PSNR qualities typically are in-between those of their constituent algorithms.
When we compare the conventional adaptive schemes with the genetic adaptive schemes, the genetic versions are better than their corresponding non-genetic versions in computational complexity. For example, GRPS is better than ERPS, and GPHS is better than PHS. GRPS is an
109
-efficient search algorithm for almost all image sequences. Therefore, the advantage offered by the adaptive switching mechanism is relatively small for the genetic searches. In contrast, the adaptive pattern switching mechanism helps the conventional searches more. Though marginally, the double level strategy further improves in both PSNR and speed.
Note that the sequences with high but regular motions, like ‘flower garden’ (FG768), are considered as moderate motion sequences because we use a very good MV predictor. In our pattern switching schemes, the MV difference to its predictor decides which pattern search to be used. We do not compare our pattern switching algorithms, DL APS or DL AGPS, with the other pattern switching algorithms because our selected constituent pattern searches differ from those used by the other existing pattern switching algorithms. Moreover, the performances of our constituent pattern searches already exceed those of many known pattern switching algorithms.
(a) ASP on 1X sequences
6.00 8.00 10.00 12.00 14.00 16.00
CR2048 CR1024 SC3072 IC1536 MB768 FM512 FM1024 FB1024 FG768 ST1024 Average ERPS
PHS
APS
DL APS
(b) ASP on 2X sequences
6.00 8.00 10.00 12.00 14.00 16.00 18.00 20.00 22.00
CR2048 CR1024 SC3072 IC1536 MB768 FM512 FM1024 FB1024 FG768 ST1024 Average ERPS
PHS
APS
DL APS
110
-(c) ASP on 4X sequences
7.00
CR2048 CR1024 SC3072 IC1536 MB768 FM512 FM1024 FB1024 FG768 ST1024 Average ERPS
PHS
APS
DL APS
Fig. 5-25 The ASP values of applying ERPS, PHS, APS and DL APS on the 1X, 2X and 4X sequences.
(a) PSNR on 1X sequences
25.00
CR2048 CR1024 SC3072 IC1536 MB768 FM512 FM1024 FB1024 FG768 ST1024 Average FS
ERPS
PHS
APS
DL APS
(b) PSNR on 2X sequences
25.00
CR2048 CR1024 SC3072 IC1536 MB768 FM512 FM1024 FB1024 FG768 ST1024 Average FS
ERPS
PHS
APS
DL APS
(c) PSNR on 4X sequences
24.00
CR2048 CR1024 SC3072 IC1536 MB768 FM512 FM1024 FB1024 FG768 ST1024 Average FS
ERPS
PHS
APS
DL APS
Fig. 5-26 The PSNR values of applying FS, ERPS, PHS, APS and DL APS on the 1X, 2X and 4X sequences.
111
-(a) ASP on 1X sequences
5.00
CR2048 CR1024 SC3072 IC1536 MB768 FM512 FM1024 FB1024 FG768 ST1024 Average
ASP GRPS ASP GPHS ASP AGPS ASP DL AGPS
(b) ASP on 2X sequences
5.00
CR2048 CR1024 SC3072 IC1536 MB768 FM512 FM1024 FB1024 FG768 ST1024 Average GRPS GPHS AGPS DL AGPS
(c) ASP on 4X sequences
6.00
CR2048 CR1024 SC3072 IC1536 MB768 FM512 FM1024 FB1024 FG768 ST1024 Average
GRPS GPHS AGPS DL AGPS
Fig. 5-27 The ASP values of applying GRPS, GPHS, AGPS and DL AGPS on the 1X, 2X and 4X sequences.
(a) PSNR on 1X sequences
23.00
CR2048 CR1024 SC3072 IC1536 MB768 FM512 FM1024 FB1024 FG768 ST1024 Average FS GRPS GPHS AGPS DL AGPS
112
-(b) PSNR on 2X sequences
23.00 25.00 27.00 29.00 31.00 33.00 35.00 37.00 39.00 41.00
CR2048 CR1024 SC3072 IC1536 MB768 FM512 FM1024 FB1024 FG768 ST1024 Average FS GRPS GPHS AGPS DL AGPS
(c) PSNR on 4X sequences
23.00 25.00 27.00 29.00 31.00 33.00 35.00 37.00 39.00 41.00
CR2048 CR1024 SC3072 IC1536 MB768 FM512 FM1024 FB1024 FG768 ST1024 Average FS GRPS GPHS AGPS DL AGPS
Fig. 5-28 The PSNR values of applying FS, GRPS, GPHS, AGPS and DL AGPS on the 1X, 2X and 4X sequences.
Discussions
To examine the correctness of the switching strategy, Fig. 5-29 shows the frequency (in percentage) that PHS is chosen when the adaptive pattern schemes, APS and DL APS, are applied to the 1X, 2X and 4X sequences. Fig. 5-30 shows the percentage that GPHS is chosen when the adaptive genetic pattern schemes, AGPS and DL AGPS, are applied to those sequences. In Fig.
5-29, the percentages of using PHS on the 4X sequences are higher than those on the 2X sequences, and in turn, the percentages on the 2X sequences are higher than those on the 1X sequences in both APS and DL APS. This is consistent with our earlier projection that the adaptive algorithms show advantages on fast moving sequences. Similar conclusion applies to the use of GPHS in both AGPS and DL AGPS.
Fig. 5-31 and Fig. 5-32 display the pattern switching thresholds (represented by the dash straight line) and the refined switching index JASP. In these figures, a yellow dot denotes the JASP
of an image frame (equivalently, an MV variance pair) and a cross denotes the JASP of an entire sequence. In Fig. 5-31, the dots in the higher-right part of JASP=0 are increasing as the sequences
113
-are getting faster. Similar situation happens in Fig. 5-32. In Fig. 5-31, the MV variance pairs -are evenly distributed on the two sides of the pattern switching threshold between ERPS and PHS. In contrast, in Fig. 5-32, most MV variance pairs are in the lower-left side of the pattern switching threshold designed for GRPS and GPHS. Consequently, the percentages of using PHS in Fig.
5-29 are higher than that of using GPHS in Fig. 5-30.
0%
Fig. 5-29 The frequency (in percentage) that PHS is chosen when the adaptive pattern schemes, APS and DL APS, are applied to the 1X, 2X and 4X sequences.
0%
Fig. 5-30 The frequency (in percentage) that GPHS is chosen when the adaptive genetic pattern schemes, AGPS and DL AGPS, are applied to the 1X, 2X and 4X sequences.
114
ERPS vs PHS (1X sequence)
variance x
ERPS vs PHS (2X sequence)
variance x
ERPS vs PHS (4X sequence)
variance x
variance y
Fig. 5-31 Pattern switching threshold (dash line), JASP (solid line) and the frame MV variance of the 1X, 2X and 4X sequences when the constituent searches are ERPS and PHS.
0 32 64 96 128 160
GRPS vs GPHS (1X sequence)
variance x
GRPS vs GPHS (2X sequence)
variance x
GRPS vs GPHS (4X sequence)
variance x
Fig. 5-32 Pattern switching threshold (dash line), JASP (solid line) and the frame MV variance of the 1X, 2X and 4X sequences when the constituent searches are GRPS and GPHS.
For our selected image sequences, the adaptive switching methods offer reasonable computation reduction and ensure robust performance in the occasional high motion cases. It provides nearly the best ASP with negligible PSNR degradation. Overall, our design methodology produces a stable and efficient fast MV search scheme that can be used for all types of motion sequences. Indeed, based on the refined ASP prediction model, we can systematically choose the nearly optimal decision threshold. In practical implementation, the non-linear ideal threshold function is approximated by a liner equation.
Section 5.5 Chapter Summary
This study tries to improve the modeling accuracy of the pattern-based motion vector search algorithms. Specifically, we propose the refined weighting function, which is defined as the average number of search points needed to find the best matched point. Based on the QMSB matching error surface assumption, we can analytically calculate the RWF for the search algorithms of our interests. RWF is a better replacement for our previously proposed weighting function. When it is used to predict the ASP performance of a new search algorithm, it reduces the
115
-average prediction errors significantly.
In the model improving process, we also find clues to further speed up the previously proposed genetic pattern search algorithms. The basic idea is that the search direction used in the previous search steps hints us in finding the next matching point. By properly prioritizing the candidate search order, we lower the average computations. Two momentum-directed genetic pattern search algorithms are thus devised. Simulation results show that the modified algorithms offer 5% to 7% average computation reduction when compared with their corresponding genetic pattern searches.
In addition, this study provides a methodology to design a robust and high performance adaptive pattern search algorithms. Our refined analytical model for pattern-based block motion estimations (PBME) serves as the foundation of the entire design process. The refined model can accurately predict the average number of search points (ASP) of a single pattern search. By using the refined model, we re-examine the critical coding tools in the adaptive pattern searches, the decision threshold and the starting point set.
Our proposed design methodology leads to a systematic procedure in choosing the appropriate threshold for selecting the pattern search. Based on the characteristics of the constituent searches, the motion vector variance is chosen as the decision metric. And the threshold function is well approximated by a linear equation to reduce computation. Accordingly, the frame-level switching strategy and the block-level switching strategy are constructed. With different constituent pattern searches, two examples of adaptive pattern search design are presented. One uses the conventional pattern searches and the other uses genetic pattern searches.
A most distinct advantage of the adaptive schemes is their robust performance (in both computation and quality) on both slow and fast motion sequences.
116