The impact of starting points or initial points on fast search algorithms have been studied by many researchers such as [14], [16], [30] and [31]. Typically the starting point is predicted by using a combination of the MVs of a few neighboring blocks. The most probable MV estimated by this type of MV predictor is used as the starting point for PBME algorithms. Although many MV predictors have been proposed, most of them are derived based on heuristic experiments. We like
57
-to design a criterion that evaluates the effectiveness of MV predic-tors and propose a systematical approach that constructs the optimal Starting Point Set (SPS). The DL AGPS and DL APS discussed in the previous section are the search algorithms used to test our starting point set in this section.
We assume that the proposed PBME model (first method in Section 3.3 [54]) is valid for different starting point selection. Then, because the MV field acquired by FS is fixed for a given video sequence, a different starting point only does a translational shift on the motion vector distribution. Given two starting points, SP1 and SP2, their difference in ASP (EASP) can be represented by (4.11).
∑
∈Let SP2 be a fixed starting point for comparison purpose; (4.11) thus becomes (4.12), in which η is a constant.
Rearrange (4.12), we obtain GASP defined by (4.13), which is proportional to the ASP using SP1.
Thus, it is used as the performance assessment criterion for starting point evaluation.
/ 1
)
(E C
GASP = ASP+η (4.13)
Because WF is fixed for a specific algorithm and only SFS_SP1(x,y) may vary, GASP is a function of MV characteristics. Herein, the MV characteristics are either the MV variances or MV standard deviations calculated from the MV w.r.t. a specific starting point (SP1). And the MVs are acquired by using FS on the selected sequence.
Fig. 4-12 shows the MV candidates that are often considered in starting point selection. They are the MVs of the neighboring spatial/temporal neighboring blocks. And the most commonly used mathematical function includes median(.) and mean(.). Combining them together, (4.14)-(4.25) are some representative MV predictors under our investigation.
58
pred median mean MV MV MV
MV = (4.18)
pred median mean MV MV MV
MV = (4.19)
pred median mean MV MV MV MV
MV = (4.20)
pred median mean MV MV MV MV MV
MV = (4.22)
Table 4-6 to Table 4-9 show the GASP of some of the well-known and best performed MV predictors ((4.14)-(4.25)) applied to the test sequences using the weighting functions of ERPS, PHS, GRPS and GPHS, respectively. We find that MVpred21 (mean value of MVU, MVL, and MVP), MVpred23 (mean value of MVU, MVL, and two MVP) and MVpred28 (mean value of MVPU, MVPD, MVPL, MVPR, MVPUL, MVPUR, MVPDL, MVPDR, and MVP) have the smallest average GASP among all the MV predictors. Together with the well-known PMV (MVpred16, identical to (2.2)) and ZMV (MVpred15, identical to (2.1)), these 5 MV predictors form the candidate set for the starting points.
59
-MVC
MVU MVUR
MVL MVUL MVP
MVPD MVPDR
MVPP MVPR
MVPUR MVPU
MVPUL
MVPL
MVPDL
Current Frame Previous Frame
Previous Previous Frame
Fig. 4-12 Motion vector predictor candidates in the current frame, the previous frame and the frame before previous frame.
Table 4-6 GASP of the MV predictors applied to the test sequences using WFERPS.
MV CT256 CT40 HL40 MD96 CG112 FM512 FM1024 FB1024 FG768 ST1024 Average pred15 6.22 9.99 9.87 10.07 8.78 14.59 13.75 32.21 9.56 19.84 13.49 pred16 6.47 10.73 10.67 10.71 8.55 11.47 10.20 30.84 10.16 17.46 12.73 pred21 6.31 10.52 10.51 10.47 8.50 10.66 9.47 27.92 9.90 15.24 11.95 pred23 6.33 10.51 10.66 10.60 8.54 10.77 9.60 29.06 9.89 15.40 12.14 pred27 6.22 9.98 10.20 10.43 8.20 11.08 10.02 30.80 9.67 15.86 12.25 pred28 6.29 10.37 10.54 10.60 8.38 10.95 9.84 28.56 9.88 15.15 12.06 MIN 6.22 9.98 9.87 10.07 8.20 10.66 9.47 27.92 9.56 15.15 11.95
Table 4-7 GASP of the MV predictors applied to the test sequences using WFPHS.
MV CT256 CT40 HL40 MD96 CG112 FM512 FM1024 FB1024 FG768 ST1024 Average pred15 9.49 11.05 11.00 11.08 10.55 13.03 12.66 21.59 10.87 15.59 12.69 pred16 9.59 11.36 11.34 11.35 10.45 11.67 11.14 20.88 11.12 14.41 12.33 pred21 9.53 11.27 11.27 11.25 10.43 11.33 10.83 19.41 11.01 13.36 11.97 pred23 9.54 11.27 11.33 11.31 10.44 11.38 10.89 19.98 11.01 13.43 12.06 pred27 9.49 11.05 11.13 11.23 10.31 11.51 11.06 20.87 10.91 13.66 12.12 pred28 9.52 11.21 11.28 11.31 10.38 11.45 10.98 19.73 11.00 13.32 12.02 MIN 9.49 11.05 11.00 11.08 10.31 11.33 10.83 19.41 10.87 13.32 11.97
Table 4-8 GASP of the MV predictors applied to the test sequences using WFGRPS.
MV CT256 CT40 HL40 MD96 CG112 FM512 FM1024 FB1024 FG768 ST1024 Average
pred15 5.30 6.28 6.27 6.32 5.96 7.70 7.43 14.39 6.18 9.27 7.51
pred16 5.36 6.48 6.50 6.50 5.90 6.73 6.36 13.87 6.35 8.53 7.26
pred21 5.32 6.43 6.45 6.43 5.89 6.50 6.16 12.69 6.28 7.83 7.00
pred23 5.32 6.42 6.50 6.47 5.90 6.53 6.20 13.15 6.28 7.88 7.06
pred27 5.30 6.28 6.36 6.42 5.81 6.62 6.31 13.83 6.21 8.01 7.11
pred28 5.31 6.38 6.46 6.47 5.85 6.58 6.26 12.93 6.27 7.79 7.03
MIN 5.30 6.28 6.27 6.32 5.81 6.50 6.16 12.69 6.18 7.79 7.00
60
-Table 4-9 GASP of the MV predictors applied to the test sequences using WFGPHS.
MV CT256 CT40 HL40 MD96 CG112 FM512 FM1024 FB1024 FG768 ST1024 Average pred15 9.16 9.68 9.67 9.69 9.52 10.34 10.22 13.20 9.62 11.20 10.23 pred16 9.20 9.79 9.78 9.78 9.48 9.89 9.71 12.96 9.71 10.80 10.11 pred21 9.18 9.76 9.76 9.75 9.48 9.78 9.61 12.47 9.67 10.45 9.99 pred23 9.18 9.76 9.78 9.77 9.48 9.79 9.63 12.66 9.67 10.48 10.02 pred27 9.16 9.68 9.71 9.74 9.44 9.84 9.69 12.96 9.64 10.55 10.04 pred28 9.17 9.74 9.76 9.77 9.46 9.82 9.66 12.58 9.67 10.44 10.01
MIN 9.16 9.68 9.67 9.69 9.44 9.78 9.61 12.47 9.62 10.44 9.99
We adopt the initial candidate set approach. That is, the proposed BME algorithm examines all MV candidates in the candidate set and then uses the best candidate as the starting point for the subsequent search procedure. As shown by (4.26), the total search point number (NTSP) equals to the size of starting point set (NSPS) plus the number of average search points (NASP) produced by a specific search algorithm minus one, where “minus one” represents the initial point count included in the NASP.
−1 +
= SPS ASP
TSP N N
N (4.26)
A well-designed starting point set should decrease NASP more than the increased size of starting point set (NSPS). We develop a systematic approach to find the optimal SPS. It is an add-on approach. At the beginning, there is only one MV in the SPS. We calculate its NTSP using a certain search algorithm. After a number of simulations, we retain a few best performers. We then add a second MV into each of these sets and evaluate their NTSP again. We continue adding new points until the NTSP does not decrease with additional MV in that set. This procedure is described by the flow chart in Fig. 4-13. In theory, this procedure does not guarantee that the final best set is globally optimal because our set is progressively constructed. However, our experiments indicate that the results are quite good.
61
-calculate GASP
of M V predictors
Select the M V predictors according GASP
to form the candidate set
A dd one M V from candidate set to the starting point set
if NTSP increases
Calculate NTSP
N
Finish selecting SSP
Y Begin
Fig. 4-13 The flow chart of constructing SPS.
We show the performance of DL APS with various starting point sets here. Due to limited space, only the better performed ones are shown. Table 4-10 is the results of DL APS with one starting point. We find that DL APS with MVpred21, MVpred23 or PMV are the best. Use each of these three MVs as the first element in three separated sets, we add a second MV. Their performance is shown in Table 4-11. The better performer for both speed and quality is the set of MVpred23 plus PMV. Based on this selection, we add one more MV into the starting point set and the results are on Table 4-12. The set of MVpred23 plus PMV and MVpred28 is the best. If we add one more MV into SPS, NTSP increases. Therefore, our SPS for DL APS is {MVpred23, PMV, MVpred28}. The order in the set is the order in search. We repeat the same SPS identification procedure for DL AGPS and the best result is PMV plus MVpred23, which gives an average ASP of 6.61. The experimental results used in constructing the set for DL AGPS are shown in Table 4-13 and Table 4-14.
Table 4-10 The performance of DL APS with only one starting point.
Normal PMV Pred21 Pred23 Pred28 ZMV
Sequence ASP PSNR ASP PSNR ASP PSNR ASP PSNR ASP PSNR
CT256 5.75 39.49 5.58 39.57 5.57 39.60 5.50 39.58 5.48 39.55
CT40 6.97 32.15 6.89 32.11 6.71 32.49 6.51 32.70 6.47 32.76
HL40 7.31 34.55 7.27 34.54 7.33 34.48 7.15 34.54 7.10 34.55
MD96 6.80 40.10 6.76 40.12 6.73 40.11 6.79 40.15 7.01 40.07
62
Table 4-11 The performance of DL APS when there are two points in the starting point set.
Normal PMV
+Pred21 PMV
+Pred23 PMV
+Pred28 PMV
+ZMV Pred21
+Pred23 Pred21
+PMV Pred21
+Pred28 Pred21
+ZMV Pred23
+Pred21 Pred23
+PMV Pred23
+Pred28 Pred23 +ZMV
Table 4-12 The performance of DL APS when there are three points in the starting point set, MVpred23 is the first starting point, and PMV is the second starting point.
Normal Pred23+PMV+Pred21 Pred23+PMV+Pred28 Pred23+PMV+ZMV
Sequence ASP PSNR ASP PSNR ASP PSNR
Table 4-13 The performance of DL AGPS with only one starting point.
Normal PMV Pred21 Pred23 Pred28 ZMV
63
-Average 6.95 33.62 6.88 33.64 6.90 33.65 7.14 33.64 8.45 33.34
Table 4-14 The performance of DL AGPS when there are two points in the starting point set.
Normal Pred21
+Pred23 Pred21
+PMV Pred21
+Pred28 Pred21
+ZMV PMV
+Pred21 PMV
+Pred23 PMV
+Pred28 PMV
+ZMV Pred23
+Pred21 Pred23
+PMV Pred23
+Pred28 Pred23 +ZMV Sequence ASP PSNR ASP PSNR ASP PSNR ASP PSNR ASP PSNR ASP PSNR ASP PSNR ASP PSNR ASP PSNR ASP PSNR ASP PSNR ASP PSNR CT256 5.24 39.54 5.26 39.55 5.22 39.57 5.21 39.58 5.25 39.52 5.24 39.62 5.22 39.59 5.22 39.60 5.24 39.59 5.24 39.56 5.21 39.56 5.20 39.55 CT40 5.76 32.60 5.76 32.52 5.69 32.68 5.68 32.73 5.78 32.40 5.72 32.67 5.69 32.61 5.67 32.73 5.75 32.62 5.72 32.61 5.68 32.70 5.67 32.72 HL40 6.30 34.53 6.26 34.53 6.21 34.51 6.21 34.55 6.24 34.51 6.25 34.54 6.21 34.57 6.20 34.55 6.30 34.54 6.24 34.53 6.23 34.57 6.21 34.55 MD96 5.95 40.07 5.89 40.07 5.90 40.10 5.95 40.08 5.91 40.08 5.91 40.12 5.89 40.09 5.94 40.09 5.92 40.11 5.88 40.14 5.90 40.10 5.93 40.09 CG112 5.98 29.11 5.89 29.13 5.94 29.12 6.40 29.08 5.89 29.12 5.88 29.12 5.89 29.11 6.41 29.11 5.99 29.11 5.87 29.11 5.97 29.11 6.43 29.08 FM512 7.05 34.00 6.82 34.10 7.03 34.02 7.52 33.91 6.84 34.07 6.78 34.09 6.89 34.08 7.39 34.01 7.03 34.00 6.78 34.09 7.06 33.99 7.52 33.91 FM1024 6.92 36.49 6.68 36.53 6.89 36.51 7.40 36.48 6.65 36.54 6.61 36.57 6.74 36.55 7.21 36.56 6.91 36.51 6.62 36.57 6.95 36.54 7.41 36.47 FB1024 11.61 34.80 10.66 35.00 11.69 34.86 13.00 34.69 10.61 34.95 10.55 35.02 10.85 35.00 12.23 34.80 11.65 34.86 10.49 35.03 11.93 34.83 13.11 34.69 FG768 6.28 26.18 6.13 26.18 6.16 26.19 6.39 26.18 6.14 26.18 6.14 26.20 6.09 26.19 6.35 26.19 6.28 26.16 6.11 26.18 6.15 26.18 6.38 26.18 ST1024 7.20 29.45 7.19 29.48 7.17 29.44 8.27 29.13 7.15 29.49 7.11 29.46 7.19 29.52 8.19 29.20 7.24 29.41 7.13 29.47 7.16 29.51 8.25 29.14 Average 6.83 33.68 6.65 33.71 6.79 33.70 7.20 33.64 6.65 33.69 6.62 33.74 6.67 33.73 7.08 33.68 6.83 33.69 6.61 33.73 6.82 33.71 7.21 33.64
Table 4-15 is a comparison of DL APS and DL AGPS with and without SPS. As discussed earlier, “DL APS + SPS” uses the 3-point SPS, “DL AGPS + SPS” uses the 2-point SPS, and the other algorithms use only PMV as the sole starting point. We find that DL APS with SPS outperforms DL APS by 8.3% in ASP (0.09dB PSNR gain). The DL AGPS with SPS outperforms DL AGPS by 5.0% in ASP (0.12dB PSNR gain).
In summary, the best SPS we identify for DL APS is {MVpred23, PMV, MVpred28}, and the best SPS for DL AGPS, {PMV, MVpred23}. With SPS, DL APS and DL AGPS can reduce ASP with slightly increased PSNR. In these two cases, the SPS size of DL AGPS is smaller. Our conjecture is that a fast-moving pattern search needs only a small SPS because the search algorithm can cover a large search area quickly without the help of additional starting points. The experiments also indicate that a 2-point (or 3-point) SPS is generally better than the single-point SPS (PMV).
Table 4-15 The effects of SPS on DL APS and DL AGPS.
Normal DL APS DL APS + SPS DL AGPS DL AGPS + SPS
Sequence ASP PSNR ASP PSNR ASP PSNR ASP PSNR
CT256 5.75 39.49 5.45 39.53 5.36 39.51 5.24 39.62
CT40 6.97 32.15 6.36 32.69 6.04 32.07 5.72 32.67
HL40 7.31 34.55 6.91 34.54 6.35 34.45 6.25 34.54
64