We have re-examed the predictioin efficiency of motion compensated prediction (MCP) and interpreted it as a motion sampler followed by the reconstruction of prediction signal. We also show that, in a statistical sense, block matching based motion estimation will result in motion vectors that are most likely to be the motion vectors sampled at block centers. With the help of motion and intensity models, the comparison of BMC, CGI, OBMC, TMP and SKIP prediction are also demonstrated both theoretically and empirically. Although TMP hardly competes with BMC, TMP is shown to outperform SKIP prediction, which explains why the bit rate can be significantly reduced when TMP is efficiently combined with SKIP prediction. Based on this theoretical framework, in this dissertation, we then apply some of these results to design a parametric solution for OBMC to suit for irregular motion sampling structures.
Chapter 3
Parametric Overlapped Block Motion Compensation
3.1 Introduction
As discussed in chapter 2, various algorithms have been proposed to improve BMC. The most straightforward technique is variable block-size motion compen-sation(VBSMC), which increases motion sampling density in areas with complex motion to compensate for the inefficiency of BMC. By contrast, Control-Grid Interpolation (CGI) [21] and Overlapped Block Motion Compensation (OBMC) [19] use more sophisticated algorithms to reconstruct the motion field without additional samples. The former improves motion interpolation by employing a triangular filter function, while the latter directly gives a linear estimate of each pixel’s intensity based on predictors derived from the current and nearby block MVs. Both are able to alleviate blocking artifacts effectively, but in practice, OBMC is preferred to CGI since the averaging of predictors also helps to reduce quantization noise [25]. To reduce and equalize prediction error within blocks, two approaches have been proposed: overlapped block motion compensation (OBMC) [3] and variable blocksize motion compensation (VBSMC) [4][5]. OBMC improves the motion compensation accuracy for every pixel by considering nearby motion estimates as different plausible hypotheses for its true motion. VBSMC, on the
other hand, extends BMC naturally to allow the use of subblocks of varying size in motion compensation. While OBMC requires no extra side information, VBSMC must additionally signal the choice of block size and motion vector. Each method has some merits and faults, and this dissertation seeks to form an optimized hybrid of the two techniques.
Motivated by the preceding observations, we are led to seek an optimized hybrid of VBSMC and OBMC, aiming to trade better prediction for fewer MVs while retaining the flexibility to adapt motion sampling structure according to variations in image statistics. However, determining OBMC weights to associate with MVs on an irregular grid poses a challenging problem. This is because the variable block-size partitioning yields spatially varying geometric relations between a prediction pixel and its nearby block centers. In this case, solving for the weights with the least-squares method would become an under-determined problem since a distinct solution has to be sought for each possible context. Clearly, there may be more parameters to be estimated than there are data points.
This problem is not new. A similar situation occurred in the development of H.263 [1]. At that time, it was resolved by treating larger blocks as a collection of smaller blocks with the same MV in each smaller block as in the larger aggregate block and by applying a fixed window function to all MVs. In an attempt to extend the notion to H.264/AVC, Wang et al. [27] additionally proposed to weight more heavily those MVs from smaller aggregate blocks, which they believed can more reliably represent the motion of neighboring blocks, although no justification was given. Both methods suffer from the same problem that inner pixels in larger blocks are not properly compensated. Essentially, the MVs utilized for OBMC of those pixels are replicated from the same (aggregate) block MVs, producing a net
effect like BMC. A third method that has recently been proposed is irregular-grid OBMC [11], which circumvents this deficiency by an adaptive window support that scales with local motion sampling density. It, however, remains unclear how to choose a proper scaling factor for each MV.
This dissertation departs from heuristic methods to approach the problem from a theoretical perspective. We formalize the notion of motion-compensated predic-tion (MCP) as a two-stage process consisting of sparse mopredic-tion sampling followed by the reconstruction of temporal predictors. Within such a framework, OBMC in its generalized form is seen to find a LMMSE estimate for every pixel’s inten-sity based on motion-compensated signals derived from MVs sampled at nearby block centers. This viewpoint allows us to derive a parametric solution, termed POBMC, for determining the optimal weights in closed form. In doing so, the signal models in [30] are adopted to describe the probabilistic structures of the underlying intensity and motion fields. One important result of our POBMC is that its parameters include only the `2 distances between the locations of the pre-diction pixel and the MVs involved–i.e., their geometric relations are all that are needed to determine the weights. This leads to a generic method of reconstructing temporal predictors from any sparsely and irregularly sampled motion data.
Although our approach has some parallels with the other parametric solution [24], the unique features that distinguish this work from it include
1. Our focus is to adapt OBMC to suit variable block-size motion partitioning, while [24] concentrates on adjusting OBMC windows, based on the use of fixed block-size partitioning, in response to variations in sequence statistics;
2. We adopt an alternative signal model [30], which not only better represents the reality but also gives a result that is considerably more intuitive and
tractable;
3. We address the uncertainty associated with a block MV’s location by in-troducing a compensation term to reflect its dispersion around the block center;
4. We propose a suboptimal yet computationally efficient implementation, which need not solve the Wiener-Hopf equation and thus eliminates the need to compute matrix inverse.
In addition, we implement the proposed scheme with KTA 2.4r1 [20] and provide a performance comparison with the recently proposed Enhanced Adaptive Inter-polation Filter (EAIF) [29] and Quadtree-based Adaptive Loop Filter (QALF) [12] together with an analysis on how they interact with each other.
In the common test conditions, our POBMC delivers better rate-distortion (R-D) performance than both the H.263 OBMC [1] and the parametric solution [24]. Relative to an H.264/AVC anchor with extended macroblock (MB) size, it achieves 3.1% (0.7-13.6%) BD-rate reductions, compared to 4.6% (0.5-10.1%) and 7.2% (1.3-18.0%) with the single use of EAIF and of QALF, respectively. Although POBMC has the least gain among these filters, it can be combined efficiently with either of the other two filters. The result is an improvement that is almost the sum of their separate effects. In particular, the combination of POBMC and QALF performs very close to or better than that of EAIF and QALF, even in cases where the single use of EAIF outperforms that of POBMC.
The rest of this dissertation is organized as follows: Section II revisits the notion of motion-compensated prediction from a perspective based on motion sampling and reconstruction. Section III presents in detail the derivation of our parametric solutions. Section IV examines their properties by contrasting
the-oretical predictions with empirical data. Section V evaluates the compression performance of POBMC from various aspects and provides a runtime analysis.
Section VI concludes this dissertation with a summary of our observations and a list of future works. Finally, the implementation details of POBMC is elaborated in Appendix.