Chapter 4 Experimental Results
4.4 Subjective Quality
This section examines the subjective quality of the interpolated frame using FA, MCI8x8, STAR and Proposed_AST.
Figure 4-4 : Foreman_CIF 4th interpolated frame. (a) FA (b) MCI8x8 (c) STAR (d) Proposed_AST
Figure 4-4 (a) is the result of FA for 4th interpolation frame of Foreman_CIF.
Blurring happened around the ear, mouth, and helmet. Figure 4-4 (b) is MCI8x8 in the same frame. Blurring effect is eliminated for motion compensation. Still, the artifact around the mouth occurred because its discontinuity of adjacent block’s motion vectors.
The figure 4-4 (c) is STAR model’s interpolation result, for maximum iteration 1 and support order is 1. It alleviated the artifact around the mouth a little, and improved overall visual quality. But it also required much computation cost than proposed method.
And a little blurring effect occurred at the edge of the helmet, since it may contain too
many unreliable spatial-temporal neighborhoods in moving filed. The figure 4-4 (d) is proposed method Proposed_AST’s interpolation result. Though the artifacts around the mouth are not totally alleviated, we improved the blurring effect at the edge of the helmet.
4.5 Time complexity
Since we use less number of variables in the proposed AR model, the LSM’s
computation loading can be significantly reduced, comparing to STAR model. The time complexity to compute the matrix inverse is Ο(𝑛4), where n is equal to the number of
variables in AR model. Assuming that support order is set to 1, the STAR model will have 22 variables for LSM; while our proposed method will use only 17 variables in average. This is because that TAR uses 18 variables, SAR uses 8 variables, and the ratio between TAR and SAR selected in our method is about 9:1. Following table shows the ratio between TAR and SAR selected in the proposed method, where the selection criteria threshold is set to 4.
Table 4-3 : Model selection ratio percentage of it with respect to the total number of training windows in the frame. The SAR column shows the number and the ratio of the training windows that are selected for applying SAR model according to the proposed selection criteria; and TAR column shows those for TAR model. The LAST column shows the number and the ratio of the
training windows whose constructed matrix is non-invertible AR. From Table 4-4, it is observed that test sequences with large motion will choose SAR as their AR model, while the sequences with low motion will choose TAR as expected.
Table 4-4 : Execution time comparison between STAR and Proposed_AST (support order
= 1)
Mother_and_daughter_CIF 2353 1411 0.60 11300 2705 0.24
News_CIF 3556 2452 0.69 15202 4521 0.30
The above table shows the number of clocks consumed by the AR process. We only consider the execution time for regression part of the STAR method and our proposed method because both methods performed the same operations (MCI) before
starting AR model process. The support order is set to 1 in this table. The STAR-IT1, STAR-IT5 means the STAR model is performed iteratively once and five times, respectively. The percentages in the table show that the proposed model consume only 36% to 75% clocks compared to STAR model for iteration once. Since STAR-IT5 (iteration 5 times) and Proposed_AST-IT2 (iteration two times) have similar visual performance, we also compare their execution times in this table and the results show that the proposed model consume only 14% to 36% clocks, compared to STAR model.
The proposed model can save up to 86% clocks in Football_CIF, because it adaptively chooses SAR about 72% in whole process, the average number 10.8 variables for LSM.
Chapter 5 Conclusion
In this thesis, an adaptive auto-regressive model for frame rate up-conversion was proposed. In this schema for frame rate up-conversion, we save a lot of computation loading from removing the unnecessary variables from the STAR model. In the experimental results, we perform our proposed method compare with the other algorithms. Also, we compare the computation efficiency with the STAR model, which states out our proposed schema can work more efficiently and stay the same visual quality level or even better. By seeing the upper bound in our experimental results, the proposed model selection criteria may still have some space to be improved.
REFERENCE
[1] Byeong-Doo Choi, Jong-Woo Han, Chang-Su Kim, and Sung-Jea Ko,
“Motion-Compensated Frame Interpolation Using Bilateral Motion Estimation and Adaptive Overlapped Block Motion Compensation”, Circuits and Systems for
Video Tchnology, IEEE Transactions on, April 2007, VOL. 17, NO. 4.
[2] Yongbing Zhang, Debin Zhao, Xiangyang Ji, Ronggang Wang, and Wen Gao, “A Spatio-Temporal Auto Regressive Model for Frame Rate Upconversion”, Circuits and Systems for Video Technology, IEEE Transactions on, September 2009, VOL.
19, NO. 9.
[3] Seok Joo Doo and Moon Gi Kang, “Generalized adaptive spatio-temporal auto-regressive model for video sequence restoration”, Image Processing, 1999.
ICIP 99. Proceedings.
[4] Efstratiadis, S.N., Katsaggelos, A.K., “A model-based pel-recursive motion estimation algorithm”, Acoustics, Speech, and Signal Processing, 1990.
ICASSP-90.
[5] Xiangjun Zhang, Xiaolin Wu, “Edge-Guided Perceptual Image Coding via Adaptive Interpolation”, Multimedia and Expo, 2007 IEEE International
Conference on, July 2007 pp. 1459 - 1462
[6] Dragoljub Pokrajac, Reed L. Hoskinson and Zoran Obradovic, “Modeling
Spatial-Temporal Data with a Short Observation History”, Knowledge and Information Systems, September 2003,Volume 5, Number 3, pp. 368-386