CHAPTER 2 THE PROPOSED METHOD
2.2 Overlapped and hole regions processing
2.2.3 Motion estimation with overlapping
Since there are hole regions produced in motion compensation, overlapping
processing block would reduce the hole regions. A frame is divided into several
blocks with half block overlapping and each block is used to do motion estimation
and compensation, as shown in Fig. 2-11. Based on this overlapping schema, the
number of hole regions is reduced, and there are fewer blocking artifact. Fig. 2-12 is
an example of result after this step.
Fig. 2-10 The interpolated frame about Fig. 2-8 after processing of the complete hole block regions by the proposed method.
Fig. 2-9 The neighboring pixels marked by white color around block.
Fig. 2-11 Motion estimation using overlapping blocks.
The hole regions
Fig. 2-12 The interpolated frame about Fig. 2-3(a) after processing of the overlapped regions with half block overlapped.
CHAPTER 3
EXPERIMENTAL RESULTS
In this section, we present some experimental results. The experiments are
focused on doubling the frame rate. The size of the marcoblock is set as 16×16 and
the search range is ± 16 pixels for both horizontal and vertical directions. The
threshold for abandoning motion vectors in motion estimation is set as 400. We
select three different video sequences for testing. They are “calendar”, “foreman” and
“table”. Test sequences are color sequences with YUV 4:1:1 format. The image size of
test sequences is 176×144 quarter common intermediate format (QCIF). In order to
show the brief and efficiency of the proposed method, we compare the results with an
algorithm of hierarchical MCFI (HMCFI) [1].
3.1 Comparison with hierarchical MCFI
Fig. 3-1 is an example of scene with multiple motions. It contains camera
zooming and panning. There are also several object motions such as horizontal
translation (train), vertical translation (calendar) and rotation (ball). The speeds of
motions are not fast. As shown in Fig. 3-1, especially Fig. 3-1(e) and Fig. 3-1(f), the
interpolated frame by HMCFI is more blur than that by the proposed method. Since
the interpolated frame by HMCFI is usually compensated by two frames with
averaging while that by the proposed method is only compensated by one frame.
(a) (b)
(c) (d)
(e) (f)
Fig. 3-1 “Calendar＂ sequences : (a) previous frame, (b) current frame, (c) frame interpolated by the HMCFI, (d) frame interpolated by the proposed method, (e) an enlarged region of (c) by a factor of two, (f) an enlarged region of (d) by a factor of two.
Fig. 3-2 is an example of scene with motion of panning. The camera moves from
left to right and the scene shifts to right with a medium speed. Although the motion is
simple, some determinations of motion vectors are difficult. It is easy to find that
(a) (b)
(c) (d)
(e) (f)
Fig. 3-2 “Foreman＂ sequences : (a) previous frame, (b) current frame, (c) frame interpolated by the HMCFI, (d) frame interpolated by the proposed method, (e) an enlarged region of (c) by a factor of two, (f) an enlarged region of (d) by a factor of two.
there are two faulted macroblocks marked by circles compensated with faulted motion
vectors on the right-up corner in the interpolated frame by HMCFI. In this case, the
proposed method provides a more precise motion estimation and a better
compensation.
Fig. 3-3 is an example of scene with motion of zooming. The camera zooms
from the near to the distant and the scene becomes larger gradually. Among the frames,
the left hand moves fast. From Fig. 3-3(c), we can see that the shape of the left hand
in the HMCFI interpolated frame is blurred seriously, even the table and the hand are
indistinct. And the shape of the hand in the proposed interpolated frame approaches
(a) (b)
(c) (d)
Fig. 3-3 “Table＂ sequences : (a) previous frame, (b) current frame, (c) frame interpolated by the HMCFI, (d) frame interpolated by the proposed method without overlapping.
the original one. The poster on the left up corner in the interpolated frame by HMCFI
is more blurred than that in the proposed interpolated frame. This is due to that
HMCFI adapt to two average compensation blocks from two frames.
Fig. 3-4 shows the PSNR for the interpolated frames by the proposed method and
HMCFI method. In order to compute the PSNR of an interpolated sequence, we drop
the frames of even number on purpose and reproduce each dropping even frame by its
previous and next odd number frames. And then we compare the interpolated frames
with original ones. Fig. 3-4(a) is the result of the “foreman” sequence. The PSNRs of
the proposed method are a little lower than those of HMCFI. It is due to that the frame
constructed by the proposed method only refers to one frame. However, the
interpolated frame by the proposed method is clearer from the visual point. Fig. 3-4(b)
is the result of the “table”. The PSNR of the 67th interpolated frame by the proposed
method is much lower since a scene change appears here. From Fig. 3-4, we found
that PSNRs of both methods in most parts are similar. However, the proposed method
provides higher quality from visual point and is simpler than HMCFI.
3.2 Effectiveness of overlapping block
In order to reduce the hole regions produced from MC and the blocking artifact,
we scan a frame by overlapping half block. To investigate how the overlapping
20
Fig. 3-4 The PSNR computation of the sequences: (a) the sequence of “foreman”, (b) the sequence of “table”.
schema affects on the perceptual quality of the interpolated frame, we simulated two
types of frame rate up-conversion:
(a) the proposed method without overlapping block
(b) the proposed method with overlapping block
Fig. 3-5 shows that the perceptual quality is improved if the overlapping schema is
used. In Fig. 3-5(e) and 3-5(f), we can see that, the interpolated frame produced with
overlapping is smoother and has fewer block artifacts. It is obvious that the block
overlapping schema does enhance the interpolation frame.
(a) (b)
(c) (d)
(e) (f)
Fig. 3-5 “Table＂sequence: (a) previous frame, (b) current frame, (c) frame interpolated by the proposed method without overlapping, (d) frame interpolated by the proposed method with overlapping, (e) and (f) enlarged regions of (c) and (d) by a factor two, respectively.
CHAPTER 4 CONCLUSIONS
In this thesis, we have presented a new motion compensated interpolated
algorithm for frame rate up-conversion. The proposed method consists of two
component: forward motion estimation and motion compensation, and overlapped and
hole regions processing. To get a naturally perceptual and non-blur scene, two new
concepts are provided. The motion vectors are abandoned when the prediction errors
are too large, and the process of compensation only refers to one frame. Moreover,
overlapping blocks make the hole regions fewer and the compensation precisely.
Experimental results show that the proposed algorithm has a better visual quality.
REFERENCES
[1] B.-W. Jeon, G.-I. Lee, S.-H. Lee, and R.-H. Park, “Coarse-to-Fine Frame
Interpolation for Frame Rate Up-Conversion Using Pyramid Structure,” IEEE
Trans. Consumer Electronics, vol. 49, no. 3, pp. 499-508, Aug. 2003.
[2] K. A. Bugwadia, E. D. Petajan, and N. N. Puri, “Progressive-Scan Rate
Up-Conversion of 24/30 Source Materials for HDTV,” IEEE Trans. Consumer
Electronics, vol. 42, no. 3, pp. 312-321, Aug. 1996.
[3] S. C. Han and J. W. Woods, “Frame-rate Up-conversion Using Transmitted Motion
and Segmentation Fields for Very Low Bit-rate Video Coding,” in Proc. Int. Conf.
Image Processing, vol. 1, pp. 747-750, Oct. 1997.
[4] ITU-T Recommendation H.263, Video Coding for Low Bit-Rate Communications.
Nov. 1995.
[5] ISO/IEC JTC1/SC29/WG11 MPEG93/N457, MPEG-2 Test Model Version5. Mar.
1993.
[6] J. N. Youn, M. T. Sun, and C. W. Lin, “Motion estimation for high performance
transcoding,” IEEE Trans. Consumer Electronics, vol. 44, no. 3, pp. 649-658,
1998.
[7] G. D. Haan, P. W. A. C. Biezen, H. Huijgen, and O. A. Ojo, “True-motion
estimation with 3-D recursive search block matching,” IEEE Trans. Circuits Syst.
Video Technol., vol. 3, no. 5, pp. 368-379, Oct. 1993.
[8] F. Dufaux and F. Moscheni, “Motion estimation techniques for digital TV: A review
and a new contribution,” Proc. IEEE, vol. 83, pp. 858-876, June. 1995.
[9] S.-H. Lee, Y.-C. Shin, S.-J. Yang, H.-H. Moon, and R.-H. Park, “Adaptive motion
compensated interpolation for frame rate up-conversion,” IEEE Trans. Consumer
Electronics, vol. 48. no. 3, pp. 444-450, Aug. 2002.
[10] B.–T. Choi, S.–H. Lee, and S.–J. Ko, “New frame rate up-conversion using
bi-directional motion estimation,” IEEE Trans. Consumer Electronics, vol. 46, no.
3, pp. 603-609, Aug. 2000.
[11] K. Hilman, H.–W. Park, and Y.-M. Kim, “Using motion compensated frame-rate
conversion for the correction of 3:2 pulldown artifacts in video sequences,” IEEE
Trans. Circuits Syst. Video Technol., vol. 10, no. 6, pp. 869-877, Sept. 2000.
[12] R. Castagno, P. Haavisto, and G. Ramponi, “A method for motion adaptive frame
rate up-conversion,” IEEE Trans. Circuits Syst. Video Technol., vol. 6, no. 5, pp.
436-446, Oct. 1996.
[13] S.-H. Lee, O. Kwon, and R.-H Park, “Weighted-Adaptive Motion-Compensated
Frame Rate Up-Conversion,” IEEE Trans. Consumer Electronics, vol. 49, no. 3,
pp. 485-492, Aug. 2003.
[14] R. Thoma and M. Bierling, “Motion compensating interpolation considering
covered and uncoverd background,” Signal Processing: Image Compression, vol.
1, pp. 192-212, 1989.
[15] M. Bierling and R. Thoma, “Motion Compensating field interpolation using a
hierarchically structured displacement estimator,” Signal Processing, vol. 11, no. 4,
pp. 387-404, Dec. 1986.