Motion adaptive de-interlacing by horizontal motion detection and enhanced ELA processing

MOTION ADAPTIVE DE-INTERLACING

BY

HORIZONTAL MOTION

DETECTION AND ENHANCED ELA PROCESSING

Shyh-Feng Lin, Yu-Lin Chang, and Liang-Gee Chen

DSPfiC Design Lab, Department

of

Electrical Engineering

National Taiwan University, Taipei, Taiwan, R.O.C.

ABSTRACT

A motion adaptive de-interlacing algorithm and its hardware architecture are presented in this paper. It consists of the direc- tional interpolation

-

ELA with median processing, and 4-field horizontal motion detection. The edges can be sharper while the ELA with median processing is adopted. Enhanced ELA with median processing can prevent the happening of the bursting pixels. Proposed 4-field horizontal motion detection discovers faster motion and makes more accurate motion detection. The proposed method achieves hardware considerations on low complexity, low memory usage, and high speed processing. The experimental results show that the image quality of the proposed algorithm is superior to previous systems, and the cost is also more efficient. It is very suitable to be integrated into any dis- play device such as projection TV and Plasma display.

1.

INTRODUCTION

De-interlacing is an important process which converts ordinary interlaced TV sequences into progressive ones in order to output on the progressive devices (e.g. CRTs, Plasma Display Panels, TFT-LCDs). Some defects such as edge-flicker, line crawling will cause visual artifacts which let audiences uncomfortable if de-interlacing is not done perfectly.

Two methods of low complexity de-interlacing methods are BOB[I] and Weave[l] which are adopted into the software DVD Player. BOB is an intra-field de-interlacing method that uses a single field to reconstruct one progressive frame. However, the vertical resolution is halved and the image is blurred. Weave is a simple de-interlacing method directly combining two interlaced fields into one progressive frame. But the line crawling effect would happen in the motion area. A directional interpolation ELA for BOB had been proposed by Kuo, Liao, and Lin [2]. The ELA algorithm enhances the sharpness of the interpolated image.

The motion adaptive de-interlacing integrates the advantages of the intra-field de-interlacing and the inter-field de-interlacing. It detects the motion areas then adopts intra-field de-interlacing into

the

motion areas and inter-field de-interlacing into the static areas. A same parity 2-field morphological motion detection mechanism has been proposed by Lin, Chang, and Chen [3], but it can not detect very high-speed motion.

Some motion adaptive techniques[4]-[7] had been presented to improve the image quality. Sugiyama and Nakamura[4] proposed a motion adaptive de-interlacing method with motion compensa- tion. They used motion compensated method combining with the adaptive interpolation to reconstruct the missing field. Hilman, Park, and Kim[5] proposed a motion-compensated frame-rate conversion algorithm with interpolation to reduce the 3:2 pull- down artifacts. Ville[6] proposed a motion adaptive technique on

a fuzzy motion detector. Patti et a1.,[7] adapt the global motion compensation into motion adaptive de-interlacing to remove the artifacts due to camera panning or zooming. All those methods can provide good image quality for de-interlacing. However, these methods take a lot of computational power and expense for consumer electronic products.

In this paper, a motion adaptive de-interlacing method with en- hanced ELA processing, and 4-field horizontal motion detection is proposed. The overview of intra-field and inter-field de- interlacing will be discussed in the section 2 . The proposed ELA with median processing, 4-field horizontal motion detection will be described in the section 3. Section 4 states the considerations for hardware implementation, and shows the experimental results. At last, the conclusion remarks the proposed methods.

2.

OVERVIEW

De-interlacing methods can be roughly divided into three catego- ries - the Intra-field de-interlacing, the Inter-field de-interlacing, and the motion adaptive de-interlacing.

2.1

Intra-field De-interlacing

The intra-field de-interlacing means to construct a whole frame just based on original one field. The easiest and the most com- mon method of intra-field de-interlacing is bilinear interpolation. The bilinear interpolation blurs the edge because it is a running average of the neighboring pixels. However, it is the easiest

method among

all

the intra-field methods.

2.2 Inter-field De-interlacing

Inter-field de-interlacing finds the best match from the forward field or the backward field to reconstruct a new frame. The video quality will be better than intra-field interpolation, especially in the static area. The best method for inter-field de-interlacing is the motion compensated method.

2.3 Motion

Adaptive De-interlacing

The most important part

of

the motion adaptive de-interlacing

is

to detect where the motion areas are. If the motion detection can be done correctly, the output image quality can be as high as possible.

3.

PROPOSED DE-INTERLACING

The block diagram of proposed methods is shown in Fig. I . Three field buffers are used to store the reference data. The 4- field horizontal motion detection calculates the difference be- tween the forward-fonvard field to current field and the forward

X

n-2

n-1

n

n+l

n+2

+

Progressive frame out

Fig. I Block Diagram ofthe Proposed De-interlacing method field to backward field then gives motion information to the de- cision block. The enhanced ELA module does the directional edge interpolation according to the current-field information. Decision block receives the motion information then uses the fonvard field in the static area, and uses the current-field by ELA in the motion area. At last, the current field and interpolation- field are combined into a progressive frame.

3.1

ELA

with Median Processing

The ELA algorithm can detect the edges in the original field and generates a new frame. The new field possesses high-contrast on the edge of the image. We enhanced the ELA to detect 5 pixels in the previous and next scan line as shown in Fig. 2. Firstly, five differences are calculated from the five directions. Then doing the interpolation along the direction which has the smallest dif- ference. The median of three pixels

-

the produced pixel, the corresponding pixels in the previous line and in the next line -

can prevent the happening of the bursting pixels. Bursting pixels may appear in our motion detection, the ELA with Median proc- essing is designed to reduce the bursting pixels. So ELA with Median processing has better image quality than usual ELA in our alogithm.

3.2

4-Field Horizontal Motion Detection

The same parity 2-field motion detection is shown in Fig. 3. The field difference means the motion area of the current field. Pixels detected to be moving will be processed by the intra-field de- interlacing. But as for this case, the motion areas of the current

k-

1

k+

1

.

Forward Field Interpolation ELA=[X(n+2,k- I)+X(n-Z,k+ 1)]12 Y=Median[X(n,k- l),X(n,k+l),ELA] Fig. 2 ELA with median processing

Prcwu'x PlCld Cuncni Firill Barkwad FlCM

\

,4

Fig. 3 The 2 field motion detection

IpL

Fig. 4 The 4 field motion detection

field are not correctly detected. The forward field would bc filled into the middle of the circle, and the artifact turns up thcrc. The 2-field difference would be insufficient for good image qual- ity. The extension of 2-field motion detection is 4-field motion detection shown

in

Fig. 4. The difference between forward- forward field and current field are introduced to obtain more motion areas and can solve the artifact which produced by the fast motion.

The severest defect on interlaced sequences displayed on pro- gressive devices is line crawling effect. A temporal prediction

Forward(<) Forward@) Foward(d) Foward(a) Forward(e) Fig. 5 4-field horizontal motion detection

could help to know where an object is going to move. So our proposed 4-field horizontal motion detection combines the tem- poral prediction and the 4-field motion detection and is shown in Fig. 5 .

The advantage of the 4-field horizontal motion detection is to remove the detection error by large motion and different parily- field check.

4.

RESULTS

This section describes the implementation issues and results of vrovosed de-interlacing methods.

Compler-

Memory Access Mom- Instrue-

I

o w tion Bilinear ELA M.C. Proposed

4.1

Complexity Comparison

The comparisons of the complexity, memory access frequency, local buffer usage, and the instruction counts between the meth- ods are described in the table 1.

The "n" in Table 1 means the pixel numbers to be processed and the "SR" means search range in the motion compensation. The complexity of bilinear interpolation and ELA is O(n) because they are just a kind of FIR filter. There are many kinds of MC de-interlacing method, but their complexities are too high for comparing with motion adaptive methods. So a common MC de- interlacing method is compared here for reference. The motion compensated de-interlacing needs full-search motion estimation, and the complexity of the full search motion estimation is O(n'x SR'). The complexity of the motion compensation in the MC de- interlacing is smaller than the motion estimation part, so its com- plexily is ignored. The complexity of the proposed method is similar to ELA, and the 4 field horizontal motion detection has a complexity of O(n). So the overall complexity of our proposed method is O(u).

Due to the pipeline architecture, the bilinear interpolation should read in one pixel into local buffer while processing each pixel. As for ELA, it needs to examine two pixels on two different scanlines. The memory access frequency of the motion estima- tion part of the MC de-interlacing varies in the different designs, but the reading frequency of the memory will be more than 10 times per pixel. Our proposed method read 5 pixels from mem- ory each time while the 4-field horizontal motion detection is utilized.

The instruction counts are profiled by I-prof with gcc 3.0.2 and the target video sequence is for CIF 30FPS. There is much to optimize in the C codes for hardware implementation. But it is shown that the proposed method needs only I.2GIPS for real- time de-interlacing. The instruction count

of

the proposed

method i s similar to those of the intra-field de-interlacing mcth- ods and is much smaller than the MC methods.

It is obviously that motion compensated de-interlacing is the most complex and time-wasting method, and the proposed algo- rithm possesses less memory access frequency than the motion compensated de-interlacing and it has the same complexity with the intra-field de-interlacing.

4.2 PSNR Comparison

Several video test sequences have been made interlaced and re- covered by the de-interlacing methods. The PSNR comparison is shown

in

the table

2.

The proposed method achieves the highest PSNR performance in many of the sequences except some se-

quences with very fast motions, but it can still retain high PSNR in these fast motion sequences.

4.3

Subject View Comparison

The sequence ''Pendulum'' emphasizes

on

the texture preserva- tion and edge sharpness while de-interlacing is done. The origi- nal sequence can divide into three parts. The most important of all is the oscillating pendulum; it swings from the left of the screen to the right back and forth. In the combined frame, the [bytes1 [MIPS)

O(") I" I" 1 481

O(") 20 I" i o 879

O(nixSR2) >Ion In 528 30.248

O(") 5" t n 25 1,224

II-698 ity Read Write Usage Counts

line crawling appears at the motion area of the pendulum. The second pan is the title "OK", which are constituted of

" 0

only in the even fields and " K only in the odd fields.

The third part of the sequence is the logo rotating at the right- down comer of the screen. The rotating logo acts like a non-rigid body, and the real appearance of non-rigid bodies are often lost by MC de-interlacing. However, motion adaptive de-interlacing methods can reconstruct the non-rigid bodies very well. The original tWo fields and the directly combined frame are shown in the Fig. 6(a) (b)(c) separately.

(a) Original even field _{fb) Orieinal odd field}

(e) Intra interpolation from

(0

Result by the 2-field mo-

(h) Enhanced ELA (g) Bilinear interpolation

(i) Proposed method Fig.6 The comparison of subject view

Edges in the picture could be recognized when intra-field de- interlacing is adopted. But the words

" 0

and " K will disappear alternatively because they exist only in the different parity fields, shown as Fig. 6(d) and Fig. 6(e).

The resulting picture of the two-field motion detection [3] is shown in Fig. 6 ( f ) Although the words " O K are successfully preserved, the line crawling still exists in the string part of the pendulum.

For the intra-field de-interlacing, the bilinear interpolation and the ELA make differences with their sharpness. The result of the bilinear interpolation is shown as Fig. 6(g), aliasing-edge can bc seen. The result of the ELA is shown as Fig. 6(h), while the cdgc is much sharper than the bilinear interpolation one.

The result of the proposed method which has 4-field horizontal motion detection and enhanced ELA is shown as Fig. 6(i). The edges of the pendulum and the logo are sharp. and the words " O K are white wards with black spaced in-between. The pro- posed method has overcome the tough hurdles of all the other de- interlacing methods.

5. CONCLUSIONS

A high quality, low cost and fast de-interlacing method is pro- posed in this paper. The ELA with Median processing keeps the sharp edgc of the original field. The 4-field horizontal amotion detection combines the techniqucs of the 4-field motion detection. The experiment results show that the proposed algorithm really has high picture quality, low memory access frequency and memory usage.

6.

REFERENCES

[ I ] htti~:lInickvsuidcs.dicital-dircsl.comiindex.htm [2] Chung J . Kuo, Ching Liao. and Ching C. Lin. "Adaptivc

Interpolation Technique for Scanning Rate Conversion,"

IEEE Trans on Circriits rrnd Systemsfor Video T e c h r w l o ~ ~ .

vol. 6, " 0 . 3 , June 1996

S.F. Lin, Y. L. Chang, and L. G. Chen, "Motion Adaptive Interpolation With Morphological Operation and 3:2 pull- downed Recovely for Deinterlacing", IEEE lnreviiutionul

Conference 011 Mdrimediu uiidExpo, Aug. 2002

Kenju Sugiyama and Hiroya Nakamura. "A Method of Deinterlacing with Motion Compensated Interpolation,"

IEEE 7 ' " s Cormuner. Elec.. vol. 4 5 , no. 3, pp.61 1-616. August 1999.

Kevin Hilman, Hyun Wook Park, and Yongmin Kim. "Using Motion-Compcnsated Frame-Rate Conversion for the Correction of 3:2 Pulldown Artifacts in Video Se- quences," IEEE Trans. Circuits Svsr. Video Techno/.. vol

IO, no. 6, September 2000.

D. Van De Villc, B. Rogge, W. Philips, and I. Lemahieu. "Deinterlacing using furry-based motion detectiun." K,zou,ledge-Bused lntelligenr 1,iforniurion Engirieeririg

S].sle!izs. 1999. Third Iritemotio,iol Co,l/erence, I Y Y Y , pp263-267.

A.J Patti, M.1 Sezan and A.M Tekalp, "Robust Methods for High Quality Stills from Interlaced Vidco in the Prcs- cnce of Dominant Motion," IEEE Trunr-. Circ. Sy.rr Video

Tech.. vol. 7, no. 2. pp. 328-342, Apr. 1997. 131 [4] [SI [6] [7]

11-699