Hardware efficient design of filter banks for video coding

(1)

1997 IEEE International Symposium ow Circuits and Systems, June 912,1997, Hong Kong

HARDWARE EFFICIENT DESIGN OF FILTER

BANKS

FOR VIDEO CODING

Po-Cheng Wu, Li,ang- Gee Chen, Yuan-Chen Liu,

and

Yeong-Kang Lai

DSP/IC

Design Lab., Department

of

Electrical Engineering

National Taiwan University

E-

mail:

{

p

cwu ,lgchen ,liu ,lai} @video.

ee.nt u .edu.

tw

Taipei, Taiwan,

R.O.C.

China

ABSTRACT

Since three-dimensional (3-D) su bband coding has been introduced, most researches on 3-D subband coding perform temporal filtering first. In this paper, we investigate the best permutation strategy for temporal, vertical, and horizontal filtering to minimize the requirement of delay elements and find that the results are opposite to our expectation.

1. I N T R O D U C T I O N

Recently, there has been rapid progress in the area of multirate digital signal processing. Applications of multirate systems include subband coding of video and audio signals, fast transforms using digital filter banks, wavelet analysis, and many others [ 1]-[3]. One of the most important applications of multirate systems is subband coding (SBC). Since it was introduced by Cro- chiere e t al. [4] in 1976, subband coding has been an effective coding approach for video and audio applications [5]-[7]. Subband coding employs a filter bank for splitting the input signal, so the filter banks are es- sential to subband coding. Because filter banks usually deal with a large number of data, high speed computing hardware is indispensable for subband coding systems. Subband coding was first extended to three dimensional (temporal, vertical, and horizontal) filtering, i.e., 3-D subband coding, by Karlsson arid Vetterli [8]. S-

ince then, most researches which deal with 3-D subband coding perform temporal filtering first, e.g., [8]- [13]. Since 3-D subband coding requires a large number of delay elements t o store the intermediate data, delay elements dominate the hardware cost in this coding scheme. Therefore, the most important task remains how t o minimize the requirement of delay elements. In this paper, we investigate the best piermutation strategy for temporal, vertical, and horizontal filtering in different conditions in order to minimise the required number of delay elements.

yI-1'

j

Figure 1: (a) Video format:

M

lines x N pixels x P

frames/sec. (b) The required dela,y elements for temporal filtering. ( c ) The required delay elements for vertical filtering. (d) The required delay elements for horizontal filtering.

2. D E S I G N S T R A T E G Y

Let us assume the video format is

i

V

lines x N pixels x

P frames/sec as shown in Fig. l ( a ) , and the numbers of filter taps for temporal, vertical, aind horizontal filtering are x + l , y + 1 , and z+1, respectively. Because the video data is read in line scan mode frame-by-frame, to perform filtering, the temporal filter needs t o store the previous x frames; the vertical filter needs t o store the previous y lines; the horizontal filter needs to store the previous z pixels. Therefore, the required numbers of delay elements for temporal, vertical, and horizontal filtering are MNx, N y , and z as shown in Fig. l ( b ) , (c), and (d), respectively. In subbamd coding, we usually employ the FIR direct form filter. The advantage of employing the FIR direct form is 'that the lowpass

(2)

I

Permutations

1)

Stage 1 (XI)

1

Stage 2 ( ~ 2 )

1

Stage 3 ( ~ 4 )

I

Total

Table 1: Comparison of the required delay elements for different filtering permutations in 3-D subband coding with eight-full bands.

Input

Figure 2: FIR direct form filter for subband coding.

filter and the highpass filter can share the same delay elements as shown in Fig. 2.

2.1. Eight-Full Bands

We will first consider 3-D subband coding with eight- full bands as shown in Fig. 3. From Fig. 3, we find that by employing the FIR direct form filters, stage 1 requires one set of delay elements; stage 2 requires two sets of delay elements; stage 3 requires four sets of delay elements. Because there are decimation operations performed along the filtering direction after filtering in each stage, the output d a t a at each branch is half the input data. Therefore, the delay elements required in each stage are different and depend on the different filtering permutations. Fig. 4 gives two examples of the required delay elements for different filtering permutations: (a) temporal, vertical, followed by horizontal filtering; (b) horizontal, vertical, followed by temporal filtering. Table 1 lists the required numbers of delay elements for six different permutations of temporal, vertical, and horizontal filtering. Note t h a t , because we use the separable FIR filters, the filtering results are identical even though the filtering permutations are d- ifferent. From Table 1, we find that for eight-full-band splitting, the cascade strategy which requires the min- imum number of delay elements is horizontal, vertical, followed by temporal filtering, i.e., (H, V, T).

Input

I

H

_

/

. I

.

L

{:_::

" L

stage1 stage2 stage3

L L H L H L L H H H L L H L H H H L H H H

Figure 3: 3-D subband coding with eight-full bands.

(b)

Figure 4: The required delay elements for: (a) temporal, vertical, followed by horizontal filtering; (b) horizontal, vertical, followed by temporal filtering.

(3)

1

Permutations

11

Stage 1 (XI)

I

Stage 2 ( ~ 2 )

1

Stage 3 ( x 3 )

1

Total

Table 2: Comparison of the required delay elements for different filtering permutations in 3-D subband coding with four-reduced bands.

I

Permutations

11

Stage 1 (XI)

I

Stage 2 ( ~ 2 )

I

Stage 3 ( ~ 3 )

1

Total

Table 3: Comparison of the required delay elements for different filtering permutations considering the finite wordlength effects in 3-D subband coding with four-reduced bands. (The unit in this table is bits.)

Input

2.3. Finite Wordlength Effects

Finally, if we further consider the finite wordlength effects to enhance the accuracy of subband coding, we assume that the wordlengths of delay elements in stage

1, stage 2, and stage 3 are a, b, and c bits, respectively. The relation between a, b, and c is usually a

5

b

5 c.

Now, we still look at 3-D subband coding with four- reduced bands. The required bits of delay elements for different permutations are listed in Table 3. From

Ta-

,LF

_-

_,;:

_{L H L}

-

stage1 stage2 stage3

ble 3, we are unable t o distinguish which permutation strategy is the best choice. Therefore, we make the fol- Figure 5: 3-D subband coding with four-reduced bands.

2.2. Four-Reduced Bands

In order to achieve a high compression ratio in 3-D subband coding, we often retain the four lower bands and discard the higher bands as shown in Fig. 5. This is wavelet-like splitting. Thus, from Fig; 5, we find that stage 1 requires one set of delay elements; stage 2 requires two sets of delay elements; stage 3 requires three sets of delay elements. The required numbers of delay elements for different permutations are listed in Table 2. From Table 2, we find that for four-reduced- band splitting, the cascade strategy which minimizes the delay elements is horizontal, vertical, followed by temporal filtering, i.e., (H, V, T).

lowing assumptions: the video format is M = 512 lines and N

=

1024 pixels; the temporal filter has x

+

1 = 2 taps (in order to minimize the delay time); the verti- cal filter has y

+

1 = 8 taps; the horizontal filter has z

+

1 = 16 taps; the wordlengths of delay elements in stage 1, stage 2, and stage 3 are a = 8 bits,

b

= 9 bits, and c

=

10 bits, respectively. After careful calculation, we find that if we consider the finite wordlength effect-

s in the above conditions, the cascade strategy which minimizes the delay elements is vertical, horizontal, followed by temporal filtering, i.e., (V, H, T).

3. CONCLUSIONS

From above discussion, we find that the results are opposite to our expectation. Performing the temporal

(4)

filtering first is not the best strategy. O n the contrary, placing the temporal filter a t final stage will save the delay elements. We also find that the best permutation strategy for temporal, vertical, and horizontal liltering depends on different conditions. Finally, for the synthe- sis filter banks or a higher number of filtering stages in 3-D subband/wavelet coding, by employing the same scheme, we can also derive the best design strategy which minimizes the requirement of delay elements.

4. REFERENCES

R. E. Crochiere and

L.

R. Rabiner, Multirate

Digi-

tal Signal Processing. Englewood Cliffs, NJ : Pren- tice Hall, 1983.

P. P. Vaidyanathan, Multirate Systems and Filter Banks. Englewood Cliffs, N J : Prentice Hall, 1993. N. J. Fliege, Multirate Digital Signal Processing. New York: Wiley, 1994.

R.

E. Crochiere, S. A. Webber, and J . L. Flanagan, “Digital coding of speech in subbands,” Bell Syst. Tech.

J.,

vol. 55, pp. 1069-1085, Oct. 1976.

J. W. Woods and S. D. O’Neil, “Subband coding of images,” IEEE Trans. Acoust., Speech, Sig- nal Processing, vol. ASSP-34, pp. 1278-1288, Oct. 1986.

H. Gharavi, “Subband coding algorithms for video applications: videophone to HDTV-conferencing,”

IEEE

Trans. Circuits Syst. Video Technol., vol. 1,

no. 2, pp. 174-183, June 1991.

E. B. Richardson and N. S. Jayant, “Subband coding with adaptive prediction for 56 kbits/s audio,” IEEE Trans. A coust.

,

Speech, Sign a1 Processing, vol. ASSP-34, pp. 691-696, Aug. 1986.

G. Karlsson and M. Vetterli, “Three dimensional sub-band coding of video,’’ in Proc. IEEE Int. Conf. Acoust., Speech, Signal Processing, 1988, p- p. 1100-1103.

J . Hartung, “Architecture for real-time implemen- tation of three-dimensional subband video coding,” in Proc. IEEE Int. Symp. Circuits Syst., vol. 3, 1992, pp. 225-228.

K. N. Ngan and W. L. Chooi, “Very low bit rate video coding using 3D subband approach,” IEEE Trans. Circuits Syst. Video Technol., vol. 4, no. 3, pp. 309-316, June 1994.

[11] C. I. Podilchuk,

N.

S . Jayant, and N. Farvwd- ~

m, “Three-dimensional subband coding of

video,a

.

IEEE

Trans. Image Processing, vol. 4, no. 2, pp.

125-139, Feb. 1995.

[12] F. Fan, S. Simon, I. Bruyland, W. Zhu,

B. D.

Canne, and M. V. Bladel, “A method for hier- archical subband HDTV splitting,” IEEE BanS.

Circuits Syst. Video Technol., vol. 5, no. 3, pp. 225-230, June 1995.

[13] C. H. Chou and C. W. Chen, “A perceptually oP timized 3-D subband codec for video communicb tion over wireless channels,’’ IEEE Trans. Circuib Syst. Video Technol., vol. 6, no. 2, pp. 143-156, Apr. 1996.