Finite-wordlength design of 2-D FIR digital filters for sampling structure conversion

(1)

SIGNAL

PROCESSING

ELSEVIER Signal Processing 55 (1996) 305-311

Finite-wordlength design of 2-D FIR digital filters

for sampling structure conversion

Jong-Jy Shyu”>*, Soo-Chang Peib, Yuan-Chih Lin”

a Department of Computer Science and Engineering, Tatung Institute of Technology. 40 Chungshan N. Rd.. 3rd Sec., Taipei, Taiwan, Republic of’ Chinu

bDepartment qf Electrical Engineering, National Taiwan Unioersitl,. Taipei, Tuiwan. Republic qf Chma

Received 19 May 1995; revised 23 July 1996

Abstract

In this paper, an effective approach is proposed for designing discrete coefficient 2-D FIR digital filters for sampling

structure conversion. After obtaining the initial continuous solution, the conventional Lagrange multiplier approach as- sociated with an appropriate tree search algorithm is used iteratively to optimize the remaining unquantized coefficients of the designed filter in the least-squares sense when one or more of the coefficients take on discrete values, till all of the filter coefficients are quantized. The method is simple and the performance is comparable with that of the existing methods.

In diesem Beitrag wird sin wirksamer Ansatz zum Entwurf zweidimensionaler FIR-Filter mit diskreten Koeffizienten zur Abtastraster-Umsetzung vorgeschlagen. Nach dem Erhalt einer kontinuierlichen Anfangslosung wird der iibliche Ansatz mit einem Lagrange-multiplikator zusammen mit einer geeigneten Baumsuche iterativ angewandt, urn die verbliebenen unquantisierten Koeffizienten des entworfenen Filters im Kleinste-Quadrate-Sinn zu optimieren; dabei nehmen einzelne Koeffizienten diskrete Werte an, bis schlieglich alle Filterkoeffizienten quantisiert sind. Das Verfahren ist einfach, und es leistet gleich vie1 wie die bekannten Methoden.

Dam cet article, on propose une approche efficace de conception de filtres FIR 2-D a coefficients discrets pour la conversion de la structure d’echantillonnage. Apres avoir obtenu la solution continue initiale, I’approche par multiplieur de Lagrange conventionnel associe a un algorithme de recherche par arbre est utilisee iterativement afin d’optimiser au sens des moindres carres les coefficients non quantifies du filtre lorsqu’un ou plus des coefficients prend des valeurs disc&es. et ce jusqu’a ce que tous les coefficients du filtre soient quantifies.

Key~vords: Sampling structure conversion; Lagrange multiplier approach; Tree search algorithm

(2)

306 J.-J. Shyu et al. /Signal Processing 55 (1996) 305-311

1. Introduction

In digital signal processing, the conversion between different periodic sampling structures is an important problem, especially for the conversion between quincunx and rectangular structures in television image processing and HDTV applications [2,7,9, lo]. The sampling matrices of the rectangular and quincunx sampling structures are generally given by

$=‘d

O

[

T2 1 and S _{LK =} KT2 LTl LTl I -KT2 ’

(1)

respectively, where Tl and T2 represent the horizontal and vertical sampling periods, and L and K are quincunx sampling parameters.

The conversion from the rectangular sampling structure to the (L,K) quincunx sampling structure, and the inverse operation, correspond, respectively, to decimation and interpolation process. To avoid aliasing error and to remove redundant images, appropriate decimation filters and interpolation filters must be used, respectively. Moreover, because human eye is particularly sensitive to the distortion on the flat areas of the reconstructed pictures in the interpolation processing, certain constraints for designing interpolation filters should be considered.

The conversion system’s operation requires numer- ous multiplications and additions. Multiplication, in particular, is extremely time consuming. So if a multiplication operation could be replaced by only a few shift operations, then the complexity of the entire conversion system could be reduced quite dramatically, such that a fast real-time system becomes feasible.

In this paper, an effective method is proposed for designing 2-D discrete coefficient FIR digital filters for sampling structure conversion systems. The method associates the Lagrange multiplier approach and a tree search algorithm, hence the constraints stated above can be incorporated into the design procedures. For each branch of the tree, the Lagrange multiplier approach is used to optimize the remaining unquantized coefficients of the designed interpolation and decimation filters in the least-squares sense when one or more

of the coefficients takes on discrete values. The proposed approach can be applied for different discrete coefficient spaces including the evenly distributed finite wordlength space and the nonuniformly distributed powers-of-two space. In this paper, the former is used only for comparison. Comparing with the existing methods, the method is simple and the performance is comparable.

2. Review of the applications of the Lagrange multiplier approach for designing continuous coefficient 2-D FIR filters for sampling structure conversion

Suppose the frequency response of the designed filter with quadrantal symmetric coefficients h(ni,nz) is given by H(o,, w2) =

5 5

h(n,, n2)e-jn~wle-j”~02 W=-N, ,Q=-& N, N, = c c a(nl,n2)COS(~lWl)COS(n202), n, =o IQ=0 (3)

where a(ni,nz) is related to h(nl,n2) by

Obn2) ifni =Oandnz=O,

4nl,n2) = 2h(nl,nz)

if (nt = 0 and n2 # 0) or (q # 0 and n2 = 0), 4h(nl,nz) otherwise.

(4)

Eq. (3) can be represented in vector form by

Ww,f32) = ~Tw31,~2) = CTbw2V, (5)

where

A = [‘4,T

AT . . . A$JT

and

(3)

in which

Ai = [a(i,O) a(i, 1) . . . a(i,NZ)lT, O<i<Nt, (8)

and

= [cos(iwl ) cos(iol ) cos(w2) . . .

cos(iw,)~os(N2a2)]~, ObidN,. (9)

Using these notations, the integrated square error is

e=

I/ R ~(W,~2)IWW,~2)

-H(w 302 >I* dw dw2

= s+PTA+ATQA, (10)

where R represents the designed bands, W(wt ,132) is the weighting function,

.S=

JJ

W(w,,W2)02(01,W2)d0,dW:!,

(11)

R P= -2~(W,~*)~(W,~*) x C(w) ~2) dw dm2 (12) and

Q=JJ

W(w > 02 )C(w > 02) R xCT(01,m2)d~t doz. (13)

Because human eye is particularly sensitive to the distortion on the flat areas of the reconstructed pictures in the interpolation processing, the following frequency constraints in the first quarter plane should be considered [8, lo]:

H(O,O) = G,

ff(wl/, 3 WZk, > = ff(&>, h2k2) = 0, (14)

where G is equal to 1 for the decimation filter design and equal to 2LK for interpolation processing,

(ml& 3 a2k, ) = (211 + l)n (2k, + 1)7t

>

L’K ’

but (h,h) # (O,O), (16)

in which 1x1 denotes the largest integer less than X. Eq. (14) can be represented in matrix form by

BTA = G, (17)

where

B = [C(O,O) C(wo,w20) . . . C(qi, 02~)

C(&o, (521) . . . CC&, 0212

)I

(18)

and

G = [G 0 . . . 0 0 . . . OIT. (19)

Hence, the design of two-dimensional filters for sampling structure conversion can be formulated as a quadratic programming problem

Minimize e=s+PTA+ATQA

subject to BTA = G, (20)

which results in the closed-from solution [5,8]

A = Q-‘B(BTQ-‘B)-‘G

+;Q-‘[B(B’Q-‘B)-‘B’Q-’ -ZIP, (21)

where Z is an (Nt + l)(Nz + 1) x (NI + l)(N2 + 1) identity matrix.

3. Design of finite-wordlength 2-D FIR filters for sampling structure conversion

Once the continuous coefficient filter is obtained, the key operation in the discrete optimization algorithm is to optimize the unquantized coefficients when some of the coefficients take on discrete values. Notice that the constraints of some coefficients to discrete values can be represented in a constrained

(4)

308 J.-J. Shyu et al. /Signal Processing 55 (1996) 305-311

matrix form. For example, the coefficients a(O,2), a(O,5) and a(O,3) should be constrained to discrete values &(0,2), Ud(O, 5) and Ud(O, 3), respectively, which can be represented by the constrained equation

[

0 0 1 0 0 0 0 ...

0000010 . ..]A= [;:::;;;I.

0 0 0 1 0 0 0 e.1

(22)

Hence, the Lagrange multiplier approach can be applied iteratively as an effective discrete optimization algorithm.

As to the branch and bound methods [l], there are two main tree search strategies, i.e. the depth-first- search strategy and breadth-first-search strategy. The former has the advantage of producing an early im- provement in the initial solution, and the latter usually leads to a better first solution but is ineffective which makes it rarely be used alone. In this paper, we use the hybrid tree search of the two [3]. The de- tails of the tree search algorithm is described as follows. After obtaining the continuous coefficient filter by (2 1 ), the coefficient with maximum absolute value is selected to take on discrete values and then fixed at I discrete values in the vicinity of the continuous optimum value. Each of the discrete values is fixed and the Lagrange multiplier approach is used to find the continuous solution under the constrained equation like (22). Hence, there are I optimization problems if the chosen coefficient is fixed at I different discrete values, which result in I sets of continuous solutions under fixing the first quantized coefficient. For each of the I sets of continuous solutions, the coefficient with maximum absolute value is chosen to take on discrete values from the unquantized coefficients. So there are I2 optimization problems when two coefficients are quantized, which also result in J2 sets of continuous solutions under fixing two quantized coefficients. For each of the obtained solutions, certain measure error is computed such that the constraints in (14) can be incorporated into the optimization. Then only I of the I2 problems, which provide the smallest error, are selected for further quantization of the coefficients (so that the number of problems should be processed would not increase step by step) and pro- duce I2 further optimization problems when a third coefficient is selected to take on discrete values. The

processes are continued until all the coefficients take on discrete values.

The procedures for designing finite-precision coefficient filters for sampling structure conversion by the proposed iterative Lagrange multiplier approach is summarized as follows:

Step 1. Find the initial optimal continuous solution by (21) and the given specifications.

Step 2. Select the coefficient with maximum absolute value, for example a(~i, Q), and fix it at I discrete values in the vicinity of u(ri, r2), say uh(q,r2), i =

1 ,..‘, I.

Step 3. Establish the constrained equation for each of the I optimization problems under fixing the first quantized coefficient

BFA= Gi, i= l,..., 1, ₍₂₃₎

where Bi, i = l,..., I, are column vectors with zero elements except the (ri + Y~(A$ + 1) + 1)th element beunit and Gi, i = l,..., I, are one-element matrices with element aa T-Z), i = 1,. . . , I, respectively.

Step 4. Find the I sets of continuous solutions by

A = Q-1Z3i(Z3~Q-1Bj)-1Gi

+~Q-‘[B~(B~Q-‘Bi)-‘B~Q-’ - ZIP. (24)

Step 5. Select the coefficient with maximum absolute value from unquantized coefficients for each of the I sets of continuous solutions and fix it at I different discrete values, for example,

uy(kt

,

k2), i = 1,. . . ,I, j = l,... , I, where i denotes the order of I discrete values in the previous quantized step and j denotes that in the latest quantized step. ’

Step 6. Establish the constrained equation for each of the I2 further optimization problems under fixing the quantized coefficients by BzA= Gij, i = l,..., I, j= l,..., 1, ₍₂₅₎ where Bij = [Bi Zj], i= l,..., 1, j = l,..., 1, (26) and

G

Gij = q , i = l,..., 1, j = l,..., Z, [

1

(27)

inwhichZj,j = O,..., 1, are column vectors with zero elements except the element be unit whose position

(5)

agrees with that of the quantized coefficient in Step 5, and q, j = 0,. . . , , I are one-element matrices with element a:(/~, , kz ), j = 0,. . . , Z, for the corresponding i.

Step 7. Find the I2 sets of continuous solutions by (24) and calculate the error value

eA =s+PTA+ATQA+W.S(IBTA-GI) (28)

for the I2 optimization problems where B and G have been originally defined in (17)-(19), W is the weighting constant and S(M) denotes the summation of the elements in the matrix M. Then select I sets provide smallest value of eA for further optimization. Notice that the error in (28) is so defined that the constraints in (14) can be incorporated into the discrete optimization.

Step 8. If all the coefficients are quantized, go to the next step; otherwise set

B/=Bi/, I=1 >.. 2 2 .I (29)

and

G, = Gij, 1 = l,..., I, (30)

where the I sets of Bij and Gij are chosen in Step 7, then go to Step 5.

Step 9. Select the set provides the smallest error value eA from the I sets of discrete solutions obtained in Step 7 as the desired solution.

Example. A 5 x 9, (L,K) = (1,2) interpolation filter is designed in this example, and the desired response

72, 0 1 2

ci

r&z 0 4X 25 0 1 :18 18 -1 2 19 6 -4

L---L

3 4 -2 -4 4 0 -2 -2

Fig. I (a) The amplitude response of a 5 x 9, (L, K) = (1,2) interpolation

(h(nl , nl) multiplied by 26),

(6)

310 J.-J. Shyu et al. /Signal Processing 55 (1996) 305-311

is given by

~(~l,co2) = 4, w + 2% <271(0.2),

0, WI + 202 Z2~(0.7). (31)

The coefficients are coded with 7 bits (sign bit in- cluded). When I = 4 and W = 1000 are used, the resultant amplitude response is shown in Fig. l(a) and the filter coefficients in the first quarter plane are listed in Fig. l(b). The related results are tabu- lated in Table 1 accompanying with those of the algorithm in [9] (integer linear programming) and direct quantization. Notice that although the approach of direct quantization gives smaller integrated square error, but it results in significant constrained error, where the integrated square error is defined in (10) and the constrained error denotes the sum of square errors between constrained values and actual values over constrained points. Also, we present the results that the coefficient with minimum absolute value to be quantized first, but the performance is not so good. Moreover, through several design examples, we rec- ommend that the quantization is processed with the

maximum absolute valued coefficient to be chosen first.

4. Discussions and conclusions

An effective method has been proposed for designing discrete coefficient FIR filters for sampling structure conversion. The method associates successfully the Lagrange multiplier approach and a tree search algorithm, which makes it attractive for designing finite-wordlength digital filters. A measure error is proposed, such that the frequency-domain constraints for sampling structure conversion system can be incorporated into the design procedures. Comparing with the existing methods, the proposed method is simple and the performance is comparable.

From Table 1 and through several examples, we generally choose the coefficient with maximum absolute value to quantize first. Comparing with other algorithms for the choice of the coefficients, the integrated square error is 0.0098 in the proposed method

Table 1

The obtained results of the designed 5 x 9, (L, K) = (1,2) interpolation filter for different approaches

Method Integrated square error Initial solution 0.0036 Proposed approach Ia 0.0098 Proposed approach IIb 0.0117 Direct quantization 0.0084 Algorithm in [I] (integer programming) 0.0166 Passband peak error 0.1558 0.1599 0.1911 0.1562 0.1167 Stopband peak error 0.1411 0.1420 0.1829 0.1577 0.1214 Amplitude response in constrained points H(O, 0) 4 4 4 3.8906 4 H(0, n) 0 0 0 0.0156 0 H(n, 5) 0 0 0 0.0156 0 Design time on 486 PC (s) 6 20 20 7

a The proposed approach I chooses the maximum absolute-valued coefficient to quantize first. b The proposed approach II chooses the minimum absolute-valued coefficient to quantize first.

(7)

which is smaller than 0.0166 if the most sensitive coefficient is quantized first,

Generally, as the statements declared in Section 1, the proposed approach can be applied for different discrete coefficient spaces including the evenly distributed finite wordlength space and the nonuniformly distributed powers-of-two space. If the former is used, the coefficients can be implemented using both of floating or fixed point arithmetic, and the multiplication operation can be replaced by a few shift operation if the latter approach is applied.

For the choice of the number of bits, it is dependent on the design requirements. If the integrated square error cannot be satisfied, the number of bits of the quantized coefficient or the filter length should be increased.

As to the choice of I, the number of discrete values in the vicinity of the continuous optimum value to be quantized, the same results are obtained in the presented example when I = 2,4 and 8. By the expe- rience of several designed examples, the performance of I = 2 is good enough, but we usually take I = 4.

References

[I] R.S. Garlinkel and G.L. Nemhauser, Integer Programming, Wiley, New York, 1972.

[2] A. Knoll, “Filter design for the interpolation of highly subsampled pictures”, Signal Processing: Image Commun- ication, Vol. 3, June 1991. pp. 239-248,

[3] Y.C. Lim and S.R. Parker, “Discrete coefficient FIR digital filter design based upon an LMS criteria”, IEEE Trans. Circuits Systems, Vol. 30, October 1983, pp. 723-739. [4] Y.C. Lim and S.R. Parker, “FIR filter design over a

discrete powers-of-two coefficient space”, IEEE Trans. Acoust. Speech Signal Process., Vol. 31, June 1983, pp. 5833591.

[5] G.W. Medlin, J.W. Adams and C.T. Leondes, “Lagrange multiplier approach to the design of FIR filters for multirate applications”, IEEE Trans. Circuits Sysiems, Vol. 35, October 1988, pp. 1210-1219.

[6] SC. Pei and J.J. Shyu, “Design of two-dimensional FIR eigenfilters for sampling structure conversion”, IEEE Trans. Circuits Systems Video Tech.. Vol. 3, April 1993. pp. 1588162.

[7] M. Renfors, T. Huuhtanen, A. Nieminen and T. Koivunen, “Linear and nonlinear filters for sampling structure conversion of two-dimensional sequences”, in: L. Chiariglione, ed., Signal Processing of HDTV, ZZ. Elsevier, Amsterdam, 1990.

[8] J.J. Shyu and SC. Pei, “Lagrange multiplier approach to the design of two-dimensional FIR digital filters for sampling structure conversion”, IEEE Trans. Signal Process., Vol. 42, October 1994, pp. 2884-2886.

[9] P. Siohan, “2-D FIR filter design for sampling structure conversion”, IEEE Trans. Circuits Systems Video Tech., Vol. 1, December 1991, pp. 337-350.

[lo] P. Siohan and D. Peli, “Sampling pattern conversion with 2-D FIR filters”, Proc. 3rd Internal. Conf: on Image Proc. and its Applications, IEE Conf. Publication No. 307. July

1989, pp. 275-279.

[l l] P. Siohan and A. Benslimane, “Finite precision design of optimal linear phase 2-D FIR digital filters”, IEEE Trans. Circuits Systems. Vol. 36, January 1989, pp. 1 l-22.