Orthogonal polynomial-basis

In [22-23], G. Y. Jiang and Kuech proposed an orthogonal polynomial adaptive filter to accelerate the convergence of the polynomial model. In general, the input signals of each channel are not mutually orthogonal, i.e. E x k x k{ ( ) ( )} 0,^{i j} ≠ ∀ ≠i j. Thus, a new set of mutually orthogonal input signal has been introduced [23]:

1

1 , 1

( ) ( )

( )

^u

( )

^{u i}

( )

u u i

i

p k x k

p k x k q x k

−

=

=

= + ∑

for 1 u K< < . The orthogonalization coefficients q can be determined using the _{u i,} Gram-Schmidt orthogonalization processing.

In addition to those approaches, step size control is also usually used to overcome the problem of low convergence rate. In Chapter 3, we will introduce several conventional step size algorithms and proposed new step size control approach.

Chapter 3

Optimum Step Size For Nonlinear AEC

In addition to [21-23], step size control is also usually used to overcome the problem of low convergence rate.

In this Chapter, we will focus on step size control in Volterra structure. We know tradeoff between fast convergence rate and small residual error power. In LMS algorithm, normally, large step size gives a faster convergence rate but large residual error power. Thus the optimum step size means that providing fast convergence rate and small residual error power at the same time.

In following sections, the conventional step size control is introduced in section 3.1. In Section 3.2, we will derive the optimum time-& tap-variant step-size LMS (OTTLMS) algorithm which is derived by introducing an optimality criterion which is given by MMSE between coefficients errors of real kernel and adaptive coefficients.

Its practical implementations are proposed in Section 3.3. The echo path change and double talk conditions are considered in Section 3.4.

3.1 Conventional step size adjustment

0 0.5 1 1.5 2 2.5 3

x 10⁴ 0

5 10 15 20 25 30 35

iteration

ERLE(dB)

Large step size

Small step sizee

Fig 3.1 Trade off of LMS algorithm

In Fig 3.1, the evaluate of echo return loss enhancement; we can obtain that due to tradeoff between fast convergence rate and small residual error in traditional constant step size, various approaches employing varying step-size in linear echo cancellation have been proposed, including time-varying [9], tap-varying [10], and both time- & tap varying [11]. In this thesis, the word “time-varying” represents all taps use identical step size which is time variant. Similarly, the word “tap-varying”

means each tap has individual and tine-invariant step size, and the word “time- &

tap-varying” means each tap has its individual time-variant step size.

In addition to linear acoustic echo cancellation, Kuech proposed a time-&

tap –varying approach in second- order Volterra structure [12] for nonlinear echo cancellation field.

These typical approaches of step size adjustment for linear AEC are summarized below:

3.1.1 Linear AEC

[A] Variable step size LMS algorithm

The VSLMS approach [9] employs a time-varying (time-variant) step-size which is controlled by the power of the error signal. This is based on using large step-size when the AEC filter coefficient is far from the optimal solution, thus speeding up the convergence rate. Similarity, when the AEC filter coefficient is near the optimum solution, small step-size is used to achieve lower MSE, thus achieving better overall performance. The variable step size LMS algorithm works as follows.

'(k 1) '( )k e k2( ) with 0 1, 0 μ + =αμ +γ < <α γ >

where the time-variant step size is controlled by

max max step-size to provide faster tracking the echo path. Similarity, when the residual error is small, the step size is decreased to yield smaller residual error.

The constant μ_max is chosen to ensure that the MSE remains bounded and μ_min is chosen to provide a minimum level of tracking ability.

[B] Exponentially weighted step size NLMS algorithm

The exponentially weighted step-size NLMS (ESNLMS) algorithm [10] uses a different step-size (tap-varying) for each tap of an adaptive filter. These step-sizes are time-invariant and weighted proportional to the expected variation of a room impulse response. As a result, the algorithm adjusts coefficients with large echo path variation in large steps, and coefficients with small echo path variation in small steps. The ESNLMS algorithm is expressed as:

₂

where U_ESNLMS is the diagonal step-size matrix to account for the tap-variant step-sizes:

The elements μ_l are time-invariant and decrease exponentially from μ₁ to

μM with the same ratio γ that depends on the decay rate of the real room impulse response

c

.

[C] Optimum time-& tap-variant step size algorithm

The OTTLMS approach [11] is employed to minimize each tap coefficient error variance at each iteration step (i.e. ( )g k ). The coefficient error is the difference _l

between real kernel and adaptive coefficients. The optimum step size can be obtained by setting the derivative of tap coefficient error variance formula with respect to

l( )k

As the optimum step-size needs to know the room impulse response to evaluate coefficient error, it is not accessible in general, thus the author employed the recursive relation of second moment coefficient error and used the room impulse response exponential decay model for practical implementation [11].

3.1.2 Nonlinear AEC

We have already introduced step size control applied to linear AEC. However, in the nonlinear AEC application, it faced low convergence rate badly due to high computational complexity. In this section, we want to introduce several step size control approaches applied to nonlinear AEC

[A] Proportionate NLMS for second-order Volterra filters

For acoustic echo cancellation, it is reasonable to assume that the echo path is sparse, i.e., many coefficients are zeros, therefore only the nonzero active coefficients

need to be identified (updated). This is the idea behind the proportionate NLMS (PNLMS) [20] algorithm. It exploits the sparseness of such impulse response to achieve significantly faster adaptation than NLMS.

Kuech [19] proposed an extension of the proportionate NLMS to second-order Volterra filters. It assumes that the unknown echo can be modeled by a finite-length second-order Volterra filter. The nonlinear echo cancellation system model is summarized in Figure 3.2; the microphone signal ( )y k is composed of echo signal '( )y k , the noise signal ( )n k accounting for background noise, and the speech

Fig 3.2 Second-order Volterra acoustic echo canceller

By (2.3) (2.4), the input/output relation of a second-order Volterra filter is given by

2 2

To obtain a compact vector representation, we define

2

M andN represent memory lengths of linear and quadratic kernel, the lengths ₂ of x⁽²⁾( )k and h⁽²⁾( )k are both L₂ =N N₂( ₂+1) / 2

The PNLMS algorithm updates each coefficient of the filter independently of the others by adjusting the adaptation step-size in proportion to the estimated filter coefficient. Thus the extension of the proportionate NLMS to second-order Volterra filters is summarized:

{ }

The step-sizes are calculated from the last estimate of the filter coefficients so that a large coefficient receives a large step-size, it is intuitive that if the someone tap of adaptive filter coefficient (i.e.h_l^{( )i} ( )k ) is large value, the coefficient error of this tap should be large, thus if we give large step size to update, it may be increase the convergence rate. Hence, PNLMS converges much faster than NLMS.

We observe that both ESNLMS and PNLMS algorithms rely on the concept of using large step-size for large tap. It is quite intuitive that large tap will produce large estimate tap coefficient error and should use a large step-size for fast tracking. This is appropriate at the stage of initial adaptation.

[B] Optimum step-size for adaptive second-order Volterra filters

The approach [A] is the direct concept of using adaptive filter coefficients to control step size. In approach [B], Kuech [12] derived the optimum step size theoretically and proposed approximated model to practical application.

The concept of the optimum step size in Kuech approach [12] is identical to OTTLMS approach which is derived by introducing minimum MSE between the coefficient errors of Volterra filter and real echo path.

The desired optimum step sizes for linear and quadratic kernel are shown below respectively The linear and quadratic kernel coefficient error in time k can be defined by

(1)

( ) k =

(1)

( ) k −

(1)

The residual echo comes from filter coefficient errors of the linear and quadratic

For a better understanding of the optimum step size, in [12], the author introduced the auxiliary step size factors:

2 2

The above definitions of step-size factors are used to factorize the optimum step sizes according to general, so the author introduce models for estimating those parameters :

1. The E{[ε^{( )i} ( )] }k ² are proportionate to the adaptive filter output of linear and quadratic kernel, respectively

( ) 2 ( )

{[ ⁱ ( )] } ( )[ _{i i}( ) ⁱ ( ) ]

E ε k ≈γ k δ β+ k z k i∈{1, 2}

2. The second-moment of coefficient error is proportionate to the magnitude of the corresponding adaptive coefficient

{ }

For comparison with [11], our work is extending [11] to nonlinear system, second-order Volterra. We employed the advantage of (second moment of coefficient error) recursive relation in [11] and proposed a different practical implementation from Kuech approach [12].

Next, in Section 3.2, we will derive the optimum time-& tap-variant step-size LMS (OTTLMS) algorithm which is derived by introducing an optimality criterion which is given by MMSE between coefficients errors of real kernel and adaptive coefficients. The practical implementation is proposed in section 3.3. The echo path change and double talk conditions are considered in section 3.4.

3.2 Derivation of optimum time-& tap-variant step-size LMS (OTTLMS) algorithm

In this section, we will extend [11] to second-order Volterra filter by getting recursive relation of coefficient errors. By this extension, we not only speed up the convergence rate in linear acoustics echo problem, but also in nonlinear echo cancellation. There our notations are identical to section 3.1 (see Fig 3.2)

We want to find out the step size in time k which can minimize each tap coefficient error variance in time k+1 i.e. MSE for each iteration step. Hence, we use diagonal matrixes U⁽¹⁾( )k and U⁽²⁾( )k to replace the step size of conventional LMS algorithm [13], thus it corresponding LMS algorithm can be rewritten as

(1)

(

k

+ = 1)

(1)

( )

k

+

(1)

( ) ( )

k e k (1)

( )

k

h h U x (3.4)

(2)

(

k

+ = 1)

(2)

( )

k

+

(2)

( ) ( )

k e k (2)

( )

k

h h U x (3.5)

(1) (1) (2) (2)

where the l element of step size matrices is chosen to minimize ^th l coefficient ^th error variance in time k+1. The criterion is summarized as

where i∈1, 2, and it means the linear and quadratic kernel, respectively.

By (3.2), (3.3), (3.6), we get recursive relation of linear kernel:

[ ]

Using (3.2) and (3.7), we may rewrite the linear kernel coefficient error v⁽¹⁾( )k

[ ]

Similar to processing in [11], we can derive the autocorrelation matrix of the linear kernel coefficient errors as follows, and by the direct-average method [13]

(1) (1) (1) From formula (3.9), the^E

{ }

^. denotes expectation. By assumption of the mutual

independence ofx k

( )

, n k( )and ( )s k , and probability density function of x k( ) is an even function, as then ^{E x k}

{

^{3( )}

}

⁼⁰ . Thus cross products terms[I U− ⁽¹⁾( )k x⁽¹⁾( )k x^(1)T( )]k v⁽¹⁾( )k , U x^{(1) (1)}( )k x^(2)T( )k v⁽²⁾( )k , U⁽¹⁾n k( )x⁽¹⁾( )k and

(1)s k

( )

(1)

( )

k

U x in formula (3.9) could be neglected.

The l’th diagonal term of autocorrelation matrix, denoting l’th mean-square of linear coefficient error, can be written as:

{ }

kernel in Volterra M and L is sufficiently large, we can approximate ₂ 4

4 x x

m ≈σ σ

≈

2 The optimum time-& tap variant step-size can be obtain by taking derivative of Eq (3.11) with respect to μ_l⁽¹⁾( )k and setting the result equal to zero.

Thus we can get the optimum time-&tap-variant step-size of linear kernel

² Analogously to linear kernel, we can get the optimal step size of quadratic kernel is given by From the result of (3.12) and (3.13), we can obtain that the optimum step sizes are direct proportion to the coefficient error variance. If the coefficient error variance large (i.e. initial state), the optimum step sizes are large; and if the coefficient error variance small, the optimum steps are become small to get small residual error, the result fits our intuition.

The numerator of (3.12) and (3.13) mean that the second moment coefficient error of linear and quadratic kernel, respectively (i.e. ^{gl(1)( )k E}

{

^{⎡⎣vl(1)( )k ⎤⎦2}

}

^,

{

²

}

(2)_j ( ) (2)_j ( )

g k E ⎡⎣v j ⎤⎦ ), and the denominator of (3.12) and (3.13) mean the summation of residual error power and near-end speech power, i.e.

{

²

}

Thus we can find that the results of (3.12) and (3.13) fit the work (3.1) in [12].

Similar to processing in [11], we substitute the optimum time-&tap-variant step size of linear and quadratic kernel back to (3.10), thus we can get that the relationship mean-square coefficient errors

(1) (1) 2 (1)

We found the results fit the works on tradition AEC [11].

Double talk condition is not considered in this section, we set ( ) 0s k = , the double talk and echo path change conditions will be considered in section 3.5, thus the approximated OTTLMS algorithm for second-order Volterra filter is summarized in Table 3.1:

(1) (1) (2) (2)

Table 3.1: OTTLMS algorithm

3.3 Extension to OTTNLMS algorithm

The above discussions are based on LMS algorithm. However, when the input is large, the LMS algorithm suffers from a gradient noise amplification problem. In order to overcome this difficulty, we extend it to the normalized LMS (NLMS) algorithm. By the approximation [13] of x^(1)T( )k x⁽¹⁾( )k =Mσ_x² and

2

Similarly, the (3.14) can be rewritten as:

(1) 2

Thus the OTTNLMS algorithm for second-order Volterra filter is summarized in Table 3.2:

Table 3.2 OTTNLMS algorithm

3.4 Practical implementations of OTTLMS algorithm

In Section 3.2, we have already derived optimum time-&tap variant step-size for LMS and NLMS algorithm in Table 3.1 and Table 3.2. Here, the OTTLMS and OTTNLMS not only need prior statistics knowledgeσ_x²,σ_x⁴, andσ_n², but also the prior knowledge of second moment of coefficient error g_l⁽¹⁾( )k and g⁽²⁾_j ( )k . Thus we must be known the real room impulse response c and second order kernel caused ⁽¹⁾ by nonlinear loudspeakerc . In general case, the echo path ⁽²⁾ c and ⁽¹⁾ c⁽²⁾ are not accessible. In section 3.3.1, we propose a model function to estimate those parameters for the application in nonlinear acoustics echo cancellation.

3.4.1 Exponential models of linear and quadratic kernel

Unlike the approximation approach of Kuech approach [12] in Section 3.1.2, we introduced the recursive formula (3.14), thus we only need to know the real envelope of real echo path (i.e.g_l⁽¹⁾(0) E h{⎡⎣ _l⁽¹⁾(0)−c_l⁽¹⁾⎤⎦²} ⎡⎣c_l⁽¹⁾⎤⎦ ), thus we proposed an ² exponentially models for implementation.

Here, we will assume reasonably the real linear and quadratic kernelc , ⁽¹⁾ c ⁽²⁾ can be modeled as an exponentially decaying envelope shown in Fig 3.3, and Fig 3.5.

Let the linear and quadratic envelope functions modeled as:

(1) (1) (1) where r⁽¹⁾ and r⁽²⁾ are linear and quadratic kernel exponential decay factors respectively.

0 50 100 150 200 250 300

Fig 3.3 Real Linear kernel and exponential model of the envelope

0

Fig 3.4 Real quadratic kernel of the nonlinear loudspeaker

0 5 10 15 20

Fig 3.5 Exponential model of the envelope of the quadratic kernel

The diagonal elements of tap coefficient error variance matrix R_v(1)

( )

k and

( 2)

( )

R

_v

k

_{are gl}⁽¹⁾_{( )k = E h[( l}⁽¹⁾_{( )k −cl}^{(1) 2}_{) ] and g}⁽²⁾_{j ( )k =E h[(} ⁽²⁾_{j ( )k −c}^{(2) 2}_{j ) ] ,}

respectively. We let the initial linear and quadratic tap coefficients to be zero.

i.e. h_l⁽¹⁾(0) 0= , h⁽²⁾_j (0) 0= , so g_l⁽¹⁾(0)=E h[( _l⁽¹⁾(0)−c_l^{(1) 2}) ]=⎡⎣c_l⁽¹⁾⎤⎦² ≈⎡⎣w_l⁽¹⁾⎤⎦ and ² quadratic kernel filter μ⁽²⁾_{j OTTLMS,} (0) , with initial step-size plugged into (3.13) we can get g_l⁽¹⁾(1) and g⁽²⁾_j (1) , and so forth . Thus, we can find μ_{l OTTLMS}⁽¹⁾_, ( )k andμ⁽²⁾_{j OTTLMS,} ( )k , recursively. The practical OTTLMS algorithm with exponentially envelope model functions can be summarized as follows:

1. Measure the exponential decay factor of linear and quadratic kernel r and ⁽¹⁾ r(2) to get w_l⁽¹⁾=w₀⁽¹⁾(r⁽¹⁾)^l and ^{1 2}

1 2

( )

(2) (2) (2)

, 2 ( )^{l l}

wl l =w r ⁺ .

2. Set up initial value g_l⁽¹⁾(0)≈ ⎣ ⎦ for 1, ,⎡w_l⁽¹⁾⎤² l= … M and

1 2

(2) (2) 2

(0) ,

j l l

g ≈ ⎣⎡w ⎤⎦ for for ,l l_{1 2} = …1, ,N₂

3. According to table.3.1

By using the exponential function to model the linear and quadratic kernel, we can practically implement the OTTLMS algorithm, whose performance will be verified in Chapter 5.

3.4.2 Exponentially approximated temporal function of step-size in OTLMS and OTNLMS

In Section 3.3, we proposed practical implement in OTTLMS. Now we would further obtain property of step size in adaptive processing.

Fig 3.6 Linear kernel step size temporal function

Fig 3.7 Quadratic kernel step size temporal function

By Fig 3.6 and 3.7, for our optimum step size in AEC, we obtain that large step size in initial times and small in converged times.

To reduce computation complexity, we want to derive the exponentially approximation temporal function of step size, with the approximated step size function, we can abbreviate the calculation of (second moment) coefficient error.

To simplify calculation, we assume that all taps of step size are equal, thus we can rewrite optimum step sizes (3.12) and (3.13) as

(1)

Similarly, the mean-square coefficient error could be rewritten as

(1) (1) 2 (1) We put the recursive relation of mean-square coefficient error (3.19) into (3.17), the OTLMS of linear kernel can be expressed as :

(1) 2 (1)

where common ratios of mean-square coefficient error are defined as

(1)( ) (1i (1)( ) )i _x2

α = −μ σ and

α

⁽²⁾( ) (1i = −

μ

⁽²⁾( ) )i

σ

_x⁴ ,for time i=1 ~k−1

By assumption of the common ratios are identical for the whole time (i.e.

(1)(k 1) (1)(k 2) ... (1)(0) (1)

Analogously to linear kernel,

Similarly, the exponential approximated OTLMS algorithm for second-order Volterra filter is summarized in Table 3.3:

(1) (1) (2) (2)

Table 3.3 Exponentially approximated temporal function of step-size in OTLMS

Similarly, for NLMS algorithm, we have:

(1) 2 4 (1)

3.5 Double Talk and Echo Path change conditions

We have already proposed OTTLMS algorithm and assumed single talk case in Section 3.2. In this section, we consider the double talk (i.e. ( ) 0s k ≠ ) and echo path change conditions.

From the (3.12) and (3.13), the determination of μ_{l OTTLMS}⁽¹⁾_, ( )k and μ⁽²⁾_{j OTTLMS,} ( )k , as statistics knowledge of near-end σ_s² are not accessible, it is intuitive that residual error variance σ_e²( )k is near to σ_n²+σ_s² in converged condition. Thus we

introduced the estimated residual error variance σˆ ( )_e² k to model the background noise and near end speech variance for practical implementation which is using the smoothed recursive algorithm from square of residual error:

σ ˆ

_e²

( ) k = λσ ˆ

_e²

( k − + − 1) (1 λ ) ( ) e k

²

where λ is constant.

Next, we look at the echo path change condition. When the echo path changes, our proposed optimum step size is not robust because our step sizes are still restricted to small value in converged single-talk.

In OTTLMS algorithm, we introduce the recursive relation to evaluate the optimum step size, rather than calculate step size repetitively. These introduction constraints that our optimum step sizes are to be small in converged condition even echo path changes suddenly.

0 0.5 1 1.5 2 2.5 3 3.5 4

Fig 3.8 OTTLMS during echo-path variations

Fig 3.8 shows the comparison of OTTLMS and LMS during echo-path variations.

The room response is changed after 12000 iterations. According to the simulation, the convergence rate of OTTLMS algorithm is increased as compared to the LMS algorithm; however, it can be observed from Fig 3.8, the result is different after echo path changed. The reason is the recursive characteristic of (3.14), whether echo path changes or not, the mean-square coefficient errors of linear and quadratic kernel are smaller and smaller, it leads to our proposed optimum step sizes are smaller and smaller, so even if the echo path changes after 12000 iterations, the step sizes are still very small at convergence.

Thus we introduce a detector [16] to detect the echo path change, by a direct measure of the adaptive filter’s convergence. Referring to Fig 3.2, the cross correlation between the desired signal ( )y k , and the residual error ( )e k is given by: The proposed convergence statistic definition from [16] is given by:

( )

measure convergence of the adaptive filter.

When the echo path change detector detects the echo path change, we re-initialize from g_l⁽¹⁾( )k tog_l⁽¹⁾(0), it means that we re-update the linear adaptive filterh⁽¹⁾, thus we can overcome the echo path change condition. The simulation result will be shown in Section 5.2.7.

3.6 Computational complexity

We have already discussed OTTLMS algorithm with an increased the convergence rate. This section we make a computation comparison of OTTLMS, LMS, and Kuech approach, we examine the number of multiplications required to make once complete iteration of the algorithm. The recursive relation of second moment coefficient error in (3.13) is need 2M +2L₂ multiplications, the update equation of OTTLMS in Table 3.1 is need 2M +2L₂ multiplications, thus the total requirement multiplication of OTTLMS is 5M +5L₂

Similarly, the multiplications of [12] are about4M+4L₂, and the EAOTTLMS (Exponentially approximation temporal function of OTTLMS) in Section 3.4.2, it needs only about2M +2L₂.

The computation complexity for OTTLMS are summarized in Table 3.3

Algorithm Multiplications/sample

LMS 2M +2L2

Kuech[12] 4M +4L2

OTTLMS 5M+5L2

EAOTTLMS 2M +2L2

Table.3.3 Computation complexity comparison of different algorithms

3.7 Summary

In this chapter, we propose the optimum step size in second-order Volterra structure, in section 3.1, we introduced the conventional step size control algorithm, in section 3.2, and the optimum step-size is derived by introducing an optimality criterion which is given by MMSE between coefficients errors of real kernel and adaptive coefficients.

In Section 3.3 we extend to NLMS algorithm. In Section 3.4 we propose exponentially model function to practical implement because the prior knowledge of echo path is not easy to be acquired. To save the computational complexity, the exponentially approximated temporal function was derived in Section 3.5.

In Section 3.6, the echo path change and double talk conditions were considered, and the computational complexity was summarized in Section 3.7. The overall of

In Section 3.6, the echo path change and double talk conditions were considered, and the computational complexity was summarized in Section 3.7. The overall of

在文檔中 根據最佳收斂步伐與通道削減來提升非線性迴音消除的收斂性 (頁 24-0)