Exponential models of linear and quadratic kernel

Unlike the approximation approach of Kuech approach [12] in Section 3.1.2, we introduced the recursive formula (3.14), thus we only need to know the real envelope of real echo path (i.e.g_l⁽¹⁾(0) E h{⎡⎣ _l⁽¹⁾(0)−c_l⁽¹⁾⎤⎦²} ⎡⎣c_l⁽¹⁾⎤⎦ ), thus we proposed an ² exponentially models for implementation.

Here, we will assume reasonably the real linear and quadratic kernelc , ⁽¹⁾ c ⁽²⁾ can be modeled as an exponentially decaying envelope shown in Fig 3.3, and Fig 3.5.

Let the linear and quadratic envelope functions modeled as:

(1) (1) (1) where r⁽¹⁾ and r⁽²⁾ are linear and quadratic kernel exponential decay factors respectively.

0 50 100 150 200 250 300

Fig 3.3 Real Linear kernel and exponential model of the envelope

0

Fig 3.4 Real quadratic kernel of the nonlinear loudspeaker

0 5 10 15 20

Fig 3.5 Exponential model of the envelope of the quadratic kernel

The diagonal elements of tap coefficient error variance matrix R_v(1)

( )

k and

( 2)

( )

R

_v

k

_{are gl}⁽¹⁾_{( )k = E h[( l}⁽¹⁾_{( )k −cl}^{(1) 2}_{) ] and g}⁽²⁾_{j ( )k =E h[(} ⁽²⁾_{j ( )k −c}^{(2) 2}_{j ) ] ,}

respectively. We let the initial linear and quadratic tap coefficients to be zero.

i.e. h_l⁽¹⁾(0) 0= , h⁽²⁾_j (0) 0= , so g_l⁽¹⁾(0)=E h[( _l⁽¹⁾(0)−c_l^{(1) 2}) ]=⎡⎣c_l⁽¹⁾⎤⎦² ≈⎡⎣w_l⁽¹⁾⎤⎦ and ² quadratic kernel filter μ⁽²⁾_{j OTTLMS,} (0) , with initial step-size plugged into (3.13) we can get g_l⁽¹⁾(1) and g⁽²⁾_j (1) , and so forth . Thus, we can find μ_{l OTTLMS}⁽¹⁾_, ( )k andμ⁽²⁾_{j OTTLMS,} ( )k , recursively. The practical OTTLMS algorithm with exponentially envelope model functions can be summarized as follows:

1. Measure the exponential decay factor of linear and quadratic kernel r and ⁽¹⁾ r(2) to get w_l⁽¹⁾=w₀⁽¹⁾(r⁽¹⁾)^l and ^{1 2}

1 2

( )

(2) (2) (2)

, 2 ( )^{l l}

wl l =w r ⁺ .

2. Set up initial value g_l⁽¹⁾(0)≈ ⎣ ⎦ for 1, ,⎡w_l⁽¹⁾⎤² l= … M and

1 2

(2) (2) 2

(0) ,

j l l

g ≈ ⎣⎡w ⎤⎦ for for ,l l_{1 2} = …1, ,N₂

3. According to table.3.1

By using the exponential function to model the linear and quadratic kernel, we can practically implement the OTTLMS algorithm, whose performance will be verified in Chapter 5.

3.4.2 Exponentially approximated temporal function of step-size in OTLMS and OTNLMS

In Section 3.3, we proposed practical implement in OTTLMS. Now we would further obtain property of step size in adaptive processing.

Fig 3.6 Linear kernel step size temporal function

Fig 3.7 Quadratic kernel step size temporal function

By Fig 3.6 and 3.7, for our optimum step size in AEC, we obtain that large step size in initial times and small in converged times.

To reduce computation complexity, we want to derive the exponentially approximation temporal function of step size, with the approximated step size function, we can abbreviate the calculation of (second moment) coefficient error.

To simplify calculation, we assume that all taps of step size are equal, thus we can rewrite optimum step sizes (3.12) and (3.13) as

(1)

Similarly, the mean-square coefficient error could be rewritten as

(1) (1) 2 (1) We put the recursive relation of mean-square coefficient error (3.19) into (3.17), the OTLMS of linear kernel can be expressed as :

(1) 2 (1)

where common ratios of mean-square coefficient error are defined as

(1)( ) (1i (1)( ) )i _x2

α = −μ σ and

α

⁽²⁾( ) (1i = −

μ

⁽²⁾( ) )i

σ

_x⁴ ,for time i=1 ~k−1

By assumption of the common ratios are identical for the whole time (i.e.

(1)(k 1) (1)(k 2) ... (1)(0) (1)

Analogously to linear kernel,

Similarly, the exponential approximated OTLMS algorithm for second-order Volterra filter is summarized in Table 3.3:

(1) (1) (2) (2)

Table 3.3 Exponentially approximated temporal function of step-size in OTLMS

Similarly, for NLMS algorithm, we have:

(1) 2 4 (1)

3.5 Double Talk and Echo Path change conditions

We have already proposed OTTLMS algorithm and assumed single talk case in Section 3.2. In this section, we consider the double talk (i.e. ( ) 0s k ≠ ) and echo path change conditions.

From the (3.12) and (3.13), the determination of μ_{l OTTLMS}⁽¹⁾_, ( )k and μ⁽²⁾_{j OTTLMS,} ( )k , as statistics knowledge of near-end σ_s² are not accessible, it is intuitive that residual error variance σ_e²( )k is near to σ_n²+σ_s² in converged condition. Thus we

introduced the estimated residual error variance σˆ ( )_e² k to model the background noise and near end speech variance for practical implementation which is using the smoothed recursive algorithm from square of residual error:

σ ˆ

_e²

( ) k = λσ ˆ

_e²

( k − + − 1) (1 λ ) ( ) e k

²

where λ is constant.

Next, we look at the echo path change condition. When the echo path changes, our proposed optimum step size is not robust because our step sizes are still restricted to small value in converged single-talk.

In OTTLMS algorithm, we introduce the recursive relation to evaluate the optimum step size, rather than calculate step size repetitively. These introduction constraints that our optimum step sizes are to be small in converged condition even echo path changes suddenly.

0 0.5 1 1.5 2 2.5 3 3.5 4

Fig 3.8 OTTLMS during echo-path variations

Fig 3.8 shows the comparison of OTTLMS and LMS during echo-path variations.

The room response is changed after 12000 iterations. According to the simulation, the convergence rate of OTTLMS algorithm is increased as compared to the LMS algorithm; however, it can be observed from Fig 3.8, the result is different after echo path changed. The reason is the recursive characteristic of (3.14), whether echo path changes or not, the mean-square coefficient errors of linear and quadratic kernel are smaller and smaller, it leads to our proposed optimum step sizes are smaller and smaller, so even if the echo path changes after 12000 iterations, the step sizes are still very small at convergence.

Thus we introduce a detector [16] to detect the echo path change, by a direct measure of the adaptive filter’s convergence. Referring to Fig 3.2, the cross correlation between the desired signal ( )y k , and the residual error ( )e k is given by: The proposed convergence statistic definition from [16] is given by:

( )

measure convergence of the adaptive filter.

When the echo path change detector detects the echo path change, we re-initialize from g_l⁽¹⁾( )k tog_l⁽¹⁾(0), it means that we re-update the linear adaptive filterh⁽¹⁾, thus we can overcome the echo path change condition. The simulation result will be shown in Section 5.2.7.

3.6 Computational complexity

We have already discussed OTTLMS algorithm with an increased the convergence rate. This section we make a computation comparison of OTTLMS, LMS, and Kuech approach, we examine the number of multiplications required to make once complete iteration of the algorithm. The recursive relation of second moment coefficient error in (3.13) is need 2M +2L₂ multiplications, the update equation of OTTLMS in Table 3.1 is need 2M +2L₂ multiplications, thus the total requirement multiplication of OTTLMS is 5M +5L₂

Similarly, the multiplications of [12] are about4M+4L₂, and the EAOTTLMS (Exponentially approximation temporal function of OTTLMS) in Section 3.4.2, it needs only about2M +2L₂.

The computation complexity for OTTLMS are summarized in Table 3.3

Algorithm Multiplications/sample

LMS 2M +2L2

Kuech[12] 4M +4L2

OTTLMS 5M+5L2

EAOTTLMS 2M +2L2

Table.3.3 Computation complexity comparison of different algorithms

3.7 Summary

In this chapter, we propose the optimum step size in second-order Volterra structure, in section 3.1, we introduced the conventional step size control algorithm, in section 3.2, and the optimum step-size is derived by introducing an optimality criterion which is given by MMSE between coefficients errors of real kernel and adaptive coefficients.

In Section 3.3 we extend to NLMS algorithm. In Section 3.4 we propose exponentially model function to practical implement because the prior knowledge of echo path is not easy to be acquired. To save the computational complexity, the exponentially approximated temporal function was derived in Section 3.5.

In Section 3.6, the echo path change and double talk conditions were considered, and the computational complexity was summarized in Section 3.7. The overall of discussion will be verified in Chapter5.

In addition to step size control of second-order Volterra, the higher-order Volterra model was not considered here. Because the optimum step size deriving processing is more complicated, for example, (3.8) will be have many cross term which is leading to hard to get the second moment of coefficient error in (3.10).

In Hammerstein model, as it is cascade structure, the joint error term produced by linear and nonlinear term, thus it is difficult to perform its optimum step size.

Chapter 4

Channel Shortening Structure For Nonlinear AEC

Another alternative is the channel shortening technique that has been proposed in [14] to overcome high computational complexity and low convergence rate disadvantages of Hammerstein structure.

In Chapter 4, we will investigate the issues of channel shortening approach. The channel shortening approach will be introduced in Section 4.1, in Section 4.2, we will perform theoretical analysis in the senses of LMS and LS in case of a linear loudspeaker to obtain the converged tendency.

In addition to theoretical linear analysis, we will propose multiple nonlinear stage update scheme to accelerate convergence rate in Section 4.3, and finally we apply the channel shortening with second-order Volterra filer in Section 4.4.

4.1 Channel shortening approach

Kun Shi [14] proposed a novel algorithm based on Hammerstein model, Fig.4.1 shows the structure of nonlinear acoustics echo cancellation, it introduced an FIR shortening filter ( )w k is introduced after the acoustics path. The purpose of shortening filter ( )w k is to “shorten” the channel, which is the convolution of the room impulse response and w( )k to have fewer number of non-negligible taps.

…

Fig4.1 Channel shortening structure for nonlinear AEC

In [14], the author performed the RLS algorithm for nonlinear polynomial coefficientaˆ( )k , and NLMS algorithm for adaptive AEC ( )h k and shortening filter ( )w k . Here we will focus the NLMS algorithm.

Suppose that the lengths of shortening filter w(k) and the AEC filter ˆ( )h k are

L and w L , respectively . Define the vectors _s

[ ]

( ) k = x k x k ( ), ( − 1), , ( x k − L

_s

+ 1)

^T

x …

The reference signal (output of shortening filter) d(k) can be defined as

( )

^T

( ) ( )

where aˆ( )k is the estimated coefficients vector of the nonlinear processor. Therefore,

ˆ( )

k

where p_i is the polynomial basis of order i , for example

2 1( ) ( ) and 2( ) ( )

p k =x k p k =x k in case of a power series expansion basis. K is the order of the polynomials. The estimated error is

( ) ( ) - ( )

The gradient of the error powere k , as derived in [15], can be calculated according ²( ) to:

2 NLMS-type adaptive algorithm is given by

2 In order to avoid trivial solutions, the author constrained that two-norm of polynomial and linear filter are equal to one (i.e. aˆ ₂ =1,

2

ˆ =1

h ).Thus, unlike the RLS [14], the NLMS algorithm for channel shortening structure was summarized as

z ₂

z ₂

We did some simulations to verify that faster converged rate in this structure. The far end signal ( )x k was generated according to an i.i.d Gaussian distribution. The room impulse response was generated by a random number generator with an exponential damping factor and we assume the length of room impulse response is equal to 350.

The nonlinear loudspeaker is modeled by polynomial function

2 3 4 5

( ) .89 0.002 - 0.3 0.001 0.5

f x = x + x x + x + x

The length of shortening filter L and linear adaptive _w L are equal to 250 and _s 100, respectively, and the nonlinear polynomial filter order isK =5.

To evaluate system performance, residual error power, performance measure of echo return loss enhancement (ERLE), and coefficient misalignment are major system performance measures for comparison purposes. With the assumption of high SNR, the (ERLE) can be formulated as

2

Fig 4.2 Comparison of classical Hammerstein and channel shortening structure

0 2000 4000 6000 8000 10000 12000

-5 0 5 10 15 20 25

iteration

ERLE

classical Hammerstein

Channel shortening approach

0 0.5 1 1.5 2 2.5 3

x 10⁴ 0

5 10 15 20 25 30

iteration

ERLE(dB)

Comparison of channel shortening and classical structure

Channel shortening structure

classical Hammerstein structure

0 50 100 150 200 250

Fig 4.3 Shortening filter, original channel and channel shortened channel In Fig 4.3, the coefficients of shortening, original and shortened channels were displayed; the shortened channel is the convolution of shortening filter and real original channel, we omit the shortened channel from 351 to 599 taps, which the amplitude value can be neglected.

We can obtain that shortening filter reduces the length of room impulse response from about 250 to 100 taps. As the reduction, the shortening structure reduces length of adaptive filter h from 350 to 100.

For computational complexity, we examine the number of multiplications required to make once complete iteration of the algorithm (4.1), (4.2) and (4.3). s( )k in (4.1) and its 2-norm need nonlinear order K and linear filter tap M multiplications respectively thus the total requirement multiplication of (4.1) is about 2M + K. For (4.2), ( ) ( )P^T k h k and its 2-norm need MK and K multiplications respectively thus

the total requirement of (4.2) is about MK+2K. Thus we know that the total requirement multiplication of classical Hammerstein structure is about MK+2M+3K.

In channel shortening structure, we reduce the linear FIR filter from M (i.e. Ls+Lw) to Ls. The requirement multiplication of (4.3) is about Lw. In order to avoid trivial solutions, the renormalized term in channel shortening approach are added, the requirement multiplication is about Ls+K.

Table 4.1 Computation complexity comparison of classical and shortening structure

By Table 4.1, it can be obtained that the main computation lies in (4.1) and (4.2), thus although it increases multiplication due to shortening filter, it still can reduce the multiplication complexity, because the dominate term of computation complexity of classical Hammerstein structure is MK, which was reduced to KLs.

Number of multiplication

Classical Hammerstein structure (Ls+Lw)K+2M+3K Channel shortening structure KLs+2Ls+3K+Lw+Ls+K

4.2 Theoretical analysis of linear echo channel

In this section, in order to discuss convergent behavior in the structure, we analyze the coefficient error under the assumption of linear loudspeaker and. By the assumption, we simply the system model in Fig 4.4.

Fig 4.4 Shortening structure for the linear loudspeaker

4.2.1 Least-square solutions

In order to discuss the Least-Square solution in this structure, we don’t take account of time index k.

In our analysis, we use the residual error

e

defined by the following equation.

* * *

= ∗ − +

e x w c x h n w

(4.4) where ∗ is linear convolution operator , c is room impulse response with finite length M , and the n denotes the background noise.

[ ... ]c1 c_M ^T

= c

[ ...1 ] where Y and X are convolution matrix version of microphone input signal y and far end signal, respectively, andL_q =M+L_w− 1

In order to get the least-square solution of wand h , we assume that the first element of shortening filter w₁= to separate (4.5) into two term form, Thus (4.5) can be 1 rewritten as:

= +

e b As

(4.6)

where

The normal equation of (4.6) can be written as

T T

A As = -A b

(4.7) Therefore, in order to minimize the coefficient error, the least-square solution of

x

can given by:

=

^{T -1 T}

s -(A A) A b (4.8)

4.2.2 Adaptive LMS algorithm and its convergence analysis

In section 4.2.1 , we analyzed the coefficient error in least-square sense , we will derived the theoretical coefficient error based on the adaptive LMS algorithm , we want to observe and discuss the result of LMS algorithm in shortening structure to see if it can achieve the least-square solution or not.

Using (4.5), the coefficient error at timek+1.

In the following analysis, we assume the step sizes of shortening filter w and FIR

filter h are equal, i.e.

μ

_w =

μ

_h =

μ

. Using LMS algorithm, we have

Using (4.9), (4.10), we rewrite the coefficient error as (k+ =1) ( )k −

μ

[ k + _Ls k ] ^T_Lq( ) ( )k k According to the direct averaging method [13], whenμ is very small, the coefficient error (ε k+1) can be approximated as follows:

where R_q =CH I_q+ _Ls×Ls,

By applying the similarity transformation,R_q is transformed into a simpler form: Hence, we get the theoretical coefficient error in LMS algorithm sense. In chapter 5, we would simulate it and compare with the least-square solution.

We will note that the theoretical mean-square error is

{

²

}

^{2 2}2

( ) ( )

_x

( )

J k = E e k = σ ε k

(4.14)

4.2.3 Non-unique converged value

To further understand the converged behavior, we discuss the converged value of

adaptive filter ( )h k and shortening filter ( )w k , we want to discuss the convergence toward the optimal parameters in the system. Indeed, the mean-square error

( ) { ( )}2

J k =E e k produces some local minima, implying the convergence toward incorrect parameters depending on the initialization of ( )h k and ( )w k .We study this phenomenon in this section. From (4.10), the square of residual error e k²( ) can be The squared error can be used to get the optimal parameter set in the minimum mean-square error sense (MMSE).To simplify analysis, we assume that the other filter is quasi-constant when we analyses one filter.

For the linear AEC filter ( )h k , the gradient of ( )J k respect to ( )h k is given by

4.3 Multiple stage update in channel shortening structure

As seen (4.16) and (4.17), in linear echo path channel, we can point out several things. First, the different initial value of filter (whatever AEC or shortening filter) will cause the shortening structure converge to different value; second, the system is not able to identify the optimal set of parameters, unless one of filter is know.

It has identical situations in nonlinear channel shortening structure (i.e. Fig 4.1).

In this section, we will try to change the way of updating in channel shortening structure, rather than joint update.

The Guerin [3] proposed the two-staged strategy which it starts with one filter, and joint adaption of all filters (i.e. polynomial filtera( )k , adaptive AEC filter ( )h k , and shortening filter ( )w k ) once the linear filter has sufficiently converged in first stage.

In channel shortening structure, the two-staged strategy means change the initial value of filters, as when the first stage finished, the initial value is identical to the converged value in first stage.

By the idea, we want to obtain and compare that the performance of different multiple update strategies (i.e. different initial value of filters). The overall of discussion will be verified in Section 5.3.

4.4 Volterra with channel shortening and OTTLMS

In this section, we try to implement the shortening structure to second order Volterra, we hope to the shortening filter shorten the linear kernel of echo path and lead to improve the convergence rate in Volterra structure.

It is identical idea with Hammerstein, the purpose of shortening filter ( )w k is to “shorten” the linear kernel, which is the convolution of the room impulse response and ( )w k to have shorter taps.

In addition to channel approach for Volterra structure, we implement the OTTLMS algorithm to the combination ( i.e. channel shortening in Volterra filter), The overall of discussion will be verified in Section 5.3.

Chapter 5

Computer Simulation

To evaluate the performance of our proposed nonlinear AEC algorithm, we provide computer simulations. In Section 5.1, we introduce the parameters of our simulation. A series of simulations and experiments on the optimum time-&tap-variant step size, will be compared and discussed in Section 5.2. In Section 5.3 we will compare simulations results with theoretical analyses in channel shortening structure.

5.1 Simulation parameters introduction

The signal to noise ratio at microphone is defined as

10log (

10 ^echo

)

noise

SNR P

=

P

where P_echo is power of the nonlinear echo and the P_noise is the power of the background noise.

For simplicity, we use a 256-tap room impulse response as shown in Fig 5.1.1. It

is generated by a random number generator with an exponential damping factor.

Nonlinear memory echo path is shown in Fig 5.1.2. In this thesis, we use a 20-memory kernel.

In our experiment, we not only use an i.i.d white Gaussian signal, but also speech signal as the input signal to examine the performance. The speech signal is sampled with 8 KHz sampling rate shown in Fig 5.1.3.

To evaluate system performance, residual error power, ERLE, and coefficient misalignment are major system performances for comparison purposes. The performance measure of echo return loss enhancement (ERLE) can be formulated in single talk condition as:

2

In section 5.3, the nonlinear loudspeaker in our simulation is modeled as the polynomial function:

2 3 4 5

( ) .89 0.002 - 0.3 0.001 0.5

f x = x + x x + x + x

In [14], Usually ERLE is usually defined as the ratio of microphone received echo power to the residual echo power. In Fig4.1, since shortening filter may change the power of receive echo, the ERLE is redefined as

2

0 50 100 150 200 250 300

Figure 5.1.1 Room impulse response

0

Figure 5.1.2 Quadratic kernel

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 x 10⁴ -1

-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

samples

amplitude

speech signal

Fig 5.1.3 Speech signal

5.2 ERLE convergent rate comparison

5.2.1 Comparison of OTTLMS, only linear OTTLMS and LMS

Figure 5.2.1 shows the convergent rate curves of LMS and OTTLMS algorithms.

The parameters settings chosen for Figure 5.2.1 are that the order of linear and quadratic kernel equal to 256 and 20 respectively, and the signal-to-noise ratio is 30dB. We simulated large and small step size in LMS algorithm, We can see that large step size provides fast convergence rate but low ERLE performance, and vice versa.

At initial state, LMS algorithm has the slowest convergence rate compared to OTTLMS with variant step-size algorithms.

0 0.5 1 1.5 2 2.5 3 x 10⁴ 0

5 10 15 20 25 30 35

iteration

ERLE(DB)

comparsion of OTTLMS and LMS algorithm

OTTLMS

LMS-step-size = 0.002 LMS-step-size = 0.0005

Fig 5.2.1 Comparison of OTTLMS and LMS algorithms (with white Gaussian input)

From result of Fig 5.2.1, in small step size that has identical convergent ERLE value to OTTLMS, we can obtain that our OTTLMS algorithm provides faster convergent rate, our approach converged after 3000 iterations which is faster than 10000 iterations in LMS algorithm. Besides compare with small step size, we can see that even if we use large step size in LMS algorithm, the OTTLMS algorithm still has a faster convergence rate than LMS algorithm.

In fig 5.2.2, we use the real speech signal in Fig 5.1.3, the parameters settings chosen for Figure 5.2.2 are as follows:

‧M =256 N =20 SNR=25 dB₂

‧LMS : u=0.01

0 0.5 1 1.5 2 2.5 3 3.5 4

comparison of OTTLMS and LMS

OTTLMS

LMS

Fig 5.2.2 Comparison of OTTLMS and LMS algorithm (with real speech) Fig 5.2.2 show that the optimum time-& tap-variant step-size LMS (OTTLMS) provides faster convergence rate than LMS algorithm in real speech input. We can obtain that our OTTLMS algorithm enhance the LMS algorithm about 10dB in initial state, after 15000 iterations, the OTTLMS still improve LMS algorithm about 5dB.

Fig 5.2.2 Comparison of OTTLMS and LMS algorithm (with real speech) Fig 5.2.2 show that the optimum time-& tap-variant step-size LMS (OTTLMS) provides faster convergence rate than LMS algorithm in real speech input. We can obtain that our OTTLMS algorithm enhance the LMS algorithm about 10dB in initial state, after 15000 iterations, the OTTLMS still improve LMS algorithm about 5dB.

在文檔中 根據最佳收斂步伐與通道削減來提升非線性迴音消除的收斂性 (頁 41-0)