Adaptive decision feedback space-time equalization with generalized sidelobe cancellation

13  Download (0)



Adaptive Decision Feedback Space–Time

Equalization With Generalized

Sidelobe Cancellation

Yinman Lee, Member, IEEE, and Wen-Rong Wu, Member, IEEE

Abstract—In wireless communications, cochannel interference (CCI) and intersymbol interference (ISI) are two main factors that limit system performance. Conventionally, a beamformer is used to reduce CCI, whereas an equalizer is used to compensate for ISI. These two devices can be combined into one as space–time equalizer (STE). A training sequence is usually required to train the STE prior to its use. In some applications, however, spatial information corresponding to a desired user is available, but the training sequence is not. In this paper, we propose an adaptive decision feedback STE to cope with this problem. Our scheme consists of an adaptive decision feedback generalized sidelobe can-celler (DFGSC), a blind decision feedback equalizer (DFE), and a channel estimator. Due to the feedback operation, the proposed DFGSC is not only superior to the conventional generalized side-lobe canceller but also robust to multipath channel propagation and spatial signature error. Theoretical results are derived for op-timum solutions, convergence behavior, and robustness properties. With the special channel-aided architecture, the proposed blind DFE can reduce the error propagation effect and be more stable than the conventional blind DFE. Simulation results show that the proposed STE is effective in mitigating both CCI and ISI, even in severe channel environments.

Index Terms—Blind equalization, channel estimation, decision feedback equalizer (DFE), generalized sidelobe canceller (GSC), space–time equalizer (STE).



N WIRELESS communications, the cochannel interference (CCI) due to multiple access and the intersymbol interfer-ence (ISI) due to multipath channels often cause severe signal distortion and limit system performance [1], [2]. In recent years, there has been a growing interest in applying adaptive antenna arrays and space–time signal processing techniques to solve these problems [3]–[5]. The common approach is to use a beamformer for CCI reduction and an equalizer for ISI compensation. These two devices can further be combined into one as space–time equalizer (STE) [6], [7]. The application of

Manuscript received February 5, 2007; revised July 26, 2007, October 8, 2007, and November 12, 2007. This work was supported by the National Science Council, Taiwan, R.O.C., under Grant NSC 96-2219-E-009-018. The review of this paper was coordinated by Prof. J. Choi.

Y. Lee is with the Graduate Institute of Communication Engineering, National Chi Nan University, Nantou 545, Taiwan, R.O.C. (e-mail: ymlee@

W.-R. Wu is with the Department of Communication Engineering, National Chiao Tung University, Hsinchu 300, Taiwan, R.O.C. (e-mail:

Digital Object Identifier 10.1109/TVT.2007.914472

the STE is beneficial to communication quality, subsequently improving the detection performance, even in severe channel environments.

The optimum STE is known to be a maximum likeli-hood sequence estimator (MLSE) operated in the space–time domain [7]. However, the MLSE is notorious for high com-putational complexity. A suboptimum approach with a hybrid of a linear filter and a Viterbi equalizer was proposed in [8]. Even so, the implementation complexity is still high, and it limits the MLSE-like structure in real-world applications. The STE performing both beamforming and equalization, as aforementioned, is what researchers consider most. The general form of the STE consists of an antenna array and a temporal filter bank [9], [10]. Either a linear equalizer or a decision feedback equalizer (DFE) can be applied to the structure. Although the performance of this structure is satisfactory, its computational complexity is quite high. The other problem is that when an adaptive algorithm is applied, the convergence is slow, particularly operating under a large number of antenna elements and a severe fading channel. To ease these problems, another form that is a hybrid (or cascade) of a spatial filter and a temporal filter was proposed for the STE [11], [12]. It requires lower computational complexity, and the convergence is faster. However, the space–time information of the received signal will not be fully exploited, and there will be some perfor-mance loss.

All the aforementioned STEs are training based. In other words, we have to transmit training sequences before actually using them. It is well known that the transmission of training sequences will reduce the bandwidth utilization efficiency and may not always be possible. In some applications, however, spatial information corresponding to a desired user is available. For instance, a receiver may perform direction of arrival (DOA) estimation before signal detection. Moreover, a space division multiple access (SDMA) system employed to increase system capacity [13]–[16] transmits or receives a signal only from a certain direction. A practical example is the broadband wireless access system, particularly for fixed wireless applications. In these cases, it is possible to utilize the a priori spatial infor-mation and avoid the requirement of training sequences. A straightforward idea is to use a generalized sidelobe canceller (GSC) [17] for CCI suppression and a blind equalizer for ISI compensation. Unfortunately, the result of this direct cascade is often far from satisfactory. The reasons are stated in the subsequent sections.


For computational complexity consideration, the GSC is often implemented with an adaptive structure. The least-mean-square (LMS) algorithm is a well-known and widely used adaptive algorithm [18]. However, the LMS-based adaptive GSC usually converges slowly. The error signal for the LMS adaptation in this case consists of the desired signal component, and so, it is large, even in a noiseless environment. This large error signal significantly magnifies the mean-squared error (MSE). To reduce the MSE, the step size (a parameter con-trolling the LMS convergence) must be small. This essentially makes the LMS algorithm converge slowly. In addition, the adaptive GSC is sensitive to constraint mismatch, which is caused by incorrect spatial information. In this case, the signal cancellation phenomenon will occur, and the performance of the adaptive GSC can seriously be degraded. In typical appli-cations, constraint mismatch can easily arise due to multipath channels and spatial signature errors.

It is well known that the DFE can have much better perfor-mance than the linear equalizer in severe fading channels. This statement is also true for blind equalization. However, the blind DFE is difficult to derive and rarely reported in the literature. The major contribution in this field is from Labat et al., who proposed an interesting blind DFE in [19]. They used an infinite impulse response (IIR) whitening filter cascaded with a blind finite impulse response (FIR) linear equalizer in the startup period. After convergence, it switches the cascading order of the IIR and FIR filters, yielding a decision feedback structure. At the same time, a decision-directed minimum MSE (MMSE) training is initiated. For easy reference, we call this blind DFE the LBDFE hereafter. One inherent problem associated with the DFE is its error propagation effect, and this will have even more impact in its adaptive implementation. Since the LBDFE uses decision-directed training, it is sensitive to error propagation.

In this paper, we propose an adaptive STE for systems with a priori spatial information. The proposed structure comprises a decision feedback GSC (DFGSC), a blind DFE, and a channel estimator. The DFGSC structure can eliminate the desired sig-nal component from the error sigsig-nal in the LMS adaptation. As a consequence, it not only improves CCI suppression but also allows the simple point distortionless constraint to be robust to multipath channel environments and spatial signature errors. The proposed blind DFE adapts a channel-aided structure, yielding better ISI compensation and higher resistance to error propagation. We will demonstrate that the proposed blind DFE performs better than the LBDFE. This paper is an extension of [20], in which only additive white Gaussian noise (AWGN) channel was considered.

This paper is organized as follows. In Section II, the space–time signal model for the STE is developed. The effect of both CCI and ISI on the desired signal is also explained. In Section III, we describe a straightforward approach of the con-ventional GSC and LBDFE in a hybrid manner. In Section IV, we propose the new adaptive STE and describe the corre-sponding operation mechanisms in detail. Section V shows that the proposed STE is robust to general space–time multipath channels and spatial signature errors. Finally, simulation results and conclusions are given in Sections VI and VII, respectively.


Consider a uniform linear array (ULA) of N antenna el-ements at the receiver. Let the N × 1 received equivalent complex baseband signal vector in continuous time be den-oted by

x(t) = [x0(t) x1(t) · · · xN−1(t)]T (1)

where xn(t) (0≤ n ≤ N − 1, for all n hereafter) is the signal

received from the nth antenna element at time t, and the super-script (·)T denotes the transpose operation. Define an N× 1

vector a(θ) as the spatial signature for the signal from the DOA θ. It is written as a(θ) = [1 eiζθ ei2ζθ· · · ei(N−1)ζθ]T, where i =√−1, and ζθ= (2πd/Lλ) sin θ, in which d is the element

spacing, and Lλ is the signal wavelength. Assume that there

are M sources (including both desired and interfering sources) coming from M different and distinguishable directions. The transmitted signal waveform of the mth source sm(t) (0≤

m≤ M − 1, for all m hereafter) can be written as sm(t) =



p(t− kT )bm(k) (2)

where bm(k) is the kth complex information symbol of the mth

source, p(t) is the pulse shape of the transmitted symbol, and T is the symbol duration. Let Lmbe the number of propagation

paths for the mth source, θmbe the DOA for the mth source,

τmlbe the delay time for the lth path (0≤ l ≤ Lm− 1, for all

l hereafter) of the mth source, and ¯hmlbe the complex channel

coefficient for the lth path of the mth source, respectively. Assume that the channel parameters for different sources are independent and remain constant over the observation period. The received signal vector in (1) can then be expressed as

x(t) = M−1 m=0 Lm−1 l=0 ¯ hmlsm(t− τml)a(θm) + n(t) (3)

where n(t) is an N× 1 complex AWGN vector. Each compo-nent in the AWGN vector is spatially white with a variance of σ2n. Substituting (2) into (3), we have

x(t) = M−1 m=0 +  k=−∞ hm(t− kT )bm(k) + n(t) (4)

where hm(t) is an N× 1 vector summarizing the total

trans-mission effect of the mth source on the information, and it is given by hm(t) = Lm−1 l=0 ¯ hmlp(t− τml)a(θm). (5)

Without loss of generality, we assume that the first M0sources (M0≤ M) are related to the desired user, i.e., b0(k) = b1(k) = · · · = bM0−1(k). The DOA θ0is the DOA of the desired signal with the strongest signal strength, and θ1to θM0−1correspond to the DOAs of the desired signal’s spatial ISI. The received equivalent complex baseband signal is sampled at the symbol rate, i.e., t = kT . The sampling clock is synchronized with the transmission clock. After sampling, the channel effect for all


Fig. 1. Hybrid of GSC and LBDFE (with LBDFE in startup period).

sources is of a finite duration within [0, (Dm− 1)T ], where

Dm is the channel order of the mth source. Here, we define

d(k) as an N× 1 vector representing the components from

the main source of the desired signal (m = 0), and i(k) as an N× 1 vector summing up the components from the spatial ISI sources (m = 1, 2, . . . , M0− 1), and they are, respectively, given by d(k) = D0−1 d=0 h0(d)b0(k− d) (6) i(k) = M0−1 m=1 Dm−1 d=0 hm(d)b0(k− d). (7) In addition, define an N× 1 vector z(k) representing the uncorrelated CCI-plus-noise components as

z(k) = M−1 m=M0 Dm−1 d=0 hm(d)bm(k− d) + n(k). (8)

Then, the expression in (4) can be rewritten as

x(k) = d(k) + i(k) + z(k). (9) We use this signal model to describe various kinds of CCI and ISI in the wireless communication environment. The main task of the STE is to suppress interference and recover the transmitted information. To simplify the notations, we write b(k) instead of b0(k) for the desired user’s information symbols in the following derivation.


In some wireless communication systems, such as SDMA applications, the main DOA (and hence the spatial signature) of the desired signal is known a priori or can be estimated. A straightforward approach, as mentioned before for CCI and ISI mitigation, is the hybrid of a conventional GSC and an LBDFE. The purpose of the conventional GSC is to suppress CCI, whereas that of the LBDFE is to compensate for ISI. With the spatial signature, no extra training sequence is required for either processing. The operation and weakness of this approach will subsequently be elaborated upon.


The conventional GSC for CCI suppression is optimized with the linearly constrained minimum variance (LCMV)

criterion [21]. The LCMV beamformer determines the N -tap weight vector w through


w w


xw subject to CHw = f (10) where Rx= E{x(k)xH(k)} is the input correlation matrix,

C is an N× U constraint matrix, and f is a U × 1 response

vector, with U being the number of constraints. The superscript (·)H denotes Hermitian transposition. With the structure of GSC, as shown in the left part of Fig. 1, the constrained optimization problem can be transformed into an unconstrained optimization problem [17]. This structure can effectively reduce the computational cost, particularly when implemented with adaptive algorithms. As illustrated in the figure, the upper path includes an N -tap quiescent signal matched filter wq. The

lower path includes an N× (N − U) blocking matrix B and an (N− U)-tap interference canceling filter wa. Then, we have

an equivalent spatial filter as w = wq− Bwa. Ideally, the span

of B is in the null space of CH. Using the constraint in (10),

wq can readily be found to be wq = C(CHC)−1f , and wa is

optimized according to the output power of the GSC as J = E  |ys(k)|2  = E(wq− Bwa)Hx(k) 2 . (11) The constrained optimization problem in (10) can then be rewritten as the following unconstrained optimization problem:

min wa

J = min wa

(wq− Bwa)HRx(wq− Bwa). (12)

The optimum wais classically solved to be [17]

wa,opt = (BHRxB)−1BHRxwq. (13)

With the optimum weight vector, the minimum value of J in (11), which is denoted as Jmin (which equals the minimum output power, which is denoted as Po,min for the conventional

GSC), can be calculated as

Jmin= Po,min (14)

= wHoptRxwopt (15) = wHq Rxwopt (16) where we let wopt= wq− Bwa,opt. Since C and f are design

parameters, wqand B can be calculated offline. The calculation

of wa,opt shown in (13), however, is much more involved. An

alternative to find it is to use the adaptive training method. The LMS algorithm, being one of the stochastic gradient methods,


is chosen here for its simple yet effective nature. Taking the sto-chastic gradient of J with respect to wa, where (·)∗denotes the conjugate operation, we can obtain the LMS update equation for waas

wa(k + 1) = wa(k) + μav(k)e∗s(k) (17)

where wa(k) is the estimate of wa,opt at the kth iteration,

v(k) = BHx(k) is the filter input vector (the output vector

from the blocking matrix), μa is the step size controlling the

convergence rate, and es(k) is an error signal between the

de-sired and actual output. For GSC applications, we have es(k) =

ys(k). When wais optimized, the error signal es(k) will chiefly

include the component from the desired signal, i.e., the desired user’s transmitted information symbols b(k). This indicates that the stochastic gradient in (17), i.e., v(k)ys(k), will be large, even for optimum weights. When the LMS algorithm is applied to estimate wa,opt, the performance will be affected due to the

large error signal used. The other problem with the conventional GSC is its sensitivity to constraint mismatch. Whenever the setting of the constraint matrix C in (10) [or the blocking matrix

B in (12)] is not fit for the actual spatial signature of the desired

signal, constraint mismatch occurs. Constraint mismatch can easily arise due to multipath channels and spatial signature errors. This seriously degrades the performance of the adaptive GSC. All these problems will be analyzed in depth later. B. LBDFE

The output of the GSC is fed into the LBDFE for equal-ization. The equalizer adaptation process is divided into two periods. In the startup period, the received signal is prewhitened by an IIR filter and then equalized by a blind FIR linear equalizer. The structure of the LBDFE in the startup period is shown in the right part of Fig. 1. In [19], the constant modulus algorithm (CMA) was used as the blind algorithm for equalization. Recently, a sophisticated blind equalization algo-rithm called the multimodulus algoalgo-rithm (MMA) was proposed [22]. Analysis shows that the MMA can provide much more stable performance, particularly with high-order constellation modulation [23]. The cost function can yield an equalized constellation rotated with a multiple of 90, eliminating the need for additional constellation phase recovery as needed in the CMA. The remaining phase ambiguity problem is classi-cally solved by differential encoding. For these reasons, we use the MMA as our blind equalization algorithm (instead of the CMA) throughout this paper. For the LBDFE, we denote the length of wf and wb as α and β, respectively. In the

startup period, let ys(k) be the output of the GSC, i.e., ys(k) =

(wq−Bwa)Hx(k), as given in (11), and let ys(k) be the

difference between ys(k) and the prewhitening filter output.

In addition, let y(k) be the input vector of wb as y(k) =

[ys(k−1) ys(k−2) · · · ys(k−β)]T. So, we have ys(k) = ys(k)− wHb y(k). The filter wb is optimized

accord-ing to the criterion minwb E{|ys(k)|

2}. As shown in Fig. 1, ys(k) serves as the input of the blind FIR

fil-ter wf as well. We define another vector as y(k) =

[ys(k) ys(k−1) · · · ys(k−α+1)]T, and, as a

conse-quence, the output of the LBDFE in the startup period

is yt(k) = wHfy(k). For the MMA, the error signal is

defined as et(k) =  y2t,r(k)− R22+y2t,i(k)− R22 (18) with R2= E  b4r(k) E{b2 r(k)} = E  b4i(k) E{b2 i(k)} (19) in which the subscripts r and i in yt(k) and b(k) denote the real

and imaginary parts of yt(k) and b(k), respectively. The cost

function for the optimization of wfis then written as

JMMA= E{et(k)}

= Ey2t,r(k)−R22+y2t,i(k)−R22 

. (20) Let μfbe the step size controlling the convergence of wf. From

[22], the stochastic gradient algorithm for the MMA of wfcan

be obtained as

wf(k + 1) = wf(k) + μfφ (yt(k)) y(k) (21)


φ (yt(k)) = yt,r3 (k) + iy3t,i(k)− R2yt(k). (22)

The startup period is expected to sufficiently open the eye pattern such that the error rate is low enough to initiate the second (tracking) period. In the tracking period, the cascading order of the IIR prewhitening filter and the FIR linear equalizer is swapped, and this turns the whole system into a conventional DFE structure. A decision-directed MMSE tracking operation, similar to that of the conventional DFE, is then activated. This approach may initially avoid the possible error propagation phenomenon and give a smooth transition strategy between blind and decision-directed equalization. Nevertheless, the sta-bility of the adaptive IIR filter makes the performance sensitive to the parameters chosen. In addition, error propagation may occur during the decision-directed mode. The behavior of the LBDFE becomes not easy to control. We will empirically show in Section VI that the performance of the hybrid of the conventional GSC and LBDFE is often far from satisfactory in severe channel environments.


In this section, we propose a new adaptive STE that is a hybrid of DFGSC and a channel-aided blind DFE (CBDFE). Fig. 2 shows the block diagram of the proposed STE. Note that in Fig. 2, a channel estimator is included. Let the coefficients of the channel estimator be denoted as wh. It is used to

model the equivalent temporal channel for the desired signal, and its operation will be explained soon. In [11], a channel estimator was also used in a training-based STE such that the corresponding beamformer can achieve better performance. In the STE scheme proposed here, the role of the channel estimator is much more involved. For spatial processing, with the help of the channel estimator, we can formulate the adaptive DFGSC,


Fig. 2. Proposed hybrid STE.

achieving better CCI suppression performance. In addition, the DFGSC can have extra robustness against constraint mismatch. For temporal processing, with the help of the channel estimator, we can formulate the CBDFE such that it is more effective in ISI compensation. This CBDFE is different from those channel-estimation-based DFEs proposed in [24] and [25], where the feedforward and feedback filters are calculated based on the estimated channel response. In the CBDFE, however, the adap-tive structure is preserved. The operation of the DFGSC and CBDFE is separately presented as follows.

A. Proposed DFGSC

Let the channel estimator wh have a dimension of γ× 1.

Here, the value of γ is chosen to be equal to or larger than the maximum value of the channel order Dm(0≤ m ≤ M0− 1). We first define ˆb(k) as the input vector of wh, i.e., ˆb(k) =

b(k) ˆb(k− 1) · · · ˆb(k − γ + 1)]T. To optimize the

inter-ference canceling filter wa and the channel estimator wh, we

propose a new cost function as J = E  |es(k)|2  = E  ys(k)− wHhb(k)ˆ  2 = E   wHq x(k) wHa wHh   BHx(k) ˆ b(k)  2  . (23)

To understand the operation mechanisms, we first study a simplified scenario in which only the temporal ISI is present. In other words, the desired signal and its ISI only come from the main DOA, i.e., M0= 1 (and so i(k) in (7) is a zero vector). Consider that the spatial signature for the desired signal is exactly known, and a distortionless constraint is set toward it. Thus, U = 1 is used in the derivation. In this case, the desired signal will be completely blocked in the lower path of the DFGSC. The general case where spatial ISI exists and the distortionless constraint may be set improperly will be discussed in the next section. Assuming that the decision is correct, i.e., ˆb(k) = b(k), we can rewrite the minimization of the cost function in (23) as

min wa,wh J = min wc wHq Rxwq− wHq [ RxB Rp]wc −wH c  BHRx RHp  wq+ wHc Rcwc (24) where we let wc=  wa wh  (25) Rc= E  BHx(k) ˆ b(k)  [ xH(k)B bˆH(k) ] =  BHR xB 0 0H σ2 bIγ  (26) Rp= E  x(k)ˆbH(k)  = σ2b[h0(0) h0(1) · · · h0(γ− 1)] (27) in which 0 denotes a zero matrix with dimension (N− 1) × γ,

Iγ denotes an identity matrix with dimension γ× γ, and σ2b

denotes the power of transmitted symbols. Since there is no correlation between BHx(k) and ˆb(k) in this case, the

off-diagonal block of Rc in (26) is zero. Again, wq is for signal

matching, and it can be set the same as for the conventional GSC. Taking the derivative of this cost function with respect to wc and setting the result to zero, we can obtain the opti-mum wcas ∂J ∂wc =−2  BHRx RHp  wq+ 2Rcwc= 0. (28) Thus wc,opt= R−1c  BHR x RH p  wq. (29)

With the special structure of Rc, we can decompose wc,opt back into the two weights

wa,opt= (BHRxB)−1BHRxwq (30) wh,opt= RH p σ2 b wq = [h0(0) h0(1)· · · h0(γ− 1)]Hwq. (31)

From (30), we observe that the expression of wa,optis the same

as that of the conventional GSC given in (13). Since wq and

wa,optremain the same, the minimum output power Po,minfor

the DFGSC is also the same. From (31), we observe that wh,opt


the DFGSC output. With wa,opt and wh,opt given earlier, the

minimum J in (23) for the DFGSC becomes Jmin= E  wH optx(k)−wh,optH b(k)ˆ  2 = wHq Rxwopt−σb2wHq [h0(0) h0(1) · · · h0(γ−1)] 2 = Po,min−σ2bwHq [h0(0) h0(1) · · · h0(γ−1)] 2 . (32) Note here that Jminis no longer equal to Po,min. We let Rd= E{d(k)dH(k)} be the input correlation matrix of the desired

signal excluding spatial ISI and Rz= E{z(k)zH(k)} be the input correlation matrix of CCI-plus-noise. The first term in (32) is equal to the total power in the DFGSC output, and the second term in (32) is the power of the desired signal in the DFGSC output, which can be defined as

Pd Δ = wHoptRdwopt = σb2wqH[h0(0) h0(1) · · · h0(γ− 1)] 2 . (33) So, Jminin (32) can be calculated as

Jmin= Po,min− Pd (34) = wHq Rzwopt. (35) Comparing (14) and (34), we can see that the desired signal is totally excluded with the help of wh,opt. Thus, we conclude that

the decision feedback only reduces the minimum J in (14), and the optimum performance is not enhanced. When an adaptive algorithm such as LMS is used to estimate wa,opt, however,

the performance of the GSC can greatly be improved by the feedback operation. The LMS update equations for the DFGSC can be written as

wa(k + 1) = wa(k) + μav(k)e∗s(k) (36)

wh(k + 1) = wh(k) + μhb(k)eˆ ∗s(k) (37)

where μa is the step size for wa, μh is the step size for

wh, v(k) is the output from the blocking matrix, and es(k) =

ys(k)− wHh(k)ˆb(k). Unlike the conventional GSC, the

steady-state es(k) will exclude the desired signal, and hence, it can

be quite small. This is where the improvement of the adaptive DFGSC stems from.

To analyze the steady-state performance for both the con-ventional adaptive GSC and the proposed adaptive DFGSC, we denote the value of J [both (11) and (23)] in the steady state as J (∞). Suppose that the decision is correct, and whis fixed at

optimum. Then

J (∞) = Jmin+ Jex(∞) (38) where Jex(∞) is the excess MSE. Note that the excess MSE is yielded by the use of the LMS algorithm. In addition, define the weight error vector as

(k) = wa(k)− wa,opt. (39)

Using the direct averaging method [18], we can have

(k + 1) = (I− μaBHRxB)(k)+ μaBHx(k)eopt(k) (40) where eopt(k) denotes the error signal produced with the opti-mum weights. Define the correlation matrix of the weight error vector as

K(k) = E(k)H(k) . (41) Invoking the independence assumption [18], we can obtain the recursive relation of K(k) as

K(k + 1) = (I− μaBHRxB)K(k)(I− μaBHRxB) + μ2aJminBHRxB. (42) Under this premise, the excess MSE, which is denoted as Jex(k), is written as

Jex(k) = tr BHRxBK(k) 

(43) where tr[·] gives the trace of the matrix inside the brackets. As k→ ∞, Jex(k) is given by Jex(∞) = Jmin N−1 l=1 μaλl(BHRxB) 2− μaλl(BHRxB) (44)

where λl(BHRxB) represents the lth eigenvalue of BHRxB. If μais small, (44) can be approximated as

Jex(∞) ≈ μaJmin 2 N−1 l=1 λl(BHRxB). (45) Thus, Jex(∞) is proportional to Jminand the step size μa. The

transient output signal-to-interference-plus-noise ratio (SINR) for the conventional GSC and DFGSC can be written as

SINR(k) =



|wH(k)x(k)− wH(k)d(k)|2. (46)

The optimum and steady-state output SINR (with the LMS algorithm) can then be calculated as

SINRopt= EwH optd(k) 2 EwH optx(k)− wHoptd(k) 2 = Pd Po,min− Pd (47) SINRLMS= EwH(∞)d(∞)2  E  |wH(∞)x(∞) − wH(∞)d(∞)|2  = Pd wH q Rxwopt+ tr [BHRxBK(∞)] − Pd = Pd Po,min+ Jex(∞) − Pd . (48)

Note that the notations w(∞), d(∞), and x(∞) are used to denote their final values and are based on the assumption of


reaching convergence under statistical expectation, just like that for Jex(n)→ Jex(∞) as n → ∞. From (47), once again, we see that the optimum performance is not enhanced by the decision feedback operation since Pdand Po,minare the same

for the conventional GSC and the DFGSC. From (45), we can see that Jex(∞) is proportional to Jmin. The corresponding Jmin values for both schemes are shown in (14) and (34), respectively. The resultant Jminfor the adaptive DFGSC will be smaller than that for the conventional adaptive GSC. As a consequence, Jex(∞) for the adaptive DFGSC will be smaller than that for the conventional adaptive GSC. Moreover, from (48), we see that the smaller Jex(∞) is, the larger the steady-state output SINR we will have. We thus conclude that with the same step size, the steady-state output SINR of the adaptive DFGSC will be higher than that of the conventional adaptive GSC. In addition, note that the step size bound for the LMS algorithm is determined by the eigenvalue spread of the input correlation matrix. The input vectors for the conventional GSC and the DFGSC are the same, and so, the step size bounds for both schemes are the same.

B. Proposed CBDFE

The MMA is conventionally applied to the blind FIR linear equalizer only. Since the DFE is a nonlinear and IIR equalizer, direct application of the MMA may result in severe error propa-gation in the startup period. It may make the performance of the blind DFE even worse than that of its FIR linear companion. Here, we make use of the previously derived channel estimator

whand propose a new structure, i.e., the CBDFE, to overcome

this problem. In FIR linear equalization, the performance of the MMA can be demonstrated to approach that of the training-based MMSE method [19], [23]. Hence, we suppose that the weights solved by the MMA are close to those solved by the MMSE criterion. Our approach uses a basic principle of the DFE, i.e., the postcursor response of the channel convolved with the feedforward filter is cancelled by the feedback filter.1 Let wf and wb be the feedforward and feedback filter

weight vectors of a conventional DFE, with length α and β, respectively. Similar to the previous section, ys(k) denotes the

output of the DFGSC. For notation simplicity, the input of wf

and wb is again written as y(k) = [ys(k) ys(k− 1) · · ·

ys(k− α + 1)]T and ˆb(k) = [ˆb(k− κ − 1) ˆb(k − κ−2) · · ·

ˆb(k− κ − β)]T, where κ is the decision delay. Note that

the convolution of the equivalent channel response and the feedforward filter results in a response of length α + γ− 1. Thus, for perfect postcursor cancellation, we must have β≥ α + γ − 2 − κ. Without loss of generality, we let β = α + γ− 2 − κ. We now prove the postcursor cancellation property mentioned earlier. Again, with correct decisions, the error signal is written as

et(k) = b(k− κ) − yt(k)

= b(k− κ) −wHfy(k)− wbHb(k) (49)

1For DFE, precursor and postcursor responses are defined as the ISI from

future and past symbols, respectively.

where yt(k) is the equalizer output. Straightforward

manipulations give the equalizer output MSE as E  |et(k)|2  = wfHRyywf− wfHRybwb− wHf pyb − wH b RHybwf+ wHb Rbbwb+ wbHpbb − pH ybwf+ pHbbwb+ σb2 (50)

with Ryy= E{y(k)yH(k)}, R

bb= E{b(k)bH(k)}, Ryb= E{y(k)bH(k)}, p

yb= E{y(k)b∗(k− κ)}, and pbb= E{b(k)b∗(k− κ)}. To obtain the optimum weights with the MMSE criterion, we set the gradient of E{|et(k)|2} with

respect to the vectors wfand wbto zero. This results in

wf,opt=  Ryy 1 σ2 b RybRHyb −1 pyb (51) wb,opt= 1 σ2 b RHybwf,opt. (52)

Let wh,opt= [ω0 ω1 · · · ωγ−1]T represent the convolution of wh,opt and wf,opt as Hwf,opt, where H is an

(α + γ− 1) × α matrix as H = ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ ω0 0 · · · · · · · · · · · · · · · 0 ω1 ω0 0 · · · · · · · · · · · · 0 .. . . .. ... ωγ−1 ωγ−2 · · · ω0 0 · · · · · · 0 0 ωγ−1 ωγ−2 · · · ω0 0 · · · 0 .. . . .. . .. ... 0 · · · 0 ωγ−1 ωγ−2 · · · ω0 0 0 · · · · · · 0 ωγ−1 ωγ−2 · · · ω0 .. . . .. ... 0 · · · · · · · · · · · · 0 ωγ−1 ωγ−2 0 · · · · · · · · · · · · · · · 0 ωγ−1 ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ . (53) We can further partition H as H = [HT

r HTp]T, where Hris

of dimension (κ + 1)× α, and Hp is of dimension (α + γ−

2− κ) × α. It is not difficult to see that Hrwf,optcorresponds

to the precursor response of Hwf,opt, whereas Hpwf,opt is

the postcursor response. With enough degrees of freedom, CCI is mostly suppressed. The DFGSC output can be modeled as

ys(k) = wHq d(k) + ν(k) (54)

where ν(k) is a white noise independent of d(k). With (54) and some manipulations, we can derive

1 σ2


RHyb= Hp. (55)

From (52), we then obtain wb,opt= Hpwf,opt. This result can

be restated as

wb,opt= post{wh,opt⊗ wf,opt} (56)

where ⊗ denotes the convolution operation, and post{·} denotes the postcursor-taking operation. This result suggests


an adaptation approach for the training-based MMSE-DFE. Let wf(k) and wb(k) be the feedforward and feedback weight

vectors at time instant k. With reference to (56), we can let

wb(k) = post{wh(k)⊗ wf(k)} (57)

in which wh(k) is the channel estimate at time instant k. If

wh(k) converges to wh,opt, wb(k) will converge to wb,opt

too. The difference between this approach and the conventional method lies in that only wf(k) is adapted [not both wf(k)

and wb(k)]. While this approach may make no difference for a

training-based DFE (since the training sequence is available), it will provide significant improvement for the blind DFE. On one hand, consider a blind DFE scenario in which both wf(k) and

wb(k) are adapted. With yt(k) = wHf(k)y(k)− wHb (k)ˆb(k),

the update equations for the MMA can be written as

wf(k + 1) = wf(k) + μfφ (yt(k)) y(k) (58)

wb(k + 1) = wb(k)− μbφ (yt(k)) ˆb(k) (59)

where φ(yt(k)) is the same as that given in (22). From (58)

and (59), we see that if there is a decision error, the error will immediately reflect to ˆb(k) and then φ(yt(k)). Note that the

adaptation of wf(k) only involves erroneous φ(yt(k)), whereas

that of wb(k) involves both erroneous ˆb(k) and erroneous

φ(yt(k)). The two error sources in (59) will make wb(k) quite

sensitive to decision errors. On the other hand, in the proposed method, only wf(k) is adapted, as given in (58). Although

the effect of decision error will also pass to es(k), which will

perturb the adaptation of wa(k) and wh(k), the influence is

smaller. This is because the convergence of wa(k) and wh(k)

in the DFGSC is much faster and more stable than that of the blind DFE (which will empirically be shown in Section VI). By using (57) to calculate wb(k), the feedback filter of the blind

DFE will perform much better. In one word, with the proposed operation, the resultant CBDFE will be less sensitive to decision errors.

In the startup period, decisions may not be trustworthy. Including decisions in the operation of the CBDFE may affect its stability. Unlike the LBDFE’s mode switching, we propose a simple mechanism allowing a smooth transition from FIR linear equalization to DFE. We let the feedback filter be multiplied by a time-varying factor f (k), where f (k)≤ 1. Initially, k is small, and we set f (k) small such that the weighting of the feedback filter is small. In this stage, the proposed equalizer will behave much like an FIR linear equalizer. As k increases and decisions become more reliable, we increase f (k). Eventually, f (k) approaches one, and the equalizer becomes a full DFE. As known in [18], the convergence of an adaptive algorithm exhibits exponential decay behavior. Thus, a natural choice for f (k) will be

f (k) = 1− e−ξk (60)

where ξ is a design parameter controlling the increase rate of this factor. With the proposed mechanism, the feedback filter is gradually taken into account, and the error propagation effect

will be reduced. The CBDFE will ultimately approach the optimum MMSE-DFE.



In this section, we will show that the proposed STE is robust to two unfavorable phenomena, i.e., spatial multipath propa-gation and spatial signature mismatch, which are frequently encountered in array signal processing. Let us first consider the spatial multipath propagation problem. Multipath propagation may cause ISI coming from different directions, which induces both coherent interference [26] and angular spread [27]. In this case, the conventional GSC with the simple point distortionless constraint tends to cancel the desired signal itself, which is referred to as signal cancellation. In the proposed STE, even with this spatial ISI, the phenomenon of signal cancellation will not exist. This is due to the use of decision feedback. In the spatial multipath environment, the desired signal will leak to the output of the blocking matrix, producing a correlation between the upper and lower paths in the DFGSC. Thus, the optimum solutions for the interference canceling filter waand

the channel estimator whare now coupled together. From (29),

we have wc,opt=  wa,opt wh,opt  = R−1c  BHR x RH p  wq (61) where Rp= E  x(k)ˆbH(k)  = σb2 M 0−1  m=0 hm(0) M0−1 m=0 hm(1)· · · M0−1 m=0 hm(γ−1)  (62) Rc=  BHR xB M MH σ2 bIγ  (63) with M = σb2BH M 0−1  m=0 hm(0) M0−1 m=0 hm(1)· · · M0−1 m=0 hm(γ−1)  (64) which is the correlation matrix between the blocking matrix output BHx(k) and the decision vector ˆb(k). Using the

inver-sion identity for subblock matrices [28], we can find the inverse of Rc, and so, wc,optin (61), i.e.,

wc,opt=  (BHRzB)−1BHRzwq M 0−1  m=0 hm(0)· · · M0−1 m=0 hm(γ− 1) H ×I− B(BHRzB)−1BHRz  wq  (65)


and thus the coupled wa,optand wh,optcan be written as wa,opt=(BHRzB)−1BHRzwq (66) wh,opt= M 0−1  m=0 hm(0)· · · M0−1 m=0 hm(γ− 1) H × (wq− Bwa,opt). (67)

The minimum J of the DFGSC, as that given in (35), can then be solved to be Jmin= wqHRxwq− wqH[ RxB Rp]R−1c  BHR x RH p  wq = wqHRzwq− wqHRzB(BHRzB)−1BHRzwq = wqHRzwopt. (68)

From (66)–(68), we see that while wa,opt and wh,opt are

changed, Jmin(and so the error signal) still contains no desired signal. The leaky desired signal in the lower path of the DFGSC is not correlated with the error signal. When optimizing wa, the

output desired signal power is not minimized, and so, no signal cancellation occurs.

We then consider the spatial signature mismatch problem. If the knowledge of the desired signal’s main DOA is erroneous,

wq no longer matches the desired signal’s spatial signature,

and B cannot obstruct the desired signal. In this case, signal cancellation may occur as well [29]. In the case of spatial multipath propagation, we have already proven that for the DFGSC, even if there is leaky desired signal in the output of the blocking matrix, wawill not cancel the desired signal. The

spatial signature mismatch can be considered a signal-leaking problem similar to that in spatial multipath propagation. Thus, no signal cancellation will occur in the DFGSC. The derivation details, however, are omitted.

For these scenarios, the expression of the optimum output SINR is the same as that given in (47), but now, Pdand Po,min

are changed. The output desired signal power becomes Pd= σb2   (wq− Bwa,opt)H × M 0−1  m=0 hm(0) M0−1 m=0 hm(1)· · · M0−1 m=0 hm(γ−1)     2 (69) which is different from (33). In the calculation of Po,min,

(15) should be used instead of (16). Moreover, due to the existence of the desired signal in the input of the interference canceling filter, the excess MSE yielded by the LMS algorithm will be different too. Let Jexd(∞) denote the excess MSE value for the desired signal in these leaky scenarios. Using (44), we can have

Jexd(∞) = Jmin

μaλ1(BHRdB) 2− μaλ1(BHRdB)

. (70)

The result in (70) is due to the fact that only one eigenvalue of


dB is nonzero. The expression of the steady-state output



SINR for the GSC structure with spatial ISI or spatial signature mismatch is then slightly modified to


Pd+ Jexd(∞)

Po,min+ Jex(∞) − (Pd+ Jexd(∞))

(71) in which Pdand Po,minshould also be changed to use (69) and

(15), respectively.


Computer simulations are conducted to demonstrate the ef-fectiveness of the proposed STE (called DFGSC-CBDFE in this section for clarity) and verify our analytic results. For com-parison, we also consider the hybrid of the conventional GSC and LBDFE (called GSC-LBDFE hereafter) and the hybrid of the DFGSC and LBDFE (called DFGSC-LBDFE hereafter). Note that the latter scheme is used for the comparison of CBDFE and LBDFE. In all cases, we assume a ULA with five omnidirectional antennas spaced half a wavelength apart. The main DOA of the desired signal is known a priori. Only the point distortionless constraint to the main DOA is used for the GSC in all three schemes. The received desired signal is corrupted by CCI, ISI, and AWGN. In the first part, we only consider channels with temporal ISI. In the second part, we consider more realistic channels where both temporal and spatial ISIs are present. In all figures, at least 500 simulation runs are averaged to obtain each simulated result.

A. Channels With Temporal ISI Only

In this set of simulations, we consider channels with temporal ISI only. We compare the performance of DFGSC-CBDFE with GSC-LBDFE and DFGSC-LBDFE. The transmitted symbols are randomly generated and then modulated by quadrature phase-shift keying. The CCI-to-noise ratio (CCINR) is set as 25 dB, and the signal-to-noise ratio (SNR) is 15 dB. The channel for the desired signal coming from 0 is [0.407 0.815 0.407] [1, p. 616], and the channel for CCI com-ing from −60◦ is [1 0 0]. The parameters used for the adaptive GSC, DFGSC, LBDFE, and CBDFE are listed in Table I. The step sizes for wh in those decision

feed-back schemes are 2× 10−3. The decision-directed MMSE training starts after 500 iterations. As mentioned, the factor f (k) for the CBDFE is used to reduce error propagation. In this case, ξ is chosen to be 0.01, and f (k) approaches


Fig. 3. Learning curves of GSC output SINR for different schemes in sup-pressing CCI.

Fig. 4. Beam patterns (enlarged region around CCI’s DOA) of different schemes in Fig. 3 after 5000 iterations.

unity at around 500 iterations. Fig. 3 shows the GSC output SINR for CBDFE, GSC-LBDFE, and DFGSC-LBDFE. In addition, the optimum SINR and the steady-state SINR obtained with the LMS algorithm are shown. From Fig. 3, we see that all schemes are comparable in convergence rate, but the SINR achieved by those schemes with the adaptive DFGSC is higher than that with the conventional adaptive GSC. As expected, the adaptive DFGSC can approach the optimum solution much more closely, which means that the effect of the excess MSE induced by the LMS algorithm is smaller. In terms of GSC output SINR and convergence rate, DFGSC-CBDFE provides the best performance. Fig. 4 reveals the beam patterns (enlarging the region around the CCI’s DOA) of those schemes used in Fig. 3 after 5000 iterations. The improvement of the decision feedback operation can clearly be seen as well. Fig. 5 shows the equalizer output MSE, i.e., E{|b(k − κ) − yt(k)|2}, for all three schemes. From the figure, we see that

both schemes with adaptive DFGSC work well in suppressing the remaining ISI. However, DFGSC-CBDFE performs better

Fig. 5. Learning curves of equalizer output MSE for different schemes.

Fig. 6. Learning curves of channel estimate MSE for different schemes.

than DFGSC-LBDFE. This means that the CBDFE is more effective in ISI suppression. In Fig. 6, we show the MSE of the channel estimate, i.e., E{|[h0(0) h0(1) · · · h0(γ 1)]Hw

q− wh(k)|2}, for the schemes with the adaptive

DFGSC. We observe that these MSE values become small when the decision is reliable (about−40 dB for DFGSC-CBDFE and about−35 dB for DFGSC-LBDFE). With the accurate channel estimate, the DFGSC can eliminate the desired signal from the error signal, and the CBDFE can construct a better feedback filter for equalization. This is also where the improvement comes from.

To show the robustness of the DFGSC against spatial signa-ture mismatch, we repeat the aforementioned scenario with a 5, DOA mismatch of the desired signal, i.e., the actual main DOA is −5◦, and the presumed main DOA is 0. We plot the resultant GSC output SINR, beam patterns, and equalizer output MSE in Figs. 7–9, respectively. In Fig. 7, we observe that the SINR performance gap between DFGSC-CBDFE and DFGSC-LBDFE becomes larger, and GSC-LBDFE completely fails due to signal cancellation. In addition, Fig. 8 shows that DFGSC-CBDFE can keep the beam pattern very close to the


Fig. 7. Learning curves of GSC output SINR for different schemes in which the actual main DOA is−5◦, and the presumed main DOA is 0.

Fig. 8. Beam patterns of different schemes in Fig. 7 after 5000 iterations in which the actual main DOA is−5◦, and the presumed main DOA is 0.

Fig. 9. Learning curves of equalizer output MSE for different schemes in which the actual main DOA is−5◦, and the presumed main DOA is 0.



Fig. 10. Learning curves of equalizer output MSE for different schemes with spatial ISI.

optimum. In Fig. 9, we see that DFGSC-CBDFE gives the smallest equalizer output MSE and, thus, performs best. All these indicate the DFGSC is better than the conventional GSC, and the CBDFE is better than the LBDFE.

B. Channels With Both Temporal and Spatial ISI

In this section, we consider a more general case where ISI comes from different directions and time instants. Again, the CCINR is fixed at 25 dB, and the SNR is 20 dB. The filter length for the blind DFE is the same as given previously. The step sizes and channel settings used are shown in Table II. From the table, we see that the ISI comes from different directions, and the effect of both coherent interference and angular spread is included. Furthermore, the step sizes for wh are 1× 10−3.

The parameter ξ is selected as 0.005, for which f (k) is close to unity around 1000 iterations. Fig. 10 shows the equalizer output MSE with 16 quadrature amplitude modulation for the three schemes. To observe the convergence behavior of these schemes, we exclude the decision-directed mode and use the MMA blind algorithm all the way first. We see that DFGSC-CBDFE outperforms the other two schemes in terms of both convergence rate and equalizer output MSE. The channel


Fig. 11. SER performance for the scenario in Fig. 10.

Fig. 12. Learning curves of equalizer output MSE for different schemes with spatial ISI (artificial errors added as five consecutive errors from the 2000th iteration, ten interleaved errors from the 3000th iteration, and ten consecutive errors from the 4000th iteration).

estimator in the proposed scheme really helps to construct a more stable and efficient blind DFE. Then, the symbol-error-rate (SER) performance for the three schemes under the same settings is given in Fig. 11. The result is observed under different iteration intervals. Again, DFGSC-CBDFE performs best and offers smaller SER in all cases. Note that GSC-LBDFE fails as well due to signal cancellation.

Finally, we repeat the same experiment with the decision-directed MMSE training in the tracking period (after 1000 itera-tions). However, some artificial errors are added in the decision process. To force an error to occur, we randomly shift the decision to a constellation point near its true value. The errors are added as follows: five consecutive errors from the 2000th iteration, ten interleaved errors from the 3000th iteration, and ten consecutive errors from the 4000th iteration. The result is presented in Fig. 12. Again, it shows that DFGSC-CBDFE performs better than LBDFE. The proposed CBDFE can reconverge after the bursty errors, but

DFGSC-LBDFE diverges. This clearly shows that the CBDFE is less sensitive to decision errors and makes the whole processing scheme more reliable. We then conclude that DFGSC-CBDFE is the best hybrid scheme for the scenario we consider.


In this paper, a new adaptive STE has been developed for the suppression of both CCI and ISI. The proposed STE introduces a hybrid of an adaptive DFGSC and an adaptive CBDFE. With the main DOA known a priori, training sequences are not required for the adaptation of the whole STE. We show that the included channel estimator can not only improve the performance of the conventional adaptive GSC but also make the blind DFE more reliable. For spatial processing, the DFGSC improves the CCI suppression capability when implemented with the LMS algorithm. In addition, the adaptive DFGSC with the simple point distortionless constraint is robust against multipath propagation environments and spatial signature er-rors. For temporal processing, the proposed CBDFE can have better performance than the LBDFE. With our special adapta-tion, the problem of error propagation is reduced. Simulation results verified our analysis and confirmed that the proposed STE can achieve good performance, even in severe channel environments.


[1] J. G. Proakis, Digital Communications, 3rd ed. New York: McGraw-Hill, 1995.

[2] R. B. Ertel, P. Cardieri, K. W. Sowerby, T. S. Rappaport, and J. H. Reed, “Overview of spatial channel models for antenna array communication systems,” IEEE Pers. Commun., vol. 5, no. 1, pp. 10–22, Feb. 1998. [3] R. A. Monzingo and T. W. Miller, Introduction to Adaptive Arrays.

New York: Wiley, 1980.

[4] L. C. Godara, “Application of antenna arrays to mobile communications. II. Beam-forming and direction-of-arrival considerations,” Proc. IEEE, vol. 85, no. 8, pp. 1195–1245, Aug. 1997.

[5] G. V. Tsoulos, Ed., Adaptive Antennas for Wireless Communications, Piscataway, NJ: IEEE Press, 2001.

[6] R. Kohno, “Spatial and temporal communication theory using adaptive antenna array,” IEEE Pers. Commun., vol. 5, no. 1, pp. 28–35, Feb. 1998. [7] A. J. Paulraj and B. C. Ng, “Space–time modems for wireless per-sonal communications,” IEEE Pers. Commun., vol. 5, no. 1, pp. 36–48, Feb. 1998.

[8] J.-W. Liang, J.-T. Chen, and A. J. Paulraj, “A two-stage hybrid approach for CCI/ISI reduction with space–time processing,” IEEE Commun. Lett., vol. 1, no. 6, pp. 163–165, Nov. 1997.

[9] N. Ishii and R. Kohno, “Spatial and temporal equalization based on an adaptive tapped delay line array antenna,” IEICE Trans. Commun., vol. E78-B, no. 8, pp. 1162–1169, Aug. 1995.

[10] M. Chiani and A. Zanella, “Spatial and temporal equalization for broad-band wireless indoor networks at millimeter waves,” IEEE J. Sel. Areas

Commun., vol. 17, no. 10, pp. 1725–1734, Oct. 1999.

[11] M.-L. Leou, C.-C. Yeh, and H.-J. Li, “A novel hybrid of adaptive array and equalizer for mobile communications,” IEEE Trans. Veh. Technol., vol. 49, no. 1, pp. 1–10, Jan. 2000.

[12] K. Hayashi and S. Hara, “A new spatio-temporal equalization method based on estimated channel response,” IEEE Trans. Veh. Technol., vol. 50, no. 5, pp. 1250–1259, Sep. 2001.

[13] G. V. Tsoulos, “Smart antennas for mobile communication systems: Bene-fits and challenges,” Electron. Commun. Eng. J., vol. 11, no. 2, pp. 84–94, Apr. 1999.

[14] J. C. Liberti and T. S. Rappaport, Smart Antennas for Wireless

Commu-nications: IS-95 and Third Generation CDMA Applications. Englewood

Cliffs, NJ: Prentice-Hall, 1999.

[15] B. Suard, G. Xu, H. Liu, and T. Kailath, “Uplink channel capacity of space-division-multiple-access schemes,” IEEE Trans. Inf. Theory, vol. 44, no. 4, pp. 1468–1476, Jul. 1998.


[16] G. M. Galvan-Tejada and J. G. Gardiner, “Theoretical model to determine the blocking probability for SDMA systems,” IEEE Trans. Veh. Technol., vol. 50, no. 5, pp. 1279–1288, Sep. 2001.

[17] L. J. Griffiths and C. W. Jim, “Alternative approach to linear constrained adaptive beamforming,” IEEE Trans. Antennas Propag., vol. AP-30, no. 1, pp. 27–34, Jan. 1982.

[18] S. Haykin, Adaptive Filter Theory, 3rd ed. Upper Saddle River, NJ: Prentice-Hall, 1996.

[19] J. Labat, O. Macchi, C. Laot, “Adaptive decision feedback equalization: Can you skip the training period?” IEEE Trans. Commun., vol. 46, no. 7, pp. 921–930, Jul. 1998.

[20] Y. Lee and W.-R. Wu, “A robust adaptive generalized sidelobe canceller with decision feedback,” IEEE Trans. Antennas Propag., vol. 53, no. 11, pp. 3822–3832, Nov. 2005.

[21] O. L. Frost, III, “An algorithm for linearly constrained adaptive array processing,” Proc. IEEE, vol. 60, no. 8, pp. 926–935, Aug. 1972. [22] J. Yang, J. J. Werner, and G. A. Dumont, “The multimodulus blind

equalization and its generalized algorithms,” IEEE J. Sel. Areas Commun., vol. 20, no. 5, pp. 997–1015, Jun. 2002.

[23] L. M. Garth, “A dynamic convergence analysis of blind equaliza-tion algorithms,” IEEE Trans. Commun., vol. 49, no. 4, pp. 624–634, Apr. 2001.

[24] P. K. Shukla and L. F. Turner, “Channel-estimation-based adaptive DFE for fading multipath radio channels,” Proc. Inst. Electr. Eng. I, vol. 138, no. 6, pp. 525–543, Dec. 1991.

[25] A. A. Rontogiannis and K. Berberidis, “Efficient decision feedback equal-ization for sparse wireless channels,” IEEE Trans. Wireless Commun., vol. 2, no. 3, pp. 570–581, May 2003.

[26] T.-J. Shan and T. Kailath, “Adaptive beamforming for coherent signals and interference,” IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-33, no. 3, pp. 527–536, Jun. 1985.

[27] S. Valaee, B. Champagne, and P. Kabal, “Parametric localization of dis-tributed sources,” IEEE Trans. Signal Process., vol. 43, no. 9, pp. 2144– 2153, Sep. 1995.

[28] G. H. Golub and C. F. Van Loan, Matrix Computation, 3rd ed. Baltimore, MD: The Johns Hopkins Univ. Press, 1996.

[29] B. Widrow, K. Duvall, R. Gooch, and W. Newman, “Signal cancellation phenomena in adaptive antennas: Causes and cures,” IEEE Trans.

Anten-nas Propag., vol. AP-30, no. 3, pp. 469–478, May 1982.

Yinman Lee (M’06) was born in Hong Kong. He

received the Ph.D. degree from National Chiao Tung University, Hsinchu, Taiwan, R.O.C., in 2006.

He is currently with the Graduate Institute of Communication Engineering, National Chi Nan University, Nantou, Taiwan. His research interests include adaptive signal processing, wireless commu-nications, and multiple antenna systems.

Wen-Rong Wu (M’89) received the B.S. degree

in mechanical engineering from Tatung Institute of Technology, Taipei, Taiwan, R.O.C., in 1980, and the M.S. degrees in mechanical and electrical engineer-ing and the Ph.D. degree in electrical engineerengineer-ing from the State University of New York, Buffalo, in 1985, 1986, and 1989, respectively.

Since August 1989, he has been a faculty mem-ber with the Department of Communication Engi-neering, National Chiao Tung University, Hsinchu, Taiwan. His research interests include statistical sig-nal processing and digital communications.


Fig. 1. Hybrid of GSC and LBDFE (with LBDFE in startup period).
Fig. 1. Hybrid of GSC and LBDFE (with LBDFE in startup period). p.3
Fig. 2. Proposed hybrid STE.
Fig. 2. Proposed hybrid STE. p.5
Fig. 6. Learning curves of channel estimate MSE for different schemes.
Fig. 6. Learning curves of channel estimate MSE for different schemes. p.10
Fig. 3. Learning curves of GSC output SINR for different schemes in sup- sup-pressing CCI.
Fig. 3. Learning curves of GSC output SINR for different schemes in sup- sup-pressing CCI. p.10
Fig. 4. Beam patterns (enlarged region around CCI’s DOA) of different schemes in Fig. 3 after 5000 iterations.
Fig. 4. Beam patterns (enlarged region around CCI’s DOA) of different schemes in Fig. 3 after 5000 iterations. p.10
Fig. 5. Learning curves of equalizer output MSE for different schemes.
Fig. 5. Learning curves of equalizer output MSE for different schemes. p.10


Fig. 9. Learning curves of equalizer output MSE for different schemes in which the actual main DOA is −5 ◦ , and the presumed main DOA is 0 ◦ .
Fig. 9. Learning curves of equalizer output MSE for different schemes in which the actual main DOA is −5 ◦ , and the presumed main DOA is 0 ◦ . p.11
Fig. 7. Learning curves of GSC output SINR for different schemes in which the actual main DOA is −5 ◦ , and the presumed main DOA is 0 ◦ .
Fig. 7. Learning curves of GSC output SINR for different schemes in which the actual main DOA is −5 ◦ , and the presumed main DOA is 0 ◦ . p.11
Fig. 8. Beam patterns of different schemes in Fig. 7 after 5000 iterations in which the actual main DOA is −5 ◦ , and the presumed main DOA is 0 ◦ .
Fig. 8. Beam patterns of different schemes in Fig. 7 after 5000 iterations in which the actual main DOA is −5 ◦ , and the presumed main DOA is 0 ◦ . p.11
Fig. 12. Learning curves of equalizer output MSE for different schemes with spatial ISI (artificial errors added as five consecutive errors from the 2000th iteration, ten interleaved errors from the 3000th iteration, and ten consecutive errors from the 400
Fig. 12. Learning curves of equalizer output MSE for different schemes with spatial ISI (artificial errors added as five consecutive errors from the 2000th iteration, ten interleaved errors from the 3000th iteration, and ten consecutive errors from the 400 p.12
Fig. 11. SER performance for the scenario in Fig. 10.
Fig. 11. SER performance for the scenario in Fig. 10. p.12