Chapter 2 LDPC codes
2.4 LDPC in 802.15.3c
In this section, we will introduce the LDPC parity-check matrix in IEEE 802.15.3c systems [7]. First, we should notice about the UEP (Unequal Error Protection) property in LDPC Codes. In [8], it mentions that bit nodes with differet degrees have different UEP. The error protection capability of irregular LDPC Codes is improved with its higher degree. We can call these higher degree bit nodes the higher error protection nodes. In the other words, with the lower numbers of
connected check nodes, the bit node has the lower error protection. Figure 2-8 is the LDPC parity-check matrix in IEEE 802.15.3c systems with different code rates.
Figure 2-8: Parity-check matrices for rate 1/2, 3/4, and 7/8 in 802.15.3c In Figure 2-8, the integrate number in a grid indicate the shifting number of a 21x21 unit matrix. And, L1, L2, L3, and L4 indicate the UEP level in the parity-check matrix; L1 has better performance than L2, and so on. The shifting operation in an unit matrix is illustrated below. For example, for a shift number S=0,1,2 in a 8x8 unit matrx, we have
Figure 2-9 shows the density function of decoded LLR’s (16QAM) in different UEP levels at SNR=25dB, and code rate=1/2. We can see that in L1, the distance between the two modes (corresponding to bit 1 or 0) is largest. In other words, the probability of decision errors will be the smallest. Figure 2-10 shows the BER for each UEP level. The result conforms the assertion we just made. In the simulation and
later chapter, we will need to use the UEP property to facilitate our analysis.
Figure 2-9: LLR density functions in different UEP level
Figure 2-10: BER for each UEP level
3 Cooperative Communication Systems
3.1 Cooperative Communication
Figure 3-1 shows a simplest three-terminal network consisting of a source, a relay and a destination, showing the basic idea behind this concept. Since each of the users sees an independent fading path to the destination, diversity is obtained by transmitting the data through the relay. By using this approach, multiple
virtual-antennas can be constructed in the transmitter. Many research works also show that considerable benefits result from signal relaying in fading environments
especially over slow fading channels, including the reduction in outage probability, high capacity, less power consumption and wider dynamic range.
Figure 3-1: The scenario of relay channel
Despite the theoretic advances in wireless user cooperation, practical signal relaying strategies have not evolved much out of the three basic forms proposed by Cover and El Gamal in 1979, namely, amplify-and –forward (AF),
decode-and-forward (DF) and compress-and-forward (CF). In section 3.3 to 3.5, we will review two basic strategies AF and DF, and also the scenarios including LDPC
Codes in these two strategies, after the system model is first given in section 3.2.
A particular powerful variation of user cooperation is coded cooperation.
Coded cooperation integrates cooperation into channel coding. The codeword will experience two independent channels before it is received by the destination.
3.2 System Model [2]
Consider the basic relay system in Figure 3-1 that comprises a source node, a relay node and a destination node. We consider the half-duplex transmit mode which means that the system cannot send and receive data at the same time. The user cooperation is operated in two stages: the broadcasting stage, where the source
broadcasts a packet of data to both the destination and the relay, and the relaying stage, where the relay processes and forwards part or all of the observations to the
destination. The destination then combines the signals received from both stages to make a best estimation of the original data. Throughout the paper, we will use subscripts S, R, D and SR, SD, RD to denote the quantities pertaining to the source, relay, and destination nodes, and those pertaining to the source-to-relay,
source-to-destination and relay-to-destination channels, respectively.
We take AWGN and block Rayleigh fading as our channel models, which are described as
( ) ( ) ( ) (3.1) the relay, y is the received signal and h is the channel state information. In the case of AWGN, h is a constant of 1. In the case of block fading, h follows a Rayleigh
distribution with a variance of 1, remains fixed over a block of fixed size, and changes
independently between successive blocks. When we consider a binary-shift keying (BPSK) modulation, xS ∈ −{1, 1} (0→ −1,1→ . The AWGNs, 1) n , SD n and SR
n , have zero mean and the variances of RD σ , SD2 σ and SR2 σ , respectively. We RD2 consider spatially independent channels among the source, the relay, and the
destination. We also assume that the instantaneous channel condition is known to the receivers, so that the decoder can exploit efficient soft decoding algorithms.
With LDPC Codes in the system, we consider the LDPC Code defined in IEEE 802.15.3c, i.e., r=1/2 and (N,K)=(672,336). In the cooperative system, we let the packet size for transmission is 672 bits. And, the transition protocol is TDMA which means that in the first time slot the source transmits a data packet to both the destination and the relay, and in the second time slot the relay forwards the packet to the destination. In the second time slot, the source neither transmits nor receives signal.
3.3 Amplify-and-Forward (AF)
In the AF scenario, the relay only amplifies and retransmits the analog signal waveform received from the source. The operation of AF is quite straightforward, requiring a lower implementation complexity in digital signal processing. More importantly, AF can operate at all times, even when the source-to-relay channel experiences outage.
In the first time slot, the relay receives the data packet from the source. Due to the channel h , the packet will experience fading and be contaminated with noise SR
n . In the second time slot, the relay just amplifies and re-transmits the received SR
signal to the destination. Finally, the destination uses the maximum ratio combining (MRC) detector to combine the received signals from the both time slots and recover
the original transmitted data
Figure 3-2: AF block diagram Mathematically, the transmit signal at the relay is formulated as
( ) SR( ) 1, 2, , (3.4)
The destination observes from the source-relay-destination (S-R-D) channel a noisy signal of the form:
Equation (3.6) makes the cascade channel behave like a single (block) fading channel
with fading coefficient RD SR
y
, and a complex Gaussian noise of variance
2 2
cooperative communication, the LLR in the destination can be calculated by an efficient way. Upon receiving a symbol, we then have to calculate the LLR of a certain bit. This is referred to soft demapping and it can be derived as follows [9] : First, if we receive a signal r, we can get equalized signal y, it will formulate as (3.7).
( ) ( ) ( ) ( ) ( ) ( ) ( ) (3.7)
( 0) (1)
For a 16 QAM symbol mapping, the function of DI,K is plotted in Figure 3-3 and the 16QAM constellation is in Figure 3-4. If we only see the area where y(i)>0, we can find that the function is nonlinear for two mapped bits. For simplicity, we can approximate it as a linear function.
Figure 3-3: Approximate versus exact LLR functions for the in-phase and quad-phase of the 16QAM constellation
Figure 3-4: Partition of the 16QAM constellation
The destination then gathers the signals received from both the cascade channel and the direct source-to-destination channel using maximal ratio combining (MRC), which in effect is to extract and combine the log-likelihood ratios (LLR) from the channels, i.e.,
Here, we prove that the LLRs of BPSK signals obtained by the summation of LLRs caculated from the direct and the relay link is equivalent to that caculated from the received signal after MRC :
Assume the destination receives ySD from the source and yRD from the relay:
0
∈ the average power of the received signals.
Then we sum two paths LLR in (3.14).
Then we arrange (3.14) to get (3.15)
2 2
Thus, the LLR obtained by the summation of LLRs calculated from the two paths is equivalent to that calculated from the received signal after MRC.
3.4 MRC and Demapping in MQAM
In the last section, we discuss the LLRs of BPSK signals obtained by the summation of LLRs calculated from the two paths is equivalent to that calculated from the received signal after MRC. In the following subsection, we try to analysis in higher level constellation, and we show that the results for these approaches are different.
3.4.1 Demapping and Combining
From the reference [9], the soft bit value can express as (3.10):
(0) (1)
For the in-phase bits of a 16QAM symbol, we have
, The definition of every symbol is as before, and we call this DC.
3.4.2 MRC and Demapping
From the last section 3.3, we find theyMRC,
Where Gc is considered as equivalent channel coefficient, and w is three noise part combination. Then we can demap theyMRC:
(1) ( 0 )
Put this relation into (3.20), and then we can find the result and here we call it MD:
( 0), (1),
2
3.4.3 Performance comparison
From Figure 3-4, if we choose a symbol (-1+j) , then the first bit is 0 and second
From the (3.23) and (3.24), we can find that the means are both zero in the noise part.
But the variance in DC is bigger than the variance in MD. Figure 3-5 shows the pdf of the first bit LLR in both approaches.
Figure 3-5: Pdf of the LLR in DC and MD
The result is MD is better than DC. In the latter simulations in Chapter 6, we will show the performance of these two different methods at the receiver.
3.5 Decode-and-Forward (DF)
In DF, when the relay has successfully decoded all the bits in the received packet, it re-encode a set of bits and re-transmit to the destination. DF typically includes an option to switch to the non-cooperation mode when the relay fails to decode the packet correctly. This is to prevent error propagation and improve overall system performance. In repetition-DF, the destination will combine the signals received from the source and the relay, i.e.
2 2
, ,
2 2
2 | | 2 | |
( ) ( ) ( ) SD RD (3.25)
DF SD RD I k I k
SD RD
h h
LLR i LLR i LLR i D D
σ σ
= + = +
The definition of received signals and symbols in (3.25) is the same in AF. Figure 3-6 is the block diagram of DF.
Figure 3-6 : DF block diagram
4 Gaussian Mixture Identification With EM Algorithm
We specify the maximum-likelihood parameter estimation problem and introduce the Expectation-Maximization (EM) algorithm to solve the parameter estimation problem in this chapter[10]. First we define the maximum-likelihood parameter estimation problem, then describe the EM algorithm, and finally use the EM algorithm to identify a Gaussian mixture.
4.1 Maximum-likelihood Estimation
The maximum-likelihood estimation problem can be defined as follows:
We have a density function ( | )p x Θ that is governed by the set of parameters Θ , and also have a data set of size N which drawn from this distribution,
1 2
{ , , ,x x xN}
Χ = K . We assume that these data are independently and identically distributed (i.i.d.) with the density function p , so the density for the data set is
1
Equation 4.1 is called the likelihood of the parameters given the data, which is thought of as a function of parametersΘ. Here, the data Χ is considered as fixed.
Then in the maximum-likelihood estimation problem, the main goal is to find
Θ which maximizes (4.1). opt
arg max ( | ) (4.2)
can just set the derivative of log( ( | ))p Χ Θ to zero, then directly findµandσ2. However, for many problems, it is difficult to find such analytical expressions, so we have to find more elaborate techniques to solve the problem.
4.2 Basic Expectation-maximization Estimation
EM algorithm is one such elaborate technique. The EM algorithm is
generally used in statistics for finding maximum-likelihood estimates of parameters in probabilistic models, which is an underlying distribution from a given data set when the data is incomplete or has missing values. There are two main applications of the EM algorithm. The first occurs when the data indeed has missing values, due to problems with or limitations of the observed process. The second occurs when optimizing the likelihood function is analytically intractable but when the likelihood function can be simplified by assuming the existence of additional but missing (or hidden) parameters. The second application is the solution what we are concerned later.
We assume that a data set Χ is observed and is generated by some distribution, and we call it incomplete data. We also assume a complete data set Z=(X,Y), and a jointly density function arises from the marginal density function
( | )
p x Θ and the assumption of hidden variable and parameter guesses:
( | ) ( , | ) ( | , ) ( | ) (4.3)
p z Θ = p x y Θ = p y x Θ p x Θ
Now we can define a complete-data likelihood function ( , Y | )p Χ Θ . Note that this function is a random variable since the missing information Y is unknown, random, and presumably governed by an underlying distribution. Since X andΘcan be seen as constants and Y is a random variable, the original complete-data likelihood function can be thought of some function asp( , Y | )Χ Θ =hΧ Θ, (Y).
The first step of the EM algorithm is finding the expected value of the complete-data log-likelihood log( ( , Y | ))p Χ Θ with respect to the unknown data Y given the observed data X and the current parameter estimation.
( 1) ( 1)
( , i ) [log( ( , Y| )) | , i ] (4.4)
Q Θ Θ − =E p Χ Θ Χ Θ −
Where Θ( 1)i− are the current parameters, Θ are the new parameter. We use Θ( 1)i−
to evaluate the expectation and optimize Θ to increase Q . Note that in (4.4), X and
( 1)i−
Θ are constants, Θ is a normal variable that we want to adjust and Y is a random variable governed by the distribution f y( | ,Χ Θ( 1)i− ). Then the right-hand side of (4.4) can be rewritten as (4.5):
( 1) ( 1) the region where the Y can take on. If this marginal distribution is a simple analytical expression of the assumed parameters Θ( 1)i− and perhaps the data, the problem will be easier to solve. However, sometimes this density function might be difficult to find.
The evaluation of this expectation is called the E-step in the EM algorithm.
The second step of the EM algorithm, M-step, has a goal to maximize the expectation we computed in the E-step. Mathematically, it can be expressed as,
( ) ( 1)
arg max ( , ) (4.6)
i i
Q −
Θ = Θ Θ Θ
Then, the two steps are iterated, and it has been shown that each iteration is
guaranteed to increase the log-likelihood. The EM algorithm will converge to a local maximum of the likelihood function. There are many works discussing the
convergence problem, but we will not pursue that here. From the description we give, it is not very clear how to exactly conduct the EM algorithm, this is because the details of the steps need to compute the given quantities which are strongly dependent
on the particular application considered.
4.3 Gaussian Mixture Identification via EM Algorithm
The mixture-density parameter estimation problem is one of the most widely used applications of the EM algorithm. For this case, we have the density to be identified as density function parameterized by θi. It can be also considered as we have M component densities mixed together with mixing coefficientsαi.
Even with the Gaussian assumption of pj(.), (4.8) is difficult to maximize since it is highly nonlinear. If we consider X as incomplete, and posit the existence of
unobserved data Y={y } whose values indicating which the component density i i=1N generated each data item, the likelihood expression can be significantly simplified and the solution is easier to obtain. Assuming that yi∈{1, 2, , }K M for each i , and
yi = if the k i sample is generated by the th k mixture component. If we know the th data Y, the likelihood can be expressed as:
1 1
which is a particular form of the component densities, and it can be easily optimized using a variety of techniques. However, we do not know the values of Y. If we assume Y is a random vector, we can proceed.
First, we must derive an expression for the distribution of the unobserved
data. In the beginning, we guess thatΘ =g {α α1g, 2g, ,K α θ θMg, 1g, 2g, ,K θMg}. GivenΘ , g we can easily compute p xj( |i θ for each i and j . Besides, the mixing jg)
parameters αj can be thought of as the priori probability of each mixture component, that isαj = p component j( ). Using Bayes’ rule, we can know:
Then, we can simplify (4.12) as
(4.12) as (4.14):
In order to maximize (4.14), we can maximize the term containing αl and the term containing θl independently since they are not uncorrelated.
To findαl, we introduce the Lagrange multiplier λ with the constraint that
l 1
lα =
∑
. The Lagrange multipliers provide a strategy for finding themaximum/minimum of a function subject to constraints. For example, if we want to maximize ( , )f x y , and the constraint is ( , )g x y = , then the cost function can be c re-defined with the Lagrange multiplier λ as follows:
Λ( , , )x y λ = f x y( , )+λ( ( , )g x y −c) (4.15) Now, we can solve the following equation forαl:
1 1
Note that in our scenario, we do not need to consider the parameters αl since we assume that all component densities mixed together with the same mixing
coefficientαl.
We then want to findθl. For some distributions, it is possible to get an analytical expression forθl. In our scenario, the distribution is a mixture of two one-dimensional
Gaussian distributions with mean µ1 = − and varianceµ2 σ12 =σ22, which is shown
Taking the logarithm of (4.19), ignoring constant terms, and substituting the result into the right side last term of (4.14), we can obtain
Taking the derivative of (4.20) with respect to µl and setting it equal to zero, 2
Finally, we can obtain the estimation forσl2:
Now, a complete EM algorithm for the Gaussian mixture identification is derived. To sum up, the E-step finds the expected value of the complete-data log-likelihood. The M-step obtains a new estimation by maximizing the expectation computed in the E-step. The estimation of the new parameters in terms of the old parameters is summarized as below:
; 1
With the EM algorithm, it is guaranteed to increase the log-likelihood and converge to a local maximum of the likelihood function. Since the EM algorithm is not guaranteed to find the global maximum, we need to choose the initial values carefully.
1
5 Compress and Forward in User Cooperation
In Chapter 3, we have described two cooperative protocols, i.e., AF and DF. In this chapter, we investigate another cooperative scheme, called compress-and-forward (CF). In CF, the relay forwards the quantized/observed/estimated version of its
observations. In [11], the CF for turbo decoder was introduced. In this chapter, we will extend its use to the LDPC decoder.
Since the SR channel may have deep fades with a high probability, the DF scheme cannot operate in the cooperative mode all the time and this will cause performance degradation. The AF does not have the problem. However, the SR channel may be noisy and retransmission will further amplify the noise. The CF can alleviate the problems mentioned above. Under the CF, the relay, whether the decoding is successful or not, retransmits the information from the source to the destination. Then the destination can combine both the LLR from the source and the relay for data detection. We will have more details in the later section.
5.1 Compress-and-forward (CF) Cooperation Strategy
Figure 5-1: Block diagram of hybrid compress-and-forward (CF)
Figure 5-1 shows the block diagram of the hybrid CF cooperative strategy, but for simplicity, we just call it CF. In DF, when the decoding in the relay fails, the relay switches to the non-cooperation mode. As a result, the destination only have the information from the source; However, in the CF scheme, when the decoding fails, the relay will switches to a mode which will retransmit the quantized LLR
information to the destination. Figure 5-2 shows the CF cooperative protocol.
Figure 5-2: CF for BPSK modulation
In Figure 5-2, xs denotes the transmit signal at the source, vs its received signal at the destination, r the LLR of a decoded bit at the relay, and w ˆ( )t (t) an modulated index for the quantized LLR (here only one bit quantization), and z (t) the received index at the destination. Then the destination collects v and z s (t) to recover the transmit data.
Here, we assume that the LDPC code is used at the source. The overall approach can be summarized as follows:
1) In the first time slot, the source broadcasts signal x to the relay and the s destination simultaneously.
2) The relay performs LDPC decoding to estimate x . If s x is decoded s
successfully, the relay use traditional DF scheme. If the decoding fails, the relay
successfully, the relay use traditional DF scheme. If the decoding fails, the relay