Chapter 2 Low-Density Parity-Check Code
2.3 Encoding of LDPC Codes
2.3.3 Quasi-Cyclic Code
As reviewed in section 2.2, the quasi-cyclic code can be described by a parity-check matrix H =[A1,A2,...Al] and each of a circulant matrix A is j
completely formed by the polynomial a(x)=a0 +a1x+....+av−1xv−1 with coefficients from its first row. A code C with parity-check matrix H can be completely characterized by the polynomials a x1( ), ( ),..., a x2 and ( )a x . As for the encoding, if l one of the circulant matrices is invertible (say A ) the generator matrix for the code l can be constructed in the following systematic form.
⎥⎥ process can be achieved with linear complexity using a v(l−1)-stage shift register.
Regarding the algebraic computation, the polynomial transpose is defined as
∑
−As an example, consider a rate-1/2 quasi-cyclic code with v=5, l =2, the first described by the polynomial
3
Figure 2.6 shows the example parity-check matrix and the corresponding generator matrix.
(a) A parity-check matrix with two circulants
⎥⎥
(b) The corresponding generator matrix in systematic form
Figure 2.6 Example of a rate-1/2 quasi-cyclic code. (a) Parity-check matrix with two circulants, where a1(x)= 1+x and a2(x)=1+x2 +x4. (b) Corresponding generator matrix in systematic form.
Quasi-Cyclic Based Code [21]
As reviewed in section 2.2, the quasi-cyclic based code can be described by a
parity-check matrix ⎥ encoding for the quasi-cyclic based structure, suppose that two of the circulant matrices Al−1 and B are invertible, we can derive two generator matrices in the l following systematic forms
[
( 2) 1]
bits, each having the same length v. The encoding procedure is partitioned into two steps.Then, combine the parity bits p1 with the message bits d to form an intermediate codeword c′ where c′=[d, p1].
Encoding Step 2: The last parity bits p2 can be derived from the generator matrix G2 and the intermediate codeword c′. That is
p2 =c′×G2. (2.30)
2.4 Conventional LDPC Code Decoding Algorithm
There are several decoding algorithms for LDPC codes. The LDPC decoding algorithms can be summarized as: bit-flipping algorithm [20], and message passing algorithm [11]. In the following, we will make an introduction of the decoding algorithms.
2.4.1 Bit-Flipping Algorithm [20]
The idea for decoding is the fact that in case of low-density parity-check matrices the syndrome weight increases with the number of errors in average until errors weights are much larger than half the minimum distance. Therefore, the idea is to flip one bit in each iteration, and the bit to be flipped is chosen such that the syndrome weight decreases. It should be noted that not only rows of the parity-check matrix can be used for decoding, but in principle all vectors of the dual code with minimum (or small) weight. In the following, we will introduce two of the bit-flipping algorithms [20].
Notation and Basic Definitions
The idea behind this algorithm is to “flip” the least number of bits until the parity check equation H x⋅ T = is satisfied.0 Suppose a binary (n,k) LDPC code is used for error control over a binary-input additive white Gaussian noise (BIAWGN) channel with zero mean and power spectral density σ2. The letter n is the code length and k is the message length. Assume binary phase-shift-keying (BPSK) signaling with unit energy is adopted. A codeword c=( , , ,c c0 1 L cn−1) {∈ GF(2)}n is mapped into bipolar sequence x=( , , ,x x0 1 L xn−1) before its transmission, where xi = ⋅2 (ci − 1), 0≤ ≤ −i n 1. Let y=( , , ,y y0 1 L yn−1)be the soft-decision received sequence at the output of the receiver matched filter. For 0≤ ≤ −i n 1, yi = + , where xi ni n is a i Gaussian random variable with zero mean and variance σ2. An initial binary hard decision of the received sequence,z(0) =(z0(0),z1(0), ,L z(0)n−1), is determined as follows
(0) 1, 0
0, 0
i i
i
z y
y
⎧ ≥
= ⎨⎩ ≤ (2.31)
For any tentative binary hard decision z made at the end of ach decoding iteration, we can compute the syndrome vector as s=H z⋅ T. One can define the log-likelihood ratio (LLR) for ear channel output yi, 0≤ ≤ − : i n 1
( 1| ) ln ( 0 | )
i i
i
i i
p c y
L p c y
= =
= (2.32) The absolute value of L , i Li , is called the reliability of the initial decision zi(0). For any binary vector v=( , , ,v v0 1L vn−1), let wt(v) be the Hamming weight of v . Let
u be the n dimensional unit vector, i.e., a vector with “1” at the i-th position and “0” i
everywhere else.
Algorithm I
Step (1) Initialization: Set iteration counter k = 0. Calculatez(0)andS(0) =wt H z( ⋅ (0)T). Step (2) If S( )k = , then go to Step (8). 0
Step (3) k←k+1. If k>kmax, where kmax is the maximum number of iterations, go to Step (9).
Step (4) For each i=0,1, ,L n−1, calculate Si( )k =wt H[ ⋅(z(k−1)+ui) ]T Step (5) Find j( )k ∈{0,1, ,L n−1} with ( ) ( )
arg(min0 )
k k
i n i
j S
≤ <
= .
Step (6) If j( )k = j(k−1), then go to Step (9).
Step (7) Calculate ( )
( ) ( 1)
k
k k
z =z − +uj and S( )k =wt H z( ⋅ ( )kT). Go to Step (2).
Step (8) Stop the decoding and return z( )k .
Step (9) Declare a decoding failure and return z(k−1) .
So the algorithm flips only one bit at each iteration and the bit to be flipped is chosen according to the fact that, in average, the weight of the syndrome increases with the weight of the error. Note that in some cases, the decoder can choose a wrong position j, and thus introduce a new error. But there is still a high likelihood that this new error will be corrected in some later step of the algorithm.
Algorithm II
Algorithm I can be modified, with almost no increase in complexity, to achieve better error performance, by including some kind of reliability information (or measure) of the received symbols. Many algorithms for decoding linear block codes
based on this reliability measure have been devised. Consider the received soft-decision sequencey=( , , ,y y0 1 L yn−1). For the AWGN channel, a simple measure of the reliability,L , of a received symbol i y is its magnitude, i yi . The larger the
magnitude yi is, the larger the reliability of the hard-decision digit z is. If the i reliability of a received symbol y is high, we want to prevent the decoding i algorithm from flipping this symbol, because the probability of this symbol being erroneous is less than the probability of this symbol being correct. This can be achieved by appropriately increasing the values S in the decoding algorithm. The i solution is to increase the values of S by the following term: i Li . The larger value
of Li implies that the hard-decision z is more reliable. The steps of the soft i version of the decoding algorithm are described in detail below:
Step (1) Initialization: Set iteration counter k = 0. Calculatez(0)andS(0) =wt H z( ⋅ (0)T). Step (2) If S( )k = , then go to Step (8). 0
Step (3) k←k+1. If k >kmax, go to Step (9).
Step (4) For each i=0,1, ,L n−1, calculate Si( )k =wt H[ ⋅(z(k−1)+ui) ]T + Li Step (5) Find j( )k ∈{0,1, ,L n−1} with ( ) ( )
arg(min0 )
k k
i n i
j S
≤ <
= .
Step (6) If j( )k = j(k−1), then go to Step (9).
Step (7) Calculate ( )
( ) ( 1)
k
k k
z =z − +uj and S( )k =wt H z( ⋅ ( )kT). Go to Step (2).
Step (8) Stop the decoding and return z( )k .
Step (9) Declare a decoding failure and return z(k−1).
It is important to point out that, in both algorithms, though the maximum number of iteration is specified, the algorithms have an inherent stopping criterion.
The decoding process stops either when a valid codeword is obtained (Step 2) or when the minimum syndrome weight at the kth iteration and the minimum syndrome weight at the (k-1)th iteration are found in the same position (Step 6).
The bit-flipping algorithm just corrects at most one error bit in one iteration. The codeword length of LDPC code is usually hundreds (or thousands) of bits. When the channel SNR (signal-to-noise ratio) is low, the decoding iteration number of the bit-flipping algorithm needs to be high to correct the erroneous bits. This will lower the throughput of the decoder. And according to Step (5), equation (2.33) is to find the minimal value of the n numbers. The value of n (codeword length) is usually large.
The hardware complexity of equation (2.33) is high.
( ) ( ) arg(min0 )
k k
i n i
j S
≤ <
= (2.33)
2.4.2 Message Passing Algorithm [11]
Since the bit-flipping algorithm is hard to be implemented in hardware, the message passing algorithm is extensively used for LDPC decoding. The message passing algorithm is an iterative decoding process. Messages between variable nodes and check nodes are exchanged back and forth. The decoder expects that error will be corrected progressively by using this iterative message-passing algorithm. At present, there are two types of iterative decoding algorithms applied to LDPC codes in general.
Sum-product algorithm, also known as belief propagation algorithm.
Min-sum algorithm
Both of sum-product algorithm and min-sum algorithm are message passing algorithms. In the following, we will discuss these two algorithms in detail. First, we explain the decoding procedure in Tanner graph below.
Decoding Procedure in Tanner Graph Form
Now we make a description of the message passing algorithm using Tanner graph form. Here is a simple example of irregular LDPC code. The parity-check matrix is shown below.
Tanner graph of this parity-check matrix is shown in Figure 2.7.
Figure 2.7 Tanner graph of the given example parity-check matrix
Assume every line in the Tanner graph has two information messages. One is expressed in a solid line and the other is expressed in a dotted line. We use the messages to decode the received signal. For convenience of explanation, we take one part of Tanner graph which is shown below.
1 1 0 1 1 0 1 1
H ⎡ ⎤
= ⎢ ⎥
⎣ ⎦
S2x1 x2 x3 x4 S1
The solid line and the dotted line are represented by qs→x and rx→s, respectively. In this example, we can get qs1→x1 by rx2→s1 and rx4→s1. Equation (2.34) shows how to compute qs1→x1.
1 1 ( 2 1 4 1)
s x x s x s
q → =CHK r → ⊕r → (2.34)
On the other hand, we can also get rx1→s1 by qs2→x1 and L , where 1 L is the 1 initialization value. The initialization value L will be discussed later. Equation (2.35) 1 shows how to compute
1 1
x s
r → .
1 1 ( 2 1 1)
x s s x
r → =VAR q → ⊕L (2.35)
There is CHK function in equation (2.34) and VAR function in equation (2.35).
The two special functions will be introduced in the following contents. In the Tanner graph, we can compute the solid line message qs→x by the dotted line messages rx→s which are connected to the same check node. In the same way, we can compute the
dotted line message rx→s by the real line messages qs→x which are connected to the same bit node. So the values of rx→s and qs→x are updated iteratively. We call this iterative decoding.
Decoding Procedure in Matrix Form
Because Tanner graph is a representation of the parity-check matrix H, we can also use the matrix form to replace Tanner graph form. Let us take the same parity-check matrix H in the previous section 1 1 0 1
1 0 1 1
H ⎡ ⎤
= ⎢ ⎥
⎣ ⎦ as an example. In equation (2.36) and equation (2.37), we define matrix Q and matrix R. The positions of the nonzero values in R and Q are the same as those of the ones in H.
The elements in the matrix Q are computed by the elements in the matrix R, for example, qs1→x1 =CHK r( x2→s1⊕rx4→s1). On the other hand, the elements in the matrix R are computed by the elements in the matrix Q. For example,
1 1 ( 2 1 1)
x s s x
r → =VAR q → ⊕L , where L is the initialization value. So the elements in 1 matrix R and Q are updated iteratively. We can also regard the CHK function as the horizontal step and VAR function as the vertical step in the decoding procedure.
In the LDPC iterative decoding procedure, there are two main functions: VAR and CHK. Equation (2.38) shows the VAR function with two inputs and equation (2.39) is the general form of the VAR function. The VAR function is fixed regardless of the decoding algorithms. It is just a summation operation.
The CHK function with two inputs can be reformulated in different forms.
There are When CHK function is in the form of equation (2.40) or equation (2.42), we call the decoding algorithm as sum-product algorithm. The fourth term
2
equation (2.42) is called the correction factor. When the check node computation is in
the form of equation (2.43), or in other words an approximate form, we call it the min-sum algorithm.
The above discussion of check node computation is only about the CHK function with two inputs. Now, we will discuss the general form of the CHK function. The general form of the CHK function can be expressed in equation (2.44).
) )...) )
( ( (...
( )
...
(L1 L2 Ll CHK CHK CHK CHK L1 L2 L3 Ll
CHK ⊕ ⊕ ⊕ = ⊕ ⊕ ⊕ (2.44)
The purpose of equation (2.44) is to unfold CHK L( 1⊕L2⊕ ⊕... Ll). The procedure is:
first, compute a1=CHK L( 1⊕L2), then a2 =CHK a( 1⊕L3), …,
1 ( 2 )
l l l
a− =CHK a− ⊕L . The computation result of equation (2.44) is al−1. This can be viewed as serial computation. Figure 2.8 shows the serial configuration for the general form of the CHK function.
Figure 2.8 Serial configuration for check node update function
The serial computation has a long critical path in the check node update unit.
From equations (2.40), (2.43), and (2.44), we can generalize the CHK function as equation (2.45) for sum-product algorithm, and equation (2.46) for min-sum algorithm.
1 2 1 2 1
( l) l ( ) [ (i ) ( ) ( l)]
i
CHK L L L sign L φ φ L φ L φ L
=
⊕ ⊕ ⊕L =
∏
+ + +L (2.45)where ( ) ln 1 1
x x
x e
φ = ⎜⎛e + ⎞⎟
⎝ − ⎠
1 2 1 2 1 2
( l) ( ) ( ) ( ) min[l , , , l ]
CHK L ⊕L ⊕ ⊕L L =sign L ⋅sign L ⋅Lsign L L L L L (2.46) Equations (2.45) and (2.46) tell us that the check node update function can also be
viewed as parallel configuration. If we derive the check node update function in parallel configuration, the critical path of the check node update function will be reduced. Figure 2.9 and 2.10 respectively show the check node updating function of the sum-product algorithm and the min-sum algorithm. These two figures neglect the multiplication of the sign symbols for an artistic view of the figures.
Figure 2.9 Check node update function of sum-product algorithm
Figure 2.10 Check node update function of min-sum algorithm
Iterative Decoding Procedure [12]
The discussion in section 2.4.2 is only part of the whole iterative decoding procedure. Now, we consider the actual decoding procedure. It means that there will involve many iterations for a decoding process. First, let us describe some notations for the iterative decoding procedure in Figure 2.11. M(l) denotes the set of check nodes that are connected to the variable node l, i.e., positions of “1”s in the l th column of the parity-check matrix. L(m) denotes the set of variable nodes that participate in the m parity-check equation, i.e., the positions of “1”s in the th m th row of the parity-check matrix. L(m)\l represents the set L(m) excluding the l th iterative decoding procedure is shown below.
⎥⎥
mindex nodeCheck
Figure 2.11 Notations for iterative decoding procedure
1. Initialization transmitted, the probability that the receiver receives a, where σ2 is the noise variance of the Gaussian channel. For every position ( lm, ) such that Hm,l =1,
m l
q → is initialized as
m l l
q → = . L (2.47)
2. Message passing
Step1 (message passing from check nodes to variable nodes): Each check node m gathers all the incoming message qm→l’s, and update the message on the variable node l based on the messages from all other variable nodes connected to the check node m.
L denotes the set of variable nodes that participate in the m parity-check th equation. )L(m can also be viewed as the horizontal set in the parity check matrix H.
Step2 (message passing from variable nodes to check nodes): Each variable node l passes its probability message to all the check nodes that are connected to it.
( )\ ( )\ that are connected to the variable node l are summed up.
( ) ( )
Hard decision is made on ql. The decoded vector xˆ is decided as Otherwise, it goes to step1 until the parity-check equation is procured or the specific maximum iteration number is reached. The whole LDPC decoding procedure can be expressed in Figure 2.12.
Figure 2.12 The whole LDPC decoding procedure
Table 2.4 Summary of sum-product algorithm 1. Initialization:
2. Message passing:
Step1: Message passing from check nodes to variable nodes. For each ml, ,
If HxT then x is the estimated codeword
or the iteration number is reached a predetermined threshold the algorithm stops
2. Message passing:
Step1: Message passing from check nodes to variable nodes. For each ml, ,
Chapter 3
Modified Min-Sum Algorithms
In this chapter, we will introduce modified LDPC decoding algorithms. As mentioned in chapter 2, the sum-product algorithm has better performance than min-sum algorithm. In the following, we will depict the difference between sum-product algorithm and min-sum algorithm. Our final goal is to modify min-sum algorithm in order to achieve decoding performances close to sum-product algorithm.
3.1 Normalization Technique for Min-Sum Algorithm [14]
Equation (3.1) is the check node updating function in the sum-product algorithm.
In equation (3.1), there is a major function ( ) ln 1 1
x x
x e
φ = ⎜⎛e + ⎞⎟
⎝ − ⎠. The function plot of ( )x
φ is shown in Figure 3.1. Implementation of the nonlinear function ( )φ x is complicated. Even the commonly adopted table-look-up scheme suffers loss in error performance because of the large quantization error, especially when x is small.
1 2 1 2
1
( ) ( ) [ ( ) ( ) ( )]
w
l i l
i
CHK L L L sign L φ φ L φ L φ L
=
⊕ ⊕ ⊕L =
∏
+ + +L (3.1)0 1 2 3 4 5 6 7 8 0
1 2 3 4 5 6
x
1 1()ln
x xe exφ⎛⎞+ ⎜⎟ −⎝⎠=
Figure 3.1 Function plot of ( )φ x
Equation (3.2) is the check node update function used in min-sum algorithm.
The key part of equation (3.2) is to find the minimal value among w numbers:
1 2
min[L , L , ,L Lw]. The value of w is decided by the row weight of the parity check matrix H and it’s usually small (say, 6 or 7). Therefore, the min-sum algorithm is more suitable to for implementation in hardware.
1 2 1 2
1
( w) w ( ) min[i , , , w]
i
CHK L L L sign L L L L
=
⊕ ⊕ ⊕L =
∏
L (3.2)As we mentioned in chapter 2, equation (3.2) is an approximate form of equation (3.1). Assume the result of equation (3.1) is A and the that of equation (3.2) is B. In [14], it proves the following two statements about the relationship between A and B.
Statements:
(1) Values A and B have the same sign, i.e.,sign A( )=sign B( );
(2) The absolute magnitude of B is always greater than that of A, i.e., B > A Statement (1) is quite straightforward because ( )φ x and min( )x are both positive functions. For convenience of proving statement (2), we assume B = Li , where i is an arbitrary number between 1 and w.
Note that because ( )φ x is a decreasing function, the comparison symbol should be changed if one takes the function ( )φ x on both inequality sides. Hence statement (2) is proved.
These two statements suggest the use of normalization to get more accurate soft values from B . In other words, one can multiply B by a factor β which is smaller than 1 to get a better approximation of A . To determine the normalization factor β, one can consider the criterion of forcing the mean of the normalized magnitude β⋅ B to equal the mean of the magnitude A [14], i.e. may not be the best, but it seems a quite reasonable choice. In the following, a theoretical value of β is derived.
It is assumed the channel is a Gaussian channel with noise variance σ2. For convenience, one denotes the set { :L ii =1, 2, , }L w . Then L are independent, and i identically distributed (i.i.d.) random variables. The probability density function (p.d.f.) of L depends on SNR and code rate. One can also write i of A and B statistically based on equations (3.4) and (3.5). The normalization factor can be obtained from equation (3.3). One can calculate equations (3.4) and (3.5) by the theory of probabilities.
First, one can calculate E B[ ]. Let Mi = Li , 1, 2, ,i= L w, so that the p.d.f. of
The second integration in (3.8) can be omitted and finally one obtains
A few lower-order terms of equation (3.12) are enough to give a very good estimation of E A[ ] in most cases. Combined with value E B[ ] given in equation (3.9), one
can obtain the theoretical value of the normalization factor β. But in practical, the theoretical value of β is hard to compute. To use the theoretical value of β for different SNR values seems to be impractical. Thus, for a specific LDPC code, one can associate a fixed normalization factor through simulations.
Now, let’s set the number of w (the input number of a check node updating function) to 6. This is because the row-weight of H is 6 or 7 in 802.16e standard (see appendix A). Assume
1 2 6
[ ( ) ( ) ( )]
A=φ φ L +φ L + +L φ L (3.13)
1 2 6
min[ , , , ]
B= L L L L (3.14)
The purpose of Figure 3.2 is to find the normalization factor β. The vertical axis of Figure 3.2 is β⋅ −B A, and the horizontal axis is β. In hardware implementation, only a certain value of β will be chosen for finite-precision representation. For example, one can set β to be a multiple of 0.125 for simple hardware implementation. Through Figure 3.2, our objective is to choose the most appropriate
β so that the value of β⋅ −B A is as small as possible. From simulations, β =0.75 is found to be a suitable value. When β is 0.75, it is shown that
B A
β⋅ − is less than 0.2.
Figure 3.2 The absolute difference between the normalization technique and sum-product algorithm, vs. the normalization factorβ
3.2 Dynamic Normalization Technique for Min-Sum Algorithm [23]
In section 3.1, one can use the normalized factor β to compensate the result of equation (3.2) so that it can approximate equation (3.1) more accurately. In [23], it shows the idea to adjust the normalized factor β dynamically to get better decoding performance. Thus the normalization factor β can have the form:
1
2
, when , when
B K
B K
β β β
⎧ <
= ⎨⎩ ≥ (3.15)
In [23], it selects two normalization factors β1 and β2 first. For convenience of hardware implementation, only certain simple values of β1 and β2 should be chosen for finite-precision realizations. For check node degree of 6, it found that
1 0.75
β = and β2 =0.875 are good choices. Then through simulations, one can find the optimum threshold value K to have the lowest decoder BER. The detailed
simulation results are in chapter 4.
3.3 Proposed Dynamic Normalized-Offset-Compensation Technique for Min-Sum Algorithm
Compared to the dynamic normalization technique, one can extend the idea by adding an additional offset factor α to equation (3.2) [6] in order to get even more accurate check-node updating values. Equation (3.16) shows the normalized-offset technique for min-sum algorithm.
1 2 1 2
1
( w) w ( ){i min[ , , , w] }
i
CHK L L L sign L β L L L α
=
⊕ ⊕ ⊕L =
∏
⋅ L + (3.16)In section 3.1, we have decided the value 0.75 of β when the check node degree is 6.
Through simulations in chapter 4, we find that for fixed value of α , the decoding performance is not always better than that of α =0. So we have the idea to adjust the offset factor α dynamically.
Now, we have the inspiration if the offset factor α can be dynamically adjusted to get better performance. Equation (3.17) shows the dynamic offset factor α .
1
2
, when , when
B K
B K
α α α
⎧ <
= ⎨⎩ ≥ (3.17)
Through simulations, we can decide the best values of α1 and α2. As we discuss in section 3.1, In hardware implementation, only certain simple values of α1and α2 will be chosen for finite-precision realizations. For check node degree of 6, we found that α1= and 0 α2 =0.125 are good choices.
Through simulations, we can decide the best values of α1 and α2. As we discuss in section 3.1, In hardware implementation, only certain simple values of α1and α2 will be chosen for finite-precision realizations. For check node degree of 6, we found that α1= and 0 α2 =0.125 are good choices.