極碼的可信度傳遞解碼排程

(1)

國

立

交

通

大

學

電信工程研究所

碩

士

論

文

極碼的可信度傳遞解碼排程

On the Scheduling of Belief Propagation Decoding for Polar Codes

研究生：劉振灃

指導教授：陸曉峯教授

(2)

極碼的可信度傳遞解碼排程

On the Scheduling of Belief Propagation Decoding for Polar Codes

研究生：劉振灃 Student：Chen-Feng Liu

指導教授：陸曉峯教授 Advisor：Prof. Hsiao-Feng Lu

國立交通大學

電信工程研究所

碩士論文

A Thesis

Submitted to Institute of Communications Engineering College of Electrical and Computer Engineering

National Chiao Tung University for the Degree of Master

in

Communications Engineering

July 2011

Hsinchu, Taiwan, Republic of China

(3)

i

極碼的可信度傳遞解碼排程

學生：劉振灃

指導教授

：

陸曉峯教授

國立交通大學電信工程研究所碩士班

摘

要

通道編碼藉由將訊息增加冗餘，讓訊息傳輸更可靠。一直以來有許多專家學者致力於研究通道編碼系統，期望可以建構出一套在傳輸速率不大於通道容量的條件下，可以可靠的傳送訊息的編碼系統。極碼（Polar codes）在使用接續消除解碼（Successive cancellation decoding）的方式下，理論上可以讓傳輸速率達到通道容量。除了接續消除解碼之外，當我們把碼長增加時，尚有其他解碼方式可以讓錯誤率衰減的更快速。可靠度傳遞解碼（Belief propagation decoding）為其中一種方法。可靠度傳遞解碼已被用來當作極碼的解碼方式，解出的訊息錯誤率相對於接續消除解碼解出的錯誤率，前者較低。為了方便解碼，接續消除解碼器忽略了某些可利用的訊息，而這些訊息則被可靠度傳遞解碼器善加利用。論文[1]說明了相對於接續消除解碼，將可靠度傳遞解碼作為極碼解碼方式的優越性。在本篇論文中，我們使用類似接續消除解碼的解碼方式，作為可靠度傳遞解碼的排程。藉由模擬二位元輸入高斯通道，我們可以看出在某些情況下，將可靠度傳遞解碼加入此排程會比單純使用可靠度傳遞解碼有較好的效能。

(4)

On the Scheduling of Belief Propagation Decoding for Polar Codes

Student：Chen-Feng Liu

Advisors：Prof. Hsiao-Feng Lu

Institute of Communications Engineering

National Chiao Tung University

Abstract

Channel coding is a scheme that adds redundancy to messages for reliable

transmission. Many works have been devoting to constructing a coding scheme

that can transmit messages reliably at rates set below the channel capacity. Polar

code is one such scheme that can achieves channel capacity theoretically under

successive cancellation (SC) decoding.

Besides SC decoding, there are other decoding algorithms such that as code

length increases, error rates of polar codes decrease more rapidly. For instance,

belief propagation (BP) decoding has been used to decode polar codes with error

rates better than the SC decoding.

BP decoder uses the information ignored by SC decoder, and its superiority

over SC decoder has been established in [1]. In this thesis, we propose a

scheduling for the BP decoding of polar codes that resembles the SC decoding.

By simulation over binary-input AWGN channel, we see that in some cases,

error performance in this schedule is better than that of BP decoding alone.

(5)

iii

Acknowledgement

I appreciate my advisor for his teaching to be a good researcher. And many

thanks to the mates in Coding science lab, during the past two years, I was glad

to do research with all of you.

I am also very thankful to my family and girlfriend. They always encouraged

and supported me patiently when I felt upset.

(6)

List of Figures

1.1 Fundamental block of recursive structure. . . 3

1.2 Vector channel P4. . . 3

1.3 Vector channel PN. . . 4

2.1 Recursive property of SC decoding function for N =4. The left pair in the butterfly pattern can be assembled from the right pair. . . 13

3.1 Factor graph of polar codes for N =8. . . 15

3.2 Factor graph of the 4th coordinate channel for N =8, P8(4), dashed lines and nodes are eliminated edges and nodes. . . 16

3.3 SC decoding diagram using factor graph, arrows represent message-passing directions in decoding process. . . 17

3.4 Comparison of SC decoding (upper curve) and BP decoding (lower curve) in terms of word error rate, when transmission takes place over binary-input AWGN channel (σ=0.97865) [1]. . . 18

4.1 Factor graph of the first block for N =8. . . 20

4.2 Factor graph of the second block for N =8. . . 20

4.3 Calculate initial LLR values of polar codes factor graph. . . 21

4.4 Update LLR from right to left in iteration 1. . . 22

4.5 Update LLR from left to right in Iteration 1. . . 22

4.6 Update LLR from right to left in iteration 2. . . 23

4.7 Error performance comparison of increasing block number in incremental updating schedule. Code rate is 1/3 and block length is 2048. . . 25

4.8 Error performance comparison of increasing block number in incremental updating schedule. Code rate is 1/2 and block length is 4096. . . 25 4.9 Error performance comparison for code rate is 1/2 and block length is 4096. 26

(9)

4.10 Error performance comparison of increasing iteration times under BP de-coding for code rate is 1/2 and block length is 4096. . . 26 4.11 Error performance comparison for code rate is 5/6 and block length is 4096. 27 4.12 Error performance comparison of increasing iteration times under BP

de-coding for code rate is 5/6 and block length is 4096. . . 27 4.13 Error performance comparison for code rate is 1/2 and block length is 8192. 28 4.14 Error performance comparison of increasing iteration times under BP

(10)

Chapter 0 Notations and Terminologies

We denote random variables by capital letters, X, and their sample values by the corresponding lower-case letters, x. Matrices are represented by boldface capitals like G, and corresponding boldface lower-case letters with underline and subscript denote row vectors of the matrix, g_k is the kth row vector of matrix G. Vector (u1, · · · , uN)

is abbreviated as uN

1 . A is a subset of integer, |A| is the size of set A. Let uA denotes

|A|-tuple vector, and indices of elements are included in A, i.e., (ui : i ∈ A). 1(E) denotes

indicator function of event E.

Consider a binary-input discrete memoryless channel (B-DMC) with input alphabet X , output alphabet Y and transition probability P (y|x), x ∈ X , y ∈ Y. The input alpha-bet X is GF(2), the output alphaalpha-bet Y and transition probability P (y|x) are arbitrary. Bhattacharyya parameter of the B-DMC is

Z =X

y∈Y

pP (y|x = 0) · P (y|x = 1)

Symmetric capacity of B-DMC is mutual information of input and output when input random variable is uniform distributed, i.e.,

I(X; Y ) = X y∈Y X x∈X 1 2P (y|x) log2 P (y|x) 1 2 P (y|x = 0) + 1 2P (y|x = 1)

(11)

Chapter 1 Introduction

Polar codes, introduced by Arikan in [2], are provable to achieve channel capacity for symmetric B-DMCs. As code length grows larger, some of coordinate channels seen by individual bits become more reliable while the others get worse. The effect is called channel polarization. Thus as code length N → ∞, fraction of coordinate channels are noiseless, and we transmit information bits via these reliable coordinate channels. Besides low encoding complexity, the SC decoder, proposed together with polar codes [2], also has low decoding complexity.

1.1 Channel polarization

Channel polarization is an operation that combines N independent copies of B-DMC into a vector channel, and then splits the vector channel into N binary-input coordinate channels. In this thesis, we denote transition probability of vector channel with suffix, i.e., PN. Besides suffix N , superscript (i) is together with transition probability of the ith

coordinate channel, P_N(i).

1.1.1 Channel combination

This operation combines two identical and independent vector channels with input alphabet XN/2_{and output alphabet Y}N/2 _{into another vector channel with input alphabet}

XN _{and output alphabet Y}N _recursively.

The first level of recursion is shown in Fig. 1.1, where u1 and u2 are used to produce

(12)

2 y 2 u x2 1 u x1 y1

P



2 P

Figure 1.1: Fundamental block of recursive structure.



P



2 x 3 x 4 x 1 x 2 v 2 u 3 v 2 y 3 y 3 u 4 v y₄ 4 u 1 y 1 v 1 u 2 P P P P 4 P 2 P 4 R

Figure 1.2: Vector channel P4.

get transition probability of vector channel , P2 : X2 → Y2,

P2(y1, y2|u1, u2) = P (y1, y2|x1, x2)

= P (y₁|x1) · P (y2|x2)

= P (y1|u1⊕ u2) · p(y2|u2)

For N = 4, channel combination is shown in Fig. 1.2. At first, the vector (u1, u2, u3, u4)

are calculated over binary field and permuted via R4; then the permuted vector (v1, v2, v3,

v4) are sent through two identical vector channels P2. Finally, (x1, x2, x3, x4) is transmitted

through four B-DMCs.

From Fig. 1.1 and Fig. 1.2, the transition probability of vector channel, P4 : X4 → Y4,

is

(13)

= P2(y1, y2|u1 ⊕ u2, u3 ⊕ u4) · P2(y3, y4|u2, u4)

= P (y₁|u1⊕ u2⊕ u3⊕ u4) · P (y2|u3⊕ u4) · P (y3|u2⊕ u4) · P (y4|u4)

To generalize channel combination, we let N = 2n_{. At first, vector (u}

1, · · · , uN) is

calculated over binary-field and then permuted in block RN. Permutation function of RN

is vi =    wk+1 2 , if k is odd, 1 ≤ i ≤ N 2 wk 2+ N 2, if k is even, N 2 + 1 ≤ i ≤ N

The permuted vector (v1, v2, · · · , vN) is decomposed into two vectors, (v1, v2, · · · , vN 2) and

(vN

2+1, · · · , vN). The former is sent through the upper vector channel, PN/2, and the latter

(14)

G= B F⊗nand F =   1 0 1 1  ,

B is called bit-reversed matrix. For an N -tuple vector, (t1, t2, · · · , tN), we represent the

indices of elements in binary sequence. For example, N = 8, (t1, t2, t3, t4, t5, t6, t7, t8)

=(t000, t001, t010, t011, t100, t101, t110, t111). After multiplied by bit-reversed matrix,

(t1, t2, t3, t4, t5, t6, t7, t8) B

= (t000, t001, t010, t011, t100, t101, t110, t111) B

= (t000, t100, t010, t110, t001, t101, t011, t111)

= (t1, t5, t3, t7, t2, t6, t4, t8)

Given the generator matrix G, the transition probability of vector channel, PN : XN →

YN _is PN(y1, · · · , yN|u1, · · · , uN) = P (y1N|x N 1 = u N 1 G) = P (y₁|x1) · · · P (yN|xN)

1.1.2 Channel splitting

Having combined N independent B-DMCs into a vector channel, the next step is to split it into N binary-input coordinate channels. The transition probability of each coordinate channel is defined as

P_N(i)(y₁, · · · , y_N, u1, · · · , ui−1|ui) = X uN i+1∈XN −i P (y₁, · · · , y_N, u1, · · · , uN) P (ui) = X uN i+1∈XN −i 1 2N −1PN(y1, · · · , yN|u1, · · · , uN) = X uN i+1∈XN −i 1 2N −1P (y1|x1) · P (y2|x2) · · · P (yN|xN), xN 1 = u N 1 G, 1 ≤ i ≤ N, N = 2 n_{, n ≥ 1,}

(15)

where the (y₁, · · · , y_N, u1, · · · , ui−1) is considered as channel output and ui is the input

of the ith coordinate channel. Bhattacharyya parameter of the ith coordinate channel is Z(P_N(i)) = X yN1∈YN X ui−11 ∈Xi−1 q P_N(i)(yN 1 , u i−1 1 |ui = 0) · P (i) N (y1N, u i−1 1 |ui = 1)

It is shown [2] that the transition probabilities of coordinate channels have the follow-ing recursive property.

P_N(2i−1)(y₁, · · · y_N, u1, · · · , u2i−2|u2i−1)

=X

u2i

1 2· P

(i)

N/2(y1, · · · , yN/2, u1⊕ u2, · · · , u2i−3⊕ u2i−2|u2i−1⊕ u2i)

· P_N/2(i) (yN/2+1, · · · , yN, u2, u4, · · · , u2i−2|u2i) (1.1) and P_N(2i)(y₁, · · · y_N, u1, · · · , u2i−1|u2i) = 1 2· P (i)

N/2(y1, · · · , yN/2, u1⊕ u2, · · · , u2i−3⊕ u2i−2|u2i−1⊕ u2i)

· P_N/2(i) (yN/2+1, · · · , yN, u2, u4, · · · , u2i−2|u2i) (1.2)

1.2 Polar coding

Now, we consider block codes that contain polar codes as a special case. Let A be an arbitrary subset of {1, 2, · · · , N }. We fix A and uAc, but let U_A be free vector; then we

get a block code taking the following form (x1, · · · , xN) = uA      g j1 ... g jK      + uAc      g l1 ... g lN −K      , K = |A| , ji ∈ A , and 1 ≤ i ≤ K, lm ∈ Ac, and 1 ≤ m ≤ N − K.

In particular, it is a coset code of a linear block code with generator matrix      g j1 ... g jK      , K = |A| , ji ∈ A , and 1 ≤ i ≤ K,

(16)

where the coset leader is uAc      g l1 ... g lN −K      , K = |A| , lm ∈ Ac, and 1 ≤ m ≤ N − K,

We denote this coset code as (N, A, uAc). We refer to A as information set and u_Ac as the

frozen vector. The code rate of this coset code is |A|/N . Given the coset code (N, A, uAc),

we encode (u1, · · · , uN) into (x1, · · · , xN), and transmit (x1, · · · , xN) through N copies of

B-DMC. Upon receiving channel output (y₁, · · · , y_N), the task is to decode (ˆu1, · · · , ˆuN).

Polar codes are specified by the rule that minimizes an upper bound of error proba-bility, and they are channel-specific, i.e., polar codes for two different channels may have different information sets.

1.3 Thesis outline

In Chapter 2, we will introduce the successive cancellation (SC) decoder for the polar codes. It is known that polar codes under SC decoding can achieve channel capacity over symmetric B-DMCs [2]. In Chapter 3, we will introduce the factor-graph representation of polar codes and coordinate channels. Another method has been developed to construct polar codes for symmetric B-DMCs, it will also be introduced in Chapter 3. Knowing that the belief propagation (BP) decoder exhibits good error performance on turbo codes and LDPC codes, it will be used to eliminate some drawbacks of SC decoder and will be shown to have good error performance. In the last section of chapter 3, we will compare the performances of SC and BP decoders at moderate code length. In Chapter 4, we will use the idea of SC decoding to modify the scheduling of BP decoder. Simulation results will be provided.

(17)

Chapter 2 Successive Cancellation Decoder

2.1 The decoding algorithm

For the coset code (N, A, uAc), since knowing u_Ac in advance, we set ˆu_Ac = u_Ac before

decoding. The output of SC decoder is ˆ ui =    h(y₁, · · · , y_N, ˆu1, · · · , ˆui−1), if i ∈ A ui, if i ∈ Ac

where the decoding function for information bits is h(y1, · · · , yN, û1, · · · , ûi−1) =      0 , if P (i) N (y1,··· ,yN,û1,··· ,ûi−1|ui=0) P_N(i)(y₁,··· ,y_N,û1,··· ,ûi−1|ui=1) ≥ 1 1 , otherwise (2.1) Decoding error occurs when ûA 6= uA. The block error probability is denoted by

Pe(N, A, uAc).

2.2 Block error of SC decoding

For a fixed information set A,

P (u1, · · · , uN, y1, · · · , yN) =

PN(y1, · · · , yN|u1, · · · , uN)

2N

and (u1, · · · , uN) ∈ XN, (y1, · · · , yN) ∈ Y N_,

Define E as the error event of SC decoding, i.e.,

(18)

ˆ Ui(yN1 , uN1 ) =    h(yN 1 , ˆu i−1 1 ), if i ∈ A ui, if i ∈ Ac

Given a fixed uAc, the error probability Pr(E|UAc=uAc) = Pe(N, A, uAc). We denote by Bi

the first SC decoding error occurs at the ith bit, i.e., Bi = {(uN1 , y N 1 ) : Û1(y1N) = u1, · · · , Ûi−1(y1N, û i−2 1 ) = ui−1, ˆ Ui(y1N, û i−1 1 ) 6= ui} = {(uN 1 , y N 1 ) : Û1(y1N) = u1, · · · , Ûi−1(y1N, u i−2 1 ) = ui−1, ˆ Ui(y1N, u i−1 1 ) 6= ui} ⊆ {(uN 1 , y N 1 ) : Ûi(y1N, u i−1 1 ) 6= ui} = {(uN 1 , y N 1 ) : P (i) N (y1, · · · , yN, u1, · · · , ui−1|ui) ≤ P_N(i)(y1, · · · , yN, u1, · · · , ui−1|ui⊕ 1)} := Ei

Note that E = ∪i∈ABi, and {Bi, i ∈ A} are disjoint. Then

Pr(E) = X uAc∈XN −K 1 2N −K · P (E|UAc=uAc) = X uAc∈XN −K 1 2N −K · X uA∈XK,yN1 ∈YN PN(yN1 |uN1 ) 2K · 1(∪i∈ABi) = X uAc∈XN −K 1 2N −K · X uA∈XK,yN1 ∈YN PN(yN1 |uN1 ) 2K · X i∈A 1(Bi) ≤ X (uN 1,y1N)∈XN×YN PN(yN1 |uN1 ) 2N · X i∈A 1(Ei) =X i∈A X (uN 1,yN1 )∈XN×YN PN(yN1 |uN1 ) 2N · 1(Ei)= X i∈A Pr(Ei) (2.2) ≤X i∈A X (uN 1,yN1 )∈XN×YN PN(yN1 |uN1 ) 2N · v u u t P_N(i)(yN 1 , u i−1 1 |ui⊕ 1) p(i)_N(yN 1 , u i−1 1 |ui)

(19)

=X i∈A X ui∈X X (yN 1 ,u i−1 1 )∈YN×Xi−1 X uN_i+1∈XN −i PN(y1N|uN1 ) 2N · v u u t P_N(i)(yN 1 , u i−1 1 |ui⊕ 1) P_N(i)(yN 1 , u i−1 1 |ui) =X i∈A X ui∈X X (yN 1 ,u i−1 1 )∈YN×Xi−1 P_N(i)(yN 1 , ui−11 |ui) 2 · v u u tP (i) N (y1N, ui−11 |ui⊕ 1) P_N(i)(yN 1 , u i−1 1 |ui) =X i∈A X ui∈X 1 2 · Z(P (i) N ) = X i∈A Z(P_N(i)) (2.3) Following from (2.2) and (2.3), we arrive at the following inequality

Pr(E) ≤X i∈A Pr(Ei) ≤ X i∈A Z(P_N(i)). From the above, the notion of polar code can be defined.

Definition 2.1. For a given code rate K/N , the coset code (N, A, uAc) will be called a

polar codeif the subset A with |A| = K is chosen such thatP

i∈AZ(P (i)

N ) is minimized.

Remark 2.2. The frozen vector uAc can be chosen at will, because for additive-noise

channel the choice is insensitive to the code performance. For symmetric channels, choos-ing 0 or 1 for frozen bits would not make any difference on error performance [2]. For simplicity, we choose 0 as frozen bits over this thesis for simulation.

In [2], the functions Z(P_N(i)) are shown to possess the following recursive property         

Z(P_N/2(i) ) ≤ Z(P_N(2i−1)) ≤ 2Z(P_N/2(i) ) − Z(P_N/2(i) )2

Z(P_N(2i)) = Z(P_N/2(i) )2

1 ≤ i ≤ N/2, 0 ≤ Z(P ) ≤ 1,

In particular, the equality holds for BEC, making the task of determining the information set much easier.

Remark 2.3. For a given B-DMC, Bhattachayya parameter of the channel approximates to 0, while symmetric capacity of the channel is much closed to 1, and vice versa [2].

(20)

On the other hand, the information theoretic view suggests I(UN 1 ; Y N 1 ) = N X i=1 I(Ui; Y1N|U i−1 1 ) = N X i=1 I(Ui; Y1N, U i−1 1 ) − I(Ui; U1i−1) = N X i=1 I(Ui; Y1N, U i−1 1 ) (2.4)

where the second equality follows from the independence among U1, · · · , UN; hence we

get I(Ui; U1i−1) = 0. Furthermore, we remark that I(Ui; Y1N, U i−1

1 ) is mutual information

of the ith coordinate channel. On the other hand, note I(UN 1 ; Y N 1 ) = I(X N 1 ; Y N 1 ) = N I(X; Y ) (2.5)

where the first equality is due to that the generator matrix G is one-to-one and onto mapping.

Since U1, · · · , UN are i.i.d. and uniform distributed, I(Ui; Y1N, U i−1

1 ) and I(X; Y ) are

symmetric capacity of the ith coordinate channel and transmitting channel respectively. Lemma 2.4 ([2]). As N → ∞, I(Ui; Y1N, U

i−1

1 ) converge to 0 or 1 almost surely.

From (2.4) and (2.5), we can get PN

i=1I(Ui; Y1N, U1i−1)/N = I(X; Y ). It follows

that the fraction of noiseless coordinate channels is symmetric capacity of transmitting channel as N → ∞. Furthermore, as N → ∞, fraction of coordinate channels whose Bhattacharyya parameters approach 0 is symmetric capacity of transmitting channel. To summarize, the above altogether shows that we can make code rate equals symmetric capacity.

As N → ∞, we can encode information bits with R ≤symmetric capacity, with SC decoding error tending to 0. We know that symmetric capacity is channel capacity for symmetric B-DMCs. So, polar codes can achieve the capacity for symmetric B-DMCs [2].

2.3 Recursive property of SC decoding

In this section we provide the recursive property for the log-likelihood ratios (LLRs) that will be used for the decoding function h (c.f. (2.8)) of SC decoding for polar codes.

(21)

From (1.1) and (1.2), we get

L(2i−1)_N (y1, · · · , yN, ˆu1, · · · , ˆu2i−2)

= lnP

(2i−1)

N (y1, · · · , yN, ˆu1, · · · , ˆu2i−2|u2i−1 = 0)

P_N(2i−1)(y1, · · · , yN, ˆu1, · · · , ˆu2i−2|u2i−1 = 1)

= 2 tanh−1{tanh[L(i)_N/2(y1, · · · , yN/2, û1⊕ û2, · · · , û2i−3⊕ û2i−2)/2]

× tanh[L(i)_N/2(y_N/2+1, · · · , y_N, û2, û4, · · · , û2i−2)/2]} (2.6)

and

L(2i)_N (y₁, · · · , y_N, ˆu1, · · · , ˆu2i−1)

= L(i)_N/2(yN/2+1, · · · , yN, û2, û4, · · · , û2i−2)

+ (−1)uˆ2i−1_{· L}(i)

N/2(y1, · · · , yN/2, û1⊕ û2, · · · , û2i−3⊕ û2i−2) (2.7)

Each value in the pair

L(2i−1)_N (y₁, · · · , y_N, ˆu1, · · · , ˆu2i−2), L (2i)

N (y1, · · · , yN, ˆu1, · · · , ˆu2i−1)

can be assembled from the same pair

L(i)_N/2(yN/21 , û1⊕ û2, · · · , û2i−3⊕ û2i−2), L (i) N/2(y N N/2+1, û2, û4, · · · , û2i−2) .

The N values L(i)_N, 1 ≤ i ≤ N , can be calculated from N values L(j)_N/2, 1 ≤ j ≤ N/2. In the following, if (a, b) is assembled from (c, d), we denote it by (a, b) ←→ (c, d). This, for example in the case of N =4,

L(1)₄ (y4 1), L (2) 4 (y 4 1, û1) ←→L(1)₂ (y₁, y₂), L(1)₂ (y₃, y₄) L(3)₄ (y4 1, û1, û2), L (4) 4 (y 4 1, û1, û2, û3) ←→L(2)₂ (y₁, y₂, û1⊕ û2), L (2) 2 (y3, y4, û2) L(1)₂ (y₁, y₂), L(2)₂ (y₁, y₂, û1⊕ û2 ←→L(1)₁ (y₁), L(1)₁ (y₂) L(1)2 (y3, y4), L (2) 2 (y3, y4, û2) ←→L(1)1 (y3), L (1) 1 (y4) The above relation is shown in Fig. 2.1.

(22)

 2

_

_

2 1, 2, ˆ1 ˆ2 L y y u u  1

1 L y  1

_

_

2 3, 4 L y y  2

_

_

2 3, 4,ˆ2 L y y u

Figure 2.1: Recursive property of SC decoding function for N =4. The left pair in the butterfly pattern can be assembled from the right pair.

The decoding function can be rewritten as h(y₁, · · · , y_N, û1, · · · , ûi−1) =    0 , if L(i)_N(y1, · · · , yN, û1, · · · , ûi−1) ≥ 0 1 , otherwise (2.8) Thus, knowing LLR values of every channel output, we can successively decode informa-tion bits by calculating N LLR values, L(i)_N, which are functions of channel LLR values and previously decoded bits.

(23)

Chapter 3 Belief Propagation Decoder

3.1 Factor graph

In [3], Forney showed normalized factor graph of RM codes. Since polar codes are sub-codes of full RM(n, n) codes, we know polar codes can also be represented by the same factor graph, rendering the BP decoder to the decoding of polar codes. There are n = log₂N sections in the factor graph of polar codes, and each section consists of N/2 Z-shaped sub-graphs. Fig. 3.1 is the factor graph of polar codes for N = 8. In addition to BP decoding, we remark that the SC decoding can also be illustrated by factor graph. From (1.1) and (1.2), we can get factor graph of the ith coordinate channel from factor graph of polar codes [4]. The factor graph of the ith coordinate channel can be obtained through the following steps.

1. In the left-most section, eliminate the degree-3 check nodes that are not connected to the ith variables nodes and the edges incident to the check nodes.

2. Eliminate the rest degree-0 and 1 variable nodes except the ith variables nodes, the edges incident to these eliminated variable nodes should also be eliminated.

3. Eliminate the rest degree-1 check nodes and the incident edges.

4. In the second left-most section, eliminate the degree-3 check nodes that are not connected to the left-most ith variable node via left path.

(24)

Then, we can get the factor graphs of coordinate channels of all N bits, uN

1 . They all are

trees. Fig. 3.2 is the factor graph of the 4th coordinate channel for N = 8.

3.2 Codes construction on symmetric B-DMCs

Let a(i)_N denotes the probability density function (pdf) of L(i)_N(yN 1 , u

i−1

1 ) conditioned on

ui = 0. In [4], it’s shown that for the symmetric B-DMCs,

Pr(Ei) = 1 2 Z ∞ −∞ a(i)_N(x)e− |x|2 + x 2 dx, 1 ≤ i ≤ N (3.1) For symmetric B-DMCs, the Bhattacharyya parameter of the channel is

Z = Z ∞

−∞

a(x)e−x2dx

a(x) is pdf of LLR of the symmetric B-DMC output conditioned on input bit is zero [5]. For symmetric B-DMCs, we know that Bhattacharyya parameters of all N coordinate channels are

Z(P_N(i)) = Z ∞

−∞

a(i)_N(x)e−x2dx, 1 ≤ i ≤ N

We can choose A to minimize P

i∈APr(Ei) or P i∈AZ(P (i) N ). + = + = + = + = + = + = + = + = + = + = + = + =

(25)

+ = + = + = + = + = + = + = + = + = + = + = + =

Figure 3.2: Factor graph of the 4th coordinate channel for N =8, P₈(4), dashed lines and nodes are eliminated edges and nodes.

3.3 SC decoding in factor graph

The factor graphs of coordinate channels, introduced in the former section, can be used to illustrate SC decoding. Since ui+1, · · · , uN are not characteristic of the ith coordinate

channel P_N(i), and ˆu1, · · · , ˆui−1 have been decoded, so these N − 1 variable nodes in the

factor graph of polar codes can be eliminated. We do the same eliminating procedure like the former section to get the factor graph of the ith coordinate channel for SC decoding. On the leaf nodes, the channel LLR values are effected by the former decoded bits. It’s illustrated below and in Fig. 3.3 for N = 4 and i = 3.

The first leaf node:

The second leaf node:

(26)

 1 2ˆ1 ˆ2 1 L y _ u u  _   3 2ˆ2 1 L y  u   2 L y  4 L y



4



1, ,ˆ ˆ1 2 L y u u + + + = = = + =

Figure 3.3: SC decoding diagram using factor graph, arrows represent message-passing directions in decoding process.

Input = lnP (y2|u3⊕ u4 = 0) P (y₂|u3⊕ u4 = 1)

= L(y₂) The third leaf node:

The forth leaf node:

Since x4 = u4, and P (y4|x4) = P (y4|u4)

Input = lnP (y4|u4 = 0)

P (y4|u4 = 1)

= L(y₄)

Under SC decoding, we decode information bits successively, and use former i − 1 decoded bits, i.e., ˆu1, · · · , ˆui−1, as information for decoding ui. Since decoding errors

maybe occur while decoding the former information bits, the errors will propagate. In order to get decoding convenience, i.e., recursive formulae (2.6) and (2.7), we assume all the frozen bits are free variables. The assumption violates the actual message probability distribution, so decoding errors will happen more frequently.

(27)

0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R

Fig. 1.

Comparison of (i) SC and (ii) BP decoder in terms of block error

probability, when transmission takes place over the BAWGNC

(σ = 0.97865).

The performance curves are shown for

n = 10 (top left), 11 (top right), 12

(bottom left), and

13 (bottom right).

x7 x6 x5 x4 x3 x2 x1 x0 u7 u6 u5 u4 u3 u2 u1 u0 (a) (b) (c)

Fig. 2.

Figure (a) shows the trellis representation used by Arıkan. Figures

(b) and (c) show equivalent representations obtained by cyclic shifts of the

3 sections of trellis (a).

in an overcomplete representation (similar to the concept used

when computing the stopping redundancy of a code [5]). For

the BEC any such overcomplete representation only improves

the performance of the BP decoder [5]. Further, the

decod-ing complexity scales linearly with the number of different

representations used. Keeping the complexity in mind, instead

of considering all the

n! factorial trellises, we use only the

n trellises obtained by cyclic shifts (e.g., see Figure 2). The

complexity of this algorithm is

O(N (log N )

2

) as compared to

O(N log N ) of the SC decoder and BP over one trellis. The

performance of the BP decoder is improved significantly by

using this overcomplete representation as shown in Figure 3.

We leave a systematic investigation of good schedules and

choices of overcomplete representations for general symmetric

channels as an interesting open problem.

C. Choice of Frozen Bits

For the BP or MAP decoding algorithm the choice of frozen

bits as given by Arıkan is not necessarily optimal. In the

case of MAP decoding we observe (see Figure 4) that the

performance is significantly improved by picking the frozen

bits according to the RM rule. This is not a coincidence;

d

min

is maximized for this choice. This suggests that there might

be a rule which is optimized for BP. It is an interesting open

question to find such a rule.

III. S

OURCE

C

ODING

In this section we show the performance of polar codes in

the context of source coding. We consider both lossless and

0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R

Fig. 3.

Comparison of (i) SC, (ii) BP, (iii) BP with multiple trellises, and

(iv) MAP in terms of block error probability, when transmission takes place

over the BEC

(ǫ =

1₂

). The performance curves are shown for n = 10 (top

left),

11 (top right), 12 (bottom left), 13 (bottom right).

0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R

Fig. 4.

Comparison of block error probability curves under MAP decoding

between codes picked according to Arıkan’s rule and RM rule. The

perfor-mance curves are shown for

n = 10 (left) and 11 (right).

lossy cases and show (in most cases) empirically that they

achieve the optimal performance in both cases. Let Ber

(p)

denote a Bernoulli source with

Pr(1) = p. Let h

₂

(

_{·) denote}

the binary entropy function and

h

−1₂

(

_{·) its inverse.}

A. Lossless Source Coding

1) Single User: The problem of lossless source coding

of a Ber

(p) source can be mapped to the channel coding

problem over a binary symmetric channel (BSC) as shown

in [6], [7]. Let

x = (x

¯

₀

, . . . , x

_N₋₁

)

T

be a sequence of

N i.i.d. realizations of the source. Consider a code of rate

R represented by the parity check matrix H. The vector

¯

x is encoded by its syndrome ¯

s = H¯

x. The rate of the

resulting source code is

1 _{− R. The decoding problem is}

to estimate

x given the syndrome ¯

¯

s. This is equivalent to

estimating a noise vector in the context of channel coding over

BSC

(p). Therefore, if a sequence of codes achieve capacity

over BSC(

p), then the corresponding source codes approach a

rate

h

₂

(p) with vanishing error probability.

We conclude that polar codes achieve the Shannon bound

for lossless compression of a binary memoryless source.

Moreover, using the trellis of Figure 2(a), we can compute

the syndrome with complexity

O(N log N ). The source coding

problem has a considerable advantage compared to the channel

coding problem. The encoder knows the information seen by

the decoder (unlike channel coding there is no noise involved

here). Therefore, the encoder can also decode and check

(a)

N = 2

10 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R

Fig. 1.

Comparison of (i) SC and (ii) BP decoder in terms of block error

probability, when transmission takes place over the BAWGNC

(σ = 0.97865).

The performance curves are shown for

n = 10 (top left), 11 (top right), 12

(bottom left), and

13 (bottom right).

Fig. 2.

Figure (a) shows the trellis representation used by Arıkan. Figures

(b) and (c) show equivalent representations obtained by cyclic shifts of the

3 sections of trellis (a).

in an overcomplete representation (similar to the concept used

when computing the stopping redundancy of a code [5]). For

the BEC any such overcomplete representation only improves

the performance of the BP decoder [5]. Further, the

decod-ing complexity scales linearly with the number of different

representations used. Keeping the complexity in mind, instead

of considering all the

n! factorial trellises, we use only the

n trellises obtained by cyclic shifts (e.g., see Figure 2). The

complexity of this algorithm is

O(N (log N )

2

) as compared to

O(N log N ) of the SC decoder and BP over one trellis. The

performance of the BP decoder is improved significantly by

using this overcomplete representation as shown in Figure 3.

We leave a systematic investigation of good schedules and

choices of overcomplete representations for general symmetric

channels as an interesting open problem.

C. Choice of Frozen Bits

For the BP or MAP decoding algorithm the choice of frozen

bits as given by Arıkan is not necessarily optimal. In the

case of MAP decoding we observe (see Figure 4) that the

performance is significantly improved by picking the frozen

bits according to the RM rule. This is not a coincidence;

d

min

is maximized for this choice. This suggests that there might

be a rule which is optimized for BP. It is an interesting open

question to find such a rule.

III. S

OURCE

C

ODING

In this section we show the performance of polar codes in

the context of source coding. We consider both lossless and

0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R

Fig. 3.

Comparison of (i) SC, (ii) BP, (iii) BP with multiple trellises, and

(iv) MAP in terms of block error probability, when transmission takes place

over the BEC

(ǫ =

1₂

). The performance curves are shown for n = 10 (top

left),

11 (top right), 12 (bottom left), 13 (bottom right).

0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R

Fig. 4.

Comparison of block error probability curves under MAP decoding

between codes picked according to Arıkan’s rule and RM rule. The

perfor-mance curves are shown for

n = 10 (left) and 11 (right).

lossy cases and show (in most cases) empirically that they

achieve the optimal performance in both cases. Let Ber

(p)

denote a Bernoulli source with

Pr(1) = p. Let h

₂

(

_{·) denote}

the binary entropy function and

h

−1₂

(

_{·) its inverse.}

A. Lossless Source Coding

1) Single User: The problem of lossless source coding

of a Ber

(p) source can be mapped to the channel coding

problem over a binary symmetric channel (BSC) as shown

in [6], [7]. Let

x = (x

¯

₀

, . . . , x

_N₋₁

)

T

be a sequence of

N i.i.d. realizations of the source. Consider a code of rate

R represented by the parity check matrix H. The vector

¯

x is encoded by its syndrome ¯

s = H¯

x. The rate of the

resulting source code is

1 _{− R. The decoding problem is}

to estimate

x given the syndrome ¯

¯

s. This is equivalent to

estimating a noise vector in the context of channel coding over

BSC

(p). Therefore, if a sequence of codes achieve capacity

over BSC(

p), then the corresponding source codes approach a

rate

h

₂

(p) with vanishing error probability.

We conclude that polar codes achieve the Shannon bound

for lossless compression of a binary memoryless source.

Moreover, using the trellis of Figure 2(a), we can compute

the syndrome with complexity

O(N log N ). The source coding

problem has a considerable advantage compared to the channel

coding problem. The encoder knows the information seen by

the decoder (unlike channel coding there is no noise involved

here). Therefore, the encoder can also decode and check

(b)

N = 2

11 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R

Fig. 1.

Comparison of (i) SC and (ii) BP decoder in terms of block error

probability, when transmission takes place over the BAWGNC

(σ = 0.97865).

The performance curves are shown for

n = 10 (top left), 11 (top right), 12

(bottom left), and

13 (bottom right).

Fig. 2.

Figure (a) shows the trellis representation used by Arıkan. Figures

(b) and (c) show equivalent representations obtained by cyclic shifts of the

3 sections of trellis (a).

in an overcomplete representation (similar to the concept used

when computing the stopping redundancy of a code [5]). For

the BEC any such overcomplete representation only improves

the performance of the BP decoder [5]. Further, the

decod-ing complexity scales linearly with the number of different

representations used. Keeping the complexity in mind, instead

of considering all the

n! factorial trellises, we use only the

n trellises obtained by cyclic shifts (e.g., see Figure 2). The

complexity of this algorithm is

O(N (log N )

2

) as compared to

O(N log N ) of the SC decoder and BP over one trellis. The

performance of the BP decoder is improved significantly by

using this overcomplete representation as shown in Figure 3.

We leave a systematic investigation of good schedules and

choices of overcomplete representations for general symmetric

channels as an interesting open problem.

C. Choice of Frozen Bits

For the BP or MAP decoding algorithm the choice of frozen

bits as given by Arıkan is not necessarily optimal. In the

case of MAP decoding we observe (see Figure 4) that the

performance is significantly improved by picking the frozen

bits according to the RM rule. This is not a coincidence;

d

min

is maximized for this choice. This suggests that there might

be a rule which is optimized for BP. It is an interesting open

question to find such a rule.

III. S

OURCE

C

ODING

In this section we show the performance of polar codes in

0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R

Fig. 3.

Comparison of (i) SC, (ii) BP, (iii) BP with multiple trellises, and

(iv) MAP in terms of block error probability, when transmission takes place

over the BEC

(ǫ =

1₂

). The performance curves are shown for n = 10 (top

left),

11 (top right), 12 (bottom left), 13 (bottom right).

0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R

Fig. 4.

Comparison of block error probability curves under MAP decoding

between codes picked according to Arıkan’s rule and RM rule. The

perfor-mance curves are shown for

n = 10 (left) and 11 (right).

lossy cases and show (in most cases) empirically that they

achieve the optimal performance in both cases. Let Ber

(p)

denote a Bernoulli source with

Pr(1) = p. Let h

₂

(

_{·) denote}

the binary entropy function and

h

−1₂

(

_{·) its inverse.}

A. Lossless Source Coding

1) Single User: The problem of lossless source coding

of a Ber

(p) source can be mapped to the channel coding

problem over a binary symmetric channel (BSC) as shown

in [6], [7]. Let

x = (x

¯

₀

, . . . , x

_N₋₁

)

T

be a sequence of

N i.i.d. realizations of the source. Consider a code of rate

R represented by the parity check matrix H. The vector

¯

x is encoded by its syndrome ¯

s = H¯

x. The rate of the

resulting source code is

1 _{− R. The decoding problem is}

to estimate

x given the syndrome ¯

¯

s. This is equivalent to

estimating a noise vector in the context of channel coding over

BSC

(p). Therefore, if a sequence of codes achieve capacity

over BSC(

p), then the corresponding source codes approach a

rate

h

₂

(p) with vanishing error probability.

We conclude that polar codes achieve the Shannon bound

for lossless compression of a binary memoryless source.

Moreover, using the trellis of Figure 2(a), we can compute

the syndrome with complexity

O(N log N ). The source coding

problem has a considerable advantage compared to the channel

coding problem. The encoder knows the information seen by

the decoder (unlike channel coding there is no noise involved

(c)

N = 2

12 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R

Fig. 1.

Comparison of (i) SC and (ii) BP decoder in terms of block error

probability, when transmission takes place over the BAWGNC

(σ = 0.97865).

The performance curves are shown for

n = 10 (top left), 11 (top right), 12

(bottom left), and

13 (bottom right).

Fig. 2.

Figure (a) shows the trellis representation used by Arıkan. Figures

(b) and (c) show equivalent representations obtained by cyclic shifts of the

3 sections of trellis (a).

in an overcomplete representation (similar to the concept used

when computing the stopping redundancy of a code [5]). For

the BEC any such overcomplete representation only improves

the performance of the BP decoder [5]. Further, the

decod-ing complexity scales linearly with the number of different

representations used. Keeping the complexity in mind, instead

of considering all the

n! factorial trellises, we use only the

n trellises obtained by cyclic shifts (e.g., see Figure 2). The

complexity of this algorithm is

O(N (log N )

2

) as compared to

O(N log N ) of the SC decoder and BP over one trellis. The

performance of the BP decoder is improved significantly by

using this overcomplete representation as shown in Figure 3.

We leave a systematic investigation of good schedules and

choices of overcomplete representations for general symmetric

channels as an interesting open problem.

C. Choice of Frozen Bits

For the BP or MAP decoding algorithm the choice of frozen

bits as given by Arıkan is not necessarily optimal. In the

case of MAP decoding we observe (see Figure 4) that the

performance is significantly improved by picking the frozen

bits according to the RM rule. This is not a coincidence;

d

min

is maximized for this choice. This suggests that there might

be a rule which is optimized for BP. It is an interesting open

question to find such a rule.

III. S

OURCE

C

ODING

In this section we show the performance of polar codes in

0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R

Fig. 3.

Comparison of (i) SC, (ii) BP, (iii) BP with multiple trellises, and

(iv) MAP in terms of block error probability, when transmission takes place

over the BEC

(ǫ =

1₂

). The performance curves are shown for n = 10 (top

left),

11 (top right), 12 (bottom left), 13 (bottom right).

0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R 0.35 0.40 0.45 10-4 10-3 10-2 10-1 PB R

Fig. 4.

Comparison of block error probability curves under MAP decoding

between codes picked according to Arıkan’s rule and RM rule. The

perfor-mance curves are shown for

n = 10 (left) and 11 (right).

lossy cases and show (in most cases) empirically that they

achieve the optimal performance in both cases. Let Ber

(p)

denote a Bernoulli source with

Pr(1) = p. Let h

₂

(

_{·) denote}

the binary entropy function and

h

−1₂

(

_{·) its inverse.}

A. Lossless Source Coding

1) Single User: The problem of lossless source coding

of a Ber

(p) source can be mapped to the channel coding

problem over a binary symmetric channel (BSC) as shown

in [6], [7]. Let

x = (x

¯

₀

, . . . , x

_N₋₁

)

T

be a sequence of

N i.i.d. realizations of the source. Consider a code of rate

R represented by the parity check matrix H. The vector

¯

x is encoded by its syndrome ¯

s = H¯

x. The rate of the

resulting source code is

1 _{− R. The decoding problem is}

to estimate

x given the syndrome ¯

¯

s. This is equivalent to

estimating a noise vector in the context of channel coding over

BSC

(p). Therefore, if a sequence of codes achieve capacity

over BSC(

p), then the corresponding source codes approach a

rate

h

₂

(p) with vanishing error probability.

We conclude that polar codes achieve the Shannon bound

for lossless compression of a binary memoryless source.

Moreover, using the trellis of Figure 2(a), we can compute

the syndrome with complexity

O(N log N ). The source coding

problem has a considerable advantage compared to the channel

coding problem. The encoder knows the information seen by

the decoder (unlike channel coding there is no noise involved

(d)

N = 2

13

1

Figure 3.4: Comparison of SC decoding (upper curve) and BP decoding (lower curve) in terms of word error rate, when transmission takes place over binary-input AWGN channel (σ=0.97865) [1].

3.4 Belief propagation decoding

Polar codes with SC decoding are asymptotically capacity achieving, but not record breaking for moderate length. Under BP decoding, it uses information provided by frozen bits, and without using the hard-decision bits decoded previously, it can avoid error propagation. We can conclude BP decoding is better than SC decoding.

BP decoding was used to compare performance of polar codes constructed on Bhat-tacharyya parameters and RM codes in [6]. Performance of polar codes under SC decoding and BP decoding was already shown in [1], we can see that BP decoding is obviously better than SC decoding with various code rates and block lengths.

(28)

Chapter 4 Scheduling of BP Decoding

Although BP decoding is better than SC decoding for polar codes, there still exists a gap between BP decoding and MAP decoding for the error rate curve [1]. Some methods have been proposed to improve error performance over BP decoding [1][7].

4.1 Incremental updating schedule

From Section 2.2, we know that if ˆu1, · · · , ˆui−1 are correctly decoded, then ui almost

can be decoded correctly for long block length. But it’s not easy to correctly decode infor-mation bit successively for moderate code length. Since SC decoding can be regarded as a case of BP decoding, the successive decoding notion can be used to modify BP decoding schedule. The proposed decoding schedule is called incremental updating schedule.

We partition the N bits including frozen bits and information bits into many equal-sized blocks according to index-order. Each block has a factor graph that consists of factor graphs associated with coordinate channels described in Section 3.1. For example, for N = 8 we partition N bits into 4 blocks. The first block is (u1, u2), second is (u3, u4)

and so on. Fig. 4.1 shows factor graph of the first block, and factor graph for the second one is shown in Fig. 4.2.

For code length is N , block size is 2l _{and l ≥ 1, there are N/2}l _{iteration times in}

incremental updating decoding schedule. The steps of decoding process are showing below, some figures are also shown for illustration:

1. We first set the initial condition using information provided by information bits and frozen bits, and calculate LLR of every path in the factor graph of polar codes from

(29)

+ = + = + = + = + = + = + = + = + = + = + = + =

Figure 4.1: Factor graph of the first block for N =8.

+ = + = + = + = + = + = + = + = + = + = + = + =

(30)

3 L u

Figure 4.3: Calculate initial LLR values of polar codes factor graph. left-most level to right-most level. It’s shown in Fig. 4.3.

2. At the first iteration, we consider the factor graph of the first block, and calculate LLR of every path in the factor graph from right-most level to left-most level. It’s shown in Fig. 4.4.

3. Update the LLR of every path in the factor graph from left-most level to the right-most level as shown in Fig. 4.5.

4. Consider the factor graph union of block 1 and block 2, revise message passing through every path of the factor graph from right-most level to the left-most level and update back. Updating from right side to left side is shown in Fig. 4.6.

5. During the ith iteration, consider the first i blocks and the associated factor graph, repeat the updating procedure described above.

6. At the last iteration, consider all block, and the associated factor graph is the factor graph of polar codes. We only update LLR from right to left, and add the information provided by frozen bits.

7. Quantize the LLR values associated with every information bit and frozen bit. Fi-nally, decide whether they are 0 or 1.

(31)

2 L y

Figure 4.6: Update LLR from right to left in iteration 2.

4.2 Simulation results

In this thesis, we simulate polar codes with code rates 1/3, 1/2 or 5/6 and code length ranging form 211 _{to 2}13 _{over binary-input AWGN channel.}

The task of finding A is according to (2.2) and (3.1), i.e., Pr(E) ≤ X i∈A Pr(Ei) Pr(Ei) = 1 2 Z ∞ −∞ a(i)_N(x)e− |x|2 + x 2 dx, 1 ≤ i ≤ N

Specifically, we simulate 10000 trials to find A. In each trial, we first calculate L(i)_N(y1, · · · ,

yN, u1, · · · , ui−1) while the sent bit ui = 0. After the 10000 trials, we find the expected

values 1 2e − |L (i) N| 2 + L(i)_N 2 , conditioned on ui = 0, for 1 ≤ i ≤ N

We note that the above expected values would converge to Pr(Ei) as the number of trials

increases. After obtaining the value associated with each i, for any specific code rate, we choose the subset of {1, · · · , N } so as to minimize P

i∈APr(Ei). Having fixed the code

rate and code length, there are two parts in the following simulations.

The first part: For a specific rate and code length, we simulate incremental updating schedule with three partition types, i.e., the decoded bits in the three types are

(33)

partitioned into 28_{, 2}9 _{or 2}10 _{blocks respectively, and show the results of increasing}

number of blocks in incremental updating schedule.

The second part: We partition decoded bits into 210_{in incremental updating schedule.}

The iteration times of BP decoding without scheduling is set to 640 so that the two decoding algorithms almost have the same complexity, i.e., the average iteration times of degree-3 nodes are the same. Besides error performance comparison of SC decoding, BP decoding without scheduling and BP decoding in incremental updating schedule, we also show the results of increasing iteration times under BP decoding without scheduling.

In both simulations we observe that the scheduling based on incremental updating per-forms better than the one without.

(34)

0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 10−3 10−2 10−1 Eb/No (dB) BER

Binary−input AWGN channel, N=2048, Rate=0.333

Incremental updating schedule with 256 blocks Incremental updating schedule with 512 blocks Incremental updating schedule with 1024 blocks

Figure 4.7: Error performance comparison of increasing block number in incremental updating schedule. Code rate is 1/3 and block length is 2048.

1.3 1.35 1.4 1.45 1.5 1.55 1.6 1.65 1.7 1.75 10−4 10−3 10−2 10−1 Eb/No (dB) BER

Incremental updating schedule with 256 blocks Incremental updating schedule with 512 blocks Incremental updating schedule with 1024 blocks

Figure 4.8: Error performance comparison of increasing block number in incremental updating schedule. Code rate is 1/2 and block length is 4096.

(35)

1.3 1.35 1.4 1.45 1.5 1.55 1.6 1.65 1.7 1.75 1.8 10−4 10−3 10−2 10−1 Eb/No (dB) BER

SC decoding

BP decoding without scheduling

BP decoding in incremental updating schedule

Figure 4.9: Error performance comparison for code rate is 1/2 and block length is 4096.

1.3 1.35 1.4 1.45 1.5 1.55 1.6 1.65 1.7 1.75 1.8 10−4 10−3 10−2 10−1 Eb/No (dB) BER

80 iteration times 160 iteration times 320 iteration times 480 iteration times 640 iteration times

Figure 4.10: Error performance comparison of increasing iteration times under BP decod-ing for code rate is 1/2 and block length is 4096.

(36)

3 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 10−4 10−3 10−2 10−1 100 Eb/No (dB) BER

Binary−input AWGN channel, N=4096, Rate=0.833 SC decoding

3 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 10−4 10−3 10−2 10−1 Eb/No (dB) BER

(37)

0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 1.6 10−4 10−3 10−2 10−1 100 Eb/No (dB) BER

SC decoding

0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 1.6 10−4 10−3 10−2 10−1 100 Eb/No (dB) BER

(38)

3 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 10−5 10−4 10−3 10−2 10−1 100 Eb/No (dB) BER

SC decoding

3 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 10−4 10−3 10−2 10−1 Eb/No (dB) BER

(39)

Chapter 5 Conclusion

In the incremental updating schedule, if we partition N bits into more blocks, the error performance will get better. In Fig. 4.7 and Fig. 4.8 we can see that the type with 210

blocks is better than the other two types. The type with 28 _{blocks is the worst one among}

all three. For BP decoding, error performance will get better as number of iterations grows. For incremental updating schedule, if we partition N bits into more blocks, error rates will get better, too. For the cases, in Fig. 4.9, Fig. 4.11, Fig. 4.13 and Fig. 4.15, error performance in incremental updating schedule is better than error performance under BP decoding.

However, in some cases like Fig. 5.1 and Fig. 5.3, decoding error rate in incremental updating schedule is not better than that under BP decoding. In Fig. 5.1, we even have partitioned with minimum block size. For a specific rate or block length, the block number we should partition into so that decoding in incremental updating schedule will be better than BP decoding is not easy to choose and calls for future works.

(40)

0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 10−4 10−3 10−2 10−1 100 Eb/No (dB) BER

SC decoding

0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 10−4 10−3 10−2 10−1 Eb/No (dB) BER

Figure 5.2: Error performance comparison of increasing iteration times under BP decoding for code rate is 1/3 and block length is 2048.

(41)

0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 10−4 10−3 10−2 10−1 100 Eb/No (dB) BER

SC decoding

0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 10−4 10−3 10−2 10−1 Eb/No (dB) BER

Figure 5.4: Error performance comparison of increasing iteration times under BP decoding for code rate is 1/3 and block length is 4096.

(42)

Reference

[1] N. Hussami, S. B. Korada and R. Urbanke, “Perfoamance of polar codes for channel and source coding,” IEEE International Sympousion on Information Theory, 2009. [2] E. Arikan, “Channel polarization: a method for constructing capacity-achieving codes

for symmetric binary-input memoryless channels,”IEEE Transactions on Information Theory, vol. 55, pp. 3051-3073, Jul. 2009.

[3] G. D. Forney Jr., “Codes on graphs: normal realizations,” IEEE Transactions on Information Theory, vol. IT-47, pp. 520-548, Feb. 2001.

[4] R. Mori and T. Tanaka, “Perfoamance of polar codes with the construction using den-sity evolution,” IEEE Communications Letters, vol. 13, no. 7, pp. 519-521, Jul. 2009. [5] T. Richardson and R. Urbanke, Modern Coding Theory. Cambridge University, 2008. [6] E. Arikan, “A perfoamance comparison of polar codes and reed-muller codes,” IEEE

Communications Letters, vol. 12, no. 6, pp. 447-449, Jun. 2008.

[7] A. Eslami and H. Pishro-Nik, “On bit error rate performance of polar codes in finite regime,” 48th Annual Allerton Conference on Communication, Control and Comput-ing, 2010.

極碼的可信度傳遞解碼排程

國

立

交

通

大

學

電信工程研究所

碩

士

論

文

極碼的可信度傳遞解碼排程

On the Scheduling of Belief Propagation Decoding for Polar Codes

研 究 生： 劉振灃

指導教授： 陸曉峯 教授

極碼的可信度傳遞解碼排程

On the Scheduling of Belief Propagation Decoding for Polar Codes

研 究 生：劉振灃 Student：Chen-Feng Liu

指導教授：陸曉峯 教授 Advisor：Prof. Hsiao-Feng Lu

國 立 交 通 大 學

電信工程研究所

碩 士 論 文

極碼的可信度傳遞解碼排程

學生：劉振灃

指導教授

陸曉峯 教授

國立交通大學電信工程研究所碩士班

摘

要

On the Scheduling of Belief Propagation Decoding for Polar Codes

Student：Chen-Feng Liu

Advisors：Prof. Hsiao-Feng Lu

Institute of Communications Engineering

National Chiao Tung University

Abstract

Channel coding is a scheme that adds redundancy to messages for reliable

transmission. Many works have been devoting to constructing a coding scheme

that can transmit messages reliably at rates set below the channel capacity. Polar

code is one such scheme that can achieves channel capacity theoretically under

successive cancellation (SC) decoding.

Besides SC decoding, there are other decoding algorithms such that as code

length increases, error rates of polar codes decrease more rapidly. For instance,

belief propagation (BP) decoding has been used to decode polar codes with error

rates better than the SC decoding.

BP decoder uses the information ignored by SC decoder, and its superiority

over SC decoder has been established in [1]. In this thesis, we propose a

scheduling for the BP decoding of polar codes that resembles the SC decoding.

By simulation over binary-input AWGN channel, we see that in some cases,

error performance in this schedule is better than that of BP decoding alone.

Acknowledgement

I appreciate my advisor for his teaching to be a good researcher. And many

thanks to the mates in Coding science lab, during the past two years, I was glad

to do research with all of you.

I am also very thankful to my family and girlfriend. They always encouraged

and supported me patiently when I felt upset.

Contents

List of Figures

Chapter 0

Notations and Terminologies

Chapter 1

Introduction

1.1

Channel polarization

1.1.1

Channel combination

P

P











R

u

u

u

u

u

u

研究生：劉振灃

指導教授：陸曉峯教授

研究生：劉振灃 Student：Chen-Feng Liu

指導教授：陸曉峯教授 Advisor：Prof. Hsiao-Feng Lu

國立交通大學

碩士論文

陸曉峯教授

_

_

_

_

_

_

_

_