Turbo Decoding

In this section, we derive the metric that will be used for turbo decoding.

Again, r is the received sequence of length ℓ. Denote by ui is the ith bit in the decision sequence. Then, we derive:

where T0 is the set of consecutive two states respectively located at trellis level i − 1 and i, of which the connecting branch is labeled with code bit 0, and subscripts ¯w and w denote the indices of the trellis states. In parallel, we also derive:

Pr (ui = 1 | r) = X

where T1 is the set of consecutive two states respectively located at trellis level i − 1 and i, of which the connecting branch is labeled with code bit 1.

Define the log-likelihood ratio (LLR) of the ith decision bit as follows:

Λ (i) = logPr (ui = 1 | r)

Define

Therefore, determination of the three functions can decide Λ(i), which in turns decides the ith bit ui.

Functions α and β can be recursively computed as follows.

α S_wⁱ

β S_wⁱ_¯

where for the additive white Gaussian noise channel, function γ is equal to:

γ S_wⁱ⁻¹_¯ , S_wⁱ

We now state the algorithmic procedures to obtain the three functions. Note that the RSC trellis we adopt has 16 states as shown in Figure 2.1.

Derivation of function α

1. Initialization: Set α (S₀⁰) = 1 and α (S_w⁰) = 0 for all 1 ≤ w ≤ 15.

2. Recursion: For 1 ≤ i ≤ ℓ, compute α (Swⁱ) = P15

¯

w=0α S_wⁱ⁻¹_¯
γ Swⁱ⁻¹¯ , S_wⁱ
.

Derivation of function β

1. Initialization: Set β S₀^ℓ
= 1 and β S_w^ℓ
= 0 for 1 ≤ w ≤ 15.

2. Recursion: For 1 ≤ i ≤ ℓ, β (Swⁱ¯) =P15

w=0β (S_wⁱ⁺¹) γ (S_wⁱ_¯, S_wⁱ⁺¹).

Derivation of function γ

1. Initialization: Give that ci is the information bit corresponding to the state transition from S_wⁱ⁻¹_¯ to S_wⁱ. Let the corresponding output vector be xi.

2. Computation: γ S_wⁱ⁻¹_¯ , S_wⁱ
= Pr (ci)^√_2πσ¹ exp{−^||ri2σ^−x2i||2}.

Chapter 3

Joint Source-Channel Block Code

In this chapter, the proposed construction approach for the joint source-channel block codes for non-uniform source distributions over additive white Gaussian noise (AWGN) channels will be presented.

3.1 Union bound

In general, it is difficult to determine the close-form formula for the symbol error probability (SEP) corresponding to a codebook and a source distribution. When a criterion for the SEP is needed, researchers will mostly derive the union bound instead. As such, we will use the union bound as a design criterion for the proposed joint source-channel code.

For the derivation of the union bound, a pair-wise error probability should be devised first.

Consider two source symbols si and sj respectively with probabilities pi and pj, and assume that they are mapped respectively to codewords c_i and c_j, between which the Hamming distance is h. In such case, an error occurs when the receiver declares the reception of sj but si is transmitted, or vice versa. Denote the previously mentioned pair-wise error probability

by Pr(si → s^j). Then, when ci is transmitted over the BSC,

Next, we denote by P_e(h) the probability that a transmitted codeword c_i is incorrectly decoded to another codeword at Hamming distance h from it. Then, P_e(h) can be bounded above by: probability Pe must then satisfy:

Pe ≤

to be the average number of codeword pairs with Hamming distance h . Finally, we have Pe ≤

ℓ

X

h=1

KhAh.

We then use the above upper bound as a design criterion for our joint source-channel coding design.

In summary, based on a given code rate R = 1/4 and system signal-to-noise ratio per information bit Eb/N0 = 10 dB, we can compute Kh. Afterwards, we will attempt to find a good joint source-channel block code of length ℓ that hopefully minimizes

ℓ

P

h=1

KhAh.

3.2 Construction of Joint Source-Channel Block Code

For a small codeword length such as ℓ = 8, we can exhaustively search for all the possi-ble code designs and find the one that minimizes P^ℓ

h=1

KhAh. However, such an exhaustive search approach may not be feasible for a larger codeword length. We therefore propose a sub-optimal but low-complexity algorithm to construct a joint source-channel code with a prohibitively small union bound value.

For given n source symbol s1, . . . , sn with probabilities of occurrence p1, . . . , pn, and for a specified codeword length ℓ , we propose to construct a joint source-channel block code as follows.

Step 0. Initialize the length index as one, namely, y = 1. Set code matrices

C =

where 0 is the all-zero matrix of proper size.

Step 1. For 1 ≤ i, j ≤ n, calculate a^i,j,y = dH(ci,1ci,2. . . c_i,y−1, cj,1cj,2. . . c_j,y−1) .

Step 2. Compute the union bound contribution for each codeword, i.e., for 1 ≤ i ≤ n,

g_i = {sⁱ}ⁿi=1 denotes the sequence of the sorted ordering.

Step 4. For 1 ≤ u < q, calculate

(d0,u = (0 ⊕ c^su,y) + asq,su,y

d_1,u = (1 ⊕ csu,y) + a_sq,su,y

and

Step 7. Compute the union bound contribution for each codeword, i.e., for 1 ≤ i ≤ n, g_i = {sⁱ}ⁿi=1 denotes the sequence of the sorted ordering.

Step 9. If y ≥ 2, compute for 1 ≤ u < q,

Step 10. Among r00, r01, r10 and r11;

if r00 is the minimum one, choose c^′sq,y−1c^′sq,y = 00;

else if r01 is the minimum one, choose c^′sq,y−1c^′sq,y = 01;

else if r10 is the minimum one, choose c^′sq,y−1c^′sq,y = 10;

else choose c^′sq,y−1c^′sq,y = 11.

Step 11. Set q = q + 1 and repeat Steps 4–8 until q = n.

Step 12. Compare the union bound of C and C^′, and let C^∗ be the one with a smaller union bound.

Step 13. Set y = y + 1, C^′ = C and C = C^∗. Then repeat Steps 1–10 until y = ℓ.

This algorithm will give us a codebook C^∗ with a satisfiable union bound value.

3.3 Decoding

For a joint source-channel block code, it has been known that the MAP decoder is optimal in the sense of minimizing the symbol error probability. The MAP decoder in general does not have an efficient implementation. Fortunately, it can be noted that the MAP decoding metric derived in Section 2.2 is strictly decreasing when being calculated in a bit by bit fashion;

hence, we can use the priority-first sequential search algorithm as a vehicle to obtain the MAP decision. In this thesis, the priority-first sequential search decoding algorithms have two forms: one for hard-decision decoding and the other for soft-decision decoding.

3.3-A) Priority-first sequential search hard-decision decoding algorithm

Step 0. Construct the binary code tree corresponding to the given codebook, in which a path form the root node to a leaf node is a codeword. Each leaf node (equivalently,

the end node of a codeword path) is associated with a probability corresponding to the probability of occurrence for the source letter that is encoded to this code-word. The probability associated with a non-leaf node is the largest one among all probabilities associated with those leaf nodes that are offsprings of this non-leaf node.

Step 1. Calculate the hard-decision sequence y. Initialize the metric of the root node as 0, and push the root node into the stack.

Step 2. Extract the node with the maximal metric value from the stack, which we denote it by e. Then, for all child nodes of the extracted node, if there is any, perform the following procedure and then repeat Step 2.

• If the child node of the extracted node is a non-leaf node, set the metric of this child node to

where ci,j is the code bit corresponding to the tree branch connecting the extracted node and the child node, and j is the tree level at which the extracted node is located. Push the child node into the stack.

• Else if the child node of the extracted node is a leaf node, set the metric of this child node to

where pi is the probability associated with this leaf node. Push the child node into the stack.

If the extracted node is a leaf node (which certainly has has no child nodes), output the codeword corresponding to the leaf node and stop the algorithm.

3.3-B) Priority-first sequential search soft-decision decoding algorithm

Step 0. The same as Step 0 in the algorithm in 3.3-A.

Step 1. Calculate the log-likelihood ratio φj for all 1 ≤ j ≤ ℓ. Initilize the metric of the root node as 0, and push the root node into the stack.

Step 2. Extract the node with the maximal metric value from the stack, which we denote it by e. Then, for all child nodes of the extracted node, if there is any, perform the following procedure, and then repeat Step 2.

• If the child node of the extracted node is a non-leaf node, set the metric of this child node to

e + (−1)^ci,jφi,

where c_i,j is the code bit corresponds to the tree branch connecting the ex-tracted node and the child node, and and j is the tree level at which the extracted node is located. Push the child node into the stack.

• Else if the child node of the extracted node is a leaf node, set the metric of this child node to

e + (−1)^ci,jφi+ 2 ln (pi) ,

where pi is the probability associated with this leaf node. Push the child node into the stack.

If the extracted node is a leaf node (which certainly has has no child nodes), output the codeword corresponding to the leaf node and stop the algorithm.

We remark that when the metric is non-increasing along all paths, the sequential search of the above two algorithms guarantee to find the optimal leaf path with the maximal metric.

However, the metrics we adopt in the two algorithms are apparently not non-increasing; so the algorithms we propose may end up with a suboptimal codeword path.

Since at low SNR, the complexity of the second sequential search algorithm may be too large to be practical, an amendment refinement to remove some of the tree nodes during the decoding search is additional proposed below.

• When extracting a node at tree level j and associated with probability q, it will be directly discarded if

2 (ℓ − j^M) − ln q^M ≤ (j^M − j) − ln q,

where qM is the largest probability associated with those nodes that have been visited thus far, and jM is the level at which the node that decides qM is located.

Chapter 4

Modified Turbo Code

4.1 Background

In Chapter 3, we have proposed a novel approach to construct a joint source-channel block code for a non-uniform source. This approach however cannot be feasibly applied when the codeword length of interest is moderately large, and hence the attainable error rates are limited. As such, in this chapter, we turn to the modification of turbo codes that were previously designed by assuming uniform prior probabilities on codewords, and target a joint source-channel turbo coding system that can provide a practically acceptable performance for non-uniform sources.

In the literature, there have been publications working on turbo code design for non-uniform source. In [4], a turbo code has been proposed for binary independent and iden-tically distributed (i.i.d.) source with non-uniform marginal distribution. In [5], the turbo code design has been extended to binary first-order Markov sources. Note that for binary i.i.d. sources, each information bit is statistically independent of all previous bits, which facilitates the derivation of the corresponding decoding metric for turbo decoder. When a first-order Markov source is considered, each bit only depends on the previous bit; nonethe-less, the extension derivation of the turbo decoding metric can make use of this statistical

structure. In this thesis, we actually consider a source of different statistical nature, where each information bit is dependent on t previous bits with t being a known function of the bit location. Specifically, an English alphabet can be binary-indexed using five bits; so the first bit is surely dependent on the next four bits, and the second bit is statistically affected by the next three bits, etc., while the fifth bit is actually independent of the next bit if the stream of English letters is i.i.d. in nature. Details will be given in later sections.

4.2 Definitions and Notations

Assume there are n symbols {s¹, s2, . . . , sn} that are generated according to an independently and identically distributed distribution with marginal probabilities {p¹, p2, . . . , pn}. Let g =

⌈log2n⌉. We can then binary-index each source symbol using g bits as s¹ = (00 . . . 00), s2 = (00 . . . 01), s3 = (00 . . . 10), etc. As a result, for 1 ≤ a < g, the (mg + a)th bit only depends on the previous a − 1 bits.

For notational convenience, we derive in the following by assuming the ith bit ui is dependent on the previous t bits u_i−1. . . u_i−t. By our setting in the previous paragraph, u_i−1 should be dependent only on bits u_i−2. . . u_i−t. Denote Ui = u_i−1. . . u_i−t= u_i−1U_i−1.

Denote by S_wⁱ the state at level i, and by Tc the set of state pair S_wⁱ⁻¹_¯ , S_wⁱ
such that the input bit ui = c will make the trellis transition from state S_wⁱ⁻¹_¯ to state S_wⁱ . Let x be the codeword sequence corresponding to the input information sequence {uⁱ}^ℓ1, and let xi

be the codeword vector portion corresponding to u_i. For convenience, we use x^j_i to denote x_ix_i+1. . . x_j. Similarly, denote by r_i the channel output due to input x_i, and use r^j_i to denote rir_i+1. . . rj. Also, abbreviate r = r^ℓ₁ = r1r₂. . . rℓ. Note that for 1/3-rate turbo coding system, xi = x^s_i, x^1p_i , x^2p_i
and ri = r_i^s, r_i^1p, r_i^2p
, where superscripts “1p” and “2p”

indicates the first and second parity-check codeword portions, respectively. We then illustrate the general scheme of a turbo decoder in Fig. 4.1, in which Λ^(j)(i) is the log-likelihood ratio

Figure 4.1: Diagram of a sample turbo decoder.

of the ith bit ui computed by the jth component code decoder, and Λ^(j)ex (i) is the extrinsic information of the ith bit ui obtained from the jth component code decoder.

4.3 Modified Decoding Metric

The derivation of the decoding metric for the turbo code in the previous chapter is based on the assumption that the source is uniform i.i.d. When the source is not uniformly distributed, the equality of (2.3) and (2.4) is no longer valid. Hence, an alternative decomposition of Pr (ui = c | r) should be done.

Let Vt be the binary bit stream of length t , i.e., Vt = vtv_t−1. . . v1, where vi ∈ {0, 1}

for every i, and denote V_t= v_tV_t−1. We then derive:

Pr (ui = c | r) = Pr (ui = c, r ) Pr (r )

= X

(^Swⁱ⁻¹¯ ,Sⁱ_w)∈T^c

Pr S_wⁱ⁻¹_¯ , S_wⁱ, r
Pr (r ) ,

where the derivation of (4.1) can be continued as follows.

X

indepen-dent of ri when given (ui, Ui), we can continued the derivation as:

and observe that when t = 0, bit u_i is independent of any previous bits and the above functions are reduced to:

Hence,

For additive white Gaussian noise channels, function γ is given by

γ c, Vt, S_wⁱ⁻¹_¯ , S_wⁱ

The basic structure of the turbo decoding metric is now done.

Now after the execution of a certain rounds of turbo decoding, we denote by r_i^ex(2) the extrinsic information from the second component decoder after de-interleaving. This r^ex(2)_i will be the input to the first component decoder. The Gaussian assumption on r_i^ex(2) from [6] [7] then gives:

Suppose ¯σ² and ¯M are the estimated variance and mean of r_i^ex(2), respectively, which are

can then be assigned as:

Pr

Hence, function γ can be given as:

γ c, Vt, S_wⁱ⁻¹_¯ , S_wⁱ

Similar to what has been introduced in the previous chapter, function α can be recursively computed through the following steps:

Case 1. i 6= mg + 1, where m ∈ Z⁺ and g = ⌈log2(n)⌉. Knowing that bit uⁱ is dependent

on the previous t bits, we get:

of any previous bits. So we can simply the above derivations to:

We next turn to the recursive computation of function β as follows.

where

where

The initial values of functions α and β are:

α (V0, S₀⁰) = 1 , α (V0, S_w⁰) = 0 for every 0 < w ≤ 16 β 0, Vk, S_w^ℓ
= β 1, Vk, S_w^ℓ
= ¹₂ for every 0 ≤ w ≤ 16

where k ≡ (ℓ − 1) mod g.

It remains to discuss about the iterative decoding scheme based on our newly derived functions. A well-known iterative decoding equation is to iteratively exchange the so-called extrinsic information as follows.

where Λ_ch(i) and Λ_ap(i) stand for the quantities due to channel and a priori information, respectively.

After updating the extrinsic information by the first component decoder, it is the turn of the second component decoder to modify the extrinsic information. Let {˜u}^ℓ1 be the input sequence after interleaving. Assume that (˜u)^ℓ₁ is i.i.d. in statistics; hence the traditional BCJR algorithm described in Section 2.4 can be applicable. Let Λ⁽²⁾(i) be the log-likelihood

ratio updated in Decoder 2. Then we have:

Based on [5], we slightly adjust the extrinsic information equation for Decoder 2 as follows. Again, let {Λ^(2)ex(i)}^ℓi=1 be the de-interleaved sequence for {˜Λ⁽²⁾ex(i)}^ℓi=1. Then,

Chapter 5

Simulation Results

In this chapter, simulation results are provided to demonstrate the performance of the FLEC that we derived in Chapters 3 and 4. Specifically, we examine what has been proposed in Chapter 3 in Section 5.1, and test the FLEC in Chapter 4 in Section 5.2. Remarks on our simulation results are given in Section 5.3.

5.1 Joint Source-Channel Block Code

Throughout this section, the code rate of FLECs examined is fixed as 1/4. The first-order Markov sources are the chosen source distributions. Three Markov source cases will be simulated.

Case 1. Transition distribution Pr (ui = 0 | ui−1 = 0) = Pr (ui = 1 | ui−1 = 1) = 0.9 with initial probability Pr (u1 = 0) = 1 − Pr (u¹ = 1) = 0.9.

Case 2. Transition distribution Pr (ui = 0 | ui−1 = 0) = Pr (ui = 1 | ui−1 = 1) = 0.95 with initial probability Pr (u1 = 0) = 1 − Pr (u¹ = 1) = 0.5.

Case 3. Transition distribution Pr (u_i = 0 | ui−1 = 0) = Pr (u_i = 1 | ui−1 = 1) = 0.55 with initial probability Pr (u₁ = 0) = 1 − Pr (u1 = 1) = 0.55.

Below we illustrate the details of all figures in Section 5.1.

1. Figure 5.1 compares the SER performances between the JSC block codes constructed by exhaustive search and those constructed by our algorithm. The source under test follows Case 1.

2. The setting of Figure 5.2 is same as that in Figure 5.1 except that the source follows Case 2.

3. The setting of Figure 5.3 is same as that in Figure 5.1 except that the source follows Case 3.

4. Figure 5.4 shows the SER performances of JSC block codes constructed by the algo-rithm introduced in Section 3.2 and decoded by the sequential soft-decision decoding algorithm introduced in Section 3.3 for three different codeword lengths. The source follows Case 1.

5. The setting of Figure 5.5 is same as that of Figure 5.4 except that the source follows Case 2.

6. The setting of Figure 5.6 is same as that of Figure 5.4 except that the source follows Case 3.

7. Figure 5.7 compares the SER performances between the JSC block code we constructed and the tandem scheme. The hard-decision decoding scheme is presumed and the source under test follows Case 1. The codeword length of the JSC block code is 64.

The tandem scheme selected for performance comparision is the concatenation of a Huffman code with an (n, k, t) = (63, 16, 11) BCH code. Since the tandem scheme becomes variable length in nature, we use the Levenshtein distance [8] to account for the symbol errors.

8. The setting of Figure 5.8 is same as that of Figure 5.7 except the source follows Case 2.

9. The setting of Figure 5.9 is same as that of Figure 5.7 except the source follows Case 3.

Remarks and observations regarding these figures will be presented in Section 5.3.

5 6 7 8 9 10 11 10^-5

10^-4 10^-3 10^-2

SER

E_b/N₀(dB)

algorithm

exhaustive search from 6dB to 10dB exhaustive search at 5dB exhaustive search at 5.5dB union bound of algorithm

union bound of exhaustive search from 6dB to 10dB union bound of exhaustive search at 5dB union bound of exhaustive search at 5.5dB

Figure 5.1: Performance comparison between the joint source-channel block code constructed by exhaustive search and that by our proposed algorithm with codeword length ℓ = 8. The source follows Case 1 and hard-decision decoding is employed.

5 6 7 8 9 10 11 10^-5

10^-4 10^-3 10^-2

SER

E_b/N₀(dB)

algorithm

exhaustive search from 5dB to 10dB union bound of algorithm

union bound of exhaustive search from 5dB to 10dB

Figure 5.2: Performance comparison between the joint source-channel block code constructed by exhaustive search and that by our proposed algorithm with the codeword length ℓ = 8.

The source follows Case 2 and hard-decision decoding is employed.

5 6 7 8 9 10 11 10^-5

10^-4 10^-3 10^-2

SER

E_b/N₀(dB)

algorithm

exhaustive search from 5dB to 10dB union bound of algorithm

union bound of exhaustive search from 5dB to 10dB

Figure 5.3: Performance comparison between the joint source-channel block code constructed by exhaustive search and that by our proposed algorithm with the codeword length ℓ = 8.

The source follows Case 3 and hard-decision decoding is employed.

0 1 2 3 4 5 6 10

^-5

10

^-4

10

^-3

10

^-2

10

^-1

SER

E

_b

/N

₀

(dB)

length=32 length=48 length=64

Figure 5.4: Performance comparison of the proposed joint source-channel block codes for different codeword length. The source follows Case 1 and soft-decision decoding is employed.

0 1 2 3 4 5 6 10

^-5

10

^-4

10

^-3

10

^-2

10

^-1

SER

E

b

/N

0

(dB)

length=32 length=48 length=64

Figure 5.5: Performance comparison of the proposed joint source-channel block codes for different codeword length. The source follows Case 2 and soft-decision decoding is employed.

0 1 2 3 4 5 6 10

^-5

10

^-4

10

^-3

10

^-2

10

^-1

SER

E

_b

/N

₀

(dB)

length=32 length=48 length=64

Figure 5.6: Performance comparison of the proposed joint source-channel block codes for different codeword length. The source follows Case 3 and soft-decision decoding is employed.

1 2 3 4 5 6 10^-5

10^-4 10^-3 10^-2 10^-1

SER

E

_b

/N

₀

(dB)

Huffman+BCH(n=63,k=16,t=11) JSC block code(length=64)

Figure 5.7: Performance comparison between the joint source-channel block code with code-word length 64 and the tandem scheme (i..e, the Huffman code + (63, 16, 11) BCH code).

The source follows Case 1 and hard-decision decoding is employed.

0 1 2 3 4 5 6 10^-5

10^-4 10^-3 10^-2

SER

E

b

/N

0

(dB)

Huffman+BCH(n=63,k=16,t=11) JSC block code(length=64)

Figure 5.8: Performance comparison between the joint source-channel block code with code-word length 64 and the tandem scheme (i..e, the Huffman code + (63, 16, 11) BCH code).

The source follows Case 2 and hard-decision decoding is employed.

2 3 4 5 6 7 8 10^-5

10^-4 10^-3 10^-2 10^-1

SER

E

b

/N

0

(dB)

Huffman+BCH(n=63,k=16,t=11) JSC block code(length=64)

Figure 5.9: Performance comparison between the joint source-channel block code with code-word length 64 and the tandem scheme (i..e, the Huffman code + (63, 16, 11) BCH code).

The source follows Case 3 and hard-decision decoding is employed.

5.2 Modified Turbo Code

For all cases in Section 5.2, the number of iterations for turbo decoder is set as 18.

Below we illustrate the details of all figures in Section 5.2.

Below we illustrate the details of all figures in Section 5.2.

在文檔中 針對非均勻訊源之固定長度整合訊源與通道編碼系統之設計 (頁 19-78)