先進錯誤控制技術及其應用之研究

(1)

先進錯誤控制技術及其應用之研究 (1/3)

Advanced Error-Control Coding Technologies and Their Applications

(1/3)

計畫編號：NSC 97-2221-E-009-082-MY3 執行期間：97 年 8 月 1 日至 98 年 7 月 31 日主持人：蘇育德教授國立交通大學電機工程學系計畫參與人員：李昌明、黃嘉偉、林坤昌、劉人仰、吳家麟、董原豪中文摘要我們針對里德所羅門碼提出了兩種新穎的隨機解碼演算法。這兩種解碼法皆應用了被稱之為交錯熵方法(the Cross Entropy Method)的遞迴式蒙地卡羅法來產生一個隨機錯誤標

示向量的集合。此集合中的每一個向量代表著接收字元串列中

n-k

個可能錯誤的位置。在每一次的迴圈中，我們先利用擦拭解碼器與接收字元串列來將這些新產生的錯誤標示向量轉變為可能的傳送碼。接著從其中選出較佳的

E

個候選者來當作下一迴圈產生新隨機錯誤標示向量的依據。針對高碼率的里德所羅門碼，以此概念設計的演算法相對於部分現有的演算法提供了性能與複雜度上的改善。關鍵字：里德所羅門碼、交錯熵方法。 Abstract

We present two novel stochastic decoding algorithms for Reed-Solomon (RS) codes. We apply an iterative Monte Carlo based the approach called the Cross-Entropy (CE) method to produce, in every iteration, a set of random error locator vectors with each indicates a set of n-k possible erasure positions within a received word. We associate each error locator vector with a candidate codeword by erasures-only-decoding the received word using the error locator vector to determine the erasures. Each iteration results in a new elite set that contains the best E candidate codewords in terms of Euclidean distance. To increase the search radius and enhance the decoder performance we use a set of randomly drawn samples, whose probabilistic distribution is determined by the elite set, to generate what we call virtual received words from which extra candidate codewords and thus candidate elite members can be obtained. The proposed algorithms offer both complexity and performance advantages over some existing algebraic decoding algorithms for high rate RS codes.

(2)

I. INTRODUCTION

The class of Reed-Solomon (RS) codes [1] is a powerful error control code that has been used in a wide variety of applications, ranging from data storage systems to deep-space and wireless communications. These codes are usually decoded by hard-decision decoding (HDD) algorithms using the Berlekamp-Massey (BM) algorithm [2] and the Euclidean algorithm [3].

Many soft-decision decoding (SDD) algorithms that use reliability information to assist HDD have been developed. Earlier proposals include Forney’s generalized minimum distance (GMD) decoding algorithm [5], the Chase II algorithm [6], and the combined Chase II-GMD algorithm [7]. These algorithms give a moderate performance improvement over HDD solutions with reasonable complexity. Guruswami and Sudan (GS) [4] invented an algebraic list decoding algorithm which corrects beyond half the minimum distance. Koetter and Vardy (KV) [8] proposed an algebraic SDD algorithm based on a multiplicity assignment scheme to improve the GS algorithm. The KV algorithm can significantly outperform HDD for low rate RS codes. However, to achieve large coding gain, the complexity can be prohibitively large.

In this report, we apply the Cross-Entropy (CE) method [9] to develop a Monte Carlo based iterative SDD algorithm which renders an improved algebraic SDD decoding performance. The CE method is an elegant practical principle for simulating rare events which approximates the probability of the rare event by means of a family of parameterized probabilistic models. Our stochastic erasure-only list decoding (SEOLD) algorithm uses the extended CE method for optimization problem by considering an optimal event as a rare event.

The rest of this report is organized as follows. Some preliminaries are given in Section II. In Section III, the general stochastic SDD algorithm based on the CE method is introduced. Two of the proposed stochastic decoding algorithms are presented in Section IV and Section V, respectively. Some simulation results and discussions are presented in Section VI.

II. PRELIMINARY

Let C be an (n, k) RS code over GF(2m_{) with minimum Hamming distance d}

min = n − k + 1.

Let c = (c0, · · · , cn−1) be a codeword in C with the binary expansion ¯c = (¯c0, ¯c1, · · · , ¯cnm−1).

Using binary phase-shift-keying (BPSK), the transmitter maps the binary imaged codeword ¯c into the bipolar vector

(3)

and sends it over an additive white Gaussian noise (AWGN) channel with zero mean and power spectral density N0/2. The received sequence at the output of the matched filter is

¯

y= (¯y0, · · · , ¯ynm−1)where ¯yj = ¯xj+ ¯wj and ¯wj’s are statistically independent Gaussian random

variables with zero mean and variance N0/2.

Let ¯z = (¯z0, · · · , ¯znm−1) be the hard decision binary vector of the received bit sequence ¯y,

i.e., ¯ zj =      0, ¯yj > 0 1, otherwise (2)

and z = (z0, · · · , zn−1)be the corresponding symbol vector. Denoted by ¯Γ = (¯γ1, · · · , ¯γnm−1)the

reliability vector of ¯y in which ¯γj is the magnitude of the log-likelihood ratio (LLR) associated

with the corresponding hard-limited bit ¯zj

L (¯cj) = log

P ( ¯cj = 0| ¯y)

P ( ¯cj = 1| ¯y)

, (3)

and define the symbol reliability vector Γ = (γ0, · · · , γn−1) of z by

γi = min

j γ¯j, j ∈ {im, · · · , (i + 1)m − 1} (4)

Assume that the ith symbol ci of c is uniformly distributed over GF(2m) and the n received

symbols are independent and uniformly drawn from GF(2m_{). Then P (c}

i = β|¯y), the probability

that ci = βwas transmitted given the observation ¯y can be easily evaluated [8]. The q ×n matrix

R = [Rβi = P (ci = β|¯y)], q = 2m, will be referred to as the reliability matrix of the received

vector ¯y.

III. STOCHASTICLIST DECODINGALGORITHM A. Algebraic Erasures-Only (EO) Decoding

It is well-known that RS codes are maximum-distance separable (MDS) which implies that any k coordinates (symbols) in an RS codeword can be used to determine the remaining n − k symbols. Hence it is sufficient to decide k correct (message) or n−k incorrect (error) coordinates of a codeword. Let EL be the collection of all combinations of n − k error coordinates,

EL= ( s= (s0, · · · , sn−1) si ∈ {0, 1}, X i si = n − k ) (5)

(4)

where si = 1 if the ith coordinate is in error. Then a straightforward decoding schedule is given

as below:

(a). For all s ∈ EL, erase the corresponding n − k error coordinates of the received word z and

decode by the erasures-only (EO) decoder. The resulting codeword set is denoted by Cz.

(b). Choose the codeword from Cz with the best score, e.g., the one whose Euclidean distance

from the received word is the smallest, as the decoder output.

It can be easily confirmed that for any c ∈ Cz, dH(c, z) ≤ n − k, where dH(c, z) is the

Hamming distance between c and z. Therefore, the transmitted codeword belongs to Cz if

the number of error symbols is less than dmin. Furthermore, (b) is equivalent to the following

minimization problem

arg min

c d (Ψ(¯c), ¯y)subject to c ∈ Cz (6)

where Ψ(·) is defined by (1) and d(¯a, ¯b) is the Euclidean distance (ED) between the nm-ary real vectors ¯a and ¯b.

B. A Stochastic List Decoding Idea

Each error locator vector (ELV) s ∈ EL represents a particular set of n − k possible error

coordinates and has a corresponding codeword cs that belongs to Cz. We denote the latter

relationship by s ∼ cs. Although more than one ELV may be associated with the same codeword,

the complexity of searching for the optimal solution c∗ in the error location domain E

L is still

extremely high because the cardinality of EL is

_n

k

and only a few (or one) elements in EL,

depending on the number and locations of the received errors, can be used to reconstruct c∗.

Suppose we model the selection of the ELV s from EL as a stochastic (vector-valued)

experiment governed by a family of parameterized distributions {f(s; u)} with u ∈ ν being a real-valued parameter vector. Usually f(s; u) is assumed to be uniformly distributed due to the lack of priori information whence the search in (6) is exhaustive unless some algebraic properties of the code are used. One way to solve (6) efficiently is to find a parameter v∗

such that f(s; v∗

) = δ(s − s∗

) where s∗ _{∼ c}∗. Then drawing one sample from f(s; v∗

) is sufficient to obtain the optimal solution c∗. To get around the difficulty that c∗ is not known,

one notices that the optimization problem (6) is related to the estimation of the probability P (d(Ψ(¯cs), ¯y) ≤ η|s ∼ cs), which is a rare event when η = η∗ = d(Ψ(¯c∗), ¯y). The connection

(5)

TABLE I

A STOCHASTICLISTDECODINGALGORITHM.

1. Define a family of probability densities {f(·; v), v ∈ ν} on the search space Rnm_{. Initialize v}(0)_{. Set t = 1.}

2. Generate a sample set S(t) _{whose N random vector samples are drawn from f(·; v}(t)_{). Regard the magnitudes as}

bit LLRs and convert them into symbol reliabilities. Erase the n − k least reliable symbols and decode the received word by EO decoding.

3. Evaluate Euclidean distances between the decoded codewords and the received word. Select the E vector samples with best metrics as the new elite set SE(t)⊂ S(t) and store the best decoded codeword d∗(t)in D(t).

4. Evaluate the new parameter v(t+1) _{by solving v}(t+1)_{= arg max} v 1 |S_E(t)_| P s(t) ` ∈SE(t)ln f (s (t) ` ; v)[9].

Update v(t+1)_{via v}(t+1)_{= ρv}(t+1)_{+ (1 − ρ)v}(t)_{where 0 < ρ < 1.}

5. Terminate decoding if the stopping criterion is met. Choose the best codeword from the list

d∗(t); ∀ t

, say ˆc∗_,

as the decoder output. Otherwise increase t by 1 and return to step 2.

comes from the fact that efficient estimation of a rare event can be achieved by the method of importance sampling and in this case the optimal importance density is f(s; v∗). Without the

knowledge of the threshold η∗, we start with a proper importance density f(s; ˆv) to generate

samples of s and compute an initial estimate ˆη for η. Ideally, we can use those drawn samples which satisfy d(Ψ(¯cs), ¯y) ≤ ˆη to obtain new parameter value ˆv0 such that f(s; ˆv0) is closest

to f(s; v∗

) in the Kullback-Leibler (KL) sense, i.e., the CE between f(s; ˆv0

) and f(s; v∗

) is minimized. Since v∗ is unknown, we choose ˆv0 such that f(s; ˆv0

) is closest to the empirical distribution of s in those samples that are generated by f(s; ˆv) and satisfy d(Ψ(¯cs), ¯y) ≤ ˆη for

this empirical distribution is likely to be a good approximation of f(s; v∗). New samples of s

are then produced by the updated importance density f(s; ˆv0

)to compute new estimate ˆη0. This

iterative procedure continues until |ˆη − ˆη0_| is less than a predetermined threshold.

The above method is known as the CE method [9] which is an iterative procedure that consists of the following two phases in each iteration.

• Generate samples from the specified importance density given by the parameters from the

previous iteration.

• Update the parameters for next iteration according to the order of the score values associated

with the drawn samples and the minimizing CE criterion.

Based on the above discussion, we propose a generic Monte Carlo based SDD algorithm as shown in Fig. 1 and in Table I with some detailed description given in Section IV.

(6)

Sample

Generator

Erasure-Only

Decoder

Sample

Evaluator

Parameter

Updator

S

(t)

v

(0)

v

(t+1)

S

_E(t)

D

(t)

Store &

Choose the

Best

d*

(t) *

ˆc

S

(t)

{f( ;v)}

Fig. 1. Flow chart of a stochastic decoder for RS codes.

C. Convergence and Complexity

Different convergence conditions and results have been discussed for the deterministic CE method and its extensions in [11] where it is also proved that convergence in distribution or η can be guaranteed but needs a proper tuning of the parameters of the algorithm such as the number of samples N, number of elites E, and smoothing factor ρ. Convergence to the global minimum is ensured only if a large sample size N is used. On the other hand, the computing complexity is related to N and is given by O(N(n − k)2_{). It is obvious that the decoding}

performance can be improved by using a larger N. As we retain the the best sample at the end of each iteration, the decoding performance is also improved by increasing the iteration number T. As an early-stopping at any iteration produces a decoded codeword, we say the algorithm converges if the sequence of decoded codewords converges. With a modest N, we found that the decoded codewords converge to the same codeword within 10 iterations in most cases. Our algorithm yields good performance although uniform convergence in distribution or η within a

(7)

limited iterations is not guaranteed.

IV. LIST DECODING BASED ONERASURE LOCATION ESTIMATION

In this section, we propose the first kind SEOLD by efficiently estimating the most possible dmin− 1 locations of erasures about the received word z, i.e., finding the solution of (6).

A. Importance Density and Sample Format

Suppose the reliability matrix R is known at the receiver. Define the distrust function, fd :

GF(2m_{) 7→ (0, ∞), of the ith coordinate z}

i of z as fd(zi) = P βRβ,i Rzi,i , β ∈GF(2m) \ {zi} (7)

The larger the value of fd(zi)is, the higher the probability that zi should be erased at the decoder.

Let s = (s0, · · · , sn−1) be a random vector where s0, · · · , sn−1 are independent Bernoulli

random variables with success probabilities p0, · · · , pn−1, i.e., P (si = 1) = 1 − P (si = 0) = pi.

We write s ∼ Ber(p), where p = (p0, · · · , pn−1). In the ELEA, si = 1 represents the ith symbol

zi of z should be erased. On the other hand, zi will be reserved because of higher reliability

when si = 0.

For the ELEA, the initial parameters p(0) ₌

p(0)₀ , · · · , p(0)_n−1 are defined as p(0)i =      1 − , fd(zi) > 1 − fd(zi), otherwise (8) where is an arbitrary real value between (0,1).

At the tth iteration, let s(t) 1 , s

(t)

2 , · · · , s (t)

N be N trials drawn from Ber

p(t) _{which satisfy} n−1 X j=0 s(t)_`,j = dmin− 1, ∀` ∈ {1, · · · , N } (9) where s(t) ` = (s (t) `,0, · · · , s (t)

`,n−1). We then collect samples from these N trials to form a sample

set S(t) _{= {s}(t)

1 , · · · , s (t) N}.

(8)

B. Update Parameters

Let d(t)

1 , · · · , d (t)

N be the output codewords of the EO decoder. We compute the ED between

each candidate codeword and the received word y and sort the corresponding random vectors according to the descending order of their associated EDs. Define the elite set SE(t) at the tth

iteration to be the E vectors with the smallest EDs to y, i.e., the corresponding codewords are more likely to have been transmitted. We always store the best one in SE(t) up to the current

iteration for the final decision when the maximum number of iteration is reached. Suppose the parameters used to generate samples at the tth iteration are p(t) ₌

p(t)0 , · · · , p (t) n−1

. The parameters for the next iteration are updated by considering the information provided by both p(t) _{and S}

E(t). More precisely, the ith parameter p(t+1)_i is obtained by [9]

p(t+1)i = (1 − ρ)p (t) i + ρ · P `∈SE(t)s (t) `,i E (10)

where ρ is a smoothing factor with real value between (0, 1).

V. LIST DECODING WITHVIRTUALRECEIVEDWORDS

In Step 2 of Table I we try to find the most likely message/error coordinates such that the associated EO-decoded codeword is closest to the received vector. Note that the random samples are used to determine the erasure locations only, and the searching sphere of the algebraic list decoding described in Section III.A is always centered at the hard-limited received word z with radius equals to n − k. To increase our search range and improve decoding performance, we include some extra codewords which lie statistically in a small neighborhood of the received word in our expanded search, such that some of them may in fact be closer in ED to the true transmitted codeword c; see Fig. 2. More specifically, the expansion is accomplished by eliminating the requirement that the search be centered at z. Instead, we randomized the search center by EO-decoding the hard-decision versions of the drawn sample vectors which we refer to as virtual received words. If the importance density does converge to the desired density δ(s − s∗), such an expanded search will eventually contract and converge to the true transmitted

codeword.

A. Importance Density and Sample Format

Let ¯s = (¯s0, · · · , ¯snm−1) be a random vector where ¯s0, · · · , ¯snm−1 are independent

(9)

c

z

₁

z

₂

z

₃

z

_{N -1}

z

_N

n-k

Fig. 2. Virtual received words are generated around the received LLR vector ¯Γ by hard-limiting the sample vectors generated by an importance probability density whose parameter values evolved according to the CE principle.

N (~µ, ~σ), where ~µ = (µ0, · · · , µnm−1) and ~σ = (σ0, · · · , σnm−1) are initialized by

µ(0)j = ¯γj (11)

σj(0) =

q

|¯γj| (12)

At the tth iteration, N random samples ¯s(t) 1 , ¯s

(t)

2 , · · · , ¯s (t)

N are drawn from N

~µ(t)_{, ~σ}(t)

to form the sample set ¯S(t)_{. Each sample vector represents the bit reliabilities of an associated virtual}

received word. By using (4) to convert the bit reliabilities into symbol reliability, the n − k coordinates with smallest symbol reliabilities are erased; the remaining bit positions are hard-limited, mapped into symbol decisions and the resulting virtual received word is then decoded by an EO decoder.

B. Update Parameters

Let d(t)

1 , · · · , d (t)

N be the output codewords of the EO decoder. We compute the ED between

each candidate codeword and the received word y and sort the corresponding random vectors according to the descending order of their associated EDs. Define an elite set ¯S(t)

E which includes

(10)

been transmitted. We always store the best one in ¯S(t)

E up to the current iteration for the final

decision when the maximum number of iteration is reached.

Then the two sets of parameters ~µ(t+1) _{and ~σ}(t+1) _{are updated by [9]}

µ(t+1)j = (1 − λ)µ (t) j + λ · P ¯s(t) ` ∈¯S (t) E s¯ (t) `,j E (13) and σj(t+1)= (1 − η)σ (t) j + η · P ¯s(t) ` ∈¯S (t) E ¯ s(t)`,j− µ (t+1) j 2 E (14)

where λ and η are real values between (0, 1) used to smooth the variation of these parameters. The algorithm described in this section is called the stochastic erasures-only listing decoding (SEOLD) algorithm.

VI. SIMULATIONRESULTS ANDDISCUSSION

In this section, some simulated performance of two SEOLD algorithms (I and SEOLD-II) are presented and compared with that of other well known RS decoding algorithms. A standard binary input AWGN channel is assumed over which the BPSK modulated codewords are transmitted. We model the receive matched filter output as the sum of a ±1−valued sequence and Gaussian sequence with zero-mean i.i.d. components. The average performance bounds on the ML error probability of RS codes over an AWGN channel developed in [12] are used as the performance lower limits.

Due to the complexity and the decoding delay considerations, the SEOLD algorithms will not terminate until convergence is assured. Instead, we limit our decoding procedure to T iterations in all simulations.

Fig. 3 shows the codeword error rate (CER) performance of the (15,11) RS code over an AWGN channel. HDD-BM refers to the performance of a hard decision bounded minimum distance decoder such as the BM algorithm. GMD and KV refer to the performance of the GMD algorithm proposed by Forney and the algebraic soft decision decoding algorithm proposed by Koetter and Vardy, respectively. Note that the KV algorithm concerned here is infinite interpolation costs, i.e., the complexity is also infinite. For both SEOLD-I and SEOLD-II, the size of the sample set N and the size of the elite set E at every iteration are set to be 20 and 6, respectively. After 10 iterations, SEOLD-I has about 0.5 dB and 0.3 dB coding gain over

(11)

1 2 3 4 5 6 7 8 10-6 1x10-5 1x10-4 10-3 10-2 10-1 100

Codeword Error Rate

E_b/N₀ (dB) HDD-BM GMD KV SEOLD-I ( N =20) SEOLD-II ( N =20) ML

Fig. 3. Codeword error probability performance of the (15,11) Reed-Solomon code; 10 iterations.

GMD and KV at a CER of 10−5_{, respectively. On the other hand, SEOLD-II outperforms all}

the previous algorithms with a performance gain of about 1.2 dB and 1.0 dB over GMD and KV at a CER of 10−5_.

In Fig. 4, SEOLD-I has a near KV performance when the N = 100 and E = 10 after 10 iterations. At the same condition, SEOLD-II still outperforms the other algorithms with reasonable complexity. The SEOLD-II has about 0.6 dB and 1.0 dB coding gain over KV where N is equal to 100 and 500, respectively. In conclusion, the proposed decoding algorithms are capable of offering good performance with modest complexity for short high rate RS codes. Its performance can be further improved by increasing the sample size N and/or the maximum iteration number T at the cost of increased decoding complexity.

REFERENCES

[1] I. S. Reed and G. Solomon, “Polynomial codes over certain finite fields,” J. Soc. Ind. Appl. Math, vol. 8, pp. 300-304, 1960. [2] E. B. Berlekamp, Algebraic Coding Theory, New York: McGraw-Hill, 1968.

[3] Y. Sugiyama, M. kasahara, S.Hirasawa, and T. Namekawa, “A method for solving key equation for decoding goppa codes,” Inform. contr., vol. 27, pp. 87-99, 1975.

[4] V. Guruswami and M. Sudan, “Decoding of Reed-Solomon codes beyond the error-correction bound,” J. complexity, vol. 13, pp. 180-193, 1997.

(12)

1 2 3 4 5 6 7 8 10-6 1x10-5 1x10-4 10-3 10-2 10-1 100

Codeword Error Rate

E_b/N₀ (dB) HDD-BM GMD KV SEOLD-I ( N =100) SEOLD-II ( N =100) SEOLD-II ( N =500) ML

Fig. 4. Codeword error probability performance of the (31,25) Reed-Solomon code; 10 iterations.

[5] G. D. Forney Jr., “Generalized minimum distance decoding,” IEEE Trans. Inform. Theory, vol. IT-12, pp.125-131, Apr. 1966.

[6] D. Chase, “A class of algorithms for decoding block codes with channel measurement information,” IEEE Trans. Inform. Theory, vol. IT-18, pp. 170-182, Jan. 1972.

[7] H. Tang, Y. Liu, M. Fosorier, and S. Lin, “On combining chase-2 and GMD decoding algorithms for nonbinary block codes,” IEEE Commun. Lett., vol. 5, no. 5, pp. 209-211, May 2001.

[8] R. K¨oetter and A. Vardy, “Algebraic soft-decision decoding of Reed-Solomon codes,” IEEE Trans. Inform. Theory, vol. 49, no. 11, pp. 2809-2825, Nov. 2003.

[9] R. Rubinstein and D. P. Kroese, The Cross-Entropy method. A unified approach to combinatorial optimization, Monte-Carlo simulation, and machine learning, Information Science and Statistics, Springer 2004.

[10] V. Guruswami and A. Vardy, “Maximum-likelihood decoding of Reed-Solomon codes is NP-hard,” IEEE Trans. Inform. Theory, vol. 51, no. 7, pp. 2249-2256,July 2005.

[11] F. Dambreville, “Cross-Entropy method: convergence issues for extended implementation,” http://www.FredericDambreville.com

[12] M. EI-Khamy and R. J. McEliece, “Bounds on the average binary minimum distance and the maximum likelihood performance of Reed-Solomon codes,” in 42nd Allerton Conf. on Communication, Control, and Computing, 2004.