IEEE SIGNAL PROCESSING LETTERS, VOL. 16, NO. 8, AUGUST 2009 691
Stochastic Erasure-Only List Decoding
Algorithms for Reed-Solomon Codes
Chang-Ming Lee and Yu T. Su, Senior Member, IEEE
Abstract—We present a novel stochastic decoding algorithm for
Reed-Solomon codes. We apply an iterative Monte Carlo based ap-proach called the Cross-Entropy method to produce, in every itera-tion, a set of random error locator vectors, each indicates pos-sible erasure positions within a received word. We associate each error locator vector with a candidate codeword by erasures-only decoding the received word, using the error locator vector to lo-cate the erasures. Each iteration results in a new elite set that con-tains the best candidate codewords. To increase the search radius and enhance the decoder performance we use the randomly drawn samples to generate what we call virtual received words from which extra candidate codewords and thus candidate elite members can be obtained. The proposed algorithms offer both complexity and performance advantages over some existing algebraic decoding al-gorithms for high rate RS codes.
Index Terms—Cross-Entropy method, list decoding, Reed–Solomon code.
I. INTRODUCTION
T
HE class of Reed–Solomon (RS) codes [1] is a powerful error control code that has been used in a wide variety of applications, ranging from data storage systems to deep-space and wireless communications. These codes are usually de-coded by hard-decision decoding (HDD) algorithms using the Berlekamp-Massey (BM) algorithm [2] and the Euclidean algorithm [3].Many soft-decision decoding (SDD) algorithms that use re-liability information to assist HDD have been developed. Ear-lier proposals include Forney’s generalized minimum distance (GMD) decoding algorithm [5], the Chase II algorithm [6], and the combined Chase II-GMD algorithm [7]. These algorithms give a moderate performance improvement over HDD solutions with reasonable complexity. Guruswami and Sudan (GS) [4] in-vented an algebraic list decoding algorithm which corrects be-yond half the minimum distance. Koetter and Vardy (KV) [8] proposed an algebraic SDD algorithm based on a multiplicity assignment scheme to improve the GS algorithm. The KV algo-rithm can significantly outperform HDD for low rate RS codes. However, to achieve large coding gain, the complexity can be prohibitively large.
Manuscript received February 13, 2009; revised May 01, 2009. First pub-lished May 12, 2009; current version pubpub-lished June 09, 2009. This work was supported in part by Taiwan National Science Council under Grant 97-2221-E009-082-MY3. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Yimin Zhang.
The authors are with the Department of Communications Engi-neering, National Chiao Tung University, Hsinchu 30056, Taiwan (e-mail: [email protected], [email protected]).
Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/LSP.2009.2022792
In this letter, we apply the Cross-Entropy (CE) method [9] to develop a Monte Carlo based iterative SDD algorithm which renders an improved algebraic SDD decoding performance. The CE method is an elegant practical principle for simulating rare events which approximates the probability of the rare event by means of a family of parameterized probabilistic models. Our stochastic erasure-only list decoding (SEOLD) algorithm uses the extended CE method for optimization problem by consid-ering an optimal event as a rare event.
The rest of this letter is organized as follows. Some prelim-inaries are given in Section II. In Section III, the general sto-chastic SDD algorithm based on the CE method is introduced. One of the proposed stochastic decoding algorithms is presented in Section IV. Some simulation results and discussions are pre-sented in Section V.
II. PRELIMINARY
Let be an RS code over with
min-imum Hamming distance . Let
be a codeword in with the binary
expan-sion . Using binary phase-shift-keying
(BPSK), the transmitter maps the binary imaged codeword into the bipolar vector
(1) and sends it over an additive white Gaussian noise (AWGN) channel with zero mean and power spectral density . The received sequence at the output of the matched filter is
where and ’s are
statisti-cally independent Gaussian random variables with zero mean and variance .
Let be the hard decision binary vector of the received bit sequence , i.e.,
otherwise (2)
and be the corresponding symbol vector.
Denoted by the reliability vector of
in which is the magnitude of the log-likelihood ratio (LLR) associated with the corresponding hard-limited bit
(3) and define the symbol reliability vector of
by
(4)
692 IEEE SIGNAL PROCESSING LETTERS, VOL. 16, NO. 8, AUGUST 2009
III. STOCHASTICLISTDECODINGALGORITHM
A. Algebraic Erasures-Only (EO) Decoding
It is well-known that RS codes are maximum-distance sepa-rable (MDS) which implies that any coordinates (symbols) in an RS codeword can be used to determine the remaining symbols. Hence it is sufficient to decide correct (message) or
incorrect (error) coordinates of a codeword. Let be the collection of all combinations of error coordinates,
(5) where if the th coordinate is in error. Then a straight-forward decoding schedule is given as below:
(a) For all , erase the corresponding error co-ordinates of the received word and decode by the era-sures-only (EO) decoder. The resulting codeword set is denoted by .
(b) Choose the codeword from with the best score, e.g., the one whose Euclidean distance from the received word is the smallest, as the decoder output.
It can be easily confirmed that for any ,
, where is the Hamming distance between and . Therefore, the transmitted codeword belongs to if the number of error symbols is less than . Furthermore, (b) is equivalent to the following minimization problem
(6) where is defined by (1) and is the Euclidean dis-tance (ED) between the -ary real vectors and .
B. A Stochastic List Decoding Idea
Each error locator vector (ELV) represents a par-ticular set of possible error coordinates and has a corre-sponding codeword that belongs to . We denote the latter relationship by . Although more than one ELV may be associated with the same codeword, the complexity of searching for the optimal solution in the error location domain is still extremely high because the cardinality of is and only a few (or one) elements in , depending on the number and locations of the received errors, can be used to reconstruct
.
Suppose we model the selection of the ELV from as a stochastic (vector-valued) experiment governed by a family of parameterized distributions with being a real-valued parameter vector. Usually is assumed to be uniformly distributed due to the lack of priori information whence the search in (6) is exhaustive unless some algebraic properties of the code are used. One way to solve (6) efficiently
is to find a parameter such that where
. Then drawing one sample from is sufficient to obtain the optimal solution . To get around the difficulty that is not known, one notices that the optimization problem (6) is related to the estimation of the probability
, which is a rare event when .
The connection comes from the fact that efficient estimation of a rare event can be achieved by the method of importance sam-pling and in this case the optimal importance density is .
Fig. 1. Flow chart of a stochastic decoder for RS codes.
Without the knowledge of the threshold , we start with a proper importance density to generate samples of and compute an initial estimate for . Ideally, we can use those drawn samples which satisfy to obtain new parameter value such that is closest to in the Kullback-Leibler (KL) sense, i.e., the CE between
and is minimized. Since is unknown, we choose such that is closest to the empirical distribution of in those samples that are generated by and satisfy for this empirical distribution is likely to be a good approximation of . New samples of are then produced by the updated importance density to com-pute new estimate . This iterative procedure continues until
is less than a predetermined threshold.
The above method is known as the CE method [9] which is an iterative procedure that consists of the following two phases in each iteration.
• Generate samples from the specified importance density given by the parameters from the previous iteration. • Update the parameters for next iteration according to the
order of the score values associated with the drawn samples and the minimizing CE criterion.
Based on the above discussion, we propose a generic Monte Carlo based SDD algorithm as shown in Fig. 1 and in Table I with some detailed description given in Section IV.
C. Convergence and Complexity
Different convergence conditions and results have been dis-cussed for the deterministic CE method and its extensions in [10] where it is also proved that convergence in distribution or can be guaranteed but needs a proper tuning of the parame-ters of the algorithm such as the number of samples , number of elites , and smoothing factor . Convergence to the global minimum is ensured only if a large sample size is used. On the other hand, the computing complexity is related to and is given by . It is obvious that the decoding perfor-mance can be improved by using a larger . As we retain the the best sample at the end of each iteration, the decoding perfor-mance is also improved by increasing the iteration number . As an early-stopping at any iteration produces a decoded coword, we say the algorithm converges if the sequence of de-coded codewords converges. With a modest , we found that the decoded codewords converge to the same codeword within
LEE AND SU: STOCHASTIC ERASURE-ONLY LIST DECODING ALGORITHMS 693
TABLE I
A STOCHASTICLISTDECODINGALGORITHM
Fig. 2. Virtual received words are generated around the received LLR vector 0 by hard-limiting the sample vectors generated by an importance probability density whose parameter values evolved according to the CE principle.
ten iterations in most cases. Our algorithm yields good perfor-mance although uniform convergence in distribution or within a limited iterations is not guaranteed.
IV. LISTDECODINGWITHVIRTUALRECEIVEDWORDS
In Step 2 of Table I we try to find the most likely message/ error coordinates such that the associated EO-decoded code-word is closest to the received vector. Note that the random samples are used to determine the erasure locations only, and the searching sphere of the algebraic list decoding described in Sec-tion III-A is always centered at the hard-limited received word with radius equals to . To increase our search range and im-prove decoding performance, we include some extra codewords which lie statistically in a small neighborhood of the received word in our expanded search, such that some of them may in fact be closer in ED to the true transmitted codeword ; see Fig. 2. More specifically, the expansion is accomplished by eliminating the requirement that the search be centered at . Instead, we ran-domized the search center by EO-decoding the hard-decision versions of the drawn sample vectors which we refer to as vir-tual received words. If the importance density does converge to the desired density , such an expanded search will even-tually contract and converge to the true transmitted codeword.
A. Importance Density and Sample Format
Let be a random vector where
are independent Gaussian random variables
with means and variances .
We write , where and
are initialized by
(7) (8) At the th iteration, random samples are drawn from to form the sample set . Each sample vector represents the bit reliabilities of an associated virtual received word. By using (4) to convert the bit reliabil-ities into symbol reliability, the coordinates with smallest symbol reliabilities are erased; the remaining bit positions are hard-limited, mapped into symbol decisions and the resulting virtual received word is then decoded by an EO decoder.
B. Update Parameters
Let be the output codewords of the EO decoder. We compute the ED between each candidate codeword and the received word and sort the corresponding random vectors ac-cording to the descending order of their associated EDs. Define an elite set which includes vectors with the smallest EDs to , i.e., the corresponding codewords are more likely to have been transmitted. We always store the best one in up to the current iteration for the final decision when the maximum number of iteration is reached.
Then the two sets of parameters and are updated by [9]
(9) and
694 IEEE SIGNAL PROCESSING LETTERS, VOL. 16, NO. 8, AUGUST 2009
Fig. 3. Codeword error probability performance of the (15,11) Reed-Solomon code; ten iterations.
where and are real values between (0,1) used to smooth the variation of these parameters. The algorithm described in this section is called the stochastic erasures-only listing decoding (SEOLD) algorithm.
V. SIMULATIONRESULTS ANDDISCUSSION
In this section, some simulated performance of the SEOLD algorithm is presented and compared with that of other well known RS decoding algorithms. A standard binary input AWGN channel is assumed over which the BPSK modulated codewords are transmitted. We model the receive matched filter output as the sum of a - valued sequence and Gaussian sequence with zero-mean i.i.d. components. The average performance bounds on the ML error probability of RS codes over an AWGN channel developed in [11] are used as the performance lower limits.
Due to the complexity and the decoding delay considerations, the SEOLD algorithm will not terminate until convergence is assured. Instead, we limit our decoding procedure to iterations in all simulations.
Figs. 3 and 4 show respectively the codeword error rate (CER) performance of the (15,11) and (31,25) RS codes over an AWGN channel. HDD-BM refers to the performance of a hard decision bounded minimum distance decoder such as the BM algorithm. Curves labelled by GMD and KV are the performance obtained by using Forney’s GMD algorithm and the algebraic soft decision decoding algorithm of Koetter and Vardy, respectively. Note that the performance of the KV algorithm shown here represents the lower bound obtained by using an infinite interpolation multiplicity. For the case of (15,11) RS codes, the size of the sample set and the size of the elite set at every iteration are set to be 20 and 6, respectively. With iterations, the SEOLD algorithm obtains 1.2 dB and 1.0 dB coding gains with respect to the GMD and the KV algorithms at CER . For the case of (31,25) RS codes, the SEOLD algorithm also outperform the
Fig. 4. Codeword error probability performance of the (31,25) Reed-Solomon code; ten iterations.
other two algorithms with reasonable complexity of ,
and , in 10 iterations. The SEOLD
algorithm has about 0.6 dB and 1.0 dB coding gains over the KV algorithm when is equal to 100 and 500, respectively. In conclusion, the proposed decoding algorithm is capable of offering good performance with modest complexity for short high rate RS codes. Its performance can be further improved by increasing the sample size and/or the maximum iteration number at the cost of increased decoding complexity.
REFERENCES
[1] I. S. Reed and G. Solomon, “Polynomial codes over certain finite fields,” J. Soc. Ind. Appl. Math, vol. 8, pp. 300–304, 1960.
[2] E. B. Berlekamp, Algebraic Coding Theory. New York: McGraw-Hill, 1968.
[3] Y. Sugiyama, M. kasahara, S. Hirasawa, and T. Namekawa, “A method for solving key equation for decoding goppa codes,” Inform. Contr., vol. 27, pp. 87–99, 1975.
[4] V. Guruswami and M. Sudan, “Decoding of Reed-Solomon codes be-yond the error-correction bound,” J. Complexity, vol. 13, pp. 180–193, 1997.
[5] G. D. Forney, Jr, “Generalized minimum distance decoding,” IEEE
Trans. Inform. Theory, vol. IT-12, pp. 125–131, Apr. 1966.
[6] D. Chase, “A class of algorithms for decoding block codes with channel measurement information,” IEEE Trans. Inform. Theory, vol. IT-18, pp. 170–182, Jan. 1972.
[7] H. Tang, Y. Liu, M. Fosorier, and S. Lin, “On combining chase-2 and GMD decoding algorithms for nonbinary block codes,” IEEE Commun.
Lett., vol. 5, pp. 209–211, May 2001.
[8] R. Köetter and A. Vardy, “Algebraic soft-decision decoding of Reed-Solomon codes,” IEEE Trans. Inform. Theory, vol. 49, pp. 2809–2825, Nov. 2003.
[9] R. Rubinstein and D. P. Kroese, The Cross-Entropy Method. A
Uni-fied Approach to Combinatorial Optimization, Monte-Carlo Simula-tion, and Machine Learning. Berlin, Germany: Springer, Information Science and Statistics, , 2004.
[10] F. Dambreville, Cross-Entropy Method: Convergence Issues for Extended Implementation [Online]. Available: http://www.Frederic-Dambreville.com
[11] M. EI-Khamy and R. J. McEliece, “Bounds on the average binary minimum distance and the maximum likelihood performance of Reed-Solomon codes,” in 42nd Allerton Conf. Communication,
