Tsun-chih Hsueh and Da-shan Shiu Graduate Institute of Communication Engineering
Department of Electrical Engineering National Taiwan University
Taipei, Taiwan
Email: [email protected], [email protected]
Abstract— Although yielding the lowest error probability, ML decoding of turbo codes has been considered unrealistic so far because efficient ML decoders have not been discovered. In this paper, we propose the Genetic Decoding Algorithm (GDA) for turbo codes. GDA combines the principles of perturbed decoding and genetic algorithm. In GDA, chromosomes are random addi-tive perturbation noises. A conventional turbo decoder is used to assign fitness values to the chromosomes in the population. After generations of evolution, good chromosomes that correspond to decoded codewords of very good likelihood emerge. GDA can be used as a practical decoder for turbo codes in certain contexts.
It is also a natural multiple-output decoder. The most important aspect of GDA, in our opinion, is that one can utilize GDA to empirically determine a lower bound on the error probability with ML decoding. Our results show that, at a word error probability of 10−4, GDA achieves the performance of ML decoding. Using GDA, we establish that an ML decoder only slightly outperforms a MAP-based iterative decoder at this word error probability for the block size we used and the turbo code defined for WCDMA.
I. INTRODUCTION
Since its invention [1], turbo codes have received a lot of interest. Turbo codes are known to achieve a performance very close to the channel capacity. Due to this strong performance and manageable decoder complexity, turbo codes have been adopted as options for forward error correction in many recent communications standards such as 3GPP and 3GPP2. Over the years, researchers have answered questions such as error floor and interleaver optimization.
Practical decoders for turbo codes are iterative decoders based on message-passing principles [2]. These decoders are suboptimal from a word error probability perspective.
Although yielding the lowest word error probability, ML decoding of turbo codes has been considered unrealistic so far because efficient ML decoders have not been discovered.
In the past several years, performance of ML decoding of turbo codes has drawn more attention. In [3] and [4], the union bound on the average error probability over the ensemble of all
[7], a decoding algorithm based on two component list Viterbi decoders was introduced. It could only yield ML decoding of turbo codes for an infinite list size.
In [8], the ”perturbed decoding algorithm” (PA) was pro-posed for a concatenated coded system consisting of an inner error correcting code and an outer error detecting code. In PA, multiple highly likely candidate codewords are obtained by feeding the inner decoder with slightly perturbed versions of the actual received signal. The candidate codewords are sub-sequently validated by an outer decoder. Among the validated codewords, the one with the highest likelihood is declared the final decoder result. As we will demonstrate in this paper, one can adapt the perturbed decoding concept for use with turbo codes. Unfortunately, due to the large code block size used by turbo codes, the direct application of PA to turbo codes so can require an enormous complexity.
Genetic algorithms (GA) constitute a class of optimization techniques that are based on the general laws of natural selec-tion and genetics [9], [10]. It has been applied to numerous intractable optimization problems. In this paper, we present the novel Genetic Decoding Algorithm (GDA) which combines the principles of GA and PA. In GDA, one maintains a pool of additive perturbation noises as the chromosomes. A conven-tional decoder is used to give scores to these chromosomes.
Specifically, one first adds a chromosome to the received signal. The perturbed signal is passed through a conventional decoder to generate a corresponding decoded codeword. The likelihood of the codeword is assigned as the score of the corresponding chromosome. Chromosomes of low scores are eliminated from the population. After several generations of evolution, good chromosomes that correspond to codewords of very good likelihoods emerge.
Although GDA is trivially guaranteed to produce decoded sequences of higher likelihood than a conventional turbo de-coder, at its current form the amount of computation required for GDA is still too large for most practical applications. The critical importance of GDA, in our opinion, lies in the fact that
2
Fig. 1. Conventional iterative decoding algorithm for turbo codes.
that GDA can outperform conventional MAP decoder by a gain of 0.05 to 0.3 dB.
The rest of this paper is organized as follows. In Sec-tion II, we briefly review the convenSec-tional iterative decoding algorithms for turbo codes and the fundamental concepts of GA and PA. In Section III, we present in detail the concept and procedure of GDA for turbo codes. In Section IV, our simulation results of ML decoding bounds for turbo codes produced by GDA are presented. The parameters and procedures for GDA are also discussed. Concluding remarks are given in Section V.
II. BACKGROUND
A. Maximal-a priori (MAP) Iterative Decoding Algorithm In this paper, by “turbo codes” we specifically mean parallel concatenated convolutional codes (PCCC). A PCCC is con-sisted of a parallel concatenation of two constituent recursive convolutional codes and a specific interleaver. For example, a rate 1/3 PCCC employed by WCDMA [11] is consisted of two identical constituent encoders and an interleaver. The two encoders share the same generator polynomial, which is given by
g(D) = [ 1 1 + D + D3
1 + D2+ D3 ]. (1) These two constituent encoders are separated by a prunable prime interleaver [12].
As illustrated in Figure 1, a conventional MAP iterative decoder is comprised of two constituent decoders linked by the interleaver. These two constituent decoders pass extrinsic information back and forth. Decoder output is obtained after several iterations. Although many other forms of decoders exist, in this paper we only consider the MAP decoder due to its excellent performance.
B. Genetic Algorithm
To apply GA to an optimization problem, the first and sometimes the most critical step is to formulate a good genetic representation. Once a genetic representation is chosen, a
˦̇˴̅̇
Fig. 2. Flowchart of GA.
selection step is a process that selects members from a pop-ulation into the mating pool. The genetic operation includes both crossover and mutation. Crossover combines the subparts of the chromosomes from two parents to produce offsprings.
Mutation, on the other hand, introduces novel variations into the chromosomes. Replacement is the step that eliminates weak members in order to remain a population of high fitness.
As mentioned in [9], [10], a typical GA procedure can be described as follows:
1) Initially, a population is generated randomly.
2) Evaluate the fitness values for each individuals of the population by calculating some objective function.
3) Select from the population some parents to produce the next generation of population (offsprings) by the genetic operations.
4) Evaluate the fitness values of offsprings in a similar fashion to their parents.
5) According to a chosen replacement strategy, some par-ents and some offsprings survive. The survivors form the population of the subsequent generation.
6) Check whether the desired termination criterion is reached or not. If not, return to step 3 and repeat the process. Otherwise, stop and produce the final solution.
We can refer to the flowchart in Figure 2 for the detailed course of the execution of GDA. Hereafter, we will tailor the objective function of the fitness value evaluation block and the
3
C. Perturbed Decoding Algorithm
Perturbed decoding algorithm (PA) is a decoding algorithm for concatenated error correction and error detection codes. In PA, the original received signal y is first decoded by the inner decoder. The decoded result is subsequently validated by an outer decoder. If the outer decoder regards the decoded result as invalid, an additive perturbation noise will be randomly generated. By adding the noise to y, one creates a perturbed received signal. This perturbed signal is decoded by the inner decoder. The decoder output, which may be different from the initial decoder output, is again validated by the outer decoder. This process is repeated until either a valid codeword is declared by the outer decoder or when some termination criterion is met. It is a practical strategy to produce multiple decoder outputs with a simple decoder.
III. GENETICDECODINGALGORITHM
A. Perturbed Decoding for Turbo Codes
The principle of PA can be extended to decoding turbo codes. Let us consider a BPSK-modulated coded transmis-sion system described as follows. The information bit se-quence is d = (d1, d2, . . . , dK), where dk ∈ {0, 1}. These bits are encoded to produce the transmitted symbols x = (x1, x2, . . . , xN), where xn ∈ {1, −1}. The received signal y = (y1, y2, . . . , yN) is disturbed by real-valued additive white Gaussian noise of variance N0/2:
yn = xn+ wn, n = 1, 2, . . . , N, (2) where wn are i.i.d. N (0, N0/2). Given y, the likelihood of a potential transmitted information sequence d0, whose corresponding coded symbol sequence is x(d0), is
L(d0) =
Consider some decoder for turbo code. It maps the received signal y into some d. If we perturb the input signal by an additive signal ν, the decoder can produce a different output dν whose corresponding likelihood is L(dν). As in the case for PA, by feeding the decoder with randomly perturbed copies of received signal y, one can produce a pool of possible decoded sequences. The highest-likelihood decoded sequence eventually converges to the ML decoder output dM L, provided that:
1) A set of perturbation signals, V, exists such that for any perturbation signal ν ∈ V, this decoder maps y + ν into dM L.
2) According to the procedure to generate the perturbation signal ν, the probability of selecting a signal ν that belongs to V is nonzero.
Whether the first condition is met depends on the particular
can be empirically determined by executing the perturbed decoding algorithm. Unfortunately, due to the large block size used by turbo codes, doing so requires an impractically enormous amount of computation!
B. Genetic Decoding for Turbo Codes
In order to avoid the complexity of PA for turbo codes, instead of randomly generating perturbation signals, we wish to extract useful information from prior perturbation-decoding attempts and subsequently apply such information to the generation of new perturbation signal. To achieve this, we combine the genetic algorithm with the PA.
In our proposed GDA for turbo codes, a chromosome is a perturbation signal ν. The genes are the individual additive perturbation noise components. Our goal is to find at least one chromosome which, when added to the receive signal y, induces the decoder to produce the ML output dM L. To this end, the fitness of a chromosome ν is defined to be the likelihood of the corresponding decoded sequence, d(ν):
F V (ν) = L(d(ν)), where y + νdecode−→ d(ν). (4) The above definition of genetic information and fitness eval-uation is the core concept of GDA for turbo codes. With them, appropriate definitions can be found for other critical functional blocks in GA: parent selection, cross over, and mutation. The overall procedure of GDA then follows that of a typical GA. In Section IV, we will present our definitions of these functional blocks.
C. ML bound using GDA
GDA is an effective tool to establish a lower and an upper bound of ML decoding even though it itself is not an ML decoder. By definition, the error performance of GDA itself is an upper bound, because no decoder offers better word error probability than ML decoding.
When any decoder produces an output with a likelihood higher than that of the actually transmitted encoded sequence, the ML decoder must either decode to this erroneous output or some other sequence with even higher likelihood. Thus we can conclude that ML decoder must also make a decoding error on the same received signal. On the other hand, when a decoder produces an output with a likelihood less than or equal to that of the actually transmitted encoded sequence, we can’t be certain whether some other sequence with likelihood higher than that of the actually transmitted encoded sequence exists. If we assume an ML decoder will decode correctly over all of these signals, we would have over-estimated the performance of ML decoding. In light of this, a lower bound of ML decoding can be empirically established using a practical decoder to be the probability when the said decoder decodes into a sequence of higher likelihood than that of the actually
4
D. Complexity Aspects for GDA
GDA involves repeated uses of a turbo decoder to compute fitness values for individuals in a population. The complexity of GDA can be several orders of magnitude higher that that of a conventional turbo decoder. Nevertheless, in certain context, GDA can be used as a practical decoder. For instances, when a turbo code is concatenated by a CRC, one can first pass received signal through a conventional turbo decoder. GDA is activated only when the conventional turbo decoder fails. The average decoding complexity of GDA is weighted down by the first-pass error probability.
IV. SIMULATIONRESULTS
A. Simulation Setup
We use the turbo code defined in the WCDMA specification in our simulation [11]. The interleaver used in the specification is prunable prime interleaver [12]. For a block size of K, the actual code rate is K/(3K + 12). Encoded bits are modulated using BPSK modulation and transmitted over an AWGN channel. The “symbol-by-symbol” MAP rule [13] is used for the turbo decoding algorithm that produces the fitness values.
Eighteen full iterations are used for the MAP decoder.
B. Parameters and Procedures for GDA
In this paper, our goal is to demonstrate the potential performance of GDA with manageable complexity. We do not claim that the choice of parameters below are optimal in terms of minimizing complexity. Actually, we surmise that our results can be materially improved if we could execute more computation.
From Section III-B, we realize that the in order to ensure a nonzero probability of obtaining a perturbation signal that induces the ML-decoding output, it is prudent to allow the perturbation signal to take any values. In light of this, we produce the first generation of perturbation noises as i.i.d. zero mean Gaussian vectors. The signal-to-perturbation noise ratio is defined as
SP N R = 10 log10(Eb
σp2). (5)
It is suggested in [8] that as long as the power of the perturbation noise is chosen reasonably, the expected number of attempts until an “ideal noise” occurs is a very weak function of SPNR. Based on empirical observation, we chose 15 dB to be the SPNR used in our simulation.
It is known that the quality of the initial population strongly affects the success of the subsequent evolution. Therefore, we randomly generate 10000 original populations and keep only the fittest 100 individuals to be the initial population. The crossover rate is selected at 80 percent, i.e. a randomly selected 80 percent of the population in a generation participates in the production of new offsprings. When two chromosomes mate, we use uniform crossover to construct offsprings. A gene is copied from each parent with equal probability. On average, each parent donates 50 percent of its genetic material to its
0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 Lower bound of ML decoding
Fig. 3. Word error probability for turbo code. The block size is 2560.
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Lower bound of ML decoding
Fig. 4. Word error probability for turbo code. The block size is 5120.
survive to the next generation. Each gene of a survivor is mutated independently with a small mutation probability of 0.5 percent. Mutation replaces the gene by a randomly generated perturbation noise with the same SPNR. The 20 percent of the population that does not participate in the mating process survives to the next generation by default.
In our simulation, we terminate the GDA if (1) the pre-defined number of generation is reached, or (2) a prepre-defined percentage of population converges to one fitness value which is the maximal value among all populations in any generation.
We choose these two thresholds at 300 and 20 percent, respectively.
In these results we also present the lower bound for ML decoding computed using the concept of III-C. Recall that the GDA output can be either the originally transmitted codeword with likelihood L(d), a codeword with likelihood lower than L(d), or a codeword with likelihood higher than L(d). In order to produce a tight lower bound, when the first two cases occur, we re-execute GDA for another two rounds.
C. Performance of GDA
The word error probability for turbo codes given informa-tion block sizes of 2560 and 5120 bits per block is shown in Figure 3 and Figure 4, respectively. The error performance of conventional decoder, GDA, and the lower bound of ML performance derived from the application of GDA are shown.
5 Lower bound of ML decoding
Fig. 5. Bit error probability for turbo code. The block size is 2560.
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Lower bound of ML decoding
Fig. 6. Bit error probability for turbo code. The block size is 5120.
of GDA, here we can determine a very important result em-pirically: the penalty of using a MAP-based iterative decoder, from a minimization of word error probability perspective, is less than 0.2 dB at a word error probability of 10−4 for the turbo codes we consider.
In addition to word error probability, we also show the in-formation bit error probability for the two block sizes in Figure 5 and Figure 6, respectively. Again the error performance of the conventional MAP decoder, GDA, and the lower bound of ML performance derived from the application of GDA are contrasted. One finds that the bit error probability follows that of the word error probability closely. Hence we surmise that the number of bit errors per word error for conventional decoders and for GDA are of approximately the same order.
V. CONCLUSION
In this paper, we present a novel decoding algorithm, the Genetic Decoding Algorithm (GDA), for use with turbo codes.
In GDA, we propose to use random perturbation noises as the genes. A conventional decoder is used to assign fit-ness values to chromosomes. Specifically, one adds the noise pattern (chromosome) to the received signal first. A well chosen decoder decodes the perturbed signal and generates
GDA can be used as an empirical tool to establish the ML decoding performance for a given specific turbo code. The performance of GDA itself can serve as an upper bound.
The probability where GDA finds a codeword with higher likelihood than the transmitted codeword can be used as a lower bound. These bounds are very tight at lower error probability regime. Our simulation results indicate that these two bounds converge at a word error probability of 10−4. From our results, we conclude that the difference between an ML decoder and a MAP-based iterative decoder is less than 0.2 dB at this word error probability and the block size we simulated.
Because GDA produces the lower bound, it achieves the ML error performance in such regime.
REFERENCES
[1] C. Berrou, A. Glavieux, and P. Thitimajshima, “Near Shannon limit error-correcting coding and decoding: turbo-codes,” Proc. IEEE Int.
Conf. Commun. ’93,, pp. 1064-1070, May 1993.
[2] Shu Lin and Daniel J. Costello, Jr., Error Control Coding: Fundamentals and applications, Second Edition, Prentice Hall, Inc., Upper Saddle River, NJ, 2004.
[3] S. Benedetto and G. Montorsi, “Unveiling Turbo-Codes: Some Results on Parallel Concatenated Coding Schemes,” IEEE Transactions on Information Theory. vol. 43, no.2, pp. 409-428, March 1996.
[4] D. Divsalar, S. Dolinar, R. J. McEliece, and F. Pollara, “Transfer Function Bounds on the Performance of Turbo Codes,” TDA Progress Report 42-122, April-June 1995, Jet Propulsion Laboratory, Pasadena, California, pp. 44-45. Auguest 15, 1995.
[5] T. M. Duman and M. Salehi, “New performance bounds for turbo codes,”
IEEE Trans. Commun. vol. 46, pp. 717-723, June 1998.
[6] I. Sason and S. Shamai, “Improved Upper Bounds on the ML Decoding
[6] I. Sason and S. Shamai, “Improved Upper Bounds on the ML Decoding