Serial Concatenation of Interleaved Codes: … Concatenation of Interleaved Codes: Performance ... allel concatenated convolutional codes known as “turbo codes ... for those applications

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 3, MAY 1998 909

Serial Concatenation of InterleavedCodes: Performance Analysis,

Design, and Iterative DecodingSergio Benedetto,Fellow, IEEE, Dariush Divsalar,Fellow, IEEE,

Guido Montorsi,Member, IEEE, and Fabrizio Pollara,Member, IEEE

Abstract—A serially concatenated code with interleaver consistsof the cascade of anouter encoder, an interleaver permutingthe outer codewords bits, and aninner encoder whose inputwords are the permuted outer codewords. The constructioncan be generalized toh cascaded encoders separated byh � 1

interleavers. We obtain upper bounds to theaveragemaximum-likelihood bit error probability of serially concatenated block andconvolutional coding schemes. Then, we derive design guidelinesfor the outer and inner encoders that maximize theinterleavergain and the asymptotic slope of the error probability curves.Finally, we propose a new, low-complexity iterative decoding al-gorithm. Throughout the paper, extensive comparisons with par-allel concatenated convolutional codes known as “turbo codes”are performed, showing that the new scheme can offer superiorperformance.

Index Terms—Concatenated codes, iterative decoding, serialconcatenation, turbo codes.

NOMENCLATURE

CC Constituent Code.PCCC Parallel Concatenated Convolutional Code.PCBC Parallel Concatenated Block Code.SCC Serially Concatenated Code.SCBC Serially Concatenated Block Code.SCCC Serially Concatenated Convolutional Code.ML Maximum Likelihood.IOWEF Input–Output Weight-Enumerating Function.CWEF Conditional Weight-Enumerating Function.SISO Soft Input Soft Output module.SW-SISO Sliding Window—Soft Input Soft Output

module.LLR Log-Likelihood Ratio.MAP Maximum a posteriori.

Manuscript received July 26, 1996; revised November 1, 1997. This workwas supported in part by NATO under Research Grant CRG 951208. Thework of S. Benedetto and G. Montorsi was supported by Italian NationalResearch Council (CNR) under “Progetto Finalizzato Trasporti (Prometheus),”by MURST (Progetto 40% “Comunicazioni con Mezzi Mobili”), and byQualcomm. The research described in this paper was performed in part atthe Jet Propulsion Laboratory, California Institute of Technology, Pasadena,CA, under Contract with the National Aeronautics and Space Administration(NASA). The material in this paper was presented in part at the IEEE Inter-national Symposium on Information Theory, Ulm, Germany, June 29–July 4,1997.

S. Benedetto and G. Montorsi are with Dipartimento di Elettronica, Politec-nico di Torino, Italy.

D. Divsalar and F. Pollara are with Jet Propulsion Laboratory, Pasadena,CA 91109 USA.

Publisher Item Identifier S 0018-9448(98)02352-9.

I. INTRODUCTION

I N his goal to find a class of codes whose probability of errordecreased exponentially at rates less than capacity, while

decoding complexity increased only algebraically, Forney [1]arrived at a solution consisting of the multilevel codingstructure known asconcatenated code. It consists of thecascade of aninner code and anoutercode, which, in Forney’sapproach, would be a relatively short inner code (typically,a convolutional code) admitting simple maximum-likelihood(ML) decoding, and a long high-rate algebraic nonbinaryReed–Solomon outer code equipped with a powerful algebraicerror-correction algorithm, possibly using reliability informa-tion from the inner decoder.

Initially motivated only by theoretical research interests,concatenated codes have since then evolved as a standardfor those applications where very high coding gains areneeded, such as (deep-) space applications and many others.Alternative solutions for concatenation have also been studied,such as using a trellis-coded modulation scheme as inner code[2], or concatenating two convolutional codes [3]. In the lattercase, the inner Viterbi decoder employs a soft-output decodingalgorithm to provide soft-input decisions to the outer Viterbidecoder. An interleaver was also proposed between the twoencoders to separate bursts of errors produced by the innerdecoder.

We find then, in a “classical” concatenated coding scheme,the main ingredients that formed the basis for the inventionof “turbo codes” [4], namely two, or more,constituentcodes(CC’s) and aninterleaver. The novelty of turbo codes, how-ever, consists of the way they use the interleaver, which isembedded into the code structure to form an overall concate-nated code with very large block length, and in the proposal ofa parallel concatenation to achieve a higher rate for given ratesof CC’s. The latter advantage is obtained using systematicCC’s and not transmitting the information bits entering thesecond encoder. In the following, we will refer to turbo codesas parallel concatenated convolutional codes(PCCC’s). Theso-obtained codes have been shown to yield very high codinggains at bit error probabilities in the range ; inparticular, low bit error probabilities can be obtained at rateswell beyond the channel cutoff rate, which had been regardedfor long time as the “practical” capacity. Quite remarkably,this performance can be achieved by a relatively simple

0018–9448/98$10.00 1998 IEEE

910 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 3, MAY 1998

iterative decoding technique whose computational complexityis comparable to that needed to decode the two CC’s.

In this paper, we consider the serial concatenation of inter-leaved codes orserially concatenated codes(SCC’s), calledSCBC or SCCC according to the nature of CC’s, that can beblock (SCBC) or convolutional (SCCC) codes. For this classof codes, we obtain analytical upper bounds to the performanceof an ML decoder, propose design guidelines leading to theoptimal choice of CC’s that maximize theinterleaver gainand the asymptotic code performance, and present a newiterative decoding algorithm yielding results close to capacitylimits with limited decoding complexity. Preliminary resultshave appeared in [5] and [6]. Extensive comparisons withturbo codes of the same complexity and decoding delay areperformed.

With this embodiment of results, we believe that SCCC canbe considered as a valid, in some cases superior, alternativeto turbo codes.

In Section II, we derive analytical upper bounds to thebit error probability of both SCBC’s and SCCC’s, using theconcept of “uniform interleaver” that decouples the outputof the outer encoder from the input of the inner encoder. InSection III, we propose design rules for SCCC’s through anasymptotic approximation of the bit error probability boundassuming long interleavers or large signal-to-noise ratios. InSection III, we compare serial and parallel concatenationsof block and convolutional codes in terms of maximum-likelihood analytical upper bounds. Section V is devoted to thepresentation of a new iterative decoding algorithm and to itsapplication to some significant codes. Performance comparisonbetween SCCC’s and PCCC’s under suboptimum iterativedecoding algorithms are presented in Section IV.

II. A NALYTICAL BOUNDS TO THE PERFORMANCE

OF SERIALLY CONCATENATED CODES

A. A Union Bound to the Bit Error Probabilityand Some General Warnings

Consider an linear block code and itsInput–OutputWeight Enumerating Function(IOWEF), defined as

(1)

where represents the number of codewords with weightgenerated by information words of weight. In (1), we have

also implicitly defined theconditional weight enumeratingfunction (CWEF)

(2)

as the function that enumerates the weight distribution ofcodewords generated by information words of a given weight

.A linear block (or convolutional) code possesses theuniform

error property[7], stating that its word and bit error probability

performance can be evaluated under the assumption that theall-zero codeword has been transmitted. Assume then thatthe all-zero codeword has been transmitted, and definethe pairwise error event as the event in which thelikelihood of a codeword with weight and generated byan information word of weight is higher than that of theall-zero codeword .

Using the union bound, the bit error probability underML soft decoding for binary phase-shift keying (PSK) (orbinary pulse amplitude modulation (PAM)) transmission overan additive white Gaussian noise channel with two-sided noisepower spectral density can be upper-bounded as

(3)

where is the code rate, is the energy perinformationbit,1 and where we have defined thebit error multiplicity

(4)

Expressions (3) and (4) suggest that two ways can be followedto improve the bit error probability performance: the first, lead-ing to the more traditional concept of good (and asymptoticallygood) codes, tries to increase the first, more significant weights

in (3); the second, forming the basis of turbo codes and alsoof serially concatenated codes, aims at reducing the bit errormultiplicities (4). To quote Forney’s 1995 Shannon lecture:

Rather than attacking error exponents, turbo codes at-tack multiplicities, turning conventional wisdom on itshead.

A more compact, but looser, upper bound, can be obtainedfrom (3) using the inequality

e (5)

which yields

(6)

From (3) and (6), we conclude that, in order to upper-boundthe bit error probability for any linear block code, we need toevaluate its CWEF. As a consequence, also for concatenatedcodes with interleavers we can use (3) and (6), providedthat we are able to compute the CWEF of the overall codeassuming that the CWEF’s of the constituent codes (CC’s)are known. This has been done already for “turbo codes,”i.e., parallel concatenated codes, in [8]. In the following, wewill show how to extend those results to the case of serialconcatenation.

1It must be noted thatEb=N0 is not the signal-to-noise ratio that can bemeasured in the channel, which, indeed, is lower by the factorRc.

BENEDETTO et al.: SERIAL CONCATENATION OF INTERLEAVED CODES 911

Fig. 1. Comparison of bit error probability curves obtained through unionbounds and simulations for two parallel concatenated convolutional codesover an additive Gaussian noise channel. Also indicated is theEb=N0 valuecorresponding to the channel cutoff rate.

Before starting the analysis leading to the evaluation of theCWEF of a serially concatenated code (SCC) with interleaver,a warning to the readers is necessary. Both (3) and (6)stem from the union bound, stating that the probability ofa union of events is less than or equal to the sum of theprobabilities of the individual events. The union bound isused extensively as an upper limit to the error probabilitiesfor digital transmission systems. The sums of the individualprobabilities in the right-hand sides of (3) and (6), however, arenot probabilities themselves, and can thus assume large valuesmuch greater than one. In fact, it is common knowledge in thefield that union bounds are very close to the true probabilityin the case of maximum-likelihood decoding for medium-high signal-to-noise ratios, whereas they tend to diverge forlow signal-to-noise ratios. A widely accepted rule of thumbis that the signal-to-noise ratio where they start to becomeunreliable is the one yielding the cutoff rate of the channel.The behavior of the bounds is illustrated as a typical examplein Fig. 1, where we plot the bounds for two different rateparallel concatenated codes and compare them with simulationresults obtained using the suboptimum, iterative decodingalgorithm proposed to decode turbo codes. Also drawn inthe figure is the corresponding to the channel cutoffrate.

Some general comments, partly based on Fig. 1, are appro-priate:

• As previously anticipated, the upper bounds based on theunion bound diverge at a signal-to-noise ratio close tothe channel cutoff rate. Obtaining tighter upper boundscapable of extending the validity interval of the unionbounds for concatenated codes is an important, and stillwidely open, topic for research. The new bounds could bebased on the technique successfully employed in [9] forconvolutional codes, or on the classical Gallager bound

[10]. A successful application of the Gallager bound toparallel concatenated codes with interleavers has beendescribed in [11], where it is shown that the new boundextends the validity of the union bound for some rangeof signal-to-noise ratios below the channel cutoff rate,typically 0.5 dB. On the other hand, those attempts wouldstill be based on the hypothesis of maximum-likelihooddecoding. Thinking of applying them to the suboptimumiterative decoding seems not realistic.

• To obtain the divergence of the union bound one needs tocompute a very large number of terms for the summationin the right-hand side of (3), or (6), and this was indeedthe case for the example to which the curves of Fig. 1refer. In that case, however, the interleaver lengthwas limited to . When becomes very large, asit is required to approach the channel capacity, only alimited number of terms in the summations (3) and (6) canbe obtained with a reasonable computational complexity.As a consequence, the obtained upper bounds are stillvery accurate above the channel cutoff rate, but maynot present the divergence at cutoff rate. In those cases,the reader should only consider as reliable the bit errorprobability values above the cutoff rate, or perhaps half adecibel below it, according to the results of [11]. A wayto overcome this drawback, and indeed to always showthe divergence of the union bound, has been proposed in[12]. It has, however, to rely on the looser bound (6).

• The main tool used in this paper to analytically predictthe performance and to find design rules about the mainCC’s parameters is a union bound. We have seen, on theother hand, that the union bound is tight only for medium-high signal-to-noise ratios. One could then question thevalidity of the approach, which suffers the paradox ofusing a bound not applicable to very low signal-to-noiseratios in order to design coding schemes intended to worknear channel capacity.

We are conscious of this inconsistency, yet, for onehand, have simply nothing better to propose, and, on theother hand, we had widely verified by simulation that theparallel concatenated codes designed on the basis of ourrules are indeed very good also at very low signal-to-noise ratios (see [13] and [14]). In the remainder of thispaper, we will show that this heuristic validation of thedesign rules also holds for serially concatenated codeswith interleavers.

• The last observation concerns still another inconsistency,resulting from the fact that we are using bounds based onmaximum-likelihood decoding to design codes that aredecoded according to a different, suboptimum algorithm.Also in this case we invoke the heuristic validation stem-ming from a large number of simulations, which showthe convergence of the simulated performance towardthe analytical bounds. In Fig. 1, where the simulatedpoints have been obtained with the suboptimum, itera-tive decoding algorithm, we see a nice example of thisbehavior.


Fig. 2. Serially concatenated(n; k; N = mp) block code.

B. Evaluating the Bit Error Probability Upper Boundfor Serially Concatenated Block Codes

We will now show how to apply the union bounds (3) and(6) to the case of serially concatenated codes with interleavers.For simplicity of the presentation, we begin considering seri-ally concatenated block codes (SCBC’s).

The scheme of two serially concatenated block codes isshown in Fig. 2. It is composed of two cascaded CC’s, theouter code with rate and theinnercode with rate , linked by an interleaver of length

that is an integer multiple of the lengthof the outercodewords. The scheme works as follows: the bits of anumber of codewords of the outer code are written into theinterleaver of length , and read in a different orderaccording to the permutation performed by the interleaver. Thesequence of bits at the output of the interleaver is then sentin blocks of length to the inner encoder.

The overall SCBC is then an code with rate

and we will refer to it as the code .In the following, we will derive an upper bound to the ML

performance of the overall code , assuming first that ,and then extending the result to the general case. We assumethat the outer and inner CC’s are linear, so that also the SCBCis linear and theuniform error propertyapplies, i.e., the biterror probability can be evaluated assuming that the all-zerocodeword has been transmitted.

In order to apply the upper bounds (3) and (6) to the SCBC,we need to evaluate the CWEF of the code, assuming thatwe know the CWEF’s of the CC’s.

If is low, we can compute the coefficients of theCWEF (2) by letting each individual informationword with weight be first encoded by the outer encoderand then, after the bits of the outer codeword have beenpermuted by the interleaver, be encoded by the inner encoder

originating an inner codeword with a certain weight. Afterrepeating this procedure for all the information words withweight , we should count the inner codewords with weight

, and their number would be the value of . Whenis large, or, in the case , when is large, theprevious operation becomes too complex, and we must resortto a different approach.

The key point, here, is that we would like to obtain asimple relationship between the CWEF’s of the two CC’s,an operation that is prevented by the fact that the knowledgeof the information word weight is not enough to obtain theweight of the inner codeword, which, instead, depends on theweight of the outer codeword and on the permutation inducedby the interleaver.

Fig. 3. The action of a uniform interleaver of length4 on sequences ofweight 2.

As in [8] and [15], a crucial step in the analysis consistsin replacing the actual interleaver that performs a permutationof the input bits with an abstract interleaver calleduniforminterleaver, defined as a probabilistic device that maps a giveninput word of weight into all distinct permutations of itwith equal probability (see Fig. 3).

Use of the uniform interleaver permits the computationof the “average” performance of the SCBC, intended as theexpectation of the performance of SCBC’s using the sameCC’s, taken over the set of all interleavers of a given length.A theorem proved in [8] guarantees the meaningfulness of theaverage performance, in the sense that there will always be, foreach value of the signal-to-noise ratio, at least one particularinterleaver yielding performance better than or equal to thatof the uniform interleaver.

Let us define the IOWEF and the CWEF of the SCBCas and . Their definition and meaningare the same as in (1) and (2).

As seen, to apply the bounds (3) and (6) to the bit errorprobability we need to evaluate the CWEF of the SCBC fromthe knowledge of the CWEF’s of the outer and inner codes,which we call and , where the firstenumerates the weight distributions of the outer codewordsgenerated by information words of weight, and the secondenumerates the weight distributions of the inner codewordsgenerated by outer codewords of weight.

To do this, we exploit the properties of the uniform inter-leaver, which transforms a codeword of weightat the outputof the outer encoder into all its distinct permutations. As aconsequence, each codeword of the outer codeof weight ,through the action of the uniform interleaver, enters the innerencoder generating codewords of the inner code . Thusthe number of codewords of the SCBC of weightassociated with an information word of weightis given by

(7)

From (7) we derive the expressions of the CWEF andIOWEF of the SCBC as

(8)

(9)


where enumerates the weight distributions of theinformation words that generate codewords of the outer codewith a given weight .

Example 1: Consider the serially concatenatedblock code obtained by concatenating the parity-checkcode to a Hamming code through an interleaver oflength . The IOWEF and ofthe outer and inner code are

so that

Through (9) we then obtain

Previous results (8) and (9) can be easily generalized to themore interesting case of an interleaver with lengthbeingan integer multiple (by a factor ) of the length of theouter codewords. Denoting by the IOWEF of thenew outer code, and similarly by theIOWEF of the new inner code, it is straightforwardto obtain

(10)

From the IOWEF’s (7)–(10), we obtain the CWEF’sand of the new CC’s, and, finally,

through (8) and (9), the CWEF and IOWEF of the newSCBC

(11)

(12)

Fig. 4. Analytical bounds for serially concatenated block code of Example2 (SCBC1 in Table I).

Fig. 5. Serially concatenated(n; k; N) convolutional code.

Example 2: Consider again the CC’s of Example 1,linked by an interleaver of length , and use (2),(3), and (11). The so obtained upper bound to the bit errorprobability is plotted in Fig. 4 for various values of theinteger . The curves show theinterleaver gain, defined as thefactor by which the bit error probability is decreased with theinterleaver length at a given signal-to-noise ratio. Contrary toparallel concatenated block codes [8], the curves do not exhibitthe interleaver gain saturation. Rather, the bit error probabilityseems to decrease regularly withas . We will explainthis behavior in Section III.

C. Serially Concatenated Convolutional Codes

The structure of a serially concatenated convolutional codeis shown in Fig. 5. It refers to the case of two convolutionalCC’s, the outer code with rate , and the innercode with rate , joined by an interleaver of length

bits, generating an SCCC with rate .will be assumed to be an integer multiple2 of . We assume,as before, that the convolutional CC’s are linear, so that theSCCC is linear as well, and the uniform error property applies.

The exact analysis of this scheme can be performed byappropriate modifications of that described in [8] for PCCC’s.It requires the use of ahypertrellishaving ashyperstatespairsof states of outer and inner codes. The hyperstatesandare joined by ahyperbranchthat consists of all pairs of paths

2Actually, this constraint is not necessary. We can choose in fact inner andouter codes of any ratesRi

c = ki=ni andRoc = ko=no, constraining the

interleaver to be an integer multiple of the minimum common multiple ofnoand ki, i.e., N = K � mcm(no; ki). This generalization, though, leads tomore complicated expressions and is not considered in the following.


Fig. 6. Analytical bounds for the serially concatenated convolutional codeof Example 3 (SCCC1 of Table II).

with length that join states of the inner code andstates of the outer code, respectively. Each hyperbranchis thus an equivalent SCBC labeled with an IOWEF that canbe evaluated as explained in the previous subsection. From thehypertrellis, the upper bound to the bit error probability canbe obtained through the standard transfer function techniqueemployed for convolutional codes [10]. As proved in [8], whenthe length of the interleaver is significantly greater than theconstraint lengths of the CC’s, an accurate approximation ofthe exact upper bound consists in retaining only the branchof the hypertrellis joining the hyperstates . In thefollowing, we will always use this approximation.

Example 3: Consider a rate SCCC formed by anouter -state convolutional code with rate and an inner

-state convolutional code with rate , joined by a uniforminterleaver of length and(SCCC1 of Table II).

Both encoders are systematic and recursive and the gener-ator matrices are reported in Table III, first and third rows.Using the previously outlined analysis, we have obtained thebit error probability curves shown in Fig. 6. The performanceshows a very significant interleaver gain, i.e., lower values ofthe bit error probability for higher values of. The interleavergain seems to behave as . This behavior will be explainedin the next section.

III. D ESIGN OF SERIALLY CONCATENATED CODES

In the previous section, we have presented an analyticalbounding technique to find the ML performance of SCBC’sand SCCC’s. For practical applications, SCCC’s are to be pre-ferred to SCBC’s. One reason is thata posteriori-probabilityalgorithms are less complex for convolutional than for blockcodes, another is that the interleaver gain can be greater forconvolutional CC’s, provided that they are suitably designed.Hence, we deal mainly with the design of SCCC’s, extendingour conclusions to SCBC’s when appropriate.

Before delving deeply into the analytical derivation, it isimportant to say a few words about the design methodology.

The performance of a concatenated code with interleaver,for both cases of parallel and serial concatenation, dependson the constituent codes and on the interleaver in a strictlyinterdependent manner. The joint design of CC’s and theinterleaver, however, is a hopeless goal, and the only wayto achieve significantly good results seems to pass through adecoupleddesign, in which one first designs the CC’s, and thentailors the interleaver on their characteristics. Our approach toachieve this goal resides once more in the use of the uniforminterleaver, which yields to an average optimization of theCC’s, followed by a customization of the actual interleaver.Only the first step, i.e., the design of CC’s, will be treated here.

Consider the SCCC depicted in Fig. 5. Its performancecan be approximated by that of the equivalent block codewhose IOWEF labels the branch of the hypertrellis joiningthe zero states of outer- and inner-code trellises. Denoting by

the CWEF of this equivalent block code, we canrewrite the upper bound (6) as3

(13)

where is the minimum weight of an input sequencegenerating an error event of the outer code, and is theminimum weight4 of the codewords of . By error eventof a convolutional code, we mean a sequence diverging fromthe zero state at time zero and remerging into the zero stateat some discrete time . For constituent block codes, anerror event is simply a codeword.

The coefficients of the equivalent block code can beobtained from (7), once the quantities and of theCC’s are known. To evaluate them, consider a rateconvolutional code with memory5 and its equivalent

block code whose codewords are all sequencesof length bits of the convolutional code starting from andending at the zero state. By definition, the codewords of theequivalent block code are concatenations of error events ofthe convolutional codes. Let

(14)

be the weight enumerating function of sequences of theconvolutional code that concatenateerror events with totalinput information weight . The coefficient representsthe number of sequences of weight, input weight , andnumber of concatenated error events. Its meaning is pic-torially clarified in Fig. 7, where it can be noticed that, by“concatenated,” we mean actually that theerror events are

3In the following, a subscript “m” will denote “minimum,” and a subscript“M ” will denote “maximum.”

4Since the input sequences of the inner code arenot unconstrainedindependent and identically distributed (i.i.d.) binary sequences, but, instead,codewords of the outer code,hm can be greater than the inner code freedistancedif .

5By memory, we mean the maximum length of the shift registers containedin the encoder.


Fig. 7. The meaning of the coefficientsAl; h; j .

adjacent, in the sense that each one starts immediately wherethe previous ends, without any gap in between.

For much larger than the memory of the convolutionalcode, the coefficient of the CWEF of the equivalent blockcode can be upper-bounded by6

(15)

where , the largest number of error events concatenated in acodeword of weight and generated by a weightinformationsequence, is a function ofand that depends on the encoder,as we will see later.

Let us return now to the block code equivalent to the SCCC.Using previous result (15) with and for theouter and inner code, respectively,7 we can write

and

Substituting them into (7), we obtain an upper bound tothe value of the coefficients of the serially concatenatedblock code equivalent to the SCCC in the form

(16)

where is the free distance of the outer code. By free distancewe mean the minimum Hamming weight of error events

for convolutional CC’s, and the minimum Hamming weightof codewords for block CC’s.

To proceed further, we need to replace the three binomialcoefficients in (16). In order to obtain an upper bound to theright-hand side of (16), we will replace the two binomials inthe numerator using the upper bound

6The upper bound is obtained neglecting the length of error eventscompared toN , and assuming that the number of waysj input sequencesproducingj error events can be arranged in a register of lengthN is N=p

j.

The ratioN=p derives from the fact that the code has ratep=n, and thusNbits correspond toN=p input words or, equivalently, trellis steps.

7In the following, superscripts “o” and “i” will refer to quantities pertainingto outer and inner code, respectively.

and the binomial in the denominator using the lower bound

Notice that the bounds are tight for largeand for ,which will be seen to always happen in our case.

Substitution of these bounds in (16) yields

(17)

Finally, substituting (17) into (13), gives the bit error proba-bility bound in the form

(18)

Using (18) as the starting point, we will obtain some im-portant design considerations. The bound (18) to the bit errorprobability is obtained by adding terms of the first summationwith respect to the SCCC weights. The coefficients of theexponentials in depend, among other parameters, on. Forlarge , and for a given , the dominant coefficient of theexponentials in is the one for which the exponent of ismaximum. Define this maximum exponent as

(19)

Evaluating in general is not possible without specifyingthe CC’s. Thus we will consider two important cases, forwhich general expressions can be found.

A. The Exponent of for the Minimum Weight

For large values of , the performance of the SCC isdominated by the first term of the summation with respect to,corresponding to the minimum value . Rememberingthat by definition, and are the maximum numberof concatenated error events in codewords of the inner andouter code of weights and , respectively, the followinginequalities hold true:

(20)

(21)

and

(22)


where is the minimum weight of codewords of theouter code yielding a codeword of weight of the innercode, and means “integer part of .”

In most cases,8 , and , so that, and (22) becomes

(23)

The result (23) shows that the exponent ofcorrespondingto the minimum weight of SCCC codewords is always negativefor , thus yielding an interleaver gain at high .Substitution of the exponent into (18) truncated tothe first term of the summation with respect toyields thefollowing result, asymptotic with respect to :

(24)

where the constant is

and is the set of input weights that generate codewordsof the outer code with weight .

Expression (24) suggests the following conclusions.

• For the values of and where the SCCC per-formance is dominated by its free distance ,increasing the interleaver length yields a gain in perfor-mance.

• To increase the interleaver gain, one should choose anouter code with large .

• To improve the performance with , one shouldchoose an inner- and outer-code combination such that

is large.

These conclusions do not depend on the structure of theCC’s, and thus they yield for both recursive and nonrecursiveencoder.

The curves of Fig. 4 showing the performance of the variousSCBC’s of Example 1 with increasing interleaver length,however, also show a different phenomenon: for a given

, there is a value of above which the bound diverges.In other words, there seem to be coefficients of the exponentsin , for , that increase with .

To investigate this phenomenon, we will evaluate the largestexponent of , defined as

(25)

This exponent will permit to find the dominant contribution tothe bit error probability for .

B. The Maximum Exponent of

We need to treat the cases of nonrecursive and recursive in-ner encoders separately. As we will see, nonrecursive encodersand block encoders show the same behavior.

8This will be seen in the examples that follow and corresponds to the mostfavorable situation.

1) Block and Nonrecursive Convolutional Inner Encoders:Consider the inner encoder and its impact on the exponent of

in (25). For a nonrecursive inner encoder, we have .In fact, every input sequence with weight one generates afinite-weight error event, so that an input sequence with weight

will generate, at most, error events corresponding to theconcatenation of error events of input weight one. Since theuniform interleaver generates all possible permutations of itsinput sequences, this event will certainly occur.

Thus from (25) we have

and interleaving gain is not allowed. This conclusion holdstrue for both SCCC’s employing nonrecursive inner encodersand for all SCBC’s, since block codes have codewords corre-sponding to input words with weight equal to one.

For those SCC’s wealwayshave, for some , coefficientsof the exponential in of (18) that increase with , and thisexplains the divergence of the bound arising, for each ,when the coefficients increasing with become dominant.

2) Recursive Inner Encoders:For recursive convolutionalencoders, the minimum weight of input sequences generatingerror events is (see [13] and [16]). As a consequence, aninput sequence of weight can generate at most errorevents.

Assuming that the inner encoder of the SCCC is recursive,the maximum exponent of in (25) becomes

(26)

The maximization involvesand , since depends on bothquantities. In fact, remembering its definition as the maximumnumber of concatenated error events of codewords of the outercode with weight generated by input words of weight, itis straightforward to obtain

(27)

Substituting now the last inequality (27) into (26) yields

(28)

To perform the maximization of the right-hand side of (28),consider first the case

integer

so that

(29)

The right-hand sideof (29) is maximized, for , bychoosing . On the other hand, for


the most favorable case is , which leads us againto the previously discussed situation. Thus the maximizationrequires . For this value, on the other hand, we havefrom (27) , and the inequality becomes an equalityif , where is the set of input weights thatgenerate codewords of the outer code with weight . Inconclusion, the largest exponent of is given by

(30)

The value (30) of shows that the exponents of in (18)are always negative integers. Thus for all, the coefficientsof the exponents in decrease with , and we always havean interleaver gain.

Denoting by , as in [8], the minimum weight ofcodewords of the inner code generated by weight-inputsequences, we obtain a different weight for even andodd values of .

even: For even, the weight associated to thehighest exponent of , is given by

since it is the weight of an inner codeword that concatenateserror events with weight .

Substituting the exponent into (18), approximated onlyby the term of the summation with respect tocorrespondingto , yields the following result, asymptotic withrespect to :

(31)

where

(32)

In (32), is the maximum input weight yielding outercodewords with weight equal to , and is the number ofsuch codewords.

odd: For odd, the value of is given by

(33)

where is the minimum weight of sequences of the innercode generated by a weight–input sequence. In this case, infact, we have

concatenated error events, of which generated byweight- input sequences and one generated by a weight-input sequence.

Thus substituting the exponent into (18) approximatedby keeping only the term of the summation with respect to

corresponding to yields the following result,asymptotic with respect to :

(34)

where

(35)

In both cases of even and odd, we can draw from (31) and(34) a few important design considerations:

• in contrast with the case of block codes and nonrecursiveconvolutional inner encoders, the use of a recursiveconvolutional inner encoder always yields an interleavergain. As a consequence, the first design rule states thattheinner encoder must be a convolutional recursive encoder.

• The coefficient , which multiplies the signal-to-noise ratio in (18), increases for increasing valuesof . Thus we deduce thatthe effective free distance ofthe inner encoder must be maximized. Both this and theprevious design rule had been stated also for PCCC’s9

[13]. As a consequence, the recursive convolutional en-coders optimized for use in PCCC’s (see tables in [13] and[17]) can be employed altogether as inner CC in SCCC’s.

From (33), however, we can infer that also the pa-rameter , namely the minimum weight of inner codesequences generated by weight-input sequences, isimportant for odd values of . This may lead to achoice for the inner code that is in partial disagreementwith the design rules obtained for PCCC’s in [13]. Thosedesign rules stated that optimum constituent encoders forPCCC’s should have a feedback connection made accord-ing to a primitive polynomial. For SCCC’s with odd ,it is often convenient to choose an inner encoder whosefeedback polynomial contains as a factor; thischoice, in fact, provided that the feedforward polynomialdoes not cancel such factor, eliminates all error events ofthe inner code originated by odd-weight input sequences,thus yielding . As a consequence, the parameter

would not be obtained from (33) anymore, but,instead, from an input sequence to the inner code withweight larger than , yielding a greater value for .In Section III, we will see an example of code designedaccording to that procedure.

• The interleaver gain is equal to for even valuesof and to for odd values of . Asa consequence,we should choose, compatibly with thedesired rate of the SCCC,an outer code with a largeand, possibly, odd value of the free distance.

9For PCCC’s, however, both CC’s had to comply with those design rules.


TABLE IDESIGN PARAMETERS OF CONSTITUENT CODES AND SERIALLY CONCATENATED BLOCK CODES FOR

THREE SERIALLY CONCATENATED BLOCK CODES

• As to other outer code parameters, and shouldbe minimized. In other words, we should have the mini-mum number of input sequences generating free distanceerror events of the outer code, and their input weightsshould be minimized. Since nonrecursive encoders haveerror events with , and, in general, less input errorsassociated with error events at free distance [18], it canbe convenientto choose as outer code a nonrecursiveencoder with minimum and . Conventionalnonrecursive convolutional codes found in books (see,for example, [7]) are appropriate.

C. Examples Confirming the Design Rules

To confirm the design rules obtained asymptotically, i.e.,for large signal-to-noise ratio and large interleaver lengths,we evaluate the upper bound (3) to the bit error probabilityfor several block and convolutional SCC’s, with differentinterleaver lengths, and compare their performance with thosepredicted by the design guidelines.

1) Serially Concatenated Block Codes:We consider threedifferent SCBC’s obtained as follows: the first is the

SCBC described in Example 2, the second isa SCBC using as outer code a parity-check code and as inner code a BCH code, and thethird is a SCBC using as outer code aHamming code and as inner code a BCH code. Notethat the second and third SCBC’s have the same rate,.

The outer, inner, and SCBC code parameters introduced inthe design analysis of Section III are listed in Table I.

In Figs. 4, 8, and 9, we plot the bit error probability boundsfor the SCBC’s 1, 2, and 3 of Table I.

Code SCBC1 has ; thus from (23), we expect aninterleaver gain going as . This is confirmed by the curvesof Fig. 4, which, for a fixed and sufficiently large signal-to-noise ratio, show a decrease in of a factor , whenincreases from to , from to , and from to .Moreover, curves of Fig. 4 show a divergence of the boundat higher for increasing . This is due to coefficientsof terms with in (18) that increase with , whoseinfluence become more important for larger.

Code SCBC2 has ; thus from (23), we expectthe same interleaver gain as for SCBC1, i.e., . This isconfirmed by the curves of Fig. 8, which also show the bounddivergence predicted in the analysis of Section II.

Code SCBC3 has ; thus from (23), we expect alarger interleaver gain than for SCBC1 and SCBC2, i.e., .This is confirmed by the curves of Fig. 9, which, for a fixedand sufficiently large signal-to-noise ratio, show a decrease in

Fig. 8. Analytical bounds for the serially concatenated block code SCBC2.

Fig. 9. Analytical bounds for the serially concatenated block code SCBC3.

of a factor , when increases from to , fromto , and from to . As to the bound divergence, wenotice a slightly different behavior with respect to previouscases. The curve with , in fact, denotes a stronginfluence of coefficients increasing with for lowerthan 7 dB.

B. Serially Concatenated Convolutional Codes

We consider four different SCCC’s obtained as follows:the first, SCCC1, is a SCCC (the same of Example3), using as outer code a-state, recursive, systematic


TABLE IIDESIGN PARAMETERS OF CONSTITUENT CODES AND OF SERIALLY CONCATENATED CONVOLUTIONAL CODES FOR

FOUR SERIALLY CONCATENATED CONVOLUTIONAL CODES

TABLE IIIGENERATING MATRICES FOR THECONSTITUENT CONVOLUTIONAL CODES

convolutional encoder and as inner code a-state, recur-sive, systematic convolutional encoder. The second, SCCC2,is a SCCC, using as outer code the same-state,

recursive, systematic convolutional encoder of SCCC1,and as inner code a-state, nonrecursive convolutionalencoder. The third, SCCC3, is a SCCC using as outercode a -state, nonrecursive, convolutional encoder, andas inner code the same-state, recursive, systematicconvolutional encoder of SCCC1. The fourth, SCCC4, finally,is a SCCC, using as outer code a-state, non-recursive convolutional encoder, and as inner code a-state,

recursive, systematic convolutional encoder, obtained byconcatenating three equal-state recursive, systematicconvolutional encoders.

The outer, inner, and SCCC code parameters introduced inthe design analysis in Section III are listed in Table II. In thistable, the CC’s are identified through the description of TableIII.

In Figs. 6, 10, 11, and 12, we plot the bit error probabilitybounds for the SCCC’s 1, 2, 3, and 4 of Table II, withinterleaver lengths InFig. 12, which refers to the code SCCC4 of Table II, thevalues of the interleaver length are different, i.e.,

This is due to the fact that theouter code for SCCC4 has a rate , instead of , and thusthe interleaver length must be changed in order to guaranteethe same input delay for all SCCC’s.

Consider first the SCCC’s employing as inner CC’s recur-sive, convolutional encoders as suggested in Section III. Theyare SCCC1, SCCC3, and SCCC4.

Code SCCC1 has ; thus from (34), we expect aninterleaver gain behaving as . This is fully confirmed bythe curves of Fig. 6, which, for a fixed and sufficiently largesignal-to-noise ratio, show a decrease in by a factor

, when increases from to . For an even moreaccurate confirmation, one can compare the interleaver gain

Fig. 10. Analytical bounds for the serially concatenated convolutional codeSCCC2.


for every pair of curves in the figure. The curves of Fig. 6do not show a divergence of the bound at higher forincreasing . This is due to the choice of a recursive encoderfor the inner code, which guarantees that all coefficientsdecrease with .

Code SCCC3 differs from SCCC1 only in the choice of anonrecursive outer encoder, which is a four-state encoder (seeTable II) with the same as for SCCC1, but with ,instead of . From the design conclusions, we expecta slightly better behavior of this SCCC. This is confirmed by


the performance curves of Fig. 11, which present the sameinterleaver gain as those of SCCC1, but have a slightly lower

(the curves for SCCC3 are translated versions of thoseof SCCC1 by 0.1 dB).

Code SCCC4 employs the same CC’s as SCCC2, but re-verses their order. It uses as outer code a ratenonrecursiveconvolutional encoder, and as inner code a raterecursiveconvolutional encoder. As a consequence, it has a lower

and a lower . Thus from (34), we expect alower interleaver gain than for SCCC1 and SCCC3, as .This is confirmed by the curves of Fig. 12, which, for a fixedand sufficiently large signal-to-noise ratio, show a decrease in

of a factor , when increases from to .On the whole, SCCC4 looses more than 2 dB in coding gainwith respect to SCCC3. This result confirms the design rulesuggesting the choice of an outer code with as largeaspossible.

Finally, let us consider code SCCC2, which differs fromSCCC1 in the choice of a nonrecursive inner encoder, with thesame parameters but with the crucial difference of . Itsbit error probability curves are shown in Fig. 10. They confirmthe predictions of Section III. We see, in fact, that for lowsignal-to-noise ratios, say below 4 dB, almost no interleavergain is obtained. This is because the performance is dominatedby the exponent , whose coefficient increases with.On the other hand, for larger signal-to-noise ratios, wherethe dominant contribution to is the exponent with thelowest value , the interleaver gain makes its appearance.From (24), we foresee a gain as , meaning four orders ofmagnitude for increasing from to . Curves in Fig.10 show a smaller gain (slightly higher than ), whichis, on the other hand, rapidly increasing with .

IV. COMPARISON BETWEEN PARALLEL

AND SERIALLY CONCATENATED CODES

In this section, we will use the bit error probability boundspreviously derived to compare the performance of parallel(“turbo codes,” [8]) and serially concatenated block and con-volutional codes.

A. Serially and Parallel Concatenated Block Codes

To obtain a fair comparison, we have chosen the followingPCBC and SCBC: the PCBC has parametersand employs two equal systematic cyclic codes withgenerator ; the SCBC, instead, isa SCCC, obtained by the concatenationof the Hamming code with a BCH code.

They have almost the same rate, and have been compared choosing the interleaver

length in such a way that the decoding delay due to theinterleaver, measured in terms of input information bits, isthe same. Since the interleaver acts on information bits forthe PCBC, its length is equal to times the length of theinformation word, i.e., . For the SCBC, instead, theinterleaver acts on the codewords of the outer code, and, asa consequence, to yield the same decoding delay measured asnumber of information bits, its length must be that of the PCBC


Fig. 13. Comparison of serially concatenated block codes and parallelconcatenated block codes with various interleaver lengths, chosen so as toyield the same input decoding delay.

divided by . As an example, to obtain a delay equal to 12input bits, we must choose an interleaver lengthfor the PCBC, and for the SCBC.

The results are reported in Fig. 13, where we plot the biterror probability bounds versus the signal-to-noise ratiofor various input delays. The results show that, for low valuesof the input delay, the performance is almost the same. On theother hand, increasing the delay (and thus the interleaver length

) yields a significant interleaver gain for the SCBC, andalmost no gain for the PCBC. The difference in performanceis 3 dB at in favor of the SCBC.

B. Serially and Parallel Concatenated Convolutional Codes

To obtain a fair comparison, we have chosen the followingPCCC and SCCC: the PCCC is a rate code obtainedconcatenating two equal rate , four-state systematic recur-sive convolutional encoders with generator matrix as in TableIII, first row. The SCCC is a rate code using as outer


Fig. 14. Comparison of serially concatenated convolutional codes and par-allel concatenated convolutional codes with four-state convolutional codes.

code the same rate four-state code as in the PCCC, and,as inner encoder, a rate , four-state systematic recursiveconvolutional encoder with generator matrix as in Table III,third row. Also in this case, the interleaver lengths have beenchosen so as to yield the same decoding delay, due to theinterleaver, in terms of input bits. The results are reported inFig. 14, where we plot the bit error probability versus thesignal-to-noise ratio for various input delays.

The results show the great difference in the interleaver gain.In particular, the PCCC shows an interleaver gain going as

, whereas the interleaver gain of the SCCC, as from (34),goes as , being the free distance of theouter code equal to. This means, for , a gainof more than 2 dB in favor of the SCCC.

Previous comparisons have shown that serial concatenationis advantageous with respect to parallel concatenation, in termsof ML performance. For medium-long interleaver lengths, thissignificant result remains a theoretical one, as ML decoding isan almost impossible achievement. For parallel concatenatedcodes (“turbo codes”), iterative decoding algorithms have beenproposed, which yield performance close to optimum, withlimited complexity. In the following section, we will present anew iterative decoding scheme capable of decoding seriallyconcatenated codes, and prove, with several examples ofapplications, that the performance gain with respect to parallelconcatenation is maintained.

V. ITERATIVE DECODING OF

SERIALLY CONCATENATED CODES

In this section, we present a new iterative algorithm fordecoding serially concatenated codes, with complexity notsignificantly higher than that needed to separately decodethe two CC’s. Because of the importance in applications,all examples will refer to SCCC’s, although the decodingalgorithm can be applied to SCBC’s as well.

The core of the new decoding procedure consists of a blockcalled SISO (Soft-Input Soft-Output). It is a four-port device,which accepts as inputs the probability distributions (or the

Fig. 15. Block diagrams of the encoder and iterative decoder for seriallyconcatenated convolutional codes.

corresponding likelihood ratios) of the information and codesymbols labeling the edges of the code trellis, and formsas outputs an update of these probability distributions basedupon the code constraints. The block SISO is used within theiterative decoding algorithm as shown in Fig. 15, where wealso show the block diagram of the encoder to clarify thenotations.

We will first explain in words how the algorithm works,according to the blocks of Fig. 15. Following that, we willgive the input–output relationships of the block SISO.

The symbols and at the input and out-put ports of SISO refer to the logarithmic likelihood ratios(LLR’s),10 unconstrained when the second argument is, andmodified according to the code constraints when it is.The first argument refers to the information symbols ofthe encoder, whereas refers to code symbols. Finally, thesuperscript refers to the outer encoder, andto the innerencoder. The LLR’s are defined as

(36)

When is a binary symbol, “” or “ ,” is generallyassumed to be the “.” When belongs to an -ary alphabet,we can choose as each one of the symbols.

Differently from the iterative decoding algorithm employedfor turbo decoding, in which only the LLR’s of informationsymbols are updated, we must update here the LLR’s of bothinformation and code symbols based on the code constraints.

During the first iteration of the SCCC algorithm,11 theblock “SISO Inner” is fed with the demodulator soft outputs,consisting of the LLR’s of symbols received from the channels,i.e., of the code symbols of the inner encoder. The secondinput of the SISO Inner is set to zero during the firstiteration, since noa priori information is available on the inputsymbols of the inner encoder.

The LLR’s are processed by the SISO algorithm,which computes theextrinsic LLR’s of the information sym-bols of the inner encoder conditioned on the innercode constraints. The extrinsic LLR’s are passed through the

10When the symbols are binary, only one LLR is needed; when the symbolsbelong to anL-ary alphabet,L� 1 LLR’s are required.

11To simplify the description, we assume for now that the interleaver actson symbols instead of bits. In the actual decoder, we deal with bit LLR’s andbit interleaver, as it will be seen later.


Fig. 16. Trellis section defining the notations used for the description of theSISO algorithm.

inverse interleaver (a block labeled “ ”), whose outputscorrespond to the LLR’s of the code symbols of the outercode, i.e.,

These LLR’s are then sent to the block “SISO Outer”in its upper entry, which corresponds to code symbols. TheSISO Outer, in turn, processes the LLR’s of itsunconstrained code symbols, and computes the LLR’s of bothcode and information symbols based on the code constraints.The input of the SISO Outer is always set to zero,which implies assuming equally likely transmitted source in-formation symbols. The output LLR’s of information symbols(which yield thea posterioriLLR’s of the SCCC informationsymbols) will be used in the final iteration to recover theinformation bits. On the other hand, the LLR’s of outer codesymbols, after interleaving are fed back to the lower entry(corresponding to information symbols of the inner code) ofthe block SISO Inner to start the second iteration. In fact, wehave

A. The Input–Output Relationships for the Block SISO

The block SISO has been described in [19]. It represents aslight generalization of the BCJR algorithm (see [20], [21],and [24]). Here, we will only recall for completeness itsinput–output relationships. They will refer, for notations, tothe trellis section of the trellis encoder, assumed to be time-invariant as we deal with convolutional codes, shown inFig. 16, where the symbol denotes the trellis edges, andwhere we have identified the information and code symbolsassociated to the edgeas , and the starting andending states of the edgeas , respectively.

The block SISO works atsymbollevel, i.e., for anconvolutional code, it operates on information symbolsbelonging to an alphabet with size and on code symbolsbelonging to an alphabet with size . We will give thegeneral input–output relationships, valid for both outer andinner SISO’s, assuming that the information and code symbolsare defined over a finite time index set .

At time , the output extrinsic LLR’s arecomputed as

(37)

(38)

The nameextrinsic given to the LLR’s computed accordingto (37) and (38) derives from the fact that the evaluationof (and of ) does not depend on thecorresponding simultaneous input (and ), sothat it can be considered as an update of the input LLR basedon information coming from all homologous symbols in thesequence, except the one corresponding to the same symbolinterval.

The quantities and in (37) and (38) are obtainedthrough theforward andbackwardrecursions, respectively, as

(39)

(40)

with initial values

otherwise

otherwise.

The quantities in (37) and (38) are normalizationconstants.

The operator performs the following operation:

e (41)

This operation, a crucial one in affecting the computationalcomplexity of the SISO algorithm, can be performed inpractice (see [22] and [23]) as

(42)

where is a correction term that can becomputed recursively using a single-entry lookup table [22],[23].

The previous description of the iterative decoder assumedthat all operations were performed atsymbol level. Quiteoften, however, the interleaver operates atbit level to be moreeffective. This is the case, for example, of all results presentedin Sections I–III.

Thus to perform bit interleaving, we need to transform thesymbol extrinsic LLR’s obtained at the output of the first SISO


into extrinsic bit LLR’s, before they enter the de-interleaver.After de-interleaving, the bit LLR’s need to be compacted intosymbol LLR’s before entering the second SISO block, and soon. These operations are performed under the assumption thatthe bits forming a symbol are independent.

Assuming an code, and denoting withthe information symbol formed by information

bits, the extrinsic LLR of the th bit within the symbolis obtained as

(43)

Conversely, the extrinsic LLR of the symbolis obtainedfrom the extrinsic LLR’s of its component bits as

(44)

The previous description should have made clear that theSISO algorithm requires the whole sequence to be receivedbefore starting. The reason is due to the backward recursionthat starts from the (supposed known) final trellis state. Asa consequence, its practical application is limited to the casewhere the duration of the transmission is short (small), or,for long, when the received sequence can be segmentedinto independent consecutive blocks, like for block codes orconvolutional codes with trellis termination [14]. It cannot beused for continuous decoding. This constraint leads to a framerigidity imposed on the system, and also reduces the overallcode rate, because of trellis termination.

A more flexible decoding strategy is offered by modifyingthe algorithm in such a way that the SISO module operates ona fixed memory span, and outputs the smoothed probabilitydistributions after a given delay . This algorithm, whichwe have called thesliding window soft-input soft-output(SW-SISO) algorithm, is fully described in [23]. In the followingsimulation results, the SW-SISO algorithm has been applied.

B. Applications of the Decoding Algorithm

We will now use the decoding algorithm to confirm thedesign rules presented before, and to show the behavior ofSCCC in the region of low signal-to-noise ratios (belowcutoff rate). Since in this region analytical bounds fail to givesignificant results, no meaningful quantitative comparisons canbe performed between simulated and analytical performance.However, we will show that the hierarchy of the simulationresults agrees with the design considerations that had beenbased on the analysis.

The following aspects will be considered:

• behavior of the decoding algorithm versus the number ofdecoding iterations;

• behavior of the decoding algorithm versus the interleaverlength;

• the effect of choosing a nonrecursive inner code;• SCCC behavior for very low signal-to-noise ratios, to

see how close serial concatenation can get to theoreticalShannon bound;

Fig. 17. Simulated bit error probability versus the number of iterations fora rate1=3 serially concatenated convolutional code obtained concatenating a4-state rate1=2 recursive convolutional code and a4-state rate2=3 recursiveconvolutional code. The concatenated code is the code SCCC1 of Table III.The decoding delay in terms of input bits due to the interleaver is 1024.

• comparison between SCCC’s and PCCC’s (turbo codes)for the same value of decoding delay imposed by the twoschemes on the input bits.

For all simulated SCCC’s, we have used purely randominterleavers.

1) Simulated Coding Gain Versus Number of Iterations:Consider the rate SCCC1 of Table II. It employs two

-state recursive convolutional encoders, the first (outer code)with rate and the second (inner code) with rate , joinedby an interleaver of length . Since the interleaveroperates on coded sequences produced by the outer rateencoder, its length of 2048 bits corresponds to a delay of 1024information bits. The simulation results are shown in Fig. 17in terms of bit error probability versus for a numberof iterations ranging from 1 to 7. The nice convergence ofthe decoding algorithm is manifest.

2) The Effect of a Nonrecursive Inner Encoder:The anal-ysis of Section III came to the conclusion that a nonrecursiveinner encoder should yield little interleaver gains. To confirmthis theoretical prediction by simulation results, we plot inFig. 18 the bit error probability versus the input decodingdelay obtained by simulating the concatenated code SCCC2of Table III. This code uses as inner encoder a-statenonrecursive encoder. The curves refer to a signal-to-noiseratio dB, and to a number of iterationsranging from 1 to 10. It is evident that the bit error probabilityreaches the floor of for a decoding delay greater than orequal to 1024, so that no interleaver gain takes place beyondthis point. For comparison, we report in Fig. 19 the resultsobtained for the code SCCC3 of Table III. The curves referto a signal-to-noise ratio of 0.75 dB, and show the interleavergain predicted by the analysis.

3) Approaching the Theoretical Shannon Limit:We showhere the capabilities of SCCC’s of yielding results close tothe Shannon capacity limit. To this purpose, we have chosena rate concatenated scheme with very long interleaver,


Fig. 18. Simulated performance of concatenated code SCCC2 of Table III.The bit error probability is plotted versus input decoding delay for differentnumber of iterations. The signal-to-noise ratio is 1.5 dB.

Fig. 19. Simulated performance of concatenated code SCCC3 of Table III.The bit error probability is plotted versus input decoding delay for differentnumber of iterations. The signal-to-noise ratio is 0.75 dB.

corresponding to an input decoding delay of 16 384. Theconstituent codes are-state codes: the outer encoder isnonrecursive, and the inner encoder is a recursive encoder.Their generating matrices are

respectively. Note the feedback polynomial in thegenerator matrix of the inner encoder, which eliminates errorevents with odd input weights. The results in terms of bit errorprobability versus signal-to-noise ratio for different number ofiterations are presented in Fig. 20. They show that the decodingalgorithm works at 0.05 dB, at 0.76 dB from theShannon capacity limit (which is in this case equal to0.817dB), with very limited complexity (remember that we are usingtwo rate codes with eight states).

Fig. 20. Simulated performance of a rate1=4 serially concatenated codeobtained with two eight-state constituent codes and an interleaver yielding aninput decoding delay equal to 16 384.

4) Comparison Between Serially and Parallel ConcatenatedCodes: Previous analytical results showed that serial con-catenation can yield significantly higher interleaver gains andsteeper asymptotic slope of the error probability curves. Tocheck if these advantages are retained when the codes areiteratively decoded at very low signal-to-noise ratios, we havesimulated the behavior of SCCC’s and PCCC’s in equal systemconditions: the concatenated code rate is, the CC’s are -state recursive encoders (rates for PCCC’s, andrates for the SCCC’s), and the decoding delays interms of input bits are 256, 1024, and 16 384, respectively. InFig. 21, we report the results, in terms of bit error probabilityversus signal-to-noise ratio, for the case of a decoding delayequal to 256, after three and nine decoding iterations. As it canbe seen from the curves, the PCCC outperforms the SCCCfor high values of the bit error probabilities. Below(for nine iterations), the SCCC behaves significantly better,and does not present the floor.12 The improvement of SCCCwith respect to PCCC becomes more visible in Fig. 22, whichrefers to a delay of 1024 and to three and seven iterations ofthe decoding algorithm. For bit error probabilities lower than

, the SCCC outperforms PCCC. In particular, the absenceof error floor is striking here. At , SCCC has an advantageof 0.7 dB with seven iterations. Finally, in Fig. 23, we reportthe results for an input decoding delay of 16 384 and six andnine decoding iterations. In this case, the crossover betweenPCCC and SCCC happens around . The advantage ofSCCC at is 0.5 dB with nine iterations.

As a conclusion, we can say that the advantages obtainedfor signal-to-noise ratios above the cutoff rate, where the unionbounds can be safely applied, are retained also in the regionbetween channel capacity and cutoff rate. Only when thesystem interest focuses on high values of bit error probability(the threshold depending on the interleaver size) the PCCCare to be preferred. PCCC’s, however, present a floor to thebit error probability, which, in the most favorable case seen

12It is customary in the turbo codes literature to call “error floor” what isactually a sensible change of slope of the performance curve.


Fig. 21. Comparison of two rate1=3 parallel concatenated convolutionalcode and serially concatenated convolutional code. The parallel concatenatedconvolutional code is obtained concatenating two equal rate1=2 4-state codes(first code in Table III); the serially concatenated convolutional code is thecode SCCC1 of Table II. The curves refer to three and nine iterations of thedecoding algorithm and to an equal input decoding delay of 256.

Fig. 22. Comparison of two rate1=3 parallel concatenated convolutionalcode and serially concatenated convolutional code. The parallel concatenatedconvolutional code is obtained concatenating two equal rate1=2 4-state codes(first code in Table III); the serially concatenated convolutional code is thecode SCCC1 of Table II. The curves refer to three and seven iterations of thedecoding algorithm and to an equal input decoding delay of 1024.

above, lies around . This floor is absent, or, at least, muchlower, in the case of SCCC.

Finally, it must be recognized that the constituent codesdesign rules presented in Section III are based on union boundconsiderations, and thus yield optimum SCCC’s above thecutoff rate. For system applications aiming at very low signal-to-noise ratios, close to the channel capacity (as, for example,in deep-space communications), a general statement is thatcomplex CC’s should be avoided, and CC’s with low numberof states (4–16) should be used. So far, finding the best codesto operate in the region of very low signal-to-noise ratios, hasbeen a matter of feeling dictated by experience and, literally,case-by-case “crafting.”

Fig. 23. Comparison of two rate1=3 parallel concatenated convolutionalcode and serially concatenated convolutional code. The parallel concatenatedconvolutional code is obtained concatenating two equal rate1=2 4-state codes(first code in Table III); the serially concatenated convolutional code is thecode SCCC1 of Table II. The curves refer to six and nine iterations of thedecoding algorithm and to an equal input decoding delay of 16 384.

VI. CONCLUSIONS

Serially concatenated codes with interleaver, a conceptbuilding on classical concatenated codes and parallel con-catenated codes known as “turbo codes,” have been studied.They consists of the cascade of anouter encoder, an in-terleaver permuting the outer codewords bits, and aninnerencoder whose input words are the permuted outer codewords.Upper bounds to theaveragemaximum-likelihood bit errorprobability of serially concatenated block and convolutionalcoding schemes have been derived. Based on those bounds,we have derived design guidelines for the outer and innercodes that maximize theinterleaver gainand the asymptoticslope of the error-probability curves. It has been shown thatthe interleaver gain, defined as the factor that decreases thebit error probability as a function of the interleaver size, canbe made significantly higher than for turbo codes. Finally, anew, low-complexity iterative decoding algorithm that yieldsperformance close to the Shannon limit has been illustrated.Extensive examples have been presented, and comparisonswith parallel concatenated convolutional codes have been per-formed. The simulation results show that, in general, SCCC’shave significantly lower changes of slope in the bit errorprobability curves than PCCC’s. On the other hand, if the biterror probabilities achievable by PCCC’s above the changeof slope are sufficiently low, they can be reached by PCCC’sat slightly lower signal-to-noise ratios than by SCCC’s. Asa conclusion, PCCC’s seem thus better suited to approachingcapacity limits for bit error probabilities above the error floor( – ), whereas SCCC’s are better suited to providenear error-free performance.

REFERENCES

[1] G. D. Forney, Jr.,Concatenated Codes.Cambridge, MA: MIT Press,1966.

[2] R. H. Deng and D. J. Costello, “High rate concatenated coding systemsusing bandwidth efficient trellis inner codes,”IEEE Trans. Commun.,vol. 37, pp. 420–427, May 1989.


[3] J. Hagenauer and P. Hoeher, “Concatenated Viterbi decoding,” inProc.4th Joint Swedish-Soviet Int. Workshop on Information Theory(Gotland,Sweden, Studenlitteratur, Lund, Aug. 1989), pp. 29–33.

[4] C. Berrou, A. Glavieux, and P. Thitimajshima, “Near Shannon limiterror-correcting coding and decoding: Turbo-codes,” inProc. ICC’93(Geneva, Switzerland, May 1993), pp. 1064–1070.

[5] S. Benedetto and G. Montorsi, “Serial concatenation of block andconvolutional codes,”Electron. Lett., vol. 32, no. 10, pp. 887–888, May1996.

[6] , “Iterative decoding of serially concatenated convolutionalcodes,”Electron. Lett., vol. 32, no. 13, pp. 1186–1187, June 1996.

[7] S. Benedetto, E. Biglieri, and V. Castellani,Digital TransmissionTheory. Englewood Cliffs, NJ: Prentice-Hall, 1987.

[8] S. Benedetto and G. Montorsi, “Unveiling turbo-codes: Some resultson parallel concatenated coding schemes,”IEEE Trans. Inform. Theory,vol. 42, pp. 409–429, Mar. 1996.

[9] G. Poltyrev, “Bounds on the decoding error probability of binary linearcodes via their spectra,”IEEE Trans. Inform. Theory, vol. 40, pp.1284–1292, July 1994.

[10] A. J. Viterbi and J. K. Omura,Principles of Digital Communication andCoding. New York: McGraw-Hill, 1979.

[11] T. M. Duman and M. Salehi, “New performance bounds for turbocodes,” inProc. GLOBECOM’97(Phoenix, AZ, Nov. 1997).

[12] A. J. Viterbi, A. M. Viterbi, and J. Nicolas, “Perspectives on inter-leaved concatenated codes with iterative soft-output decoding,” privatecommunications, May 1997.

[13] S. Benedetto and G. Montorsi, “Design of parallel concatenated con-volutional codes,”IEEE Trans. Commun., vol. 44, pp. 591–600, May1996.

[14] D. Divsalar and F. Pollara, “Turbo codes for PCS applications,” inProc.ICC’95 (Seattle, WA, June 1995).

[15] S. Benedetto and G. Montorsi, “Average performance of parallel con-catenated block codes,”Electron. Lett., vol. 31, no. 3, pp. 156–158,Feb. 1995.

[16] G. D. Forney, Jr., “Convolutional codes I: Algebraic structure,”IEEETrans. Inform. Theory, vol. IT-16, pp. 720–738, Sept. 1970.

[17] D. Divsalar and R. J. McEliece, “Effective free distance of turbo codes,”Electron. Lett., vol. 32, no. 5, Feb. 1996.

[18] P. Thitimajshima, “Systematic recursive convolutional codes and theirapplication to parallel concatenation,” Ph.D. dissertation, Univ. deBretagne Occidentale, Dec. 1993, in French.

[19] S. Benedetto, D. Divsalar, G. Montorsi, and F. Pollara, “A soft-inputsoft-output APP module for iterative decoding of concatenated codes,”IEEE Commun. Lett., vol. 1, no. 1, pp. 22–24, Jan. 1997.

[20] L. R. Bahl, J. Cocke, F. Jelinek, and J. Raviv, “Optimal decoding oflinear codes for minimizing symbol error rate,”IEEE Trans. Inform.Theory, pp. 284–287, Mar. 1974.

[21] R. J. McEliece, “On the BCJR trellis for linear block codes,”IEEETrans. Inform. Theory, vol. 42, pp. 1072–1091, July 1996.

[22] P. Robertson, E. Villebrun, and P. Hoeher, “A comparison of op-timal and sub-optimal MAP decoding algorithms operating in thelog domain,” in Proc. ICC’95 (Seattle, WA, June 1995), pp. 1009–1013.

[23] S. Benedetto, D. Divsalar, G. Montorsi, and F. Pollara, “Soft-inputsoft-output building blocks for the construction and distributed iterativedecoding of code networks,”Europ. Trans. Telecommun., April 1998,invited paper.

[24] J. Hagenauer, E. Offer, and L. Papke, “Iterative decoding of binaryblock and convolutional codes,”IEEE Trans. Inform. Theory, vol. 42,pp. 429–445, Mar. 1996.

Documents

Serial Concatenation of Interleaved Codes: … Concatenation of Interleaved Codes: Performance ... allel concatenated convolutional codes known as “turbo codes ... for those applications