Algorithmic Complexity in Coding Theory

and the Minimum Distance Problem

Alexander Vardy

Coordinated Science Laboratory

University of Illinois

1308 W. Main Street, Urbana, IL 61801

vardy@golay. csl .uiuc .edu

Abstract. We startwithan overviewof algorithmiccomplexityproblemsin coding theory We then show that the problemof com-puting the minimumdiktanceof a binaryIinwr code is NP-hard,andthe correspondingdeci~”onproblemis W-complete. Thisconstitutesa proof of the conjecture Bedekamp, McEliece,vanTilborg, datingback to 1978. Extensionsand applicationsof this result to otherproblemsin codingtheqv are discussed.

1. Introduction. This paper is organized in a mannercontrapositive to the talk. In the talk, we give a detailedoverview of algorithmic complexity issueain codhg theory.Herein,westart with an overviewof the prominentrole thatcomputational complexity plays in coding theory: we brieflysurveythe numerousalgorithmic complexity issuesthat arisein code construction and decoding. Subsequently,we focuson one specific problem. Namely, we consider the problemof computing the minimum distance of a linear code, whichis also equivalentto the problem of finding the shortestcyclein a linear matroid over a finite field [90]. A long-standingconjecture [12] says that this problem is NP-hard, and wewill settle this conjecture in the ai%rmative. We providea detailed proof. We hope that our proof illustrates someof the elegant techniques used in coding theory today, suchas the construction of MDS codes via Vandermondematri-ces [15, 59, 72]and concatenated coding [23,31], for instance.It is interesting that algebraic techniques of this kind cartbe employed to answer an essentially combinatorial ques-tion. The complexity of computing the minimum distanceand the proof of the conjecture of Berlekamp,McEliece, andvanTilborg [12] will be only briefly mentioned in the talk.

2. Complexity in coding theory

In discussing the close ties between coding theory and com-plexity theory, there are two general categories of naturalquestions: one involving the application of codes to compu-tational complexity and the other focusing on the computa-tional complexity of coding itself. In the first category,codeshavebeen usedextensivelyboth to deviseefficientalgorithmsin a varietyof contexts and to prove that no such algorithms

“This workwas supportedby the Packard Foundation, the Na-tional Science Foundation, and the JSEP grant NOOO14-961O129.

Permission 10 nmkc digiull/ll:lrd ct~pics OFall or p;lll ol’lhis maleri:ll lilr

pemomd or clmwoonl we is grantixi $vitllwl (LCprovided IIml the coplcs,arenot madeor di.slnlw[cd I’orprolil or cwnmercial ;idvoi~[ogc.[he cop},-

right noliw. lhe title ot Illc p(llll]c:llitm find 11sd:llc nppcar, atld oo[ice is

exist. For example, the elegant theory of probabiliiticrdlycheckableproofs [6] uses algebraic codes in an essentialwayto resolve Iong-staudIngopen questions about the hardnessof approximation for such basic optimization problems asgraph-coloring and clique-size. This mea of research has bynow accumulated a sizable body of interestingresults. How-ever, we will not even attempt to discuss the use of codes incomputational complexity. We refer the reader to [4, 6, 79],and especially [27], for a detailed overviewof this subject.

On the other hand, in what follows, we brieflysurveysome ofthe fascinating computational problems that arise in codingtheory itself. We will try to make this overview accessibleto as broad an audience as pcsssible,and will not assumeany prior knowledge in coding theory. Our overview is byno means comprehensive;when [10] is finally ready, a moredetailed survey would hopefully become available.

The bkth of the subject of codkg for data transmission oc-curredat the time of the announcementof Shannon’s codingtheorems [73], which not only established the limits of thegains possible with codktg, but also proved the existence ofcodes that could effectivelyreach these limits. Shannon [73]showed that for every communication channel, there is aconstant C, called the copocity of the channel, which has thefollowing fundamentalsignificance: if one wishes to commu-nicate over the channelat a rate R (in bits per channel use),then one can do so as reliably as desired, if and only if R < C.Specifically, for every & >0 there exists a sufficiently longerror-correctingcode C of rate R, such that the probabilityof error in maximum-likelihood decoding of C is at most e.

It was recognized early on that the trouble with Shannon’scoding theorems, from a practical point of view, is essentiallycomputational in nature. Although Shannon [73]settled thequestion “Do good codes exist?” in the atlirmative,his workled to two other questions “How can we find such codes?”and “How can we decode them?”. In a sense, coding theory is all about these two questions, and both questions arefundamentallycomputational.

It is trivial to find the codes promised by Shannon usinga superexponentialsearch, but in practice we would like toconstruct a code in polynomial time. This leads to the prob-lem of code construction, which is d~cumed in the next sutr-section. Furthermore, Shannon used maximum-likelihood

givw thal wp~iglll is h! permission ol’llle ,-\~h[. Im. ‘1’0topy olhcrw isc.

tO republish, 10 posl011scllws or (0dislnhulc [0 Iisls. t’cqllires Spccilicpermissionand/or Lx

.W()(”” ‘ 97 El Paso, “1-cxnsLis<\

Copyrighl 1997 ACM 0-8979 I -XXX-(W71(J5 ..$3.50

92

decoding in the proof of his theorems [73], which can betrivially accomplished by an exponential brute-force search,but in practice we would like to decode more efficiently. Thisleads to the questions of complexity of decoding, which wediscuss in more detail in a later subsection.

2.1. Complexity of code construction

A code C of length n over the finite field IFq is a subsetof IF;, the vector space of n-tuples over lF~. The primaryexample of a finite field, in theory aa well aa in practice, isIFz = {O, 1}, and codes over IFz are called binary. A codeis said to be linear if it is a subspace of IF:, and we willbe concerned with linear codes unless stated otherwise. Therate of a linear code C ~ lF~nis defined as R = k/n, wherek = dim C Given a code C of length n and dimension kover IFg, we can encode an arbitrary information word u 6 IF}into a codeword c E C via a one-to-one mapping S : IF) ~ Ccalled an encoder for C. Thus the rate R is the numberof information symbols per code (channel) symbol, and wewould like the rate to be high.

A linear code C can be compactly specified as the row-spaceof a k x n generator matrix G or as the kernel of a (n – k) x nparity-check matrix H. Both G and H are usually assumedto have full row-rank, and can be computed from each otherin time 0(n3 ) using standard linear algebra techniques. No-tice that a generator matrix G also specifies an encoder for Cvia &(u) = UG. Thus encoding is easy, it can be always im-plemented in time 0(n2) as a vector-matrix product.

The error-correction capability of a code has to do with itsminimum Hamming distance

where the second equality is true for linear codes. Here, theHamming distance d(z, y) is the number of positions wherez and y differ, while the Hamming weight wt(z) = d(z, O)is the number of nonzero positions in z. Thus an error-correcting code can be viewed m a packing of disjoint spheresof radius t= l(d–1)/2j in the space IF; endowed with theHamming metric. If codewords of C are transmitted over anoisY channel, then errors in any < tpositions may be cor-rected at the receiver end by identifying the unique sphereto which the error-corrupted channel output belongs. Ev-idently, we would like t, and hence also d, to be large. Itshould be obvious that attaining a high rate R = log~ 1~/nand a large distance d are conflicting goals. Hence the codesof interest for communications are those that achieve a goodtradeoff between these two parameters.

Reed-Solomon codes. Here is an example of a simplepolynomial-time construction that produces excellent codes.As is often done in coding theory, we will specify a code Cby describing an encoder for C. For each information wordu = (Uo,ul,. ... Uk_ ~) c F;, we first consider the polyno-

k–1mid ~.(Z) = Uk-1$ + ~~. + UIZ + UO. Then the code-word c=(cl, cz, ..., ~) E C of length n = g – 1 correspond-ing to u is the evaluation of /U(z) at all the nonzero ele-ments crl, az, . . . . crg-..l of the field lF~, that is c~ = fw(ai).

Since a nonzero polynomial of degree s k – 1 vanishes forat most k – 1 elements of JFg, we immediately obtain that

d z n – (k – 1) in view of (l). On the other hand, it is notdifficult to show that d < n – k + 1 for any linear code. Thisresult is known as the Singleton bound [59, p.33], and codesthat attain this bound with equality are called maximum-distance separable (MDS). MDS codes are related in manyways to various problems in combinatorics and linear alge-bra (cf. [59, Chapter 11]), and the codes we have just con-structed, known as Reed-Solomon codes [71], constitute theprimary example of MDS codes. This is also an example ofthe general class of polynomial codes, used extensively in thetheory of probabilistically checkable proofs [4, 27, 79]. Reed-Solomon codes can be found today in almost every household(compact disk players use these codes) and on the outskirtsof the solar system (in the hardware of the Voyager probe).

A major problem with Reed-Solomon codes is that they areshort. The one hint given to us by Shannon [73], with regardto code construction, is that achieving channel capacity caflsfor long codes, namely n should be large. On the other hand,for practical considerations, we would like to keep the size qof the code alphabet fixed and small, preferably q = 2. Yet,for Reed-Solomon codes n ~ q. There are several ways ofmodifying the original Reed-Solomon construction [71] toalleviate this problem, as discussed in what follows.

Algebraic-geometry codes. One option is to consider poly-nomials in more than one variable. For example, a poly-nomial ~(z, y) over lF~ can be evaluated at up to q2 dis-tinct points, leading to a code of length n s q2 rather thann ~ q. The problem here is that the known bounds onthe number of points in the plane lt?~x IFq at which such apolynomial might vanish lead to rather weak codes. Thesebounds become much stronger if we evaluate polynomialsalong an algebraic curve. For example, if q = T2, we can eval-uate f(z, y) at the rational points of the Hermitian curve,namely at all the solutions (z, y) c Fq x Fq to the equation~T+l – y’ – y = O. It is easy to see that there are exactly r3distinct solutions, and this procedure produces the Hermi-tian code of length n = r3 > g which is nearly MDS. Namelyd = (n– k+l) –g, where g = l/2r(r– 1) is the genus of thecurve. This is a simple example of a generaf type of codes,known aa algebraic-geometry codes [83, 84].

Algebraic-geometry codes are arguably the most powerfulcodes known today. These codes are constructible in polyno-mial time, but the complexity of constructing the best codesin this class tends to be too high for practiczd purposea:0(n30) was originally reported in [60], which was recentlyimproved to 0(n17 ) in [56]. A construction that would pro-duce a generator matrix for such codes in time 0(n3 ), say,would be a significant achievement, and there is reason tobelieve that a result of this nature is forthcoming [70].

Another drawback of algebraic-geometry codes is that theirsymbol alphabet is still quite large. Algebraic-geometryconstructions are successful (in particular, better than theGilbert-Varshamov bound discussed below) only for q >49.

BCH codes. On the other hand, here is how one can getbinary codes from Reed-Solomon codes: given an (n, k, d)Reed-Solomon code C over I13m, let C be the (n, k*, d*)

93

binary subfield subcode of C, that is C’ = Cfl IF;. This pro-duces the important class of binary BCH codes [59, Chap-ter 9]. These codes are easily constructible in polynomialtime, as we can compute a parity-check matrix for the BCHcode O in time O(n log n) from a parity-check matrix forthe Reed-Solomon code C. It is obvious that d* ~ d butk“ < k, and the question is whether a code with param-eters (n, k“, d*) is still a good code. It turns out that forshort lengths, say up to n < 128, BCH codes are amongthe best binary codes known. However, as n -+ cc, eitherd*/n or k* /n tend to zero. The latter result, due to Lin andWeldon [54], shows that BCH codes are asymptotically bad.

Asymptotically good codes. This brings us to the subject ofasymptotic properties of codes. For i = 1, 2, ..., let G be an(~i, ki, di) linear code over a fixed field IFq. The infinite se-quence of codes Cl, C2,. . . is said to be asymptotically goodif ni + cm with i, while kl/n~ + R and d~/n~ ~ b for somenonzero R and 6. Thus all we require is that the rate andthe relative distance are both asymptotically non-vanishing.This defining property being so generous, it is rather sur-prising that it took nearly 25 years to come up with thefirst example of an asymptotically good sequence of binarycodes that is constructible in polynomial time [46]. The ba-sic idea of Justesen [46] was a clever use of a concatenatedcoding technique [31], which is described in detail in thenext section. Today, many such constructions are known:Zyablov codes [94], codes constructed by Shen [74] fromHermitian curves using a variation of Justesen’s concatena-tion, and codes constructed using expander graphs by Alonet al. in [3] are just a few examples. Asymptotically good bi-nary codes with polynomial-time complexity of constructionand the best known parameters were obtained by Vl&du~,Katsman, and Tsfasman in [89]. The parameters of thesecodes are depicted in Figure 1. The approach of [89] com-bines powerful algebraic-geometry codes, constructed fromDrinfeld’s modular curves, with concatenated coding.

0.5

04JPL upparbound

Gilbott.Varehamovlowerbound(expopantial-tknacqratruclion)

0.2

Waduta-KatamawTafaanuncodaa(polvmmlal-tknaconstruction)00 0.25 0.5 0.75

‘R

Figure 1: Parameters of mymptotically good binary codes

The Gilbert-Varshamov bound. Not surprisingly, if we aflowexponential-time complexity of construction, things becomemuch easier: a simple greedy algorithm works and produces

parameters R and J much better than those of [89]. Since alinear code C is the kernel of its parity-check matrix H bydefinition, the minimum distance of C is just the minimumnumber of linearly dependent columns of If, in view of (1).We construct an (n – k) x n binary parity-check matrix Hwith the property that every d – 1 columns of H are linearlyindependent, column-by-column, using greedy search. Afteri columns of H have been already chosen, there are at most

(:)+o+”+(’adistinct linear combinations of these i columns, taken d – 2or fewer at a time. If this number is less than 2n-k – 1we can always find another nonzero column, different fromthese linear combinations, and append it to H. We cankeep doing this, and complete the construction of all the ncolumns in H, provided

1+C1)+(3+”””+(0<2“-’‘2)Thus if (2) holds, we can construct an (n, k, d) binary codein exponential time. This simple result is known as theGilbert-Varshamov bound. Further evaluating this boundfor n + co produces asymptotically good binary codeswhose parameters R and d lie on the curve R = 1 – H2(8),where H2(z) = zlog2 z-1+ (1–z)log2(l–x)-l is the binaryentropy function. A long-standing conjecture says that thiscurve is the best possible for binary codes. Proof of this con-jecture is one of the major open problems in coding theorytoday. Such a proof would establish a rather remarkable factthat the simple greedy algorithm described above is (asymp-totically) the best possible construction method for binarycodes, regardless of the available computational resources.

An even more significant achievement, at least for practicalpurposes, would be a polynomial-time construction of bi-nary codes whose parameters lie on the Gilbert-Varshamovcurve R = 1 – Hz(6). A considerable amount of effort hasbeen devoted to this problem, which nevertheless remainsopen. The lack of such construction is particularly surpris-ing in view of the fact that almost all linear codes attainthe Gilbert-Varshamov bound [83, p.77]. Thus it is easyto devise randomized algorithms that with high probabilityproduce codes lying on the curve R = 1 – H2 (6), Such algo-rithms could indeed be used for code construction, if therewere a polynomial-time procedure for computing the mini-mum distance of a general linear code. However, as we willshow in the next section, computing the minimum distanceof linear codes is NP-hard.

In this subsection, we discussed the complexity of code con-struction, while completely ignoring the complexity of de-coding the resulting codes. Although, historically, this hasafso been the prevailing approach in coding theory for sometime, it is worth emphasizing that the two problems – codeconstruction and decoding – are closely related to each other:we would generally like to have a good code which can bealso efficiently decoded. In this context, it might be appro-priate to mention the recent work of Spielman [76, 77] whoconstructs asymptotically good codes that can be decodedin linear time. The approach of [76, 77] is based on certain

94

results from the theory of expander graphs, and it would bedesirable to resolve the random nature of these results. Wedefer a more detailed discussion of the expander codes ofSpielman [76, 77] to the following subsection.

2.2. Complexity of decoding

Suppose that a codeword c of a linear code C c lJ?~is trans-mitted over a noisy communication channel with output al-phabet A, and a vector y E An is observed at the channeloutput. Decoding is the task of trying to determine what cwas, given y, ~ and a probabilistic model for the channel.There is a large number of various approaches to this task,which can be roughly classified into four categories, accord-ing to whether they are maximum-likelihood or bounded-distance, hard-decision or soft-decision. Of course, manyuseful decoding algorithms do not fall squarely into eithercategory, as we discuss later in this subsection.

Hard-decision vs. soft-decision decoding. The terms “hard”and “soft” decision are largely historical; in our context theseterms have to do with the channel model assumed for decod-ing purposes. In both cases, the channel is usually assumedto be memoryless. This means that the noise is an i.i.d.random process: the probability of error is the same at alltimes (codeword positions), and what happens at one timeis independent of what happens at all other times.

To illustrate the difference between hard and soft decisiondecoding we will, for the sake of simplicity, restrict our at-tention to bhmry codes. That is, we consider the case wherethe channel input is {O, 1}. In hard-decision decoding, thechannel output A is also {O, 1}. The most useful modelin this case is the binary symmetric channel: a bit at thechannel input is either transmitted as is, or inverted with afixed probability p. In soft-decision decoding, the channeloutput A is a large set, most often A is the real line. Thechannel is characterized by two known probability-densityfunctions fo(.) and fl (), where f,(a) is the probability ofreceiving a E A given that z ~ {O,1} was transmitted. Thisis a very general channel model. The most useful specialcase, known as the additive white Gaussian noise (AWGN)channel, is when fO(.) and fl (.) are the Gaussian dktribu-tions IV(+1, a2) and N(–-1, Oz), respectively. In this case, itis convenient to think of codewords as embedded in the Eu-clidean space R“ via the mapping {O,1} ~ {+1, –l}. Underthis mapping, a binary code C of length n becomes a subsetof the 2“ vertices of the hypercube [+1, —1]”. Furthermore,the logarithm of the probability (log-likelihood) of receivingy E R“ becomes proportional to the squared Euclidean dis-tance from y to the transmitted codeword c E C c [+1, –1]’.

Usuafly, soft-decision decoding is a much more challengingtask than hard-decision decoding, but the potential rewardsare great. Loosely speaking, for a given code, maximum-likelihood soft-decision decoding requires twice less energyper bit than maximum-likelihood hard-decision decoding, toachieve the same probability of decoding error [17].

Maximum-likelihood vs. bounded-distance decoding. Giventhe channel output y E An, the optimal decoding strategy is

to find the codeword c E C that maximizes the probabilityPr{c]y} that c was transmitted given that y was received.We may usually assume, without loss of generality, that thecodewords of C are transmitted with equal a priori probabil-ity l/l Cl. In this case, by a simple application of the Bayesrule, the optimal decoding strategy is equivalent to findingthe most likely codeword c E C that maximizes the probabil-ity Pr{ylc} that y would be received if c was transmitted.A decoder for C that afways finds the most likely codeword(or one of the most likely codewords, if there are ties) is saidto be a maximum-likelihood decoder.

As discussed above, maximum-likelihood decoding is a com-putational task that means different things for differentchannels. On a binary symmetric channel, the most likelycodeword is obviously the one closest to y G IF; in theHamming metric. In fact, the same conclusion easily ex-tend to the more general case of g-ary symmetric channels.Thus hard-decision maximum-likelihood decoding is a near-est neighbor search in the space IF; endowed with the Ham-ming metric. On the other hand, it is not difficult to showthat soft-decision maximum-likelihood decoding (for binarycodes) is equivalent to finding the codeword c c C whichmaximizes the log-likelihood sum M(c), given by

n

M(c) = ~(-l)c’pi (3)i=l

where pi = log ~O(yi) – log ~1(yi) is the log-likelihood ra-

tio for the i-th position. A similar expression exists formaximum-likelihood soft-decision decoding of non-binarycodes. In the important special case of AWGN channels,maximizing (3) again reduces to the nearest neighbor search,except that now C is viewed as a subset of [+1, – l]” and thesearch is in the space R“ endowed with the Euclidean met-ric. Thus in most (but not all) cases, maximum-likelihooddecoding is equivalent to nearest neighbor decoding.

Although maximum-likelihood decoding is the optimal de-coding strategy, it is NP-hard for the general class of linearcodes [12]. Moreover, polynomial-time maximum-likelihooddecoding algorithms are not known today for any specificfamily of useful codes, such as the binary BCH codes, forinstance. Thus we are often interested in decoding strate-gies that are sub-optimal, but permit more efficient decod-ing algorithms. The major example of such decoding strat-egy is bounded-distance decoding. A decoder is said to bebounded-distance if there exists a constant t >0 such thatthe decoder always finds the closest codeword to a channeloutput y, provided the distance from y to that codeword is atmost t. There is no guarantee on what a bounded-distancedecoding algorithm does if the distance from y to the closestcodeword c exceeds t:it may still find c, or it may outputanother codeword, or it may simply halt indicating failure.

In hard-decision bounded-distance decoding, we usually havet = l(d– 1)/2], while in soft-decision bounded-distance de-coding (of binary codes on the AWGN channel) we usuallyhave t= W. In both cases, t is equal to half the minimumdistance between distinct codewords, in the correspondingmetric. This means that t is the largest constant for whichwe can guarantee the correction of all error patterns of

95

weight (norm) at most t, even if we were to use maximum-likelihood decoding. Bounded-distance decoders of this kindare said to achieve the error-correction radius of the code.

Algebraic bounded-distance decoding. For many importantfamilies of codes, in particular BCH codes, Reed-Solomoncodes, and most algebraic-geometry codes, we now havepolynomial-time hard-decision bounded-distance decodingalgorithms that achieve the error-correction radius of thecode. The availability of such bounded-distance decoders isthe result of a considerable body of work in coding theory,see [11, 42, 83] and references therein.

The most notable contribution to the development of thesedecoders is arguably the Berlekamp-Massey [11, 61] algo-rithm which found applications also outside of coding the-ory, in such fields as cryptography and dynamic control. Wewill illustrate the idea for Reed-Solomon codes. Supposethat c= (co, cl, . . ..cl–l ) is a codeword of a Reed-Solomoncode C over lF~ = IF~+l, and let e = (eo, el, . .,e~–1) bethe channel error-vector, so that y = c + e. Consider theassociated polynomials C(Z) = C.– 1z n-1 + ..+ CIZ+COand e(z) = en–lzn–l +.. + elz + eo. Notice that the sumy(z) = c(z) + e(z) is known to the decoder. Furthermore, itis not difficult to see that the (n, k, d) Reed-Solomon code C,defined in the previous subsection, is precisely the set ofall c E IF:, such that the associated polynomial C(X) satis-fies c(crl) = C(crz) = . ~. = C(ad-’) = O, where a is a fixedelement of order n in lF~. The decoder can thus compute

S, ~f e(ai) = e(ai) + C(CY;) = y(ai) (4)

for all i = 1,2, . . . . d– 1. Moreover, observe that the entireerror-vector e can be reconstructed from its n syndromesSI, S2,.. ., S., defined by (4). The problem is that onlythe first d – 1 of these n syndromes are known to the de-coder. Fortunately, it can be shown that if wt(e) ~ t thenthe syndrome sequence S1, S2,. . ., S~ satisfies a linear re-currence of order ~ t. This recurrence, and with it the en-tire sequence S1, S2, . . . . S., can be recovered from the firstd– 1 syndromes S1, S2,... , S,j- 1, provided d – 1 ~ 2t. Thisis precisely what the Berlekamp-Massey algorithm does: itis a general purpose procedure for efficiently determiningthe linear recurrence of lowest order that produces a givensequence (over any field). If the sequence S1, %, ..., Sd- 1of known syndromes satisfies a linear recurrence of ordert s L(d– 1)/2], then the Berlekamp-Massey algorithm findsthis recurrence in time 0(t2 ). The complexity of the syn-drome computation in (4) is O(nt), which is also the com-plexity of reconstructing the error-vector e given the recur-rence generating S1, S2, . . . . Sn.

It is obvious that the Berlekamp-Massey algorithm is in-herently a hard-decision decoding algorithm. However, For-ney and Vardy [36] have recently shown how this algorithmcan be employed for polynomial-time bounded-distance de-coding in Euclidean space. The results of [36] are actuallymore general. Forney and Vardy [36] show that wheneveran (n, k, d) code C has a hard-decision bounded-distance de-coder which achieves the error-correction radius [(d–1)/2j,it can be used to design a soft-decision bounded-distance de-coder which achieves the error-correction radius W in the

Euclidean space !Rn. The increase in complexity from hard-decision to soft-decision decoding is by a factor of O(d).

It is also clear from our discussion of the BerIekarnp-Masseyalgorithm that it may fail whenever the weight of the er-ror vector exceeds [(d– 1)/2]. A number of papers [30, 40]have been devoted to developing polynomial-time algorithmsthat guarantee decoding to the closest codeword beyond thisbound. For Reed-Solomon codes, the best results have beenrecently obtained by Sudan [80], using a new approach. Thealgorithm of [80] decodes to the closest codeword wheneverthe weight tof the error vector is bounded by

t < ‘=a,b:(a+l)(b+l )+~b(.t-l)(b+l)>n {n-a-b(k-l)-l}

This may be well approximated as t ~ n – (2nk)112. Thedecoding complexity is still polynomial, and the foregoingupper bound on t becomes far better than the conventionalts l(d–1)/2j for Reed-Solomon codes of rate R < ~.

Finally, our discussion of the Berlekamp-Massey afgorithmillustrates the idea of reconstructing the “unknown” syn-dromes tkom the known ones. Once all the syndromes havebeen computed, the entire error vector can be recovered.Loosely speaking, this general idea underlies most of theknown bounded-distance decoding algorithms for algebraic-geometry codes. These decoding algorithms involve deep r~suits from algebraic geometry, and we will not even attemptto describe them here. We refer the reader to H@holdt andPellikaan [42] for an excellent recent survey of this subject.

The hardness of maximum-likelihood decoding. Maximum-likelihood decoding of a binary linear (n, k, d) code C,both hard-decision and soft-decision, can be trivially ac-complished in time 0(2~ ) by simply comparing the chan-nel output y with all the 2k codewords of C. It is notdifficult to show that this task can be also accomplishedin time 0(2”– k). The following result, due to Berlekarnp,McEliece, and van Tilborg [12], shows that for the generalclass of binary linear codes, polynomial-time algorithms areunlikely to exist (unless P = NP), even for the simpler caseof hard-decision maximum-likelihood decoding. To describethis result, we first need to express the maximum-likelihooddecoding problem in slightly different terms. Given the chan-nel output y E IF; and an (n – k) x n parity-check matrix Hfor a binary code ~ we define the syndrome of y as s = Hyt.Then the most likely codeword, that is the one closest to y inthe Hamming metric, is given by c = y + e, where Hei = sand e c IF; is the minimum-weight vector with this prop-erty. Hence maximum-likelihood decoding is equivalent tofinding e, given H and s. The corresponding decision prob-lem can be stated w follows:

Problem: MAXIMUM-LIKELIHOODDECODING

Instance: A binary m x n matrix H, a vector s E lF~,and an integer w >0.

Question: Is there a vector z E IF; of weight <w, suchthat Hzt = s?

Berlekamp, McEliece, and van Tilborg [12] show that thisproblem is NP-complete using a reduction horn 3-DIMEN-

SIONALMATCHING,a well-known NP-complete problem [39].

96

The complexity of MAXtkNJM-LikelihOOd DECODING is fun-damental in coding theory, and it was analyzed in severalcontexts by several authors [5, 9, 16, 20, 25, 78]. Fang,Cohen, Godlewski, and Battail [25] used the results ofBerlekamp, McEliece, and van Tilborg [12] to establish therather obvious fact that soft-decision maximum-likelihooddecoding of linear codes is afso NP-hard. Barg [9] showedthat MAXIMUM-LIKELIHOODDECODINGis NP-complete forcodes over an arbitrary alphabet of fixed size q, not onlyfor the case q = 2. Stern [78] and Arora, Babai, Stern,Sweedyk [5] proved that approximating the solution toMAXIMUM-LIKELIHOODDECODINGwithin a constant factoris NP-hard.

Bruck and Naor [16] considered a slightly different contextof maximum-likelihood decoding with pre-processing. Theyassume that the parity-check matrix H is not part of theinstance; rather it is fixed and available to the decoder inadvance (as is the case in practice). The decoder can thusperform computations on the matrix H, prior to being pre-sented the instance consisting ofs c lF~-k and w >0. Bruckand Naor [16] prove that MAXIMUM-LIKELIHOODDECODINGremains hard even with unlimited amount of pre-processingon H, in the following sense: the existence of a polynomial-time afgorithm for this version of MAXIMUM-LIKELIHOODDECODING implies that the polynomial hierarchy collapsesat the second level, namely U~l X: = X;. The proof of [16]is baaed on the theorem of Karp and Lipton [47] which showsthat the polynomial hierarchy collapses in this way if an NP-complete problem can be solved in polynomial time withpolynomial advice (belongs to the class P/poly). The resultof the pre-processing on H constitutes the polynomial advicein the argument of [16].

Downey, Fellows, Vardy, and Whittle [20] recently provedthat MAXIMUM-LIKELIHOODDECODING is hard for theparametrized complexity class W[l]. Namely, it is un-likely that there exists an algorithm which solves MAXIMUM-LIKELIHOODDECODINGin time /(w)nc, where c is a con-stant independent of w and f(. ) is an arbitrary function.Many NP-complete problems are fixed-parameter tractable.For example VERTEX COVER, a well-known NP-completeproblem [39] which asks whether a graph G on n verticeshas a vertex cover of size at most k, can be solved [8] intime O(kn + (4/3) kk2). Loosely speaking, the parametrizedcomplexity hierarchy FPT = WIO] ~ WII] c W[2] c . . . in-troduced by Downey and Fellows [18, 19] distinguishes be-tween those problems that are fixed-parameter tractable andthose that are not. The result of [20] shows that MAXIMUM-LIKELIHOODDECODINGis not likely to be fixed-parametertractable. Furthermore, this result implies that bounded-distance decoding for the class of binary linear codes is hardin the following sense: if a polynomial-time algorithm forbounded-distance decoding exists then the parametrized hi-erarchy collapses with IV[l] = FPT.

Although we have, by now, accumulated a considerableamount of results on the hardness of MAXIMUM-LIKELIHOODDECODING,the broad worst-case nature of these results isstill somewhat unsatisfactory. For example, as discussedin the previous subsection, it is known [83, p.77] that al-most alI linear codes attain the Gilbert-Varshamov bound.

However, there is no proof so far that the hard instancesof MAXIMUM-LIKELIHOODDECODINGdo not belong to thevanishing fraction of codes that do not meet the Gilbert-Varshamov bound. It is very unlikely that this is so, butwe would like to have a proof. Thus, it would be worth-while to establish the hardness of MAXIMUM-LIKELIHOODDECODINGin the average sense [53], or for more narrowclasses of codes. A first step along these lines was taken byBarg [9], who showed that MAXIMUM-LIKELIHOODDECOD-INGis NP-hard for multilevel (or generalized concatenated)linear codes. However, the class of multilevel codes consid-ered in [9] is still rather “broad,” and further results of thisnature would be desirable.

Trellis decoding and its complexity. As discussed in the fore-going paragraph, it is unlikely that there exist polynomial-time maximum-likelihood decoding algorithms for Iinearcodes, either soft-decision or hard-decision. Nevertheless, wecan do substantially better than (9(2k ) or 0(2n–k). Namely,it is possible to achieve full maximum-likelihood soft-decisiondecoding of certain linear codes in exponential time, butwith complexity exponent much less than n min{R, 1–R}.Despite their exponential nature, such algorithms are of con-siderable interest in coding theory, due to the significant gapin performance between hard-decision bounded-distance de-coding and soft-decision maximum-likelihood decoding: onmany useful channels, a maximum-likelihood soft-decisiondecoder for a short code might achieve a lower probabilityof error than a hard-decision bounded-distance decoder fora much longer code.

Trellis decoding is a primary example of exponential-timemaximum-likelihood decoding algorithms of this type. Trel-lises can be also used to implement randomized “sequential”decoding algorithms [26, 33], whose performance is close tomaximum-likelihood and whose running time is polynomialwith appreciably high probability.

A trellis T is an edge-labeled directed graph T = (V, E, A)with the following property: the set of vertices V can bepartitioned into disjoint subsets Vo, VI,..., Vn, such that ev-ery edge e E E begins at a vertex v E Vi and ends at a vertexv’ E V,+l for some i = O,1, . . . . n–l. It is usually assumedthat the subsets VO,V. C V each consist of a single vertex,called the root and the toor, respectively. Clearly, the se-quence of edge labels along each path from the root to thetoor in T defines an n-tuple (al, az, . . . an) over the label al-phabet A. We say that T represents a code C over A if theset of afl such rwtuples is equal to the set of codewords of C.

Given a code C over Il?gand a trellis T = (V, E, IFq) that rep-resents ~ maximum-likelihood soft-decision decoding of Ccan be accomplished with exactly 21El – IVI+ 1 operations.The decoding procedure that accomplishes this is known asthe Viterbi algorithm [32]. Basically, the Viterbi afgorithmis a simple application of dynamic programming [68]. Werefer the reader to [63, 86] for a detailed description of theViterbi algorithm. Here, we are more concerned with theproperties of the trellis itself.

It is obvious that every trellis T represents a unique code,which can be determined by readng the edge labels of eachpath in T. However, we usually need to solve the converse

97

problem: namely, given a code C we wish to construct atrellis T which represents C. It is easy to see that there arealways many different trellises representing the same code.Hence, we would generally like to construct the ‘best’ trellisrepresenting a given code. This problem has two importantaspects, discussed in what follows.

If we think of a code C as a fixed set of codewords, thenconstructing the best trellis for C is relatively straightfor-ward. It is shown in [65, 49] that for every linear code ~there exists a unique, up to graph isomorphism, minimaltrellis representing C. It is now known [63, 65, 87] that theminimal trellis simultaneously minimizes all the conceivabletrellis complexity measures, such as 21EI – IVI + 1, IVl, I.El,maxi lV~l,aad lVil for each i = O,1, ..., n, among all possibletrellis representations for C. Here is one way to construct theminimal trellis, due to Babl, Cocke, Jelinek, and Raviv [7].Let H=[hl, h2,..., hn] bean (n–k) x n parity-check matrixfor C. Then the set of vertices Vi is given by

~ = {Clhl+... +C{h; :( Cl, Ci, ci+l,i,Cn)EC, Cn)EC

fOr SOmeci+l, . . ..~CIFq}

fori=l,2,..., n, with the convention VO= {O}. There is aedge fkom a vertex v c Vi to a vertex v’ 6 Vi+l if and only ifthere exists a codeword (cl, cz,. ... c~) c C, such that

Clhl +-. +&-1hi-1 +Ciha = V

Clhl +... + Cihi + Ci+lhi+l = v’

The label of this edge is ci GlF~. Today, at leaat three otherconstructions of the minimal trellis are known [62, 34, 49]. Ingeneral, it is fair to say that minimal trellises for linear codesare by now well understood, and all the questions pertainingto the minimal trellis have been already answered.

Surprisingly, however, the innocuous operation of permut-ing the symbols in each codeword of C according to a fixedpermutation n seems to assume a fundamental significancein the context of trellis complexity, and leads to a numberof challenging problems. Two codes C and T(C) that dif-fer by a permutation of coordinates are called equivalent incoding theory. Until recently, such codes were consideredaa essentially same. They certainly have the same parame-ters (n, k, d) and share all other important properties, ezcepttrellis complexity. It turns out that a permutation of coor-dinates can drastically change the number of vertices in theminimal trellis representation of a given code ~ often by anexponential factor. The problem of minimizing the trelliscomplexity of a code via coordinate permutations, termedthe “art of trellis decoding” by Massey [62] has attracted alot of interest recently (for example, seven papers in [28] aredevoted to this problem). Nevertheless, the problem remainsessentially unsolved.

We now describe the present state of knowledge on thepermutation problem, with an emphasis on the asymptoticresults. To do so, we need to introduce a precise mea-sure of the “complexity” of a trellis. The most widely ac-cepted [34, 51, 52, 65] trellis complexity measure may be defined ass = maxi=o,l,...,n logq IUI, for a trellis T = (V, ~, ~g)with V = V. U V1 U . . . U Vn. We can also define the relativetrellis complexity as s = sfn. A binary code represented

by a trellis with relative complexity s can be decoded intime 0(2nf), using the Viterbi algorithm. A simple upperbound, due to Wolf [93] and Massey [62], is given by

f ~ min{R, 1 – R} (5)

This bound is true for the minimal trellis of any linearcode, for any permutation of the code coordinates. The bestknown lower bound is due to Lafourcade and Vardy [51, 52].For n + co this bound takes the following form: for myasymptotically good sequence of codes with rate fixed at Rand relative distance d/n iixed at b, we have

(6)

where Rm= (.) is the JPL bound (see Figure 1). The boundin (6) holds for all linear codes, and for all possible coor-dinate permutations. It is depicted in Figure 2 for the caseR = 1 – Hz (6), that is for binary codes meeting the Gilbert-Varshamov bound. The best known upper bound in this

< 0.5

0,45

0.4

0.35

0.3

0.Z5

0,2

0,15

0.1

o.r6

o0 0.1 0.2 0,3 0.4 0.5 0.6 0.7 0,6 0.9 1R

Figure 2: Asymptotic bounds on trellis complexity

context was obtained by Dumer [22] and Kudryashov-Zakha-rova [50]. They show that there exist bhry linear codes onthe Gilbert-Varshamov curve R = 1 – Hz(6), whose trelliscomplexity satisfies

(

1 – Hz(c$) O ~ R ~ l–Hz(l/d)<~

Hz(2J) – If2(6) l–Hz(l/d) ~ R ~ 1

for some coordinate permutation. This bound is alsopitted in Figure 2. We observe that both bounds are signif-icantly below the triangle ~ = min{R, 1 — R}, which corre-sponds to the Wolf-Massey bound of (5) and to straightfor-ward maximum-likelihood decoding.

Tanner graphs and iterative decoding. There is a differentway of representing a code by a graph, that has a polynomialrather than exponent ial complexity. That is, the numberof vertices and edges in this graph is 0(rz2) for afl linearcodes, and O(n) for a certain important sub-class of low-density codes [38, 76]. This idea datea back to the work ofTanner [81] in 1981, but has recently attracted a considerable

98

renewed interest [20, 35, 37, 57, 58, 76, 77, 91, 92]. Stayingtrue to the original source [81], we will refer to such graphsas Tanner graphs in what follows.

Let H bean (n–k) x n parity-check matrix for a binary linearcode C. Then the corresponding Tanner graph for C is thebipartite graph on a set of n – k red vertices 7? and a set of nblue vertices B, having H as its red-blue adjacency matrix.Thus a blue vertex /3 G L?corresponds to a code coordinate,while a red vertex p c 73 corresponds to a parity-check on itsblue neighbors. If X ~ 23is the support of a vector z c IF;,then x G C if and only if afl the checks are satisfied: everyred vertex has an even number of neighbors in X.

It is clear that every bipartite graph G is a Tanner graphfor some code, that is the binary linear code defined as thekernel of the red-blue adjacency matrix of G. Thus one canconstruct codes from bipartite graphs with certain proper-ties. Spielman [76, 77] uses this idea to construct a fam-ily of asymptotically good binary linear codes, for whichbounded-distance decoding can be accomplished in lineartime. The underlying Tanner graph is (h, 1)-regular, namelyevery blue vertex is adjacent to h red vertices (checks), andevery red vertex is adjacent to 1 = hn/(n —k) blue vertices(coordinates). The decoding algorithm, originally due toGallager [38], is surprisingly simple. Given the assignmentof coordinate values corresponding to the channel outputy 6 IF;, some of the n– k checks are satisfied while some oth-ers are unsatisfied. Examine the coordinates in some fixedorder, and invert the first coordinate that appears in morethan h/2 unsatisfied checks; recompute the checks and re-peat. It is obvious that for (h, 1)-regular graphs, with h fixed,this algorithm runs in time O(n). Spielman [76, 77] showsthat if the underlying Tanner graph is a sufficiently goodexpander, then the algorithm is guaranteed to correct anerrors, where a is a constant fraction depending on h, n, kand the expansion properties of the graph. This means thatwe have a linear-time bounded-d~tance decoder for the cor-responding code C, although this decoder does not usuallyachieve the error-correction radius of C.

It is interesting that the Tanner graph for a code C can beafso used for soft-decision decoding of C, using a variantof the Viterbi algorithm known as the sum-product algo-rithm [35, 81, 91]. It has been recently shown by severaf au-thors [37, 57, 58] that the sum-product algorithm is actuallya version of “probability propagation” or “belief propaga-tion” in Bayesian networks, a topic well-studied in the fieldof expert systems and artificial intelligence [69]. If the under-lying Tanner graph is cycle-free, then the sum-product algo-rithm converges to the optimal maximum-likelihood solutionin time 0(n2). Unfortunately, Trachtenberg and Vardy [82]have recently shown that the question “Which codes havecycle-free Tanner graphs?” has a disappointing answer: es-sentially, only trivial ones. If the Tanner graph has cycles,the sum-product algorithm generally regresses into itera-tions. It is presently not clear under which conditions theseiterations converge, and if they do, whether they convergeto the maximum-likelihood solution. In fact, it is easy toconstruct pathological counter-examples [64] where no con-vergence is possible. Nevertheless, iterative decoding tech-niques based on Tanner graphs and similar graph represen-

tations of a code (cf. Wiberg [91]) seem to work very well inpractice. Of particular interest in this context are graphs oflarge girth. For such graphs, iterative decoding techniquesappear to perform reasonably close to maximum-likelihoodsoft-decision decoding, with polynomial-time complexity. Infact, the “turbo” decoding algorithm of [13], considered bymany a major breakthrough in coding theory, is a specialcase of iterative decoders of this kind [37, 91].

Although these recent iterative decoding techniques consti-tute what is arguably the most promising type of decodersavailable today, a major concern is that we do not yet un-derstand why they work as well as they do. A first attemptto analyze such decoders was made by Wiberg [91]. Furtherinvestigation of iterative decoding on graphs with cycles iscertainly an important project for future research.

3. The Minimum Distance problem

Berlekamp, McEliece, and van Tilborg [12] showed in 1978that two fundamental problems in coding theory, namelymaximum-likelihood decoding and computation of the (non-zero terms in the) weight distribution, are NP-hard for theclass of binary linear codes. They conjectured, but wereunable to prove, that the following decision problem:

Problem: MINIMUMDISTANCE

Instance: A binary m x n matrix H and an inte-ger w >0.

Question: Is there a nonzero vector z E IF; ofweight ~ w, such that Hzt = O?

is also NP-complete. It is easy to see that the NP-comple-teness of this problem would imply that computing the min-imum distance of a binary linear code is NP-hard. Indeed,let C be a linear code defined by the parity-check matrix H,and let d denote the minimum distance of C. If d is known,then one can answer the question of MINIMUMDISTANCEbysimply comparing d and w. On the other hand, if one cansolve MINIMUMDISTANCEin polynomial time, then one canalso find d in polynomial time by successively running analgorithm for MINIMUMDISTANCEwith w = 1,2,..., untilthe first affirmative answer is obtained.

The MINIMUM DISTANCEproblem has a long and convo-luted history. To the best of our knowledge, it was firstmentioned by Welsh [90] at an Oxford Conference on Com-binatorial Mathematics in 1969. In the printed version [90]of his paper, Welsh calls for an efficient algorithm to findthe shortest cycle in a linear matroid over a field IF. Itis easy to see that for IF = IFq this is equivalent to find-ing the minimum weight codeword in a linear code over IF~.Hence the NP-completeness of MINIMUMDISTANCEimpliesthat a polynomial-time algorithm for the problem posed byWelsh [90] is unlikely to exist. Following the publicationby Berlekamp, McEliece, and van Tilborg [12] of their con-jecture, the MINIMUMDISTANCEproblem was mentioned asopen in the well-known book by Garey and Johnson [39,p. 280]. Three years later, it was posed by Johnson [44] as

99

an “open problem of the month” in his ongoing guide to NP-completeness column. The problem remained open despiterepeated calls for its resolution by Johnson [45] and others.

Determining whether computation of the minimum Ham-ming distance of a linear code is NP-hard is important notonly because this is a long-standing open problem. Thereare several more compelling reasons. First, for a host ofproblems in coding theory there is an easy reduction fromMINIMUMDISTANCE. A few examples of such problems arepresented in the next section. Hence if MINIMUMDISTANCEis computationally intractable, then all these problems areintractable as well. Secondly, as mentioned in the previoussection, a polynomial-time procedure for computing the min-imum distance of a linear code would imply the existence ofpolynomial-time randomized algorithms for constructing bi-nary codes that attain the Gilbert-Varshamov bound. Thusit is important to know that such a polynomial-time proce-dure is unlikely to exist.

Due to these and other reasons, the conjecture of Berlekamp,McEliece, and van Tilborg [12] sparked a remarkable amountof work, most of it unpublished. In particular, in an attemptto establish the NP-completeness of MINIMUMDISTANCE,agreat number of closely related problems were proved to beNP-complete. For example, the problems of finding the max-imum weight of a codeword, finding a codeword of minimumweight which is not a multiple of a given integer, determiningthe existence of a codeword of weight in/2j, and computingthe minimum weight of a codeword which is non-zero in aspecified position, were shown to be NP-hard by Ntafos andHak~mi [67], Calderbank and Shor (see [21]), and Lobsteinand Cohen [55], respectively. At least eight different prob-lems of this kind are presently known to be NP-complete.It is pointed out in [55] that all the eight problems are tan-talizingly close to MINIMUMDISTANCE,and hence providefurther evidence to support the conjecture of [12] that MIN-IMUMDISTANCEis NP-complete. The ensemble of all theseproblems, however, does not suffice to prove this conjecture.

Our main goal in this paper is to prove that MINIMUMDIS-TANCEis NP-complete. To this end, we exhibit a polynomialtransformation to MINIMUMDISTANCEfrom MAX]MUM-LIK-ELIHOODDECODING. Thus we settle the conjecture of Ber-lekamp, McEliece, and van Tilborg [12] in the affirmative,using a reduction from the main result of [12].

3.1. Preliminaryobservations

The transformation from MAXIMUM-LIKELIHOODDECODINGto MINIMUMDISTANCEthat we will use places certain minorrestrictions on MAXIMUM-LIKELIHOODDECODING. Hence,our goal herein is to observe that MAXIMUM-LIKELIHOODDECODINGremains NP-complete under these restrictions.

First, we slightly modify the question of MAXIMUM-LIKELI-HOODDECODINGby requiring that the solution to Hxt = sis nofzzero. This restriction makes a difference only forthe special case s = O, for if s # O then obviously anysolution x to Hxi = s is nonzero. We therefore observe

that the proof in [12] of the NP-completeness of MAXIMUM-LIKELIHOODDECODING,based on the transformation from3-DIMENSIONALMATCHING,uses only the special case whereS=(ll. .l)t. Hence the same proof establishes that theminor variation of MAXIMUM-LIKELIHOODDECODING&j-cussed above is also NP-complete.

Next, as pointed out in [12], one may assume w.1.o.g. thatthe m x n matrix H at the input to MAXIMUM-LIKELIHOODDECODINGis full-rank. This implies that the columns of Hcontain a basis for IFzm,and we may further assume w.1.o.g.that w ~ m – 1. Indeed, if H is full-rank and w ~ m,then the answer to the question of MAXIMUM-LIKELIHOODDECODINGis trivially “Yes.”

We also assume w.1.o.g. that the columns of H are distinct. Ifthis is not so, then we can form in polynomial time an m x n’matrix H’ by retaining a single representative from each setof equal columns of H. It is easy to see that Hzt = s hm asolution of weight at most w, if and only if so does H’x~ = s,providing s # O. But the case s = O may be safely excludedfrom the input, as discussed above. The assumption that allthe columns of H are distinct further implies that n ~ 2-.These are all the assumptions that we will need.

The key idea in the transformation from MAXIMUM-LIKE-LIHOODDECODINGto MINIMUMDISTANCEis to regard thecolumns of the m x n parity-check matrix H aa elementsal, az, ..., an in the finite field ll?z~. The syndrome vectors E IFzmmay be also regarded as an element ~ in lF2~. Withthis notation, taking into account the restrictions discussedin the foregoing paragraphs, we may re-phrase MAXIMUM-LIKELIHOODDECODINGas the finite-field version of SUBSETSUM,a well-known NP-complete problem [39, p.233]. Specif-ically, consider the following problem:

Problem: FINITE-FIELDSUBSETSUM

Instance: An integer m ~ 2, a set of n s 2m distinctelements 0s, crz, . . ., an C IFw, a nonzero element@ c IFZWI,and a positive integer w S m – 1.

Question: Is there a subset {~,1 , cri,, . . . . ~,$} of theset{crl, a2, ..., cYn}, such that CYil+.. .+cu$ = @and b<w?

According to the discussion in this subsection, we con-clude that the NP-completeness of MAXIMUM-LIKELIHOODDECODING, established in [12], immediately implies thatFINITE-FIELDSUBSETSUM is NP-complete.

3.2. NP-completenessfor codesof characteristictwo

Given the input m, CM,CY2,..., cr~, ~, and w to FINITE-FIELD SUBSETSUM, we first construct a series of matricesAl, Az, . . . . AW, which may be thought of aa paxity-checkmatrices for the codes Cl, Cz, . . . . ~ over IFzm. Thesematrices are constructed in such a way (see Lemma 1 be-low) that the minimum distance of ~ is equal to 6 + 1 iffl~l+~:~+.. . + ~i~ = /3 for some il,iz, . . . ,i~. Otherwise

100

the minimum distance of ~ is equal to 6 + 2, and o is anMDS code [59, p.317]. The matrix AI is given by

[AI= 1 l“’~n;

01 CY2 . . .1

(7)

and it is easy to see that the minimum distance of Cl iseither 2 or 3, according as ~ = cri for some i = 1, 2, . . . . n ornot. In general, for J = 2,3, . . . . w, the matrix A6 is given by

Aa = (8)

Notice that for all 6 = 1,2,. . . ,w, the matrix Ah has n + 1columns and d + 1 linearly independent rows. Hence thedimension of ~ is n – 6, and its minimum distance is atmost (n+ 1) - dim ~ + 1 = d+ 2 by the Singleton bound [59,p.33], mentioned in the previous section.

Lemma 1. Let dh denote the minimum distance of G.Then dh = 6+ 1, if

and dh = C$+ 2 otherwise.

Proof Let M be a (6+1) x (6+1) square matrix consistingof some 6 + 1 columns of A6. If the last column of Ah,namely (O. . . Ol@t, is not among the columns of M, then Mis a Vandermonde matrix [59, p.1 16]. Since al, cw, . . . . anare all distinct, M is non-singular in this case. Otherwise,assuming w.1.o.g. that (O... 01~)~ is the last column of M,we expand along this column to obtain:

where [ - Iwith respect to a matrix denotes the determinant.The first determinant on the right-hand side of the aboveexpression is again a Vandermonde determinant, while thesecond one is a simple first-order alternant [14, 66]. Alter-ants were studied by Muir [66] and many others. In general,it is well-known that

1 1 . . 1Xl XZ . Xk

. .. .

x; x; ... x;

= ~k.j(x) ~ (Xiz – Xil )

forj=l,2,. ... k– 1, where ~,(. ) is the r-th elementary sym-metric function in the indeterminate X = Xl, X2, . . . . xk.

A proof of the above expression may be found in Muir [66,VO1.111,Chapter 5], for instance. The elementary symmetricfunction .S, (.) is defined by

Sr(X) ‘gf~~<ll<;z<...<;r<kxilxiz “ ‘ “ ‘i.

and in particular S1(X) = Xl + X2 + ..- + Xk. In our case,we indeed have r = k–j = 6 – (6–1) = 1, and the foregoingexpression for det M reduces to

det M = –(ail +~i2 + . . -+aiJ –p) ~ (@ib–c%) (9)l~a<b~6

Since al, cr2,. . . , an are distinct, the Vandermonde factorin (9) is non-zero, which implies that det M = O if andonly if ail + ~iz + . . . + ~a~ = /3. Thus if no subset ofexactly 6 elements of {al, az, . . . . cr~} sums up to ~, thenevery 6 + 1 or less columns of A,5 are linearly independent.In this case dh = J + 2 by the Singleton bound, and ~ isMDS. On the other hand, if ail + craz+ . . . + ~it = /3 forsome il, iz, . . . , ah, then obviously dh < d + 1. Now, deletingthe last row of Ah, we obtain the parity-check matrix A;which defines the code ~ that contains ~ as a subcode. Itis easy to verify (cf. [59, p.323]) that ~ is an MDS code,andhence dh~d~=(n+l)-dim~6 +1=6+1. I

We observe that the MDS codes discussed in Lemma 1 are ofindependent interest; they were studied by Roth and Lempelin [72]. We also point out that the counterpart of Lemma 1over the positive integers was proved by Khachiyan in [48].In our context, it follows immediately from Lemma 1 that ifwe could find the minimum distance of a linear code over afield of characteristic 2 in polynomial time, we could solveFINITE-FIELD SUBSET SUM in polynomial time. Formally,consider the following problem:

Problem: MINIMUMDISTANCEOVER lF2-

Instance: A positive integer m, an r x n matrix %over lFzI., an integer u >0.

Question: Is there a nonzero vector x of length nover IF2M,such that Hzt = O and wt(x) < w?

One might argue that the operations in MINIMUMDISTANCEOVER lFzm, in short MDv, are over the finite field IFw,whereas the operations in MAXIMUM-LIKELIHOODDECOD-INGare over IFz. If one were to implement the operations inlFz~ using a table of the field, for example, then this wouldrequire exponential memory. However, if we implement theoperations in 11% as polynomial addition and multiplicationmodulo an irreducible polynomial g(z) of degree m, thenonly linear memory is required, and each operation in IFzmcan be carried out in polynomial time using operations in IFZ.

Proposition 2. Existence of a polynomial-time algorithmfor MDWI implies the existence of a polynomial-time algo-rithm for FINITE-FIELDSUBSETSUM.

Prooj. Suppose that @ is a polynomial-time algorithmfor MDz.m. Then, given the input to FINITE-FIELD SUB-SETSUM, we construct the matrices Al, Az, . . . , AWas in (7)and (8). ‘We then run @ with N = A6 ~d w’=> + 1, for15=1,2 ,. ... w. It follows from Lemma 1 that if @ returns

101

“Yes” in at least one of these queries, then the answer to thequestion of FINITE-FIELD SUBSETSUM is ‘(Yes,” otherwisethe answer is ‘[No.”

It is also easy to see that in each of the w queries, the lengthof the input to MDz~ is bounded by a polynomial in thelength of the input to FINITE-FIELDSUBSETSUM. If the in-putcll, a2, ..., G and @ to FINITE-FIELDSUBSETSUMtakesm(n+l) = O(n2) bits, then the number of bits required tospecify each matrix A6 is O(rz3), and the number of bitsrequired to specify all of them is at most 0(n4 ). Further-more, each of these matrices can be obviously constructedin polynomial time from al, az, . . . . ci~ and /3, using oper-ations in IFVI. The only thing that is not entirely obviousis that IFzrnitself, namely an irreducible polynomial g(z)of degree m that defines IFz~, can be constructed in deter-ministic polynomial time. However, Shoup [75] provides adeterministic algorithm for this purpose, whose complexityis strictly less than 0(rn5 ) operations in IFz. I

The procedure used in the proof of Proposition 2 is called“Turing reduction” in Garey and Johnson [39]. It uses apolynomial number (namely w, in our case) of queries toan oracle @ for MDWI. This shows that MDzm is NP-hard, but not necessarily NP-complete, at leaat accordingto how this terminology is used in Garey and Johnson [39].There are at least two alternative ways to establish theNP-completeness of MDWI. One way is to reduce directlyfrom 3-DIMENSIONALMATCHING. The key observation hereis that the reduction from 3-DIMENSIONALMATCHINGtoMAXIMU?..I-LIKELIHOODDECODINGin Berlekamp, McEliece,and van Tllborg [12] holds without change if we replace thephrase “of weight ~ w“ with the phrase “of weight ex-actly w“ in the question of MAXIMUM-LIKELIHOODDECOD-ING. This eliminates the need for multiple queries to @ in theproof of Proposition 2, and establishes a polynomial trans-formation from 3-DIMENSIONALMATCHINGto MDVI. How-ever, we find some intrinsic merit in reducing to MDz~,and hence also to MINIMUM DISTANCE, from MAXIMUM-LIKELIHOODDECODING rather than from 3-DIMENSIONALMATCHING. Therefore, in what follows, we describe a sim-ple construction which shows that a single query to @ wouldsuffice to solve FINITE-FIELD SUBSETSUM, and hence alsoMAXIMUM-LIKELIHOODDECODING.

As before, given al, cm,. . . . an, ~, and w, we first constructthe matrices Al, Az,. ... Aw, given by (7) and (8), whichdefine thecodes Cl, Cz,..., G. Next, forb = 1,2,..., w,we let ~6 denote the linear code obtained by repeating eachcodeword of ~ exactly 16times. Thus a parity-check matrixfor CTdis given by

I IA6

Zn+l –In+l

H; = l.+l –In+l (lo)

‘.

In+l –In+l

where l.+l is the (n+l) x (n+l) identity matrix and blanksdenote zeros. Clearly, the length of ~J is 1~(n + 1), its di-

mension is n – 6, and its minimum distance is dj = l~d~which is equal to either lJ(J + 1) or la(6 + 2) by Lemma 1.The integers 11,12,. ... lW are defined, recursively, as follows:

w(w+l)_llW = 2+3+.. +w = 2 (11)

and

“= r6+$:~2)lfor d = w–l, w–2,. ..,1 (12)

Finally, we define the code @ over lFWI as the direct sumof the codes C?l,~, ..., ~. A parity-check matrix for @is given by

[

H;Hi

H# =.1

(13)“.

1where H{j H~, ..., H; are given by (10), and blanks againdenote zeros. That is, a typical codeword of@ is obtainedby independently selecting one codeword each from the codesCI, G,..., ~, and then concatenating them all together.Clearly, the length of@ is

n#= (11+12+ . ..+lw)(n+l)

its dimension is (n – 1) +.. ~+ (n – w) = O(n2), and its min-imum dktance is given by dx = rein{ fldl, 12d2,. . ., lwdw}.

We now show that the number of bits required to specify H#is bounded by a polynomial in n. It is easy to see from (12)that ll>lz >.. > lU, and therefore

Using the relation (J+1)16 < (3+2)lt+l +(6+1) which followsfrom (12), it can be readily verified by (reverse) inductionthat the following holds for all 6 = w–l, w–2,. ..,1

(6+1)13 < (w+l)lw +(d+l)+(c$+2)+. +w

Substituting 6 = 1 in this expression yields

211 < (W+l)lW +(2+.. + w) = (w+ 2)/w < W3 (14)

where the last two inequalities follow from (11 ). Hence, wehave n# < lln2 < w3n2 = 0(n5), and the number of bitsrequired t: specif~ H# is at most n# (n# – k#) m = O(nll ).Since the expressions in (10) and (13) are straightforward,this argument is all we need to prove that H# can be con-structed from the input al, az, ..., an, p, and w to FINITE-FIELD SUBSETSUM in polynomial time.

We are now ready to prove that FINITE-FIELDSUBSETSUMcan be solved with a single query to an oracle for MDz-.

Theorem 3. MDv is NP-compiete.

Proof. Clearly, MDv is in NP, since given a putative so-lution z, we can verify ‘Hzt = O and wt(z) < win polynomialtime. We exhibit a polynomial transformation from FINITE-FIELD SUBSETSUMto MDWI as follows. Given the input toFINITE-FIELDSUBSETSUM,we construct in polynomial timethe matrix H# in (13), and then run the oracle @ for MD2m

102

with ?-f = H# and u = 211. By the definition of the integersll,lz,. ..,lU in (12), we have (c$+ 1)16 ~ (d+ 2)1J+I for allJ = 1,2,. . . ,w–1. This implies

211 ~ 312 ~ . . Z (~+l)lw (15)

311 ~ 412 > . . ~ (W+2)1W (16)

Now, suppose that the answer to the question of FINITE-FIELD SUBSETSUM is “Yes.” Then it follows from Lemma 1that da = d + 1 for at least one rS= 1,2,..., w. Therefore

dx = min{lldl,lzdzl . . ..l~d~}

< max{21~,31z,. ... (w+ l)lW} (17)

= 211

in view of (15), and @ will necessarily return “Yes.” Onthe other hand, suppose that the answer to the question ofFINITE-FIELD SUBSET SUM is “No.” Then, by Lemma 1, wehave d8=6+2for a116=l,2,..., w, and

dx = min{ildl,. ..,l~d~}

= min{311,. ... (w+2)lW} (18)

= (w+ 2)1W >21,

where the third equality follows from (16), and the last in-equality is precisely (14). Hence, in this case, @ will neces-sarily return “No.” I

Obviously, the NP-completeness of MDWI is a weaker resultthan the NP-completeness of MINIMUMDISTANCE,since theset of inputs to MINIMUMDISTANCEis a special case of theset of inputs to MDY-. However, Theorem 3 is a useful step-ping stone in the proof of the NP-completeness of MINIMUMDISTANCE,which is the subject of the next subsection.

3.3. NP-completenessfor binarycodes

Given the transformation from FINITE-FIELD SUBSETSUMto MDzm in Theorem 3, the NP-completeness of MINIMUMDISTANCEwould follow if we could map, in polynomial time,the code @ constructed in (13) onto a binary linear code C,in such a way that the minimum distance d# of @ can bedetermined from the minimum distance d of C. A mappingof this kind is exhibited in this section.

Certain simple mappings from codes over IFv to binarycodes are well-known [59, pp.207–209]; however none of thesemappings is adequate for our purposes. For example, wecould let C be the binary subfield subcode of @, as iscommonly done in obtaining BCH codes from Reed-Solomoncodes (see the discussion in Section 2.1). In this cased ~ d#.Alternatively, one could let C be the trace code (cf. [59,p.208]) of @, in which case d < dx. Yet another option isto represent each element of fl?zmas a binary rn-tuple (cf. [59,p.298]), using a fixed basis for lF2m over IFz. In this case,we again have d ~ dx. All these mappings establish boundson d#. Furthermore, it can be shown that these bounds arereasonably tight [59]. However, such mappings are not suf-ficient to determine the value of d# exactly, which is whatwe need in the present context.

Instead, we will employ a concatenated coding scheme [23,31], using I@ as the outer code. We let 0 denote the(n*, k*, d*) binary linear code used as the inner code in theconcatenation: namely, we require that k“ = m and repre-sent each element of IFzmby a codeword of C’. Specifically,fix a basis /31,~2,. . . . /3m for IFw1 over IF2 and a generatormatrix G* for C. Then a one-to-one mapping q : IF.w ~ U7takes each element ~ = bl ~1 + bz~z +.. ~+ b~@~ of 11% into

w(~) = (bl, bz, . . .. bm)G* (19)

which is a binary n“-tuple. When this mapping is appliedto@, the result is a binary linear code C of length n“ n# anddimension mkx. If @ = [~i,j] is the TX x n# parity-checkmatrix for @ in (13), and the code C’ is systematic (as wemay assume without loss of generality), then a parity-checkmatrix for C is given by

H=

P“

[?’1,11’

1:[-m]’

[7r#,J’

1-1

P*

[W]’

[-ml’

[’Yr#,21’

1I “.

. . .

. . .

. . .

P*

[-fl,?#l’

[72,n#l’

[-rr#,n#r

I—

(20)

where blanks denote zeros, If” = [P* 11] is a parity-checkmatrix for C in a systematic form, and [-y] standa for them x m companion matrix (cf. [59, p.106]) of -yc lF.z~ withrespect to the baais /31,~Z, ..., ~~. Henceforth, we let ddenote the minimum distance of the code C defined by (20).The following lemma provides an upper bound on d, in termsof d#, n*, and k“= m.

Lemma 4.d ~ ~*d# 2m–1

Zm–1

Proof. Since @ is a linear code over IFIYII,if it containsa codeword c of weight d#, then it contains 2m —1 suchcodewords, namely all the multiples of c by the nonzero ele-mentsoflFzm. Letcl, cz, ..., CWI– I E @ denote these 2m– 1codewords, and consider the (2m – 1) x n# matrix M having

c m_ 1 as its rows. It is obvious that each of the d#C1,C2,..., 2non-zero columns of M contains each of the 2m —1 non-zeroelements of IPv exactly once. Now let c{, c;, . . . ,cjm-l Ecbe the images of c1, cz, . . . , C2WI– 1 under the mapping p(.)in (19), and consider the (2m – 1) x n“nx matrix M’ havingC;, cj, . . . ~c~m– 1 as its rows- If some n* columns of M’ corre-spond to a non-zero position of c E @, then every non-zerocodeword of C’ appears exactly once in these n“ columns.It follows that the weight of each non-zero column of M’ isprecisely 2m- 1, and there are at most n*d# such columns.Thus the total weight of M’ is at most n*d#2”’- 1. Thelemma now follows by observing that M’ has 2m– 1 rows. 1

We note that Lemma 4 is just a variation of the well-knownPlotkin bound [59, p.41]. Yet, it provides exactly the kind of

103

instrument we need for our purposes. Indeedj suppose thatd# <21, as in (17), where 11 is defined by (11) and (12).The; Lemma 4 implies that

yn–1d < 211n*—

Zm–1(21)

On the other hand, suppose that d# ~ 211+ 1 as in (18).Then, since d ~ d*d# by construction, we obviously have

(22)

In the present context, one is more interested in the vice-versa interpretation of the bounds in (21) and (22). Namely,given d (say, by an oracle for MINIMUMDISTANCE),we wouldlike to distinguish between the two possibilities for d#. For-tunately, if

211 2m-1d*>n” ——

2/1+12~–l(23)

then the right-hand side of (22) is strictly greater than theright-hand side of (21). Thus our goal can be achieved, pro-vided the minimum distance of V is sufficiently large.

We now observe that 211< U13in view of (14), and w < m– 1as discussed in Section 3.1. Thus, in order to satisfy (23), itwould certainly suffice to require that

* m’ - ~ Zm-lL>—— 1 2m -‘3 (24)n“ m3 Zm–1

= 0.5 – ~ 2(2m – 1)

These considerations may be translated into a specific set ofconditions relating to the code C’ used as the inner code inour construction:

PI. The length of C is bounded by a polynomialin n, and a generator matrix for C’ can beconstructed in polynomial time.

P2. The dimension of C“ is at least m (if dim ~is strictly greater than m, then any subcodeof CY will suffice for our purposes).

P3. The ratio of the minimum distance of C’ toits length satisfies (24).

Less formally, what we need is a sequence of binary lin-ear codes, whose relative distance approaches the Plotkinbound d*/n* ~ 0.5, and whose rate tends to zero onlypolynomially fast aa function of their dimension. Further-more, we should be able to construct each code in the se-quence in polynomial time. This rules out codes that at-tain the Gilbert-Varshamov bound [59, p.557], as well asZyablov codes [94], since the complexity of Zyablov’s con-struction [94] becomes exponential at low rates. Neverthe-less, many other known constructions of asymptotically goodfamilies of low-rate codes have the desired properties. Forexample, all the polynomial-t ime constructions discussed inSection 2.1, with the exception of Justesen [46] concatena-tion, would suffice for our purposes. In what follows, how-ever, we shall use a simple construction due to Noga Alon [2],which is concise enough to be described in one paragraph.

Alon’s construction: Given an integer v ~ 2 anda non-negative integer s ~ 2V – 2, consider a con-catenation of the (2”, s + 1, 2“ — s) Reed-Solomon

code over IFz. with the (2” — 1, u, 2“–1) binary sim-plex code [59, p.30]. The result is a binary linearcode C (v,s) with the following parameters

n“ = 2“(2” – 1) (25)

k“ = U(S+ 1) (26)

d“ = 2“-1(2” – s) (27)

Noga Alon [2] observes that a generator matrix G*for C’ (v,s) maybe specified directly as follows. Thecolumns of this matrix are indexed by pairs (z, y),where z, y c IF’w and y # O, while its rows are in-dexed by integer pairs (i, j), where i = 0,1,... ,sandj=l,2, . . ..u. Letal, as, . . ..a”beabaais forIF2. over IF2. Then the entry in row (i, j) and column(z, y) is defined as (~j~i, y), where ~jxi is computedin IFw, and (., .) denotes the inner product of ~jx’and y as binary v-tuples.

We takes = m and u = [5 log2 ml in the foregoing construc-tion. Then C’ = C’ (v,s) trivially satisfies property P2,since k“ = v(nz + 1) ~ m. Furthermore

n’ = 2U(2V_ 1) < 22t5}0g2~+lJ = 4m1° = O(nl”)

so that C’ also satisfies property PI. Thus the length n“ n#of the concatenated code C is upper bounded by

n“nx < l/.7n10(m-l)4(n+l) = 0(n15)—

and the matrix H in (20) can be specified in polynomialtime using at most 0(n30) bits. Now, it follows from (25)and (27), along with our choice of s and v, that

d“ _ 2“-1(2” – s)~— 2“(2” – 1)

> o.5–& >0.5–$

1 2m–m3~ 0.5 – ~ Z(’p – 1)

where the last inequality holds for all m > 10 (and followsstraightforwardly from the fact that 2m ~ m3 + mz + m + 1for such m). Thus C’ also satisfies property P3. With both& and (7 at hand, we are ready to prove our main result.

Theorem 5. MINIMUMDISTANCEis NP-compIete,

Proof. Clearly, MINIMUM DISTANCEis in NP. A poly-nomial transformation from FINITE-FIELD SUBSET SUM toMINIMUMDISTANCEcan be described as follows. Given theinput al, crz,..., cr~,~ E IFz- and w to FINITE-FIELD SUB-SET SUM, we answer the question of FINITE-FIELD SUB-SET SUM by exhaustive search if m < 10. Otherwise, weconstruct in polynomial time the matrix H in (20), withH* = [F’” II ] specified by Alon’s construction. We thenquery an oracle for MINIMUMDISTANCEfor the existence ofa codeword of weight at most

Zll ~. 2m–1Zm–1

= 1,2”(2”– l)&

where 11is defined by (11) and (12), and v = [5 logz ml. Bythe foregoing discussion, the oracle for MINIMUMDISTANCE

104

will return “Yes” if and only if the answer to the question ofFINITE-FIELDSUBSETSUM is “Yes.” I

This concludes the proof of the conjecture of Berlekamp,McEliece, and van Tilborg [12]. In the next section, we dis-cuss certain extensions and consequences of this result.

4. Further results

We note that our proof

and concluding remarks

of Theorem 5 can be immediatelyextended to codes over an arbitrary, fixed, finite field IFq.This is based on the observation [9] that the transformationfrom 3-DIMENSIONALMATCHINGto MAXIMUM-LIKELIHOODDECODING in [12] holds without change if the input toMAXIMUM-LIKELIHOODDECODINGis an m x n matrix Hover IFQ,rather than a binary matrix. Given the NP-comp-Ieteness of MAXIMUM-LIKELIHOODDECODINGover IFg, onecan essentially go through the proof in the previous section,replacing each instance of 2 by q. There are a few intricatepoints along the way, that require some explanation.

First, in rephrasing MAXIMUM-LIKELIHOODDECODINGasFINITE-FIELD SUBSET SUM, one should leave the expres-sion ~i~ + Ct’i~+ . . . + ai~ = /3 in the question of FINITE-FIELD SUBSET SUM as is, rather than ask whether @ isa linear combination of cql, ~i2, . . . . ai$. This is certainlynot the question that one would be concerned with for de-coding purposes, but it is legitimate in an NP-completenessproof given the specific transformation horn 3-DIMENSIONALMATCHINGto MAXIMUM-LIKELIHOODDECODINGin [12]. (Itis easy to see that a vector z 6 IFq”of weight s m/3 satisfiesHzf = (11 . . . I)t for the m x n incidence matrix H con-structed in [12] only if all the m/3 nonzero positions in z areequal to 1.) Secondly, the bound in Lemma4 becomes

m—1d < n*d# q

9m–l+qm–2. ..+q+l

and one has to modify equation (24) accordingly. Fortu-nately, Alon’s construction [2] works in this case as well.Here, the columns of G* would be indexed by x, y E lF~., sothat equation (26) remains without change, equation (25)becomes n“ = q“ (q” – 1), and equation (27) becomes

d* > (q– I)q”-l(q” –S) (28)

The key observation in the proof of (28) is as follows: if<, y c IFg. and ~ # O, then as y ranges over all the elementsof IFQU,the inner product (~, y) takes each value in lF~ ex-actly q“– 1 times. (Alternatively, this can be viewed as aconcatenation of the (q”, s + 1, q“ – s) Reed-Solomon codeover lF~uwith the (q” – 1, v, (q – l)qv– L) first-order general-ized Reed-Muller code over lF~,see [11, p.362]. ) To completethe proof, one can again take s = m and v = [5 log~ ml inthis construction.

The complexity of approximation algorithms for NP-hardproblems haa been extensively investigated recently [41], andit is natural to ask whether approximating the minimum dk-tance of a linear code is still hard. Since our proof of the

NP-completeness of MINIMUMDISTANCEis based on a trans-formation from MAXIMUM-LIKELIHOODDECODINGand it isknown [5, 78] that MAXIMUM-LIKELIHOODDECODINGre-mains RTP-complete under approximation within a constantfactor, it is plausible that the same should be true for M] N-IMUMDISTANCE.We leave a more rigorous investigation ofthis question as an open problem.

Another immediate consequence of our proof is that certainuseful computational tasks in coding theory are NP-hard, aathere is an easy reduction from MINIMUMDISTANCEto eachof these tasks. There is a large number of computationalproblems of this kind; we will give just three examples here.

First, we note that determining whether a given linear codeis MDS is NP-hard: namely, the following decision problem

Problem: MDS CODE

Instance: A prime p z 2, positive integers m and r,andan Txprn matrix H over lFP_.

Question: Is there a nonzero vector x of length pmover lFPm,such that Hxt = O and wt(x) < T?

is NP-complete. The fact that MDS CODE is NP-hard, evenfor p = 2, follows immediately from Lemma 1. The NP-completeness of MDS CODE then follows from the obser-vation that the phrase “of weight < w“ in the question ofMAXIMUM-LIKELIHOODDECODINGcan be changed to thephrase “of weight exactly w,” as discussed in Section 3.2.

As another example, consider the problem of determiningthe trellis complexity of a linear code. More precisely, thecomputational task is to find a coordinate permutation thatminimizes the number of vertices at a given time z in theminimal trellis for a binary linear code. For more details onthis problem, and on trellises in general, see Section 2.2. Itis not difficult to see that the corresponding decision prob-lem [43] may be posed as follows:

Problem: PARTITIONRANK

Instance: A binary k x n matrix H, and positive in-tegers i and w.

Question: Is there a column permutation that takesH into a matrix H’ = [Ai 113m_a],such that Aa isan k x i matrix and rarrk(Ai) + rank(Bn-i) < w?

Horn and Kschischang [43] recently proved that this problemis NP-complete, using an ingenious and elaborate transfor-mation from SIMPLEMAX CUT [39, p.210] which spans overfive pages. On the other hand, given the NP-completenessof MINIMUMDISTANCE,this result can be established in afew lines as follows. First, we observe that the least integer ifor which

rank(Ai) + rank(Bn_i) < rank(H) + z

is equal to min{d, d~ }, where d, d~ denote, respectively, thedistance and the dual distance of the code defined by H.Notice that it does not matter whether H is viewed as aparity-check or as a generator matrix in this problem, Now,suppose that C is an (n, k, d) binary linear code whose mini-mum distance we would like to determine, and let d~ denote

105

the dual distance of C. Given C, we first construct a binarylinear Reed-Muller code ~ of length 2m and order r, where~ = 2[log2~]+1~d T= [10g2~].Then ~ is an (n’, k’, d’)self-dual code, where

n’ = 22(1OK2nl+l < 8n2—

k’ = n’/2 ~ 4n2

d’ = 2m-” = Zrlwz ~1+1 > Zn—

We then use the well-known Kronecker product construc-tion [59, p. 568] to obtain a generator matrix for the productcode G = CL @ ~, where CL is the dual code of C. Evi-dently, the length of C7 is n* = nn’ < 8n3, and its minimumdistance is

d*=dAd’~2ndL~n>d

On the other hand, it is easy to see that the dual distanceof C is the minimum of the dual distances of Cl and ~,namely min{d, d’} = d. Hence, running a polynomial-timealgorithm for PARTITION RANK with the input H being agenerator matrix for C’, we can determine d in polynomialtime. The foregoing firing reduction from MINIMUMD]s-TANCEshows that, given a linear code C, computing eitherthe minimum dktance d or the minimum dual distance d~is NP-hard. This furthermore proves that PARTITIONRANKremains NP-hard, even if the input is restricted to

w = rank(H) + i – 1

In other words, even if all we would like to know is whetherthere exists a permutation of a linear code ~ such that]Og2lU1# z in the minimal trellis T = (V, E, F2) for C,the computational task of determining this is still NP-hard.This is a somewhat stronger result than the one reportedin [43]. Moreover, we believe that the NP-completeness ofMINIMUMDISTANCEcan be now used to show that deter-mining the mtimum trellis complexity of a code, namelys = maxi logz IVaI as defined in Section 2.2, is also NP-hard.

As a third example, we mention the problem of finding thelargest subcode with a prescribed contraction index [88].Namely, given a k x n generator matrix for a binary linearcode C, and a positive integer A, we wish to find the largestsubcode ~ ~ C which has a generator matrix with at mostA + dlm C’ distinct columns. This problem is of importancein soft-decision and majority-logic decoding (see [88] for anextensive treatment ), and it is possible to show that it isNP-hard using a transformation from MINIMUMDISTANCE.The proof of this is a bit tedious, and we omit the details.

Finally, we would like to mention two important problemsin coding theory, for which we do not have a polynomialtransformation from MINIMUMDISTANCE,but believe thatit should be possible to find one.

The first problem is that of bounded-distance decoding ofbinary linear codes. While the intractability of maximum-likelihood decoding has been thoroughly studied (see, for ex-ample [5, 9, 12, 16, 20, 78], and the discussion in Section 2.2),most of the decoders used in practice are bounded-distancedecoders. It is still not known whether bounded-distancedecoding is NP-hard for the general class of binary linear

codes. For bounded-distance decoding that achieves theerror-correction radius of the code, as defined in Section 2.2,the corresponding decision problem can be formulated as

Problem: BOUNDED-DISTANCEDECODING

Instance: A positive integer d, a binary m x n ma-trix H, such that every set of d – 1 columns of His linearly independent, a vector s E lF2~, and apositive integer w s l(d– 1)/2].

Question: Is there a vector z E IFn of weight ~ w,such that Hxt = s?

Notice that BOUNDED-DISTANCEDECODINGis not likely tobe in NP, since in view of our main result in this paper,verifying that every d – 1 columns of H are linearly independent is NP-hard. Thus BOUNDED-DISTANCE DECODING

is an example of a “promise” problem (cf. [24]). Neverthe-less, we could ask whether BOUNDED-DISTANCEDECODINGis NP-hard. We concur with the remark of Barg [9], andconjecture that this is so. Furthermore, we believe that theNP-completeness of MINIMUMDISTANCEshould be instru-mental in trying to prove this conjecture. We note that ahardness result in a somewhat different context was recentlyestablished in [20]; see the discussion in Section 2.2. How-ever, the problem as posed above, is still open.

The second problem we would like to mention is that of find-ing the shortest vector (in the Euclidean norm) in a sublat-tice of 2’. As already observed by Johnson in [45], the over-all status of computational problems for lattices is remark-ably similar to the situation with linear codes. P. van ErodeBoas [85] proved in 1980 that finding the nearest vector(which is equivalent to maximum-likelihood decoding) in asublattice of 23’ is NP-hard, and conjectured that finding theshortest vector should be hard aa well. Formally, van ErodeBoas [85] conjectured that the following problem:

Problem: SHORTESTVECTOR

Instance: A basis VI, VZ,. ... v~ E Z“ for a lattice A,and an integer w >0

Question: Is there a nonzero vector z in A, suchthat IIz112< VJ?

is NP-complete. In spite of a considerable amount of work(see [5] for a recent survey), the proof of this conjecture re-mains elusive. Arora, Babai, Stern, and Sweedyk [5] classifythis as a “major open problem.” Moreover, this conjecturebecomes particularly significant in view of the celebratedrecent result of Ajt ai [1], who showed that hard instantesof the SHORTESTVECTOR problem can be efficiently gen-erated, provided that the problem itself is hard. Thus theNP-completeness of SHORTESTVECTOR is of utmost impor-tance in cryptographic applications.

We notice that if the phrase “]IxI12< w“ in the question ofSHORTESTVECTOR is replaced with “IIzI12= w,” the prob-lem becomes NP-complete. This result, which was recentlyestablished by Fellows and Vardy [29], shows that the coun-terpart of the WEIGHT DISTRIBUTIONproblem, which wasproved to be NP-complete for binary linear codes in [12], isNP-complete for lattices. It is, therefore, all the more plau-sible that the counterpart of MINIMUMDISTANCEshould bealso NP-complete for lattices.

106

Intuitively, finding the shortest vector in a lattice is at leastas “difficult” as finding the minimum-weight vector in a bi-nary linear code. Thus it is reasonable to suggest that therewould be a polynomial transformation from MINIMUMDIS-TANCEto the SHORTESTVECTOR. Specifically, we pose thefollowing problem: given a binary linear code C construct,in polynomial time, a lattice A ~ Z“ so that the minimumdktance of C can be determined from the minimum normof A. In view of our main result, solving this problem wouldamount to proving that SHORTESTVECTOR is NP-complete.

Acknowledgement. I would like to acknowledge helpfuldkicusaions with Noga Alon, Alexander Barg, Yoram Bresler,Shuki Bruck, Ilya Dumer, Herbert Edelsbrunner, Mike Fel-lows, Nabil Kahale, Moni Naor, Ronny Roth, Dilip Sarwate,Leonard Schulman, and Vijay Vazirani. I am especially in-debted to Noga Alon for referring me to his construction,which is used in Section 3. Finally, I would like to thankHagit Itzkowitz for her invaluable help.

References

p]

[2]

[31

[4]

[5]

[6]

{7]

[8]

[9]

[10]

M. AJTAI, Generating hard instances of lattice prob-lems, in Proc. 28-th Annual ACM Symp. Theory ofComputing, pp. 99-108, Philadelphia, PA, May 1996.

N. ALON, Packings with large minimum kissing num-bers, preprint, 1996.

N. ALON, J. BRUCK, J. NAOR, M. NAOR, and R. ROTH,Construction of asymptotically good low-rate error-correcting codes through pseudo-random graphs, IEEETrans. Inform. Theory, vol. 38, pp. 509-516, 1992.

S. ARORA, Probabilistic checking of proofs and hard-ness of approximation problems, Ph.D. thesis, Univer-sity of California, Berkeley, CA., 1994.

S. ARORA, L. BABA1, J. STERN, and Z. SWEEDYK,Thehardness of approximate optima in lattices, codes, andsystems of linear equations, in Proc. 34-th AnnuaJSymp. Found. Computer Science, pp. 724-733, PaloAlto, CA, 1993.

S. ARORA and C. LUND, Hardness of approximations,in Approximation Algorithms for NP-Hard Problems,D.S. Hochbaum (Ed.), Boston: PWS Publishing Co.,pp. 399-446, 1997.

L.R. BAHL, J. COCKE, F. JELtNEK,and J. RAVtV, Opti-mal decodhg of linear codes for minimizing symbol er-ror rate, IEEE Trans. Inform. Theory, vol. 20, pp. 284–287, 1974.

R. BALASUBRAMANIAN,M. FELLOWS, and V. RAMAN,An improved fixed-parameter algorithm for vertex co-ver, to appear, 1997.

A. BARG, Some new NP-complete coding problems,Prob~. Peredachi Informatsii, vol. 30, pp. 23-28, 1994,(in Russian).

A. BARG, Complexity issues in coding, survey chapterto appear in Handbook of Coding Theo~, R.A. Brualdi,C. Huffman, and V. Pless (Ed.), Amsterdam: Elsevier.

[11]

[12]

[13]

[14]

[15]

E.R. BERLEKAMP,Algebmic Coding Theory, New York:McGraw-Hill, 1968.

E.R. BERLEKAMP, R.J. MCELIECE, and H.C. A. VANTILBORG,On the inherent intractability of certain cod-ing problems, IEEE !lkans. Inform. Theory, vol. 24,pp. 384-386, 1978.

C. BERROU,A. GLAVIEUX,and P. THITIMAJSH]MA,N-ar Shannon limit error-correcting coding and decoding:turbo codes, in Proc. IEEE Int. C’onf. on Communica-tions, pp. 1064-1070, Geneva, Switzerland, 1993.

M. BLAUM, J. BRUCK, and A. VARDY, On MDS codesand alterants over certain rings, in Proc. 900-th Meet-ing, American Math. Sot., Chicago, IL., March 1995.

M. BLAUM,J. BRUCK, and A. VAR~Y, MDS array codeswith independent parity symbols, IEEE Trans. Inform.Theory, vol. 42, pp. 529-542, 1996.

[16] J. BRUCKand M. NAOR, The hardness of decoding lin-ear codes with preprocessing, IEEE Ihns. Inform. The-ory, vol. 36, pp. 381–385, 1990.

[17] G.C. CLARK and J.B. CAIN, Error-Correction Codingfor Digital Communications, New York: Plenum Press,1981.

[18] R.G. DOWNEY and M.R. FELLOWS, Fixed parametertractability and completeness I: b~ic theory, SIAM J.Comput., vol. 24, pp. 873-921, 1995.

[19] R.G. DOWNEY and M.R. FELLOWS, Fixed parametertractability and completeness II: completeness for W[l],Theoret. Computer Sci. A vol. 141 pp. 109-131, 1995.

[20] R.G. DOWNEY,M. R. FELLOWS,A. VARDY, and G. WHI-TTLE, On the parametrized complexity of certain fun-damental problems in coding theory, preprint in prepa-ration, 1997.

[21] P. DIACONK and R.L. GRAHAM, The Radon transformon 73$,Pacific J. Math., vol. 118, pp. 176–185, 1985.

[22] 1.1. DUMER, On complexity of maximum-likelihood de-coding of the best concatenated codes, in Proc. 8-th AJ1-Union Conf. on Coding Theory and Information The-ory, Moscow-Kuibishev, pp. 66-69, 1981, (in Russian).

[23] 1.1. DUMER, Concatenated codes and their general-izations, to appear in Handbook of Coding Theory,R.A. Brualdi, W.C. Huffman, V. Plesa (Ed.), Amster-dam: Elaevier.

[24] S. EVEN and Y. YACOBI, Cryptography and NP-comp-leteness, Lect. Notes Comp. Science, vol. 85, pp. 195-207, Springer-Verlag, 1982.

[25] J. FANG, G.D. COHEN, PH. GODLEWSKI, and G. BAT-TAIL, On the inherent intractability of soft decisiondecoding of linear codes, Lect. Notea Comp. Science,vol. 311, pp. 141–149, Springer-Verlag, 1988.

[26] R.M. FANO, A heuristic discussion of probabilistic de-coding, IEEE Tkans. Inform. Theory, vol. 9, pp. 64–73,1963.

[27] J. FEIGENBAUM,The use of codkig theory in com-putational complexity, in Proc. Symp. Appl. Math,A.R. Cafderbank (Ed.), Providence, RI: AMS Press,pp. 203-229, 1995.

107

[28] J, FmGmw3Auhi, G.D. FORNEY, B. MARCUS, R.J. McELIECE, and A. VARDY, Special issue on “Codes andComplexity,” IEEE Trans. Inform. Theory, vol. 42,November 1996,

[29] M.R. FELLOWSand A. VARDY, The hardness of thetaseries in lattices, preprint in preparation, 1997.

[30] G.-L. FENG and K.K. TZENG, A new procedure for de-coding cyclic and BCH codes up to actual minimum dis-tance, IEEE Trans. Inform. Theory, vol. 40, pp. 1364–1374, 1994.

[31] G .D. FORNEY,JR., Concatenated Codes, Cambridge,MA: M.I.T. Press, 1966.

[32] G.D. FORNEY,JR., The Viterbi algorithm, Proc. IEEE,VO].61, pp. 268–278, 1973.

[33] G.D. FORNEY,JR., Convolutional codes III: sequentialdecoding, Inform. Control, vol. 25, pp. 267-297, 1974.

[34] G.D. FORNEY,JR., Coset codes II: Binary lattices andrelated codes, IEEE ‘Ikms. Inform. Theory, vol. 34,pp. 1152–1187, 1988.

[35] G.D. FORNEY,JR., The forward-backward algorithm,in Proc. 34-th Allerton Conference on Comm., Con-trol, and Computing, Monticello, IL., pp. 432-446, Oc-tober 1996.

[36] G.D. FORNEY,JR. and A. VARDY, Generalized mini-mum distance decoding of Euclidean-space codes andlattices, IEEE Trans. Inform. Theory, vol. 42, pp. 1992-2026, 1996.

[37] B.J. FREY and F.R. KSCHtSCHANG,Probability propa-gation and iterative decoding, in Proc. 34-th AllertonConference on Comm., Control, and Computing, Mon-ticello, IL., pp. 482–493, October 1996.

[38] R.G. GALLAGER,Low Density Parity-Check Codes, Ca-mbridge: M.I.T. Pressl 1962.

[39] M.R. GAREY and D.S. JOHNSON,Computers and htra-ctability: A Guide to the Theoy of NP- Completeness,San l+ancisco: Freeman, 1979.

[40] C.R.P. HARTMANand K.K. TZENG, Decoding beyondthe BC!H bound using multiple sets of syndrome se-quences, IEEE Tkns. Inform. Theory, vol. 20, pp. 292–295, 1974.

[41] D.S. HOCHBAUM,(Editor), Approximation Algorithmsfor NP-Hard Problems, Boston: PWS Publishing Co.,1997.

[42] T. H@HOLDT and R. PELLIKAAN, On the decoding ofalgebraic-geometric codes, IEEE Tkans. Inform. The-ory, vol. 41, pp. 1589–1614, 1995.

[43] G.B. HORN and F.R. KSCHISCHANG,On the intractabi-lity of permuting a block code to minimize trellis com-plexity, IEEE Zkns. Inform. Theory, vol. 42, pp. 2042–2048, 1996.

[44] D.S. JOHNSON,The NP-completeness column: An on-going guide, J. Algorithms, vol. 3, pp. 182–195, 1982.

[45] D.S. JOHNSON,The NP-completeness column: An on-going guide, J. Algorithms, vol. 7, pp. 584-601, 1986.

[46] J. JUSTESEN, A class of constructive asymptoticallygood algebraic codes, IEEE Thans. Inform. Theory,vol. 18, pp. 652–656, 1972.

[47] R. KARP and R. LIPTON, Some connections betweennonuniform and uniform complexity classes, in Proc.12-th Annual Symp. Theory of Computing, pp. 302-309, 1980.

[48] L. KHACHIYAN,On the complexity of approximating ex-tremal determinants in matrices, J. Complexity, vol. 11,pp. 138-153, 1995.

[49] F.R. KSCHISCHANGand V. SOROKINE, On the trellisstructure of block codes, IEEE Tkans. Inform. Theory,vol. 41, pp. 1924–1937, 1995.

[50] B.D. KUDRYASHOVand T.G. ZAKHAROVA,Block codesfrom convolutional codes, Problemy Peredachi Infor-matsii, vol. 25, pp. 98–102, 1989, (in Russian).

[51] A. LAFOURCADEand A. VARDY, Asymptotically goodcodes have infinite trellis complexity, IEEE Trans. In-form. Theory, vol. 41, pp. 555-559, 1995.

[52] A. LAFOURCADEand A. VARDY, Lower bounds on trel-lis complexity of block codes, IEEE 2hns. Inform. The-ory, vol. 41, pp. 1938–1954, 1995.

[53] L.A. LEWN, Average case completeness, SIAM J. Com-put., vol. 15, pp. 285-286, 1986.

[54] S. LIN and E.J. WELDON, Long BCH codes are bad,Inform. and Control, vol. 11, pp. 445-451, 1967.

[55] A. LOBSTEINand G.D. COHEN, Sur la complexit6 d’unprob16me de codage, Theoretical Informatics AppL,VO1.21, pp. 25–32, 1987.

[56] B. L6PEZ JIMENEZ, Plane models of Drinfeld modu-lar curves, Ph.D. thesis, University of Complutense,Madrid, March 1996.

[57] D.J.C. MACKAY and R.M. NEAL, Near Shannon limitperformance of low-density parity-check codes, Elect.Lett., to appear, 1997.

[58] D.J.C. MACKAY and R.M. NEAL, Good error-corre-cting codes based on very sparse matrices, IEEE Tkans.Inform. Theory, submitted for publication, 1996.

[59] F.J. MACWILLIAMS and N.J.A. SLOANE,The Theory ofError Correcting Codes, Amsterdam: North-Holland,1977.

[60] Yu.I. MANIN and S.G. VL.kDUTS, Linear codes andmodular curves, J. Soviet Math., vol. 30, pp. 2611–2643,1985, (in Russian).

[61] J.L. MASSEY, Shift-register synthesis and BCH decod-ing, IEEE fians. Inform. Theory, vol. 15, pp. 122–127,1969.

[62] J.L. MASSEY, Foundation and methods of channel en-codhg, Proc. ht. Conf information Theory and Sys-tems, NTG-Fachberichte, Berlin, 1978.

[63] R.J. MCELIECE, On the BCJR trellis for linear blockcodes, IEEE Tkans. Inform. Theory, vol. 42, pp. 1072–1092, 1996.

108

[64] R.J. MCELtECE, E. R. RoDEM]cH, and J.-F. CHENG,The turbo decision algorithm, in Proc. 33-rd AllertonConference on Comm., Control, and Computing, Mon-ticello, IL., pp. 366-379, October 1995.

[65] D.J. MUDER, Minimal trellises for block codes, IEEETkans. Inform. Theory, vol. 34, pp. 1049-1053, 1988.

[66] T. MUIR, Tkentise on the Theory of Determinants, NewYork: Dover, 1960.

[67] S.C. NTAFOS and S.L. HAKIMI, On the complexity ofsome coding problems, IEEE Trans. Inform. Theory,vol. 27, pp. 794–796, 1981.

[68] J.K. OMURA, On the Viterbi decoding algorithm, IEEE?kans. Inform. Theory, vol. 15, pp. 177-179, 1969.

[69] J. PEARL, Probabilistic Reasoning in Intelligent Sys-tems: Networks of Plausible Inference, San Mateo, CA:Kaufmann, 1988.

[70] R. PELLIKAAN, Asymptotically good sequences of cur-ves and codes, in Proc. 34-th Allerton Conferenceon Comm., Control, and Computing, Monticello, IL.,pp. 276–285, October 1996.

[71] 1.S. REED and G. SOLOMON, Polynomial codes over cer-tain finite fields, SIAM J. Appl. Math., vol. 8, pp. 300-304, 1960.

[72] R.M. ROTH and A. LEMPEL, A construction of non-Reed-Solomon type MDS codes, IEEE Tkans. Inform.Theory, vol. 35, pp. 655-657, 1989.

[73] C.E. SEIANNON,A mathematical theory of communi-cation, Bell Syst. Tech. J., vol. 27, pp. 379-423 andpp. 623-656, 1948.

[74] B.-Z. SHEN, A Justesen construction of binary con-catenated codes that asymptotically meet the Zyablovbound for low rate, IEEE Tkans. Inform. Theory,VOI.39, pp. 239–242, 1993.

[75] V. SHOUP,New algorithms for finding irreducible poly-nomials over finite fields, Math. Computation, vol. 54,pp. 435-447, 1990.

[76] M. SIPSERand D.A. SPIELMAN,Expander codes, IEEETkans. Inform. Theory, vol. 42, pp. 1710–1722, 1996.

[77] D.A. SPIELMAN,Linear-time encodable and decodablecodes, IEEE Ihms. Inform. Theory, vol. 42, pp. 1723–1731, 1996.

[78] J. STERN, Approximating the number of error locationswithin a constant ratio is NP-complete, Lect. NotesComp. Science, vol. 673, pp. 325-331, Springer, 1993.

[79] M. SUDAN,Efficient checking of polynomials and proofsand the hardness of approximation problems, Ph.D.thesis, University of California, Berkeley, CA., 1992.

[80] M. SUDAN, Decoding of Reed-Solomon codes beyondthe error-correction bound, J. Complexity, vol. 1, 1997,to appear.

[81] R.M. TANNER,A recursive approach to low-complexitycodes, IEEE Trans. Inform. Theory, vol. 27, pp. 533–547, 1981.

[82] A. TRACHTENBERGand A. VARDY, Which codes havecycle-free Tanner graphs?, preprint, 1997.

[83] M.A. TSFASMAN and S.G. VLADUTS, Algebmic Geome-try Codes, Dodrecht: Kluwer Academic, 1991.

[84] M.A. TSFASMAN, S.G. VLADUTS, and T. ZINK, Mod-ular curves, Shimura curves, and Goppa codes bet-ter than the Varshamov-Gilbert bound, Math. IVach-richten, vol. 104, pp. 13–28, 1982.

[85] P. VAN EMDE BOAS, Another NP-complete partitionproblem and the complexity of computing short vectorsin a lattice, Tech. Report 81-04, Dept. of Mathematics,Univ. of Amsterdam, 1980.

[86] A. VARDY, Tkellis structure of block codes, to appear inHandbook of Coding Theory, R. Brualdi, W.C. Huifman,and V. Pless (Ed.), Amsterdam: Elsevier.

[87] A. VARDY and F.R. KSCHISCHANG, Proof of a conjec-ture of McEliece regarding the expansion index of theminimal trellis, IEEE Tkans. Inform. Theory, vol. 42,pp. 2027-2033, 1996.

[88] A. VARIX, J. SNYDERS, and Y. BE’ERY, Bounds on thedimension of codes and subcodes with prescribed con-traction index, Linear Algebra Appl., vol. 142, pp. 237–261, 1990.

[89] S.G. VLADUTS, G.L. KATSMAN, and M.A. TSFASMAN,

Modular curves and codes with polynomial complex-ity of construction, Problemy Peredachi In formatsii,vol. 20, pp. 47–55, 1984, (in Russian).

[90] D.J.A. WELSH, Combinatorial problems in matroidtheory, pp. 291–307 in Combinatorial Mathematics andits Applications, D.J.A. Welsh (Ed.), London: Aca-demic Press, 1971.

[91] N. WIBERG, Codes and decoding on general graphs,Ph.D. thesis, University of Linkoping, Sweden, 1996.

[92] N. WIBERG, H.-A. LOELIGER, and R. KOTTER, Codesand iterative decoding on general graphs, Euro. Trans.Telecommun., vol. 6, pp. 513-526, 1995.

[93] J.K. WOLF, Efficient maximum-likelihood decoding oflinear block codes using a trellis, IEEE Trans. Inform.Theory, vol. 24, pp. 76-80, 1978.

[94] V.V. ZYABLOV, An estimate of the complexity of con-structing binary linear concatenated codes, ProblemyPeredachi Informatsii, vol. 7, pp. 5-13, 1971.

109