Soft Decision Decoding Algorithms of Reed-Solomon CodesSoft Decision Decoding Algorithms of Reed-Solomon Codes

Jing Jiang and Krishna R. Narayanan

Department of Electrical Engineering

Texas A&M University

Historical Review of Reed Solomon Codes

Date of birth: 40 years ago (Reed and Solomon 1960)

Related to non-binary BCH codes (Gorenstein and Zierler 1961)

Efficient decoder: not until 6 years later (Berlekamp 1967)

Linear feedback shift register (LFSR) interpretation (Massey 1969)

Other algebraic hard decision decoder:Euclid’s Algorithm (Sugiyama et al. 1975)

Frequency-domain decoding (Gore 1973 and Blahut 1979)

Wide Range of Applications of Reed Solomon Codes

NASA Deep Space: CC + RS(255, 223, 32)

Multimedia Storage:CD: RS(32, 28, 4), RS(28, 24, 4) with interleavingDVD: RS(208, 192, 16), RS(182, 172, 10) product code

Digitial Video Broadcasting: DVB-T CC + RS(204, 188)

Magnetic Recording: RS(255,239) etc. (nested RS code)

Basic Properties of Reed Solomon Codes

12

)()( where,)()()( :form polynomialGenerator (2)tb

bi

ixxgxmxgxc

)12)(1(12

)1(

1

1

:matrixcheck parity ThetbNtb

bNb

H

1N where),(over defined K) RS(N, mm qqGF

))(),...,(),((),...,,()( :form evaluation Polynomial (1) 11021 NN fffxxxxc 1

110 ...)( where K

K xfxffxf

)(in elements nonzerodistinct are s mi qGF

Properties of BM algorithm:

Decoding region:

Decoding complexity: Usually

Basic Properties of Reed Solomon Codes (cont’d)

min2 dfe

Properties of RS code:

Symbol level cyclic (nonbinary BCH codes)

Maximum distance separable (symbol level): 112min KNtd

)( 2No2mind

Motivation for RS Soft Decision Decoder

Hard decision decoder does not fully exploit the decoding capability

Efficient soft decision decoding of RS codes remains an open problem

RS Coded Turbo Equalization System

-

+

a priori

extrinsicinterleaving

a priori

extrinsic

ΠΣ

source

RS Encoder

interleaving

PR Encoder

sink

hard decision

+

AWGN+

RS Decoder

Channel Equalizer

de-interleaving

Π

1Π

Σ

Soft input soft output (SISO) algorithm is favorable

Presentation Outline

Iterative decoding for RS codes

Symbol-level algebraic soft decision decoding

Simulation results

Binary expansion of RS codes and soft decoding algorithms

Applications and future works

Symbol-level Algebraic Soft Decision Decoding

Reliability Assisted Hard Decision Decoding

Generalized Minimum Distance (GMD) Decoding (Forney 1966):New distance measure: generalized minimum distanceSuccessively erase the least reliable symbols and run the hard decision decoderGMD is shown to be asymptotically optimal

Chase Type-II decoding (Chase 1972):Exhaustively flip the least reliable symbols and run the hard decision decoderChase algorithm is also shown to be asymptotically optimal

Related works:Fast GMD (Koetter 1996)Efficient Chase (Kamiya 2001)Combined Chase and GMD for RS codes (Tang et al. 2001)Performance analysis of these algorithms for RS codes seems still open

Bounded distance + 1 decoding (Berlekamp 1996)

Beyond decoding for low rate RS codes (Sudan 1997)

Decoding up errors (Guruswami and Sudan 1999)

A good tutorial paper (JPL Report, McEliece 2003)

Algebraic Beyond Half dmin List Decoding

2/mind

NKN

Outline of Algebraic Beyond Half Distance Decoding

Complexity:Interpolation (Koetter’s fast algorithm): Factorization (Roth and Ruckenstein’s algorithm):

)( 42mNO

)(NKO

Factorization Step: generate a list of y-roots, i.e.:

Pick up the most likely codeword from the list L

}))(deg(),,(|))(( :][)({ KxfyxQxfyxFxfL

)(ˆ xf

Decoding:

Basic idea: find f(x), which fits as many points in pairs

))(),...,(),(( :codeword dTransmitte 21 Nfff ), ... , ,( : vectorReceived 21 N

)),(( iif

),( yxQ

Interpolation Step: Construct a bivariate polynomial of minimum (1,K-1) degree, which has a zero of order at , i.e.:m Nlll ,...,1 ),,(

mjiyxQ ll than less degree of termno involves ),( if

Algebraic Soft Interpolation Based List Decoding

Koetter and Vardy algorithm (Koetter & Vardy 2003)Based on the Guruswami and Sudan’s algebraic list decodingUse the reliability information to assign multiplicitiesKV is optimal in multiplicity assignment for long RS codes

Reduced complexity KV (Gross et al. submitted 2003)Re-encoding technique: largely reduce the cost for high rate codes

VLSI architecture (Ahmed et al. submitted 2003)

Basic idea: interpolating more symbols using the soft information

The interpolation and factorization is the same as GS algorithm

Sufficient condition for successful decoding:

The complexity increases with , maximum number of multiplicity

Soft Interpolation Based Decoding

)()1(2)( MCKcSM

Definition:Reliability matrix:

Multiplicity matrix:

Score:

Cost:

cMcSM ,)(

q

i

n

j

jimMC

1 1

,

2

1)(

Nq

)(gM

2)(MC

Recent Works and Remarks

The ultimate gain of algebraic soft decoding (ASD) over AWGN channel is about 1dB

Complexity is scalable but prohibitively huge for large multiplicity

The failure pattern of ASD algorithm and optimal multiplicity assignment scheme is of interest

Recent works on performance analysis and multiplicity assignment:Gaussian approximation (Parvaresh and Vardy 2003)

Exponential bound (Ratnakar and Koetter 2004)

Chernoff bound (El-Khamy and McEliece 2004)

Performance analysis over BEC and BSC (Jiang and Narayanan 2005)

Performance Analysis of ASD over Discrete Alphabet Channels

Performance Analysis over BEC and BSC (Jiang and Narayanan, accepted by ISIT2005)

The analysis gives some intuition about the decoding radius of ASD

We investigate the bit-level decoding radius for high rate codes

For BEC, bit-level radius is twice as large as that of the BM algorithm

For BSC, bit-level radius is slightly larger than that of the BM algorithm

In conclusion, ASD is limited by its algebraic engine

Binary Image Expansion of RS Codes and Soft Decision Decoding

Binary Image Expansion of RS Codes over GF(2m)

bm xGFx tor binary vec dim-man as expressed becan )2( known that isIt

],...,,[ 1101 NN cccC ],...,,,...,,...,,[ )1(1

)1(1

)0(1

)1(0

)1(0

)0(0)1(

m

NNNm

Nmb ccccccC

Km) (Nm,RSexpansion binary a has )2(over code K)RS(N, ly,Consequent bmGF

NKNKNKN

N

NKN

HHH

HHH

H

),1(1),1(0),1(

1,01,00,0

)(

1,1)(1,1)(0,1)(

1,01,00,0

)(

NmmKNmKNmKN

Nm

NmmKNb

hhh

hhh

H

Bit-level Weight Enumerator

“The major drawback with RS codes (for satellite use) is that the present generation of decoders do not make full use of bit-based soft decision information” (Berlekamp)

How does the binary expansion of RS codes perform under ML decoding?

Performance analysis using its weight enumerator

Averaged ensemble weight enumerator of RS codes (Retter 1991)

It gives some idea about how RS codes perform under ML decoding

Performance Comparison of RS(255,239)

Performance Comparison of RS(255,127)

Remarks

RS codes themselves are good code

However, ML decoding is NP-hard (Guruswami and Vardy 2004)

Are there sub-optimal decoding algorithms using the binary expansions?

Trellis based Decoding using BCH Subcode Expansion

Maximum-likelihood decoding and variations:

Partition RS codes into BCH subcodes and glue vectors (Vardy and Be’ery 1991)

Reduced complexity version (Ponnampalam and Vucetic 2002)

Soft input soft output version (Ponnampalam and Grant 2003)

Subfield Subcode Decomposition

Remarks:Decomposition greatly reduces the trellis size for short codesImpractical for long codes, since the size of the glue vectors is very large

Related work:Construct sparse representation for iterative decoding (Milenkovic and Vasic 2004)

Subspace subcode of Reed Solomon codes (Hattori et al. 1998)

BCH subcodes

Glue vector

4321

000

000

000

000

~

glueglueglueglue

b

GGGG

B

B

B

B

G

Reliability based Ordered Statistic Decoding

Reliability based decoding:Ordered Statistic Decoding (OSD) (Fossorier and Lin 1995)Box and Match Algorithm (BMA) (Valembois and Fossorier 2004)Ordered Statistic Decoding using preprocessing (Wu et al. 2004)

Basic ideas:Order the received bits according to their reliabilitiesPropose hard decision reprocessing based on the most reliable basis (MRB)

Remarks:The reliability based scheme is efficient for short to medium length codesThe complexity increases exponentially with the reprocessing orderBMA algorithm trade memory for time complexity

Iterative Decoding Algorithms for RS Codes

How does the panacea of modern communication, iterative decoding algorithm work for RS codes?

Note that all the codes in the literature, for which we can use soft decoding algorithms are sparse graph codes with small constraint length.

A Quick Question

How does standard message passing algorithm work?

bit nodes…………. ………..

. . . . . . . . . …………….

check nodes

…………….

erased bits

? If two or more of the incoming messages are erasures the check is erasedFor the AWGN channel, two or more unreliable messages invalidate the check

A Few Unreliable Bits “Saturate” the Non-sparse Parity Check Matrix

000000000000000000000bc

Iterative decoding is stuck due to only a few unreliable bits “saturating” the whole non-sparse parity check matrix

011110010001111101100

001111101100011110010

111101100011110010001

001011111110101010100

100001011111110101010

011111110101010100001

bH

Binary image expansion of the parity check matrix of RS(7, 5) over GF(23)

Consider RS(7, 5) over GF(23) :

1.15.01.08.02.09.06.01.02.09.05.01.03.04.08.01.10.17.06.19.08.0 r

Sparse Parity Check Matrices for RS Codes

Can we find an equivalent binary parity check matrix that is sparse?

For RS codes, this is not possible!

The H matrix is the G matrix of the dual code

The dual of an RS code is also an MDS Code

Each row has weight at least (K+1)

Typically, the row weight is much higher

Iterative Decoding for RS Codes

Recent progress on RS codes:Sub-trellis based iterative decoding (Ungerboeck 2003)

Stochastic shifting based iterative decoding (Jiang and Narayanan, 2004)

Sparse representation of RS codes using GFFT (Yedidia, 2004)

Iterative decoding for general linear block codes:Iterative decoding for general linear block codes (Hagenauer et al. 1996)

APP decoding using minimum weight parity checks (Lucas et al. 1998)

Generalized belief propagation (Yedidia et al. 2000)

Recent Iterative Techniques

Sub-trellis based iterative decoding (Ungerboeck 2003)

Self concatenation using sub-trellis constructed from the parity check matrix:

Remarks:Performance deteriorates due to large number of short cyclesWork for short codes with small minimum distances

011110010001111101100

001111101100011110010

111101100011110010001

001011111110101010100

100001011111110101010

011111110101010100001

bH

Binary image expansion of the parity check matrix of RS(7, 5) over GF(23)

Recent Iterative Techniques (cont’d)

Stochastic shifting based iterative decoding (Jiang and Narayanan, 2004)

Due to the irregularity in the H matrix, iterative decoding favors some bits

Taking advantage of the cyclic structure of RS codes

],,,,,,[ 4321065 rrrrrrr ],,,,,,[ 6543210 rrrrrrr

Stochastic shift prevent iterative procedure from getting stuck

Best result: RS(63,55) about 0.5dB gain from HDD

However, for long codes, the performance deteriorates

Shift by 2

1011001

0110011

0001111

H

Proposed Iterative Decoding for RS Codes

Iterative Decoding Based on Adaptive Parity Check Matrix

transmitted codeword 0011010c

Idea: reduce the sub-matrix corresponding to unreliable bits to a sparse nature using Gaussian elimination

For example, consider (7,4) Hamming code:

parity check matrix

1010110

1100101

1101010

H

1011001

0110011

0001111

H

1011001

0110011

0001111

H

We can make the (n-k) less reliable positions sparse!

received vector 1.01.02.14.11.06.01.1 r

Adaptive Decoding Procedure

bit nodes…………. ………..

. . . . . . . . . …………….

check nodes

…………….

unreliable bits

After the adaptive update, iterative decoding can proceed

Gradient Descent and Adaptive Potential Function

The decoding problem is relaxed as minimizing J using gradient descent with the initial value T observed from the channel

J is also a function of H. It is adapted such that unreliable bits are separated in order to avoid getting stuck at zero gradient points: ),( )0( THH

Geometric interpretation (suggested by Ralf Koetter)

Define the tanh domain transform as:

The syndrome of a parity check can be expressed as:

Define the soft syndrome as:

Define the cost function as:

1ijH

ji rs

11

)(ijij H

jH

ji LTS

kn

iiSTHJ

1

),(

)2/tanh()( jjj LLT

Two Stage Optimization Procedure

Proposed algorithm is a generalization of the iterative decoding scheme proposed by Lucas et al. (1998), two-stage optimization procedure:

The damping coefficient serves to control the convergent dynamics

)))(()(( 1 },1|{

)(1)(1)1(

kn

i mjHjj

tm

tm

tm

ij

TTT

),( )()0()( tt THH

Avoid Zero Gradient Point

Adaptive scheme changes the gradient

and prevents it getting stuck at zero

gradient points

Zero gradient point

Variations of the Generic Algorithm

Connect unreliable bits as deg-2

Incorporate this algorithm with hard decision decoder

Adapting the parity check matrix at symbol level

Exchange bits in reliable and unreliable part. Run the decoder multiple times

Reduced complexity partial updating scheme

Simulation Results

AWGN Channels

AWGN Channels (cont’d)

Asymptotic performance is consistent with the ML upper-bound.

AWGN Channels (cont’d)

AWGN Channels (cont’d)

Interleaved Slow Fading Channel

Fully Interleaved Slow Fading Channels

Fully Interleaved Slow Fading Channels (cont.)

Turbo Equalization Systems

Embed the Proposed Algorithm in the Turbo Equalization System

RS Coded Turbo Equalization System

-

+

a priori

extrinsicinterleaving

a priori

extrinsic

ΠΣ

source

RS Encoder

interleaving

PR Encoder

sink

hard decision

+

AWGN+

RS Decoder

BCJR Equalizer

de-interleaving

Π

1Π

Σ

Turbo Equalization over EPR4 Channels

Turbo Equalization over EPR4 Channels

RS Coded Modulation

RS Coded Modulation over Fast Rayleigh Fading Channels

Applications and Future Works

Potential Problems in Applications

Respective problems for various decoding schemes:Reliability assisted HDD: Gain is marginal in practical SNRsAlgebraic soft decoding: performance is limited by the algebraic natureReliability based decoding: huge memory, not scalable with SNRSub-code decomposition: only possible for very short codesIterative decoding: adapting Hb at each iteration is a huge cost

General Problems:Coding gain may shrink down in practical systemsConcatenated with CC: difficult to generate the soft informationPerformance in the practical SNRs should be analyzed

“In theory, there is no difference between theory and practice. But, in practice, there is…” (Jan L.A. van de Snepscheut)

A Case Study (System Setups)

Forward Error Control of a Digital Television Transmission Standard:

Modulation format: 64 or 16 QAM modulation (semi-set partitioning mapping)

Inner code: convolutional code rate=2/3 or 8/9

Bit-interleaved coded modulation (BICM)

Iterative demodulation and decoding (BICM-ID)

The decoded bytes from inner decoder are interleaved and fed to outer decoder

Outer code: RS(208,188) using hard decision decoding

Will soft decoding algorithm significantly improve the overall performance?

A Case Study (Simulation Results)

Future Works

How to incorporate the proposed ADP with other soft decoding schemes?

Taking advantage of the inherent structure of RS codes at bit level

More powerful decoding tool, e.g., trellis

Extend the idea of adaptive algorithms to demodulation and equalization

Apply the ADP algorithm to quantization or to solve K-SAT problems

Thank you!