Lecture5Bpapenlab1.ucsd.edu/~coursework/WES/slides/Lec5B-Modulation_Coding1.pdf · Codes Algebraic Block Codes CheckMatrix ErrorPer-formance Repetition Codes Syndrome CommonChannelCodingArrangement

Lecture 5B

Preview ofNextQuarter

ChannelCodingDiversity

Soft and HardDecisionDecoding

Types ofChannelCodes

OverallSystem

CodeStructureand CodeRate

Classes ofCodes

AlgebraicBlockCodesCheck Matrix

Error Per-formance

RepetitionCodes

Syndrome

Lecture 5BCoding-1

Wireless Embedded Systems - Communication Systems Lab - Winter 2018 Lecture 5B 1

Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Preview of Next Quarter

Baseband Signal

FrequencyTranslation Demodulation

s(t) s(t) r(t)

Codeword Sensewordor sensedcodeword

DetectedCodeword

BasebandModulation

ModulatedSignal

Transmitter

WirelessChannel

ReceivedSignal

ElectricalSignal

Filtering/Sampling

SymbolDetection

Channel Receiver

Continuous Discrete Discrete

r

s s

CodewordDetection

SampleSequence

r(t) rk

Lab 1, 2 Lab 5 Lab 3 Lab 6Lab 6 Lab 2, 4

Block Diagram of a Wireless Communication System

Five labs next quarter are1 Coding2 OFDM-I3 OFDM and synchronization4 MIMO-I5 MIMO-II Space-time coding

Overall theme - using multiple degrees of freedom (diversity) to increaseinformation rate in the presence of fading


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Channel Coding

Channel coding is the intentional introduction of symbol dependencieswithin a data sequence.

This type of coding is different than source coding used to removeredundancies in source data files such as video (MPEG) or pictures(JPEG).

These dependencies are a flexible degree of freedom that can be used inconjunction with the modulation format to reduce errors caused bychannel distortion and the effect of noise.

Channel codes are designed for the type of error.

Errors may occur randomly and independently or a burst of correlatederrors may occur.

The kind of error determines the kind of symbol dependencies that areincorporated into the code.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Diversity

The use of the available spatial, temporal, and polarization subchannelsthat the channel supports create the symbol dependencies is calleddiversity .

The resulting dependent symbol structure “spreads” the transmittedinformation across the available subchannels.

The redundant information conveyed in these subchannels can producereliable communication when individual subchannels are corrupted.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Types of Detection/Decoding

The sequence of noisy sensed symbols values is called the sensedcodeword or senseword .

Assume synchronization so that the symbols in the senseword match thesymbols in the transmitted codeword.

There are two types of detection. In hard-decision detection, eachreceived symbol is detected separately to a logical symbol value.

Logical values used for a hard-valued senseword that is passed to thedecoder.

Called a hard-decision detection with hard-decision decoding.

Senseword can also have “soft” quantized valuesSenseword may be considered as essentially continuous -valued sequencethat is passed to the decoder.

Called soft-decision detection with soft-decision decoding.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Types of Channel Codes

A data transmission code controls errors by using knowledge about theanticipated types of noise and other channel impairments

Early examples of error-control codes were designed to decodehard-detected symbols.Modern channel decoders most commonly work with soft-detectedsymbols, and the notion of a symbol error is not relevant.

Data modulation codes use knowledge about the propagationcharacteristics of the channel to prevent specific undesired patterns oftransmitted symbols.

Examples include controlling the spectral characteristics of the modulatedwaveform

producing a null in the transmitted power density spectrumAlso be used to prohibit problematic symbol patterns.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Common Channel Coding Arrangement

Data TransmissionEncoder

DatawordIn

Data ModulationEncoder

DemodulatorData ModulationDecoder

Channel

DatawordOut

Data TransmissionDecoder

Modulator


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Organization of Codes

A dataword is first encoded using a data transmission code - typicallyproduces discrete outputs (outer code.)

These discrete symbols are encoded using a modulation code (inner code)with the output typically being a sequence of symbols that is used by themodulator

The modulated waveform propagates through a channel whereimpairments such as distortion, noise, and interference are introduced.

The Rx signal is first demodulated, producing hard-detected orsoft-detected symbols.

Symbols are input first to a data modulation decoder then to a datatransmission decoder.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Overall System Including Coding

DAC

ADC coderEC

coderSource

Source EC

Mul

tiple

xing

De-modulator

Modulator

decoding decoding Detection Equaliser

MEDOMCEDOC

pulse shaping

linecoding/

Dem

ultip

lexi

ng

2

Codec uses both source coding (ie. JPEG) and error control (EC) codingbefore modem

Modem applies additional line code and pulse shaping

Receiver compensates for effects of channel (equalization)


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Datawords, Codewords and Code Rate

Encoding maps a dataword into a codeword.A dataword d is a sequence of length k of symbols from a specifiedalphabet (set of logical symbols)

Binary alphabet {0, 1} or the q-ary alphabet {0, 1, . . . , q}.

A codeword c is a sequence of length n of symbols in the same alphabet,where n is larger than k for a data transmission code.The code rate is defined as the ratio Rc

.= k/n.

The code rate has units of data symbols per code symbol (or data bitsper codebit)


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Example

A binary code for data transmission maps binary datawords of length kinto binary codewords of length n.Out of the 2n possible binary words of length n, there are 2k codewords.Thus, for n = 7 and k = 4, of the 128 possible seven-bit words, onlysixteen are codewords.The remaining 112 words are not codewords, and cannot be transmitted.The encoder maps each of the 16 four-bit datawords into one of the 16seven-bit codewords, and sends the codeword to the channel.This example is a very small code.A modern code may have k and n in the thousands.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Effect of the Channel

Propagation through a noisy and impaired channel and the subsequentdemodulation and detection creates a received sequence of noisy symbolvalues called the noisy sensed codeword or the senseword .In hard-decision detection, each received symbol is separately detectedto produce a logical value that is passed to a hard-decision decoder.In soft-decision detection, each received sample is separately quantized.

this be regarded as equivalent to the continuous sample

Sample is passed to a soft-decision decoder.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Performance of a Code

The performance of a channel code is characterized by the probability pe

of a decoding error.The performance is also characterized by the coding gain, — reduction inthe energy used for the codeword as compared to the energy used for anuncoded dataword that achieves the same probability of a codeword error.To achieve reliable transmission, coding requires an adjustment of eitherthe bandwidth, the data rate, or the signal constellation.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Performance of a Code

Modern coding theory recognizes three broad classes of codes:Algebraic block codesConvolutional codesComposite codes

Each kind is designed for a certain class of decoderSpherical decoders (for block codes)Sequential decoders (for convolutional codes),Iterative decoders (for composite codes).


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Code Rates for Each Class of Code

Each class is most suitable for a particular range of code rates defined bythree fundamental rates:

The critical rate Rcrit (practical limit of spherical decoding of blockcodes.)The cutoff rate R0, (fundamental limit of sequential decoding ) andThe capacity C. (Maximum information rate for a channel.)

These rates satisfy the inequalities

0 < Rcrit < R0 < C.

There are three intervals of the real line implied by the inequalities.The nature of the coding problem is different for code rates in each ofthe three intervals.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Range of Rates for each Type of Code/Decoder

Spherical decoding based on minimum distance is typically used coderates Rc smaller than the critical rate Rcrit.

In this regime, algebraic block codes, such as Reed-Solomon codes, aresuitable.

Sequential decoding based on the maximum-likelihood principle ispractical only for code rates Rc smaller than the cutoff rate R0. Exampleis a convolutional code.Iterative decoding based on the componentwise maximum-posteriorprinciple is practical for code rates Rc smaller than the capacity C.

In this regime, composite codes such as turbo codes (Berrou codes) orlow-density parity-check codes (Gallager codes) are suitable.

For code rates larger than the capacity C, every code is bad.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Nesting of Codes

Algebraic codes are useful primarily for low code rates such as code ratessmaller than Rcrit.For this reason, two layers of coding are sometimes used

OuterEncoder

Userdata

InnerEncoder

To the Information

Channel

Data Transmission Encoder

The inner code may be a composite code, a convolutional code, or evenan algebraic code suitable for the actual information channel.The outer encoder sees this new information channel, possibly in themultibit alphabet obtained by converting the binary datastream into anm-bit wide parallel datastream.The outer encoder sees an different channel, possibly with a critical ratecloser to the capacity.From an operational point of view, this structure can be understood asfollows.

The inner decoder reduces the error rate.The outer decoder then deals with the infrequent, but difficult, errorpatterns that are incorrectly decoded by the inner decoder.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Example - Coding for DVB-S Standard

Functional block diagram of the System

Source coding is MPEG-2 transport streamPacketized version of MPEG-2 on DVD - more robust to transmissionerrorsOuter coder is block code

Inner coder is convolutional - (modern standards replaced by turbo code orLDPC code.)


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Binary, linear, block codes

Linear codes Binary codes where the modulo-2 addition of twocodewords A + B produces another codeword C

Codes use linear algebra on an finite field (can only have specificnumber of letters)

Operation + represents the modulo-2 addition or the Boolean XORoperation. The componentwise modulo-two multiplication of two binarycodewords is the componentwise AND operation.Truth tables

× 0 10 0 01 0 1

+ 0 10 0 11 1 0

If the check bits are added either before or after the data bits, then theencoding process is systematic.

If the check bits are interleaved with the data bits, then the encodingprocess is nonsystematic.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Hamming Weight and Distance

The properties of linear block codes are described by the following terms:

A linear block code is a block code for which the componentwise sum ofany two codewords, c1 and c2, is equal to another codeword denotedc1 + c2.

The Hamming weight w(c) of the codeword c is the number of nonzerocomponents of c.

The minimum Hamming weight wmin of a code is the smallest Hammingweight of any nonzero codeword of the code.

The Hamming distance d between two codewords is the number of placesat which the two codewords differ.

The minimum Hamming distance dmin is the minimum of the Hammingdistance over all pairs of distinct codewords.A maximum-distance code is a code that satisfies dmin = n− k + 1.

A code is cyclic if whenever c = {c0, c1, ...cn−1} is a codeword,c′ = {cn−1, c0, ...cn−2} is also a codeword.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Generation of Codewords

The codewords are generated using a set of linear equations and themodulo-two addition and multiplication operations.

This encoding operation can be expressed as a modulo-two matrixmultiplication

a = dG, (1)

d is the dataword of length k expressed as a row vectora is the codeword of length n,G is a k by n matrix with linearly independent rows that is called thegenerator matrix.

The generator matrix specifies the encoding operation.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Decoding

The decoding operation can also be specified by a matrix equation. Everycodeword a satisfies the following equation

aHT = 0, (2)

where H is a n− k by n matrix called the check matrix.

The check matrix is related to the generator matrix G by

GHT = 0. (3)

Given that the rows of G are linearly independent, H is uniquely definedby (3)

A linear (n, k) code is completely specified by the check matrix H.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Check Matrix

For a binary linear block code, the elements of H are ones and zeros.The elements of H that are equal to one define the dependencies betweenthe bits in the code.

A low-density parity-check code is a linear block code for which the checkmatrix has a low density of ones.

These concepts are illustrated by describing two trivialsingle-error-correcting binary repetition code.

This code simply repeats the value of a single data bit n times to createthe codeword.

There are only two codewords for this code—the all-zero codeword andthe all-one codeword.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Repetition Code

This code simply repeats the value of a single data bit n times to createthe codeword.

There are only two codewords for this code—the all-zero codeword andthe all-one codeword.

This code has a minimum Hamming distance of n, a code rateRc = 1/n, and is often described as a (n, 1, n) code where the thirdnumber specifies the minimum Hamming distance. If n = 3, then thegenerator matrix is

G =[

1 1 1], (4)

and the check matrix is

H =

[1 0 10 1 1

]. (5)

We will learn that this (3,1,3) code has a rate Rc = 1/3, a minimumHamming distance dmin = 3, and can correct one error in a block of three.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Hamming Code

A binary Hamming code is a binary linear block code with a datawordblocklength k = 2m − 1−m where m is an integer not smaller thanthree, and a codeword blocklength n = 2m − 1.

The number of check bits is n− k = m. For m = 3, this is the (7,4)Hamming code with a rate Rc = 4/7.

A check matrix of the (7,4) Hamming code in systematic form is

H =

[ 1 0 0 0 1 1 10 1 0 1 0 1 10 0 1 1 1 0 1

].

The three rows of the matrix give the three check equations which are

c1 + c5 + c6 + c7 = 0 (6)c2 + c4 + c6 + c7 = 0 (7)c3 + c4 + c5 + c7 = 0. (8)


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Error Detection and Correction for Spherical Decoding

Consider a transmitted codeword c that is corrupted producing one ormore errors in the hard-detected senseword.

These errors can be represented as an error pattern e such that thesenseword r is given by

r = c + e

We first consider the conditions for which the decoder can determine if thereis an error in the senseword.

An error cannot be detected if the error pattern e transforms atransmitted codeword into another codeword.

If the code has a minimum Hamming distance of dmin, then if etransforms one codeword into another codeword, the number of errorst ≥ dmin.

If dmin > t, then a properly designed decoder can detect an error.Alternatively, the number of errors the decoder can detect with certainty isdmin − 1


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Spherical Decoding Figure

t

dmin

(a) (c)

Hammingsphere

Possiblesensewords

Codeworddmin

r

c2

r

(b)

c1

c1

c2

(a) Hamming sphere about each codeword.(b) A senseword that produces a codeword error.(c) A senseword that produces a decoding failure.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Number of Errors and Minimum Hamming Distance

In order for the decoder to correct a detection error, it must determinewhich codeword c is closest to the senseword r as measured by theHamming distance.

Each error increases the distance between the senseword r and thetransmitted codeword c.

Suppose that the number of errors is such that the noisy word r ishalfway between two codewords a and c separated by dmin.

Figure on previous slide.

Therefore, error correction is only possible if t < dmin/2 or 2t + 1 ≤ dmin.

The error correction capability of a block code is then

t = b(dmin − 1)/2c . (9)


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Example Repetition Codes

A repeat n binary code copies the value of a single bit n times to producethe codeword.

There are only two codewords for this code − the all-zero codeword andthe all-one codeword.

This (n, 1) code has a Hamming distance of n and a rate Rc = 1/n.

For example, if n = 3, then the generator matrix is

G =[

1 1︸︷︷︸PT

1︸︷︷︸Ik

], (10)

The check matrix is

H =

[1 00 1︸︷︷︸In−k

11︸︷︷︸P

](11)

This (3,1) code has a rate Rc = 1/3 and a minimum Hamming distancedmin = 3.

Therefore, this code can detect two errors and correct one error becauset = b(3− 1)/2c = 1.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Sensewords for Repetition Code

The complete list of all possible three-bit sensewords of the (3, 1, 3) repetitioncode for the all-zero codeword is given in Table 1 along with the kind ofoutput from the spherical decoder.

000010100001

}Correct decoding

011110101111

}Decoding error

Table: Table of all possible sensewords for the (3, 1, 3) repetition code along with thekind of output.

This kind of code always generates a codeword although the codeword may bein error.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Decoding Repetition Code

The decoding may be accomplished by a majority logic decoding rule.

This rule determines the output value of the decoder based on themajority of the three code bits.

Any single bit error will only invert one of the three code bits. Themajority logic decoding rule will correct these errors, which are listed inthe first column of Table 1.

If two or three bits are inverted, which is the second column of Table 1,then the transmitted codeword (000) is decoded as (111). This is adecoding error.

For this kind of code, all eight possible sensewords are decoded.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Syndrome-1

Algebraic codes have a simple function of the senseword, called thesyndrome S

depends only on the error pattern, and not on the codeword.

The syndrome equals zero if there are no errors.

Using r = c + e, where c is a codeword and e is an error pattern, andbecause cHT = 0, it follows that rHT = eHT .

The left side of this equation is easily computed from the senseword r.

The right side of this equation, which is a vector of length n− k,depends on only the error pattern.

It is the syndrome of the error pattern given by

S = eHT , (12)

where S has length n− k, and the error pattern e has length n andweight at most t.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Syndrome-2

All error patterns e inside a decoding sphere must have a uniquesyndrome

An error pattern e of weight at most t can be recovered from thesyndrome S by inverting (12) to give a solution for a Hamming weight atmost t.

This inversion is nontrivial because H is not square.

Once e is recovered, the codeword c is simply r− e.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Syndrome-example

As an example, suppose that a (7,4,3) Hamming codeword is transmittedwith check matrix

H =

[ 0 1 1 1 1 0 01 0 1 1 0 1 01 1 0 1 0 0 1

]. (13)

and that there is a single bit error.

Applying (12), each error pattern e that has a single error produces aunique syndrome S

equal to the corresponding row of HT .For e = (0100000), expression (12) produces the syndrome S = (011),

second row of HT given in (13).

This means that there is an error in the second bit of the sensewordbecause every column of H is distinct.

This error can be corrected by inverting the second bit in the senseword.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Syndrome-example-2

Now suppose that there are two errors.

The form of (12) means that the syndrome S is now generated from thesum of two rows of HT .

For example, the double error pattern e = (1001000) produces thesyndrome S = (111), which is the fourth row of HT given in (13).

This is the same syndrome that would be generated by a single bit errorin the fourth component of the senseword, and would be miscorrected assuch.

Therefore, for this small code, more than one error will always bemiscorrected.


Lecture 5B





OverallSystem


Classes ofCodes


Error Per-formance

RepetitionCodes

Syndrome

Calculation of Syndrome

Although S is a linear function of e, the inverse calculation of e from S isnot linear because HT is a nonsquare n by n− k matrix.For large codes, this computation can be nontrivial.

One reason for the popularity of Reed-Solomon codes is that their uniquealgebraic structure is amenable to tractable algorithms that can solve(12) to determine the lowest Hamming-weight error pattern e from S

even though HT is large and nonsquare.

This lowest-weight error pattern is the most likely pattern that producedan error.


Documents

Lecture5Bpapenlab1.ucsd.edu/~coursework/WES/slides/Lec5B-Modulation_Coding1.pdf · Codes Algebraic Block Codes CheckMatrix ErrorPer-formance Repetition Codes Syndrome CommonChannelCodingArrangement