EDI042 – Error Control Coding 2 (Kodningsteknik) Lentmaier EDI042 – Error Control Coding: Chapter 1 9/26 Application examples 2D Barcodes: I Quick response codes (QR codes) use

EDI042 – Error Control Coding(Kodningsteknik)

Chapter 1: Introduction

Michael Lentmaier

September 2, 2013

Outline

1 Course Information

2 Introduction

3 History of coding

4 The channel capacity limit

5 Content of the course

6 Connection to other fields

Michael Lentmaier EDI042 – Error Control Coding: Chapter 1 1 / 26

Course overview

I Lectures:Michael Lentmaier, [email protected], E:2375Mondays 10.15 – 12.00 and Tuesdays 15.15 – 17.00

I Seminars:Saeedeh Moloudi, [email protected] 13.15 – 15.00

I Problem solving:Saeedeh MoloudiThursdays 15.15 – 17.00

All these events take place in room E:3139.I Secretary:

Doris Glöck, E:3152hI Web: http://www.eit.lth.se/kurs/EDI042


Course overview

Projects:I Two Matlab projects to solve at homeI Week 4: decoding of convolutional codesI Week 6: Reed-Solom codesI To be handed in two weeks after handoutI May be done in groups of two

Final exam:I Written examI Thursday, October 24, 8.00 – 13.00I Open book exam:

you are allowed to bring books and notes with youI Oral exam per request (only grade 3 or fail)


Recommended literatureI Lecture slides and some other material will be made available

during the course on the course webpage or as handouts.I The course does not strictly follow any book, but if you are

interested in additional reading the following books arerecommended:

Martin Bossert, “Channel Coding forTelecommunications”, Wiley, 1999,ISBN 978-0-471-98277-7

Shu Li, Daniel J. Costello, Jr., “ErrorControl Coding”, Prentice Hall, 2004,ISBN 978-0-130-42672-7


Outline


2 Introduction

3 History of coding





What is error control coding?

The term coding is used in different contexts:

I Cryptography:encoding (encryption) of messages into an unreadablecyphertext

I Source coding:compression of data, either lossless (e.g., ZIP, FLAC) or lossy(e.g., JPG, MP3)

I Error control coding:reliable transmission over unreliable channels

Error control coding is also called channel coding or forward errorcorrection (FEC).


Communication System Model

Communication: transmission of data from a source to a sink

Source'

Sink'

Source'Encoder' Encryp/on' Channel'

Encoder'

Source'Decoder'

Channel'Decoder'

Decryp/on'

Modulator'

Demodulator'

Physical'Channel'

Equivalent'Channel'

While the source encoder removes redundant parts of the data,the channel encoder creates redundancy in a controlled way.


Communication System Model

The source and the sink of the transmission can be separatedI in spaceI in timeI in space and time

Examples of physical channels:I wireless link: radio or free-space opticalI wire or optical fibreI hard disk (magnetic, flash), CD, DVD (optical)

Modulation: conversion of digital signal to waveforms that aresuitable for transmission over the physical channel

Channel coding: protection of the information to enablereliable transmission


Application examplesCoding for audio compact disc (CD):

I protection against dirt and scratches on surfaceI Reed-Solomon codes are used for error detection and

error correctionCoding in mobile communications:

I fading leads to burst errors of several hundreds of bitsI GSM system uses convolutional codes with interleavingI UMTS / LTE: more modern turbo codes with iterative decoding

Coding in deep space communications:I Voyager mission to Jupiter, Saturn, Uranus, and NeptuneI concatenation of Reed-Solomon codes and convolutional codesI Modern CCSDS standard (Consultative Committee for Space

Data Systems): uses LDPC (low-density parity-check) codes


Application examples

2D Barcodes:

I Quick response codes (QR codes)use Reed-Solomon error correction

I different levels of error protectionare possible

Data centers: (Google, DropBox, etc.)I storage of massive data is currently a very hot topicI efficient handling of failures is required

! on the fly replacement of damaged hard disksI how can the data be efficiently reproduced?

! error control coding over several disks


Why is error correction possible?

Example

“Educafion is wzat rempins aftqr onx hay porgotten uhat kne

has lehrned in sctool.”

Albert Einstein

I Despite of 11 erroneous letters the text in this example can stillbe well understood

I Language contains redundancy which allows the reader todecode the errors

I Shannon showed that English prose has a redundancy of morethan 50%


From error detection to error correction

Detection of an error can be achieved by simple addition of a paritybit xn = Ân�1

i=1

xi: (addition is carried out modulo-2)

1010010 ! 10100101

A single error can be detected by Âni=1

xi 6= 0.An extension to higher order fields (non-binary numbers) is possible.

Hamming, 1947

“Damn it, if the machine can detect an error, why can’t it

locate the position of the error and correct it?”


Hamming’s idea:use several parity-checks simultaneously

u1

u2

u3u4

p1 p2

p3

p1

= u1

+u2

+u3

p2

= u1

+u3

+u4

p3

= u2

+u3

+u4

Hamming codes are able to correct arbitrary single errors.


An experiment:1. Source: picks one of 16

possible messages2. Encoder:

selects codeword from table3. Channel:

flips one arbitray symbol4. Decoder:

chooses closest codewordand outputs correspondingmessage

Assume that the following vector isreceived:

1001101

Which message was transmitted?

message codeword0000 00000000001 11010010010 01010100011 10000110100 10011000101 01001010110 11001100111 00011111000 11100001001 00110011010 10110101011 01100111100 01111001101 10101011110 00101101111 1111111


Outline


2 Introduction

3 History of coding





History of coding theory (selection)

1948 Shannon: foundation of information theory1949 Golay: first paper on coding1950 Hamming: publishes his codes (known before 1948)1954 Elias: product codes1954 Reed-Muller codes1955 Elias: convolutional codes1960 Reed Solomon codes / BCH codes1962 Gallager: LDPC codes1967 Viterbi algorithm1969 Berlekamp Massey algorithm1978 Imai/Hirakawa: multilevel coded modulation1981 Tanner: codes on graphs1982 Ungerböck: trellis coded modulation1993 Berrou et al: turbo codes1999 Jimenez/Zigangirov: LDPC convolutional codes


Outline


2 Introduction

3 History of coding





Information theory

Shannon introduced a mathematical theory of communication thatmodels information by means of probabilities.

Data Source

Channel Encoder

Channel Channel Decoder

Data Sink

U X Y U

pY |X(y|x)

Shannon’s measure of information

H(X) = �Âxi

PX(xi) log

2

(PX(xi)) uncertainty / entropy

can be used to characterize the channel by its mutual information

I(X;Y) = H(Y)�H(Y|X) = H(X)�H(X|Y)


Information theory

Theorem (Channel Coding Theorem)

For every code rate

R < C = max

p(X)I(X;Y)

there exists a code for which the probability of error after decoding

approaches zero.

Conversely, if R > C, then significant distortion must occur.

I the parameter C is called channel capacityI it represents a fundamental limit on the achievable transmission

rate (bits per channel use) for reliable communicationDrawbacks:

I encoding over long sequences of information is assumedI a constructive method for efficient encoding/decoding

is not provided


Channel capacity for discrete constellationschair

Modern Coding Theory Lecture Slide 2

Channel Capacity and Coding

Source: Bernd Friedrichs,„Kanalcodierung“, Springer 1995.

M-ASK modulation(amplitude shift keying)

without coding

AWGN channel

I ASK modulation(amplitude shift keying)

I dots show uncodedtransmission for givenPb = 10

�5

I Source:Bernd Friedrichs,“Kanalcodierung”,Springer 1995.

Coded modulation:coding gains possible withoutincreasing the bandwidth!

Within this course we considerbinary constellations only.


The coding theory challenge chair

Modern Coding Theory Lecture Slide 5

Development of Coding Systems

Source: CostelloSource: D.J. Costello, Jr., “Modern Coding Theory”, Lecture at the Third Canadian Summer School on Communicationsand Information Theory, Banff, Alberta, Canada, August 19, 2008


Outline


2 Introduction

3 History of coding





Outline of the course

Content:I Chapter 1: IntroductionI Chapter 2: Principles of Error Control CodingI Chapter 3: Optimal Decoding MethodsI Chapter 4: Reed-Solomon CodesI Chapter 5: Concatenated Codes

After this course you should understand:I general principles of codingI important coding schemes: binary block codes, RS codes,

convolutional codes, concatenated codesI common methods of decoding: algebraic decoding,

ML/MAP decoding, iterative decoding


Outline


2 Introduction

3 History of coding





Connection to other fields

Detection and estimation theory (signal processing)I Coding can be considered as a detection problem. In the linear

case it has the form:y = A ·x+n

where n is the noise, y the observations and x the dataI Due to the discrete nature of the data, a linear detector

(zero-forcer or linear MMSE) is far from optimal

Mathematical optimizationI Finding the linear least squares solution can also be casted as a

linear optimization problem (linear programming)I Problem: integer least squares is known to be NP-hardI Solving relaxations of this problem for codes with sparse

representation (e.g., LDPC codes) is currently anactive area of research


Connection to other fields

Detection in communication systemsI Detectors for discrete linear models occur frequently in

communication systemsExamples: channel equalization, MIMO detection

I Methods that originate from coding can be used to solve theseproblemsExamples: Viterbi equalization, sphere detection

Iterative communication receiversI Codes on graphs and iterative decoding has gained a lot of

interest in the last decadesI The success of iterative decoding has stimulated iterative

processing between various different components of acommunication system (turbo processing)

I Analysis tools that originated in coding are nowadaysfrequently used to design efficient iterative systems


Documents

EDI042 – Error Control Coding 2 (Kodningsteknik) Lentmaier EDI042 – Error Control Coding: Chapter 1 9/26 Application examples 2D Barcodes: I Quick response codes (QR codes) use