Linear Block Codes

Outline

• Definition of Linear and systemic codes.

• Generator Matrix.

• Parity check matrix / encoding.

•• Dual code.

• Standard Arrays decoding.

• Syndrome decoding.

• Error detecting and correcting capabilities.

EELE 6338 2

Definition of Linear Codes

• Let u be a k-bit information sequence;

v be the corresponding n-bit codeword.

• A total of 2k n-bit codewords constitute (n,k) code.• A total of 2 n-bit codewords constitute (n,k) code.

• Linear code: The sum of any two codewords is a

codeword.

• Observation: The all-zero sequence is a codeword

in every linear block code.

EELE 6338 3

Systematic codes

• Any linear block code can be put in systematic

form

EELE 6338 4

n-k

check bits

k

information bits

Generator Matrix

• All 2k codewords can be generated

from a set of k linearly independent

codewords.

• Let g0, g

1, …, g

k-1be a set of k

= 1

0

g

g

G

• Let g0, g

1, …, g

k-1be a set of k

independent codewords.

• v = u·G

=

−1

1

kg

g

MG

EELE 6338 5

Generator Matrix (cont’d)

• The simplest choice of this set is the kcodewords corresponding to the information sequences that have a single nonzero element.

• Illustration: The generating set for the (7,4) • Illustration: The generating set for the (7,4) code:

1000 ===> 1101000

0100 ===> 0110100

0010 ===> 1110010

0001 ===> 1010001

EELE 6338 6

Linear Code Example

EELE 6338 7

Generator Matrix (cont’d)

• Every codeword is a linear combination of these 4 codewords.

That is:

v = u∙ G

1 1 0

0 1 1

1 0 0 0

0 1 0 0

• Storage requirement reduced from 2k(n+k) to k(n-k).

EELE 6338 8

[ ]G P | Ik

=

=

× − ×

0 1 1

1 1 1

1 0 1

0 1 0 0

0 0 1 0

0 0 0 1

k n k k k( )

124 34 1 24 34

Generator matrix example (3.2)

EELE 6338 9

Parity-Check Matrix

For G = [ P | Ik], define the matrix H = [I

n-k| PT]

(The size of H is (n-k)xn).

Check that GHT = 0.

Since v = uG, then vHT = uGHT = 0.

EELE 6338 10

H =

1 0 0 1 0 1 1

0 1 0 1 1 1 0

0 0 1 0 1 1 1

=

1000101

0100111

0010110

0001011

G

Encoding Using H Matrix

(Parity Check Equations)

[ ]6543210

011

100

010

001

vvvvvvv =

0

EELE 6338 11

[ ]

3212

2101

3200

6542

5431

6530

6542

5431

6530

6543210

uuu=v

uuu=v

uuu=v

vvv=v

vvv=v

vvv=v

0vvv+v

0vvv+v

0vvv+v

101

111

110

++

++

++

⇒

++

++

++

⇒

=++

=++

=++

information

Encoding Circuit

EELE 6338 12

The Dual Code

• The H matrix, formed from G, has (n-k) rows

of length n that are linearly independent.

• The H matrix can therefore be used to generate

an (n,n-k) linear code C .an (n,n-k) linear code Cd.

• Cd

is called the dual code of the code C

generated by G.

EELE 6338 13

More on the Minimum Distance of Linear Codes

• DF: The weight of a codeword v , denoted by w(v), is the number of the nonzero elements in the codeword.

• DF: Hamming distance between v and w, denoted by d(v,w), is the number of locations where they differ.Note that d(v,w) = w(v+w)

• DF: The minimum weight of a code, w , is the smallest • DF: The minimum weight of a code, wmin

, is the smallest

weight of the nonzero codewords in the code.wmin

= min {w(v): for every v in C; v ≠0}.

• DF: The minimum distance of the code d

min= min {d(v,w): v,w in C, v ≠ w}

• TH3.1: In any linear code, dmin

= wmin

EELE 6338 14

Minimum Distance, Cont.

• TH3.2 For each codeword of Hamming weight l there exists l

columns of H such that the vector sum of these columns is

zero. Conversely, if there exist l columns of H whose vector

sum is zero, there exists a codeword of weight l.

• COL 3.2.2• COL 3.2.2

dmin

of C is equal to the minimum numbers of columns in H

that sum to zero.

• Demo:

H matrix of (7,4)

Hamming code.

EELE 6338 15

=

1110100

0111010

1101001

H

Standard Array (�)

k

k

22322212

2321

vev+ev+ev+e

vev+ev+ev+e

vvv0v

+

+

=

L

L

L

correctab

le error p

atterns

EELE 6338 16

kknknknkn

k

22322212

23332313

veveveve

vev+ev+ev+e

----++++

+

MMMM

L

correctab

le error p

atterns

TH. No two n-tuples in the same row of a standard array are identical. Every

n-tuple appears in one and only one row.

Standard Array (�)

1. List the 2k codewords in a row, starting with the all-zero

codeword c0.

2. Select an error pattern e1

and place it below c0. This error

pattern will be a correctable error pattern, therefore it should

be selected such that:be selected such that:

(i) it has the smallest weight possible (most probable error)

(ii) it has not appeared before in the array.

3. Repeat step 2 until all the possible error patterns have been

accounted for. There will always be 2 n-k rows in the array.

Each row is called a coset. The leading error pattern is the

coset leader.

EELE 6338 17

Standard Array Decoding (�)• For an (n,k) linear code, standard array decoding is able to

correct exactly 2n-k error patterns, including the all-zero error

pattern.

• Illustration 1: The (7,4) Hamming code

# of correctable error patterns = 23 = 8# of correctable error patterns = 23 = 8

# of single-error patterns = 7

Therefore, all single-error patterns, and only single-error

patterns can be corrected.

EELE 6338 18

Standard Array Decoding (�)

Illustration 2: The (6,3) code defined by the H matrix:

H =

1 0 0 0 1 1

0 1 0 1 0 1

Codewords

000000

110001

EELE 6338 19

0 0 1 1 1 0

c = c c

c = c c

c = c c

5

4

4

1 6

2 6

3 5

+

+

+

101010

011011

011100

101101

110110

000111

3dmin=

Standard Array Decoding (�)

• Can correct all single errors and one double error pattern

EELE 6338 20

The Syndrome (�)

EELE 6338 21

The Syndrome (�)

• Ex (3.5), For the (7,4) code, let v=(1001011) and r=(1001001).

• s = rHT=111.

EELE 6338 22

The Syndrome (�)

Error Pattern Syndrome

0000000 0001000000 100

EELE 6338 23

1000000 1000100000 0100010000 0010001000 1100000100 0110000010 1110000001 101

Syndrome Decoding (table-

lookup)Decoding Procedure:

1. For the received vector r, compute the syndrome s = rHT.

2. Using the table, identify the error pattern e.

3. Add e to r to recover the transmitted codeword c.3. Add e to r to recover the transmitted codeword c.

Example:

r = 1110101 ==> s = 001 ==> e = 0010000

Then, c = 1100101

• Syndrome decoding reduces storage memory and the

searching time considerably.

EELE 6338 24

Hardware Implementation

• Let r = r0

r1

r2

r3

r4

r5

r6

and s = s0

s1

s2

• From the H matrix for the (7,4) code:

s0

= r0

+ r3

+ r5

+ r6

s1

= r1

+ r3

+ r4

+ r5

s = r + r + r + rs2

= r2

+ r4

+ r5

+ r6

• From the table of syndromes and their corresponding

correctable error patterns, a truth table can be constructed. A

combinational logic circuit with s0

, s1

, s2

as input and

e0

, e1

, e2

, e3

, e4

, e5

, e6

as outputs can be designed.

EELE 6338 25

Decoding Circuit for the (7,4) HC

Error Pattern Syndrome

0000000 0001000000 100

EELE 6338 26

1000000 1000100000 0100010000 0010001000 1100000100 0110000010 1110000001 101

Error detecting capability

EELE 6338 27

Error Correcting Capability

• # of errors that can be corrected = t =

• A codeword with dmin

can detect all error patterns of weight

dmin

– 1 or less. It can detect many higher error patterns as

well, but not all. Therefore,

−

2

1min

d

EELE 6338 28

∑+=

−−

≤

n

ti

ini

upp

i

nP

1

)1(

Probability of undetected error (�)

EELE 6338 29

Probability of undetected error (�)

EELE 6338 30

Probability of undetected error

(�)

31

Simple Upper Bound on Pu

• Knowing weight distribution

∑=

−

−=

n

di

ini

iuppAP

min

)1(

• The weight distribution is only known to few codes.

• As an upper bound,

)(2

2

12 kn

n

k

uP

−−

≅−

≤

=di min

EELE 6338 32

Next lecture

• We will cover several classes of important

linear block codes.

EELE 6338 33