exercise in the previous class (1)

Consider an “odd” parity check code C whose codewords are(x1, …, xk, p) with p = x1+…+xk+1. Is C a linear code?

No. x1=1, x2=x3=...=xk=0 ⇒ p = 0, and 100...00 is a codeword

x2=1, x1=x3=...=xk=0 ⇒ p = 0, and 010...00 is a codeword

the sum of the two codewords = 110...0, not a codeword

1

exercise in the previous class (2)

Construct a 2D code for 6-bit information (a1, ..., a6) as follows.

determine the generator and parity check matricesencode 011001 using the generator matrixcorrect an error in the sequence 110111001010

2

a1 a2 a3a4 a5 a6

p1p2

q1 q2 q3 r

(a1, ..., a6)→ (a1, ..., a6, p1, p2, q1, q2, q3, r)

p1 = x1 + x2 + x3p2 = x4 + x5 + x6q1 = x1 + x4q2 = x2 + x5q3 = x3 + x6r = x1 + x2 + x3 + x4 + x5 + x6

parity symbols:

exercise in the previous class (3)

3

p1 = x1 + x2 + x3

p2 = x4 + x5 + x6q1 = x1 + x4

q2 = x2 + x5

q3 = x3 + x6

r = x1 + x2 + x3 + x4 + x5 + x6

111000000111100100010010001001111111

coefficients( 係数 )

111000 100000000111 010000100100 001000010010 000100001001 000010111111 000001

100000 101001010000 100101001000 100011000100 011001000010 010101000001 010011

H:G:tr

ansp

ose as is

(0 1 1 0 0 1)G= (0 1 1 0 0 1 0 1 0 1 0 1)

H(1 1 0 1 1 1 0 0 1 0 1 0)T

= (0 1 1 0 0 1)T = the 4-th column110111001010 110011001010

in the previous class...

linear codes: definition, encoding, decoding

one-bit error at the i-th symbol position⇔ 　 syndrome equals the i-th vector of H

if several column vectors in H are the same, then we cannot correct one-bit errors in a codeword.

if all column vectors in H are different, then we can correct all one-bit errors in a codeword.

4

design of error correcting codes

Construct a parity check matrix with all column vectors differ,then we have a one-bit error correcting code.

5

101100110010011001

H= 101001101001111101001

H= 101001010011110100010101

H=OK examples:

110100101010010001

H= 101001111001111101001

H=NG examples:

C = {v | HvT = 0 mod 2},the discussion is easier if the right-submatrix of H is an identity matrix...

construction of a code

6

coefficients101 100110 010011 001

H= 101110011

100 110010 011001 101

G=

transposeas is

p1 = x1 + x3p2 = x1 + x2p3 = x2 + x3

000000,001101,010011,011110,

100110,101011,110101,111000.

codewords

the “shape” of a check matrix

a parity check matrix with m rows and n columns a code with...

7

length# of information symbols# of parity symbols

= n= n – m (= k)= m

100001111

010001010

001000101

000101100

000010011

H

n = 9

m = 5

code length 9= 4 information symbols + 5 parity symbols

“vertically longer” H means more parity symbols in a codeword less number of information symbols not efficient, not favorable...NG good

Hamming code

To design a one-bit error correcting code with small redundancy,construct a horizontally longest check matrix (all columns differ).

Hamming codedetermine m, # of parity check symbolslist up all nonzero vectors with length muse the vectors as columns of H

(any order is OK, but let the right-submatrix be an identity)

8

Richard Hamming1915-1998

m = 3:

1101000

1010100

0110010

1110001

G,

1001101

0101011

0010111

Hlength 7= 4 information + 3 parity

Parameters of Hamming code

Hamming codedetermine m, # of parity check symbolsdesign H to have 2m – 1 different column vectors

H has m rows and 2m – 1 columnslength# of information symbols# of parity symbols

9

n = 2m – 1k = 2m – 1 – mm

m234567

n37

153163

127

k14

112657

120(n, k) code:

code with length n, andk information symbols

comparison of codes

two codes which can correct one-bit errors:(7, 4) Hamming code(9, 4) 2D code

which is the “better”?Hamming code is more efficient (small redundancy)Hamming code is more reliable

correct data transmission with BSC with error prob. p:– Hamming code: (1 － p)7 + 7p(1 － p)6

– 2D code ： (1 － p)9 + 9p(1 － p)8

“shorter is the better”

10

= 0.85= 0.77if p=0.1

codes better than Hamming code?

(7, 4) Hamming code3 parity bits are added to correct possible one-bit errors

Is there a one-bit error correcting (6, 4) code, with only 2 parities?No. Assume that such a code exists, then...

there are 24 = 16 codewords# of vectors decoded to a given codeword = 1+6=7# of vectors decoded to any one of codewords = 7×16 = 112# of vectors with length 6 = 26 = 64, which is < 112

11{0, 1}6

contradiction! ( 矛盾 )

Hamming code is perfect

(7, 4) Hamming codethere are 24 = 16 codewords# of vectors decoded to a given codeword = 1+7=8# of vectors decoded to any one of codewords = 8×16 = 128# of vectors with length 7 = 27 = 128

12

{0, 1}7

all of 128 vectors are exactlypartitioned to 16 classes with 8 vectors

Hamming code is a perfect one-bit error correcting code:

2𝑘((𝑛0 )+(𝑛1 ))=2𝑛

advanced topic: multi-bit errors?

Hamming code is a perfect one-bit error correcting code.Are there codes which correct two or more errors?

Yes, there are many...

one-bit error: syndrome = one column vector of Htwo-bits error: syndrome = sum of two column vectors of H

different combinations of t columns in H results in different sums, the code corrects t-bits errors.

13

advanced topic: two-bits error correcting code

if different two-bits errors results in different syndromes

received 11111000 the syndrome is 110111⇒ T= h2 + h6

⇒ errors at the 2nd and 6th bits

⇒ 10111100 should be transmitted

14

.

10000010

01000010

00100001

00010001

00001011

00000111

H

ability of the code

Error-correcting capability of a code is determined by the relation of column vectors of a parity check matrix.

It is not easy to consider all the combinations of columns.More handy and easy means is needed.

For linear codes, we can useminimum distance, or minimum weight

15

similarity of vectors

a codeword u is sent, errors occur, and v is received:In a practical channel, the distance between u and v is small.

16

u = 000 v = 000, with probability 0.729BSC with error prob. 0.1

v = 001, with probability 0.081v = 011, with probability 0.009

v = 111, with probability 0.001

If there is another codeword u’ near u,then v = u’ occurs with notable probability.

safe not safe

Hamming distance

a=(a1, a2,..., an), b=(b1, b2,..., bn): binary vectors

the Hamming distance between a and b, dH(a, b)

= the number of symbols which are differ between a and b

17

dH(0100,1101) = 2 dH(000, 011) = 2dH(000, 111) = 3 dH(011, 011) = 0

If a vector u with length n is sent over BSC with error prob. p,then a vector v with dH(u, v) = i is received with prob. (1 – p)n–ipi.

inverse correlation between the distance and the probability( 逆相関 )

dH(a, b) = dH(a + b, 0)

Hamming distance between codewords

code with length 4 ... vectors = vertices of a 4-dimensional hyper-cubecodewords = subset of vertices

18

C1={0000, 1100, 0011, 1111}

two or more edgesbetween codewords

good

C2={0000, 0001, 1110, 1111}

some codewords areside-by-side

bad

minimum distance

the minimum Hamming distance of a code C:

19

}.,,|),(min{min bababa Cdd H

C1={0000, 1100, 0011, 1111}

dmin = 2

C2={0000, 0001, 1110, 1111}

dmin = 1

computation of the minimum distance

consider a linear code whose generator matrix is

20

110100

101010

011001

G

000000001011010101011110100110101101110011111000

00000003343443

00101130434334

01010134034334

01111043303443

10011034430334

10110143343043

11001143343403

11100034434330

dmin = 3 for this code

Do we need to consider all of 2k×2k combinations?

minimum Hamming weight

the Hamming weight of a vector u: wH(u) =dH(u, 0)...# of 1s

the minimum Hamming weight of a code C: wmin=min{wH(u) : uC, u 0}

Lemma: if C is linear, then wmin = dmin.

proof of wmin≤dmin:

let u and v be codewords with dmin = dH(u, v).

u + v C, and wmin ≤ wH(u + v) = dH(u + v, 0) = dH(u, v) = dmin.

proof of dmin≤wmin:

let u be the codeword with wmin = wH(u).

dmin ≤ dH(u, 0) = wH(u) = wmin. 21

u

v

0

u + v

examples of minimum Hamming weight

22

(9, 4) 2D code： the minimum Hamming weight is 4000000000

0010010011

4100010101

4110000110

4

0001010114

0101110004

1001111106

1101011016

0010011014

0110111106

1010110004

1110010116

0011001104

0111101016

1011100116

1111000004

(7, 4) Hamming code: the minimum Hamming weight is 30000000

00001011

3

10001013

10011104

01001114

01011003

00101103

11010014

00101103

00111014

10100114

10110003

01100013

01110104

11101004

11111117

general case of Hamming code

lemma: The minimum Hamming weight of a Hamming code is 3.

proof sketch:Let H = (h1, ..., hn) be a parity check matrix:

{h1, ..., hn} = the set of all nonzero vectors

if codeword u with weight 1, then HuT = hi = 0...contradiction

if codeword u with weight 2, then HuT = hi + hj= 0

...this means that hi = hj, contradiction

23

no codewords with weight 1 or 2

proof (cnt’d)

lemma: The minimum Hamming weight of a Hamming code is 3.

proof sketch:Let H = (h1, ..., hn) be a parity check matrix:

{h1, ..., hn} = the set of all nonzero vectors

Choose x, y as you like, and choose z so that hx + hy = hz.

Let u be a vector having 1 at the x-th, y-th and z-th positions,then HuT= hx + hy + hz = 0, meaning that uC.

24

codewords with weight 3 are constructible

minimum distance and error correction

What does dmin=3 mean?

Any two codewords are differ at three or more symbols.At least three-bits errors are needed to change

a codeword to a different codeword.

25

error error error error error

{u’ | dH(u, u’)=1, uC} {v’ | dH(v, v’)=1, vC} =

We can distinguisha result of one-bit error from a codeword u, anda result of one-bit error from other codeword v.

u v u vu’ v’

decoding territory

dmin=3 ： define territories around codewords

radius = 1... territories do not overlapradius = 2... territories do overlap

rule of the error correction:if a received vector r falls in the territory of

a codeword u, then r is decoded to u.

if dmin=3, then the maximum radius of the territory is at most 1.

the code can correct up to one-bit errors

26

general discussion

define ( is the largest integer ≤ x)

27

tmax tmax

territories do not overlap if the radius ≤ tmax

tmax tmax

⇒ C can correct up to tmax bits errors in a codeword.

dmin=7, tmax=3

dmin=8, tmax=3

examples

28

dmin tmax

3

4

5

6

7

8

1

1

2

2

3

3

about tmax

tmax is the maximum radius that is allowed

we can consider smaller territories with radius < tmax

29

tmaxt

vectors which do not belong to any territory

detect errors, but do not correct them

advantage and disadvantage

30

t

sent codeword

received

decoded to thecorrect codeword

decoded to awrong codeword

error detectiononly

radius correct wrongdetectlargesmall

large small largesmall large small

The radius should be controlled according to applications.

familiar image?

31

A

B

C

D

AB

CD

P(A) …largeP(B), P(C), P(D) …largeP(miss) …small

A: award of 10,000 YenB, C, D: penalty of 1,000,000 Yen

P(A) …smallP(B), P(C), P(D) …smallP(miss) …large

summary of today’s class

Hamming codeone-bit error correctingperfect code

the minimum distance and minimum weighthandy measure of the error correcting capabilitylarge minimum distance means more power

32

exercise

Define (one of) (15, 11) Hamming code:construct a parity check matrix, anddetermine the corresponding generator matrix

Let C be a linear code with the following parity check matrix. Show that the minimum distance of C is 4.

33

.

10001110

01001101

00101011

00010111

H