View
218
Download
0
Category
Preview:
Citation preview
exercise in the previous class (1)
Consider an “odd” parity check code C whose codewords are(x1, …, xk, p) with p = x1+…+xk+1. Is C a linear code?
No. x1=1, x2=x3=...=xk=0 ⇒ p = 0, and 100...00 is a codeword
x2=1, x1=x3=...=xk=0 ⇒ p = 0, and 010...00 is a codeword
the sum of the two codewords = 110...0, not a codeword
1
exercise in the previous class (2)
Construct a 2D code for 6-bit information (a1, ..., a6) as follows.
determine the generator and parity check matricesencode 011001 using the generator matrixcorrect an error in the sequence 110111001010
2
a1 a2 a3a4 a5 a6
p1p2
q1 q2 q3 r
(a1, ..., a6)→ (a1, ..., a6, p1, p2, q1, q2, q3, r)
p1 = x1 + x2 + x3p2 = x4 + x5 + x6q1 = x1 + x4q2 = x2 + x5q3 = x3 + x6r = x1 + x2 + x3 + x4 + x5 + x6
parity symbols:
exercise in the previous class (3)
3
p1 = x1 + x2 + x3
p2 = x4 + x5 + x6q1 = x1 + x4
q2 = x2 + x5
q3 = x3 + x6
r = x1 + x2 + x3 + x4 + x5 + x6
111000000111100100010010001001111111
coefficients( 係数 )
111000 100000000111 010000100100 001000010010 000100001001 000010111111 000001
100000 101001010000 100101001000 100011000100 011001000010 010101000001 010011
H:G:tr
ansp
ose as is
(0 1 1 0 0 1)G= (0 1 1 0 0 1 0 1 0 1 0 1)
H(1 1 0 1 1 1 0 0 1 0 1 0)T
= (0 1 1 0 0 1)T = the 4-th column110111001010 110011001010
in the previous class...
linear codes: definition, encoding, decoding
one-bit error at the i-th symbol position⇔ syndrome equals the i-th vector of H
if several column vectors in H are the same, then we cannot correct one-bit errors in a codeword.
if all column vectors in H are different, then we can correct all one-bit errors in a codeword.
4
design of error correcting codes
Construct a parity check matrix with all column vectors differ,then we have a one-bit error correcting code.
5
101100110010011001
H= 101001101001111101001
H= 101001010011110100010101
H=OK examples:
110100101010010001
H= 101001111001111101001
H=NG examples:
C = {v | HvT = 0 mod 2},the discussion is easier if the right-submatrix of H is an identity matrix...
construction of a code
6
coefficients101 100110 010011 001
H= 101110011
100 110010 011001 101
G=
transposeas is
p1 = x1 + x3p2 = x1 + x2p3 = x2 + x3
000000,001101,010011,011110,
100110,101011,110101,111000.
codewords
the “shape” of a check matrix
a parity check matrix with m rows and n columns a code with...
7
length# of information symbols# of parity symbols
= n= n – m (= k)= m
100001111
010001010
001000101
000101100
000010011
H
n = 9
m = 5
code length 9= 4 information symbols + 5 parity symbols
“vertically longer” H means more parity symbols in a codeword less number of information symbols not efficient, not favorable...NG good
Hamming code
To design a one-bit error correcting code with small redundancy,construct a horizontally longest check matrix (all columns differ).
Hamming codedetermine m, # of parity check symbolslist up all nonzero vectors with length muse the vectors as columns of H
(any order is OK, but let the right-submatrix be an identity)
8
Richard Hamming1915-1998
m = 3:
1101000
1010100
0110010
1110001
G,
1001101
0101011
0010111
Hlength 7= 4 information + 3 parity
Parameters of Hamming code
Hamming codedetermine m, # of parity check symbolsdesign H to have 2m – 1 different column vectors
H has m rows and 2m – 1 columnslength# of information symbols# of parity symbols
9
n = 2m – 1k = 2m – 1 – mm
m234567
n37
153163
127
k14
112657
120(n, k) code:
code with length n, andk information symbols
comparison of codes
two codes which can correct one-bit errors:(7, 4) Hamming code(9, 4) 2D code
which is the “better”?Hamming code is more efficient (small redundancy)Hamming code is more reliable
correct data transmission with BSC with error prob. p:– Hamming code: (1 - p)7 + 7p(1 - p)6
– 2D code : (1 - p)9 + 9p(1 - p)8
“shorter is the better”
10
= 0.85= 0.77if p=0.1
codes better than Hamming code?
(7, 4) Hamming code3 parity bits are added to correct possible one-bit errors
Is there a one-bit error correcting (6, 4) code, with only 2 parities?No. Assume that such a code exists, then...
there are 24 = 16 codewords# of vectors decoded to a given codeword = 1+6=7# of vectors decoded to any one of codewords = 7×16 = 112# of vectors with length 6 = 26 = 64, which is < 112
11{0, 1}6
contradiction! ( 矛盾 )
Hamming code is perfect
(7, 4) Hamming codethere are 24 = 16 codewords# of vectors decoded to a given codeword = 1+7=8# of vectors decoded to any one of codewords = 8×16 = 128# of vectors with length 7 = 27 = 128
12
{0, 1}7
all of 128 vectors are exactlypartitioned to 16 classes with 8 vectors
Hamming code is a perfect one-bit error correcting code:
2𝑘((𝑛0 )+(𝑛1 ))=2𝑛
advanced topic: multi-bit errors?
Hamming code is a perfect one-bit error correcting code.Are there codes which correct two or more errors?
Yes, there are many...
one-bit error: syndrome = one column vector of Htwo-bits error: syndrome = sum of two column vectors of H
different combinations of t columns in H results in different sums, the code corrects t-bits errors.
13
advanced topic: two-bits error correcting code
if different two-bits errors results in different syndromes
received 11111000 the syndrome is 110111⇒ T= h2 + h6
⇒ errors at the 2nd and 6th bits
⇒ 10111100 should be transmitted
14
.
10000010
01000010
00100001
00010001
00001011
00000111
H
ability of the code
Error-correcting capability of a code is determined by the relation of column vectors of a parity check matrix.
It is not easy to consider all the combinations of columns.More handy and easy means is needed.
For linear codes, we can useminimum distance, or minimum weight
15
similarity of vectors
a codeword u is sent, errors occur, and v is received:In a practical channel, the distance between u and v is small.
16
u = 000 v = 000, with probability 0.729BSC with error prob. 0.1
v = 001, with probability 0.081v = 011, with probability 0.009
v = 111, with probability 0.001
If there is another codeword u’ near u,then v = u’ occurs with notable probability.
safe not safe
Hamming distance
a=(a1, a2,..., an), b=(b1, b2,..., bn): binary vectors
the Hamming distance between a and b, dH(a, b)
= the number of symbols which are differ between a and b
17
dH(0100,1101) = 2 dH(000, 011) = 2dH(000, 111) = 3 dH(011, 011) = 0
If a vector u with length n is sent over BSC with error prob. p,then a vector v with dH(u, v) = i is received with prob. (1 – p)n–ipi.
inverse correlation between the distance and the probability( 逆相関 )
dH(a, b) = dH(a + b, 0)
Hamming distance between codewords
code with length 4 ... vectors = vertices of a 4-dimensional hyper-cubecodewords = subset of vertices
18
C1={0000, 1100, 0011, 1111}
two or more edgesbetween codewords
good
C2={0000, 0001, 1110, 1111}
some codewords areside-by-side
bad
minimum distance
the minimum Hamming distance of a code C:
19
}.,,|),(min{min bababa Cdd H
C1={0000, 1100, 0011, 1111}
dmin = 2
C2={0000, 0001, 1110, 1111}
dmin = 1
computation of the minimum distance
consider a linear code whose generator matrix is
20
110100
101010
011001
G
000000001011010101011110100110101101110011111000
00000003343443
00101130434334
01010134034334
01111043303443
10011034430334
10110143343043
11001143343403
11100034434330
dmin = 3 for this code
Do we need to consider all of 2k×2k combinations?
minimum Hamming weight
the Hamming weight of a vector u: wH(u) =dH(u, 0)...# of 1s
the minimum Hamming weight of a code C: wmin=min{wH(u) : uC, u 0}
Lemma: if C is linear, then wmin = dmin.
proof of wmin≤dmin:
let u and v be codewords with dmin = dH(u, v).
u + v C, and wmin ≤ wH(u + v) = dH(u + v, 0) = dH(u, v) = dmin.
proof of dmin≤wmin:
let u be the codeword with wmin = wH(u).
dmin ≤ dH(u, 0) = wH(u) = wmin. 21
u
v
0
u + v
examples of minimum Hamming weight
22
(9, 4) 2D code: the minimum Hamming weight is 4000000000
0010010011
4100010101
4110000110
4
0001010114
0101110004
1001111106
1101011016
0010011014
0110111106
1010110004
1110010116
0011001104
0111101016
1011100116
1111000004
(7, 4) Hamming code: the minimum Hamming weight is 30000000
00001011
3
10001013
10011104
01001114
01011003
00101103
11010014
00101103
00111014
10100114
10110003
01100013
01110104
11101004
11111117
general case of Hamming code
lemma: The minimum Hamming weight of a Hamming code is 3.
proof sketch:Let H = (h1, ..., hn) be a parity check matrix:
{h1, ..., hn} = the set of all nonzero vectors
if codeword u with weight 1, then HuT = hi = 0...contradiction
if codeword u with weight 2, then HuT = hi + hj= 0
...this means that hi = hj, contradiction
23
no codewords with weight 1 or 2
proof (cnt’d)
lemma: The minimum Hamming weight of a Hamming code is 3.
proof sketch:Let H = (h1, ..., hn) be a parity check matrix:
{h1, ..., hn} = the set of all nonzero vectors
Choose x, y as you like, and choose z so that hx + hy = hz.
Let u be a vector having 1 at the x-th, y-th and z-th positions,then HuT= hx + hy + hz = 0, meaning that uC.
24
codewords with weight 3 are constructible
minimum distance and error correction
What does dmin=3 mean?
Any two codewords are differ at three or more symbols.At least three-bits errors are needed to change
a codeword to a different codeword.
25
error error error error error
{u’ | dH(u, u’)=1, uC} {v’ | dH(v, v’)=1, vC} =
We can distinguisha result of one-bit error from a codeword u, anda result of one-bit error from other codeword v.
u v u vu’ v’
decoding territory
dmin=3 : define territories around codewords
radius = 1... territories do not overlapradius = 2... territories do overlap
rule of the error correction:if a received vector r falls in the territory of
a codeword u, then r is decoded to u.
if dmin=3, then the maximum radius of the territory is at most 1.
the code can correct up to one-bit errors
26
general discussion
define ( is the largest integer ≤ x)
27
tmax tmax
territories do not overlap if the radius ≤ tmax
tmax tmax
⇒ C can correct up to tmax bits errors in a codeword.
dmin=7, tmax=3
dmin=8, tmax=3
examples
28
dmin tmax
3
4
5
6
7
8
1
1
2
2
3
3
about tmax
tmax is the maximum radius that is allowed
we can consider smaller territories with radius < tmax
29
tmaxt
vectors which do not belong to any territory
detect errors, but do not correct them
advantage and disadvantage
30
t
sent codeword
received
decoded to thecorrect codeword
decoded to awrong codeword
error detectiononly
radius correct wrongdetectlargesmall
large small largesmall large small
The radius should be controlled according to applications.
familiar image?
31
A
B
C
D
AB
CD
P(A) …largeP(B), P(C), P(D) …largeP(miss) …small
A: award of 10,000 YenB, C, D: penalty of 1,000,000 Yen
P(A) …smallP(B), P(C), P(D) …smallP(miss) …large
summary of today’s class
Hamming codeone-bit error correctingperfect code
the minimum distance and minimum weighthandy measure of the error correcting capabilitylarge minimum distance means more power
32
exercise
Define (one of) (15, 11) Hamming code:construct a parity check matrix, anddetermine the corresponding generator matrix
Let C be a linear code with the following parity check matrix. Show that the minimum distance of C is 4.
33
.
10001110
01001101
00101011
00010111
H
Recommended