Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
1
E.1
Error-control coding
A good communication system requires transmittinginformation from one end of the system at a rate and a levelof reliability and quality that are acceptable to a user at theother end.– Two key systems parameters are transmitted signal power
and channel bandwidth. In general,• Signal power ↑ error rate ↓• Bandwidth ↑ error rate ↓
– In practice, we often arrive at a modulation scheme andfind that it is not possible to provide acceptable dataquality.
– ∴ Error-control coding
E.2
Feed-Forward Error Correction (FEC)
The channel encoder in the transmitter accepts message bitsand adds redundancy according to a prescribed rule,thereby producing encoded data at a higher bit rate.
The channel decoder in the receiver exploits the redundancyto decide which message bits were actually transmitted.
2
E.3
Automatic-Repeat Request (ARQ)
FEC is used for both the detection and correction of errors.– Only a one-way link between the transmitter and receiver
is required.
ARC uses redundancy merely for the purpose of errordetection. Upon the detection of an error in a transmittedcode word, the receiver requests a repeat transmission of thecorrupted code word, which need a return path.– Used only on half-duplex or full-duplex links– widely used in computer-communication systems
E.4
Classification of channel coding
Block Codes– has no memory– adding redundant bits that are algebraically related to the
message bits.
Convolutional Codes– has memory– discrete time convolution of the input sequence with the
impulse response of the encoder.
3
E.5
Channel Coding Theorem
Channel Capacity bit/sec
If a discrete memoryless channel has capacity C bit/s and asource generates information at a rate less than C bit/s, thenthere exists a coding technique such that the output of thesource may be transmitted over the channel with anarbitrarily low probability of symbol error.
– This theorem gives a fundamental limit on the rate atwhich the transmission of reliable messages can takeplace over a discrete memoryless channel.
)/1(log 2 NSBC +=
E.6
Binary operations
Channel encoding and decoding involves the binaryarithmetic operations of modulo-2 addition andmultiplication for binary codes {0,1}:
0 + 0 = 0 0 0 = 00 + 1 = 1 0 1 = 01 + 0 = 1 1 0 = 01 + 1 = 0 1 1 = 1
Modulo-2 addition is the Exclusive-OR (XOR) operationModulo-2 multiplication is the AND operation
⋅⋅
⋅⋅
4
E.7
Linear Block Code
Linear – the modulo-2 sum of any codewords is another valid codeword.
A code of 8 possible codewords– ‘Linear’ means code 1 + code 2 = either code 1, 2, 3, 4,
5, 6, 7 or 8.– Binary code is linear (e.g. 101+111 =010)
8
2
76
54 3
1
E.8
Block
Block– the input message is divided into blocks of k bits and
coded into n-bit codewords.
message check bits
k bits
n bits
k bits(n-k) bits
message
Message block m Channelencoder
Code Block c
5
E.9
Message block m: Code block c: =
This is known as a systematic (n, k) linear block code.– A k-bit message is coded into a n-bit codeword consisted
of (n - k) parity-check bits and k message bits.
– Code rate (or Rate efficiency) =
[ ]110 −kmmm L[ ]110 −nccc LLLLL
[ ]110110 −−− kkn mmmbbb LL
[ ]mbM=
nk /
E.10
Encoding: Coefficient matrix P and Generator matrix G
Now, the n-bit code word is
Encoding
G Codeword cMessage m
11
0k-n
11-k-n
11
00
:
:
−−
−−
=
==
==
kn
kn
mc
mcbc
bcbc
6
E.11
The (n - k) parity-check bits are linear sum of the k messagebits, i.e.,
where
The coefficient are chosen in such a way that the rowsof the generator matrix G are linearly independent.
1,11100 ... −−+++= kikiii mpmpmpb
=otherwise0
on depends if1 jiij
mbp
ijp
E.12
In a matrix form,
where
and
mPb =
=
−−−−−
−−
−−
1,11,10,1
1,11110
1,00100
...
...
...
knkkk
kn
kn
ppp
pppppp
MMMMP
[ ][ ][ ]
mGIPmmmP
mbc
k
====
M
M
M (P is called coefficient matrix)
7
E.13
c = mG
(1 x n) (1 x k) (k x n)
where
G = k
n
⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅
−−−−−
−−
−−
1000
00100001
1,11,10,1
1,11110
1,00100
LML
M
M
M
LML
LML
knkkk
kn
kn
ppp
pppppp
E.14
Example 1
Consider a (6, 3) linear block code
=
m - 8 possible 3-bit messages
For m = [1 1 1]
c= = [0 0 0 1 1 1]
[ ]3IPG M=
100011010101001110
M
M
M
mGc =
[ ] 1 1 1
1 0 0 0 1 1 0 1 0 1 0 1 0 0 1 1 1 0
8
E.15
Example 1
Similarly, m c
0 0 0 0 0 0 0 0 00 0 1 1 1 0 0 0 10 1 0 1 0 1 0 1 00 1 1 0 1 1 0 1 11 0 0 0 1 1 1 0 01 0 1 1 0 1 1 0 11 1 0 1 1 0 1 1 01 1 1 0 0 0 1 1 1
Encoding: needs to store G or P
E.16
For a k-bit message block, there are possible messages:
k2
12
1
0
12
1
0
−−
=
kk c
cc
mGc
m
mm
MM
k bits and 2k possible combinations
n bits and 2k possible combinations(there are 2n-k - unused combinations)
9
E.17
Sum of codewords
To prove the sum of 2 codewords is another codeword, takecodeword ,
Since the mod-2 sum of and must result in anothermessage vector within the set of 2k message, say . So,
sum of two codewords give another codeword
ji cc +
Gmm
GmGmcc
ji
jiji
)( +=
+=+
im jmkm
kkji cGmGmm ==+ )(
kji ccc =+∴
E.18
Decoding: Parity-check matrix H
Let H denote an (n-k) x n Parity-check matrix, defined as is a (n - k) x k matrix
=
][ Tkn PIH M−= TP
⋅⋅⋅⋅⋅⋅⋅⋅⋅
⋅⋅⋅⋅⋅⋅⋅⋅⋅
−−−−−−−
−
−
−
1,11,11,0
2,11202
1,11101
0,11000
1000
010000100001
knkknkn
k
k
k
PPP
PPPPPPPPP
L
L
L
L
ML
M
M
M
ML
ML
ML
10
E.19
Property of H
Consider =
=
Thus,
Therefore, for any codeword c
and this equation is called parity-check equation
THG
−
k
TT
kn IPPI ][ M
[ ]0PP TT =+
0=THG )0( =⇒ TGH
cHT = mGHT= 0
E.20
Example
e.g., =
= =
TGH
221202
211101
201000
100 010 001
PPPPPPPPP
M
M
M
100010001221202
211101
201000
PPPPPPPPP
+++++++++
222212120202
212111110101
202010100000
PPPPPPPPPPPPPPPPPP
[ ]0
11
E.21
Summary
i. c = mG Encodingii. GHT = HGT = 0iii. cHT = mGHT= 0 Parity-check equation
Decoding
HT Null vector 0 Received code c
Encoding
G Codeword cMessage m
E.22
Syndrome s
Let r be the received codeword that results from sending thecodeword c overy a noisy channel.
Decoding
HT Syndrome s Received code r
Encoding
G Codeword cMessage m
12
E.23
Let be the received code word where e =
Define the Syndrome s as
No error: and r is a valid codeword∴ c = r
Error occurred:
ecr +=[ ]1210 −neeee L
=otherwise0
location at the occurrederror an if1 th
ii
e
TrHs =
0
)(
==
+=
+=
TT
TT
T
cHeHeHcHHec
Q
[ ] [ ]00000 =⇒= se L
[ ] [ ]00000 ≠⇒≠ se L
E.24
Example 2
Consider a (7, 4) Linear Block Code
For m = [1 0 1 1], c = mG = [0 0 1 1 0 1 1 ]No error, r = c, and s = cHT= [0 0 0]
=
=
110110010110100111001
1000110010010100100110001111
M
M
M
M
M
M
M
H
G
13
E.25
Let c suffered an error in the transmission such that thereceived vector r = [0 0 0 1 0 1 1] = [0 0 1 1 0 1 1] + [0 0 1 0 0 0 0] = c + e
s = cHT = [0 0 0 1 0 1 1] = [0 0 1] (= eHT )
Note: Error in the 3rd bit results in s corresponding to the 3rd
row of HT.
110101011111100010001
E.26
For this (7, 4) code, a single error in the ith bit would give sequal to the ith row of HT. The code is capable of correctinga single error.
Double errors: e = [0 1 1 0 0 0 0]
s = c = e = [0 1 1] ≠ [0] = 7th row of HTand is interpreted as an error in the 7th
bit of the received word.
– Error is detectable but not correctable.– More than 2 errors might give s = [0] and in that case the
errors are not detectable!
14
E.27
Syndrome decoding for multiple errors: Problem
In syndrome decoding, the receiver forms s = rHT.s = rHT = 0 implies transmitted codeword c = rs = rHT ≠ 0 Find the corresponding error pattern(s) that
gives rise to s and c = r + e
The process is a bit more complicated as was illustrated for adouble error in the (7, 4) code.
E.28
Consider the (7, 4) code with
G =
For r= [1 0 0 0 0 1 1] s = rHT = [1 0 0]
But s = rHT = eHT, let e =
1 0 0 0 1 1 0 0 1 0 0 1 0 1 0 0 1 0 0 1 1 0 0 0 1 1 1 1
M
M
M
M
[ ] 7654321 eeeeeee
15
E.29
s = =
3 equations and 7 unknowns. ∴ Unique solution for e isnot possible.
For s = [1 0 0], the possible error patterns are
[ ] 0 0 1 [ ]7654321 eeeeeee
1 0 00 1 00 0 11 1 01 0 10 1 11 1 1
7431
6421
5321
0 0 1
eeeeeeeeeeee
+++=+++=+++=⇒
MM
1110110000011100010010010000
2
7654321
4
eeeeeee
E.30
Error detection and error correction capabilities
Hamming weight of a code word c, w(c), is the number ofnonzero elements in the code vector. For example,
c = [11001010111] ⇒ w(c) = 7
Hamming distance, d(c1,c2), between a pair of code vectorsand that have the same number of elements is defined as thenumber of location in which their respective elements differ.Say,
c1=[1011001] , c2 =[0111101] ⇒ d(c1, c2) = 3
16
E.31
Minimum distance
Minimum distance, dmin, of a block code is the smallestdistance between any pair of code words in the code. Forexample,Codewords weight w(c)
000000 0 d(c1,c2)= 3 110001 3 d(c1,c2) = 3 101010 3 d(c1,c2) = 4 011011 4 : 011100 3 : 101101 4 Look for : 110110 4 the smallest distance d(c1,c2) 000111 3
E.32
Hamming weight and Minimum distance
TheoremMinimum distance of a linear block code is the smallestHamming weight of the nonzero code vectors in the code.
The smallest Hamming weight from the previous exampleexcept c= [0 0 0 0 0 0 0] is 3, therefore, dmin= 3
17
E.33
Proof– .The minimum distance is the same as the smallest
Hamming weight of the difference between any pair ofcode vector (i.e. )
– From the closure property of linear block code, the sum(or difference) of two code vectors is another codevector.(i.e. )
Note: The best strategy for the decoder is to pick the codevector closest to the received vector, that is, the one forwhich the Hamming distance d(ci,r) is the smallest.
{ } { })(min),(minmin iji wdd ccc ==
{ } { })(min),(min jiji wd cccc −=
{ } { })(min)(min iji ww ccc =−
E.34
TheoremA linear block code with a minimum distance dmin cancorrect up to [(dmin - 1) / 2] errors and detect up to (dmin–1)errors in each code word where [*] denotes the largestinteger no greater than *, i.e., [3.5] = 3. Thus, in theprevious example,
dmin = 3 ∴ [(3 - 1) / 2] = 1 (3 - 1) = 2
Therefore, this (7, 4) block code can correct 1 error anddetect up to 2 errors.
18
E.35
Interpretation:
Case I
error can be corrected
(t is the distance between received codeword r and thetransmitted codeword c)
12),( +≥ td ji cc
icjc
t r
E.36
Case II
error cannot be corrected but can be detected
1),( +≥ td ji cc
icjc
rt
t+1
19
E.37
Case III
error cannot be detected
1),( +< td ji cc
ic jc
r
E.38
Syndrome decoding for multiple errors: Solution
Consider a (n, k) codeThere are 2k possible codewords: c1, c2, c3,…, c2k
Received vector r has 2n possible values. Decoder has torelate r with one of the 2k valid code words.
Divide the 2n possible received codewords into 2k disjointsubsets in such a way that each subset contains only 1 codevector ci. Then the received codeword r is decoded into ci ifit is in the ith subset.
20
E.39
Construction of subsets: standard array
The subsets constitute is constructed as follows:1. The 2k code words are placed in a row with the all-zero codeword c1 as the left most element.2. An error pattern e2 is picked and placed under c1, and a
second row is formed by adding e2 to each of the remaining code vectors in the first row; it is importantthat the error pattern chosen as the first element in a rowhas not previously appeared in the standard array.
3. Step 2 is repeated until all the possible error patternshave been accounted for.
The array is called standard array.
E.40
The rows of the standard array are called co-sets and the firstelement in each row is called co-set leader. The co-setleaders are chosen to be the error patterns that are mostlikely to occur. In the case of a binary symmetric channel,the smallest the Hamming weight of an error pattern themore likely it is to occur. Accordingly, the standard arrayshould be constructed with each coset leader having theminimum Hamming weight in its coset.
1c 2c 3c … k2c
2e 2ec +2 2ec +3 … 2ec +k2
3e 32 ec + 33 ec + … 32ec +k
M M M … Mkn−2
e kn−+2
ec2 kn−+2
ec3 … knk −+2
ec2
21
E.41
Thus, the procedure for decoding a linear block code is
1. Compute s = rHT.
2. Within the coset characterized by the syndrome s,identify the coset leader (i.e., the error pattern with thelargest probability of occurrence); call it eo.
3. Compute c=r+ eo .
E.42
Example 3
Consider the (6, 3) linear code generated by the followingmatrix: .The standard array of this code is
coset leader
=
100011010101001110
G
000000 011100 101010 110001 110110 101101 011011 000111111100 001010 010001 010110 001101 111011 100111001100 111010 100001 100110 111101 001011 010111010100 100010 111001 111110 100101 010011 001111011000 101110 110101 110010 101001 011111 000011011110 101000 110011 110100 101111 011001 000101011101 101011 110000 110111 101100 011010 000110111000 001110 010101 010010 001001 111111 100011
100000010000001000000100
000010000001100100
22
E.43
Example 4
Construction of a single-error correction codeneed dmin = 3 (or more)
when single error occurs, the ith bit corresponds to the ith
row of HT
Therefore,i. Choose the rows of HT to be all distinct.
ii. Make sure that the first (n-k) rows of HT is In-k . iii. Do not use [000…0] for any row of HT.
E.44
Since there are (n-k) columns, we can have a possible of 2n-k
distinct rows. So, use the first rows for , then fill in the restof the matrix arbitrarily but make sure that all the rows aredifferent. Finally, make sure that none of the rows is [0 0 0 .. . 0].
TH =
−
PI kn =
n
k
kn
PPPP
PPPPPPPP
kn
knkkkk
kn
kn
−
−
−−−−−−
−−
−−
444444 3444444 21L
MM
L
L
L
MMM
L
L
1,1121,10,1
1,1121110
1,0020100
1000
00100001
23
E.45
Question: Given k, how can n be determined?
Now, n rows of HT must be distinct.
∴Given k, the minimum value of n is .
nkn ≥−⇒ − 12( ) ( )1log2 +≥− nkn
( )1log2 ++≥ nkn
( )1log2 ++ nk
E.46
Design a (n, 8) code with single error correction capability,
Minimum value of n that satisfies the equation is 12. Hence a(12,8) code.
)1(log + 8 2
+≥ nn
24
E.47
=
=∴−
P
IH
knT L
LLLL
11100111101001011001110001100011
1000010000100001
E.48
Check: For the 8-bit data m = [1 1 1 0 1 0 1 1] , the 12-bit codeword c is
[ ]
[ ]1101011101011 0 0 0 0 0 0 0 1 1 1 00 1 0 0 0 0 0 0 0 1 1 10 0 1 0 0 0 0 0 1 0 1 00 0 0 1 0 0 0 0 0 1 0 10 0 0 0 1 0 0 0 1 0 0 10 0 0 0 0 1 0 0 1 1 0 00 0 0 0 0 0 1 0 0 1 1 00 0 0 0 0 0 0 1 0 0 1 1
11010111
M
M
M
M
M
M
M
M
M
=
== mGc
25
E.49
Assume that the received bit is in error, i.e.,r = [ 1 0 1 0 0 1 1 0 1 0 1 1 ]
]1100[
01111110010110101001001101101100
0001001001001000
]111010011010[ =
==
LL
TrHs
E.50
s = [ 1 1 0 0 ] corresponds to the 5th row of .HT
∴the bit of r is in error and
c = r+e = [101001101011]+[000010000000] = [101011101011] and the message m is
[11101011]