Chapter 3

Linear Block CodesDr.Mohammed Taha El Astal

Outline

• Introduction – Definition of Linear, block and systemic codes.

• Encoding process– Generator matrix and parity-check matrix

– Encoding circuit

• Decoding process– Standard Arrays decoding.

– Syndrome decoding.

• Error detecting and correcting capabilities.

• Probability of decoding error over BSC.

• Examples: Single-Parity-check codes, repetition codes, and self-dual codes

2

Introduction

• As u is k-tuple, there are a total of 2k distinct

message.

• v is referred as codeword/ code vector of the

message u.

• Therefore, corresponding to the 2k possible

messages, there are 2k codewords.

• A total of 2k n-bit codewords (called block code)

constitute (n, k) code. 3

Sequence of binary digits

01010111101100….

Segmentation (i.e. k=4) u

(0101)(0111)……

Transformation: According to certain

rules, each u transforms in to block of n-tuple (v)

Cont.

• There should be a total of 2k distinct

codewords, why?

• Unless certain special structure was

used, the encoding will be complex

for a large k and n, why?

• What do you propose to avoid this?

How?

4

To ease the recovery (decoding). Otherwise, it is impossible to

recover data

Need to store 2k(n+k) digits

Definition of Linear Codes

• Linear code: The sum of any two codewords

is a codeword.

• Observation: The all-zero sequence is a

codeword in every linear block code.

5

Linear Code Example

6

• Check if linear? By definition: Its

codes words form 4-dimenstional

subspace of V7 over GF(2)yes

• By properties :

• The sum of any two codewords is a

code word yes

• The all-zero sequence is a code word

yes

• Result: These two properties are

results of the definition conditions.

Systematic codes

• Any linear block code can be put

in systematic form

• It is referred as linear systematic

block codes

• Table 3.1: 3 left-most bit for

parity-check digits and 4 right-

most bits for message digits

7

n-k

check bits

k

information bits

Generator Matrix

• All 2k codewords of an (n,k) linear code C can be

generated from a set of k linearly independent codewords,

why and why k?

• Let g0, g1, …, gk-1 be a set of k independent codewords.

8

Because an (n,k) linear code C is a k-dimensional subspace of the vector space Vn of all the

binary n-tuples, it is possible to find k linearly independent codewords, g0,g1,…gk-1 in C

such that every codeword v in c is alinear combination of these k codewords

Generator Matrix (cont)

• Encoding:

• Clearly, the rows of G generate (or span) the (n,k) linearcode C. This is it called Generator Matrix

• It can be noted that an (n,k) code C is completely specifiedby the k rows of a G matrix. Why it is big feature?

9

Storage requirement reduced from 2k(n+k) to k(n-k).

Example 3.1

10

• Find G for the code shown in Table 3.1?

• Using G, find v if u =1101 (check with Table 3.1)

• N.B. The simplest choice of this set is the k codewords

corresponding to the information sequences that have a

single nonzero element.

Generator Matrix (cont)

Systematic linear block codes

11

Generate identical as the

message information u ( k-tuple)

Generate the parity-check bits

(n-k)-tuple

• P matrix is created using the parity-check equation of the code

and vice versa.

Generator Matrix (cont’d)

• As shown in Example 3.2, G follows the systematic structure of

G

12

G P | Ik

1 1 0

0 1 1

1 1 1

1 0 1

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

k n k k k( )

Example 3.2

13

• Using G of Example 3.1, find the parity-check

equations?

• Using these equation, find v if u=1011, check

using table 3.1

Parity-Check Matrix

14

• Hence, any (n,k) linear block code can be represented in alternative way as follow:

– An n-tuple v is a codeword in the code C generated by G iff v. HT =0

• The code is said to be the null space of H.

• This matrix H is called a parity-check matrix of the code C.

Parity-Check Matrix

Example: Find H for thee code given in Example 3.1?

• Check that GHT = 0? Generalize?

• Using H, find the parity-check equations?15

H

1 0 0 1 0 1 1

0 1 0 1 1 1 0

0 0 1 0 1 1 1

1000101

0100111

0010110

0001011

G

Encoding Using H Matrix

(Parity Check Equations)

16

3212

2101

3200

6542

5431

6530

6542

5431

6530

6543210

uuu=v

uuu=v

uuu=v

vvv=v

vvv=v

vvv=v

0vvv+v

0vvv+v

0vvv+v

101

111

110

011

100

010

001

vvvvvvv

0

information

Encoding Circuit

17

Example : Draw the encoding circuit of the code

given in Example 3.1

18

The Dual Code

• The H matrix, formed from G, has (n-k) rows

of length n that are linearly independent.

• The H matrix can therefore be used to generate

an (n, n-k) linear code Cd.

• Cd is called the dual code of the code C

generated by G.

19

Syndrome and error detection

• Because of the channel noise, r may be different

from v. The vector sum:

is referred as error vector/ error pattern.

20

v=(v0,v1,….vn-1) Noisy

channelr=(r0,r1,….rn-1)

Syndrome and error detection (cont.)

• Define the syndrome of r to be

– Note that r=v+e.

– When s≠ 0, the presence of error has been detected.

– When s= 0, the rx. assign r as the transmitted v.

• Its possible that the errors are not detectable (undetectable error

pattern). When?How many in (n, k) code?

• The syndrome depends only on the error pattern but not on the

transmitted codeword, show?

21

If e is identical to a nonzero codeword

Because there are 2k-1 nonzero codeword, there are 2k-1 undetectable error pattern

Syndrome and error detection (cont.)

22

Syndrome circuit

23

Example: Draw syndrome circuit of code given in

Table 3.1

24

Example 3.5: For the (7, 4) code, let v=(1001011) and

r=(1001001). Decode r?

• s = rHT=111.

25

H

1 0 0 1 0 1 1

0 1 0 1 1 1 0

0 0 1 0 1 1 1

More on the Minimum Distance of Linear Codes

• DF: The Hamming weight of a codeword v , denoted by w(v),

is the number of the nonzero elements in the codeword.

• Example: Let v={1001011}, then w(v)=?

• DF: Hamming distance between v and w, denoted by d(v,w),

is the number of locations where they differ.

• Example:Let v={1001011}and w={0100011}then d(v,w)=?

26

4

3

N.B. Hamming distance satisfies the triangle inequality. Let v,w and x be three n-tuples then d(v,w)+d(w,x) ≥ d(v,x)

N.B. d(v,w)=w (v+w), why?Example?

This is result from the definition of d and the definition of module-2 addition

v={1001011}, w={1110010}, check?

More on the Minimum Distance of Linear Codes

• DF: The minimum weight of a code, wmin, is the smallest

weight of the nonzero codewords in the code.wmin = min {w(v): for every v in C; v ≠0}.

• DF: The minimum distance of the code dmin = min {d(v,w): v,w in C, v ≠ w}

• TH3.1: In any linear code, dmin = wmin, why?

27

If C is a linear block

code, the sum of two

codewords is also a

codeword.

Minimum Distance, Cont.

• Theorem 3.2: For each codeword of Hamming weight ℓ there

exists ℓ columns of H such that the vector sum of these

columns is zero. Conversely, if there exist ℓ columns of H

whose vector sum is zero, there exists a codeword of weight ℓ.

• Corollary 3.2.2: dmin of C is equal to the minimum numbers of

columns in H that sum to zero.

• Example:

Given H matrix of (7,4), find dmin ?

28

1110100

0111010

1101001

H

Error detecting capability

• A block code with dmin is capable of detecting all error patterns of dmin -1 or

fewer errors. Show?

• However, it cannot detect all the error patterns of dmin or more errors,

why?

29

v=(v0,v1,….vn-1) Noisy

channelr=(r0,r1,….rn-1) with ℓerror pattern d(v,r)=ℓ

• Given that the code employed is with dmin, this mean any two codewords

of C differs in at least dmin places. For this code, no error pattern of dmin -1or

fewer can change one codeword into another.

Therefore, any error pattern of dmin -1 or fewer will result in a received vector

r that is not a codeword of C, we say that errors are detected.

• Because there exists at least one pair of codewords that differ in dmin

places and there is an error pattern of dmin errors that will carry one into

the other

Cont.

• How many error patterns can be detected? show?

• For large n, small fraction of error patterns pass through the decoder

without being detected.

30

In an (n, k) linear code, the detecting capability is 2𝑛 − 2𝑘 error

patterns of length n.

• All error patterns =2n-1

• 2k-1 are identical to codewords and hence if added will results a

codeword and then syndrome will be zero (no errors are

detected)

• Total=2n-1-(2k-1)= 2n-2k

Cont.

• Let 𝐴𝑖 the number of codeword of weight i in C

(weight distribution).

• If 𝑃𝑢(𝐸) denotes the probability of an undetected

error over BSC,

𝑃𝑢 𝐸 =

𝑖=1

𝑛

𝐴𝑖𝑝𝑖 1 − 𝑝 𝑛−𝑖

where 𝑝𝑖 is the transition probability of the BSC.

• Example: Given that 𝑷 = 𝟏𝟎−𝟐, determine 𝑷𝒖 𝑬of C shown?

31

𝐴0 = 1, 𝐴1 = 𝐴2 = 0, 𝐴3 = 7, 𝐴4 = 7, 𝐴5 = 𝐴6= 0, 𝑎𝑛𝑑 𝐴7 = 1⇒ 𝑃𝑢 𝐸 = 7𝑝3 1 − 𝑝 4 + 7𝑝4 1 − 𝑝 3 + 𝑝7

If 𝑃 = 10−2 ⇒ 𝑃𝑢 𝐸 = 7 × 10−6.This mean 7 codewords out of 1 million can’t be

detected if errors involved

Error Correcting Capability

• dmin is either odd or even

2𝑡 + 1 ≤ 𝑑𝑚𝑖𝑛 ≤ 2𝑡 + 2 (t be + integer)

• Next, we show that the code C is capable of correcting all error

patterns of t or fewers errors:

• Let v and r be the Tx. and Rx. codewords, respectively

– d(v,r)+d(w,r) ≥ d(v,w)….3.21 (Traingle inequality).

– Suppose error pattern of t’ errors occur: d(v,r)=t’

– Because v and w are codewords in C, we have

d(v,w) ≥ dmin ≥ 2t+1……(3.22)

32

Error Correcting Capability– Combining (3.21) and (3.22) and using the fact of d(v,r)=t’ :

d(w,r) ≥2t+1-t’

– assuming t ≥t’ d(w,r)>t

• This says that if there is an error pattern of t or less , r is still close to

the transmitted codeword v it self. (The decoder is still capable od

recovering the trasnmitted code word.

• In contrast, the code is not capable of correcting all the error patterns

of ℓ errors with ℓ > 𝑡. (see textbook page for details).

• In summary, # of errors that can be corrected t =

33

2

1min

d

ExampleConsider the code given in Table 3.1, Find the

detecting and correcting capabilities of the

code shown

• dmin=3

• Any error patterns with 2 errors can be

detectd

• Any single error can corrected

34

Hard vs. Soft decision decoding• Hard-decision : for each received symbol, Rx. make hard decision (0 or 1).

• Soft-decision: Rx. can be designed to declare a symbol erased when it

received unreliably (0 or 1 or erasures).

• A code with dmin is capable of correcting any pattern of v errors and e erasures

provided the following condition is satisfied

• Show?

35

N.B: dmin≥e+1 mean that all codewords are different in e+1 digits and hence

there is at least a single digit in the rx vector (which also is correct for sure)

can determine the corresponding tx vector

Decoding schemes

• Decoding mission is to find a rule to

partition the 2n possible received

vectors in to 2k disjoint subset D1,

D2,...D2k such that the codeword vi is

contained in the subset Di for 1≤i≤2k.

• Thus, each subset Di is one-to-one

correspondence to a codeword.

• Decoding is correct iff the r is in the

subset Di that corresponds to the

codeword transmitted.

v=(v0,v1,….vn-1) Noisy

channelr∈2n n-tuples over

GF(2)

1. Standard Array

37

Standard Array Construction

1. List the 2k codewords in a row, starting with the all-zero

codeword v1.

2. From the remaining 2n-2k , select an error pattern e2 and

place it below v1. The selected pattern should:

(i) has the smallest weight possible (most probable error) why?

(ii) has not appeared before in the array.

3. Complete the row entries by finding the corresponding

addition results {e2+v2, e2+v3,…. e2+v2k} and place them.

4. Repeat step 2-3 until all the possible error patterns have

been accounted for.

38

Standard Array Construction

39

kknknknkn

k

k

k

22322212

23332313

22322212

2321

veveveve

vev+ev+ev+e

vev+ev+ev+e

vvv0v

- - - -

TH3.3 No two n-tuples in the same row of a standard array are

identical. Every n-tuple appears in one and only one row. Proof?

(Result): There are 2n/2k=2n-k disjoint rows in the standard array and

each row consist of 2k distinct elements.

Any element in a coset can be used as its coset leader. This does not

change the element of the coset, it simply permutes them.

Coset leader

(representative)

40

Standard Array Construction

Example: Consider the (6,3) linear code generated by

G, find the standard array?

41

Decoding using Standard Array

• Still we didn’t show how rx vector can be decoded

using standard array, can you guess?

• Example r=110100, then e=000010 and hence v*=110110

Decoding using Standard Array

• N.B. The decoding is correct iff the error pattern caused

by the channel is a coset leader. Why?

• Lets the experienced error pattern through the channel is x and it is not

a coset leader (Say it is in 𝑙th coset and under vi)

• This mean x=e𝑙+vi

• Suppose that vj is transmitted.

• r=vj+x

• and using x=e𝑙+vi this results that

• r=vj+e𝑙+vi

=e𝑙+vs and hence the Rx. will specify vs as the transmitted codeword

• It should be now clear why the error pattern with the

minimum weight should be placed as coset leaders (place

the most probable (most probable=minimum weight)

patterns in positions that can be corrected). In otherwords,

this achieve the maximum likelihood decoding. Proof?

Proof:

Be noted that the el has been chosen to be the vector with the minimum weight in the coset

Decoding Error Probability

• Let α denote the # of coset leaders of weight i.

• α0, α1, α2…. αn are called the weight distribution of the

coset leaders.

• Because a decoding error occurs iff the error pattern is not

a coset leader, the error probability for a BSC is

Useful Properties

Proof:?

Proof:?

• Using the one-to-one correspondence that shown by Theorem

3.6, a decoding table which is simpler that standard array can be

formed

• This table consist of the 2n-k coset leaders and its corresponding

symdrome

2. Syndrome Decoding (table-lookup)

Decoding Procedure:

1. For the received vector r, compute the syndrome s = rHT.

2. Using the decoding table, identify the error pattern e.

3. Add e to r to recover the transmitted codeword c.

47

2. Syndrome Decoding (table-lookup)

48

2n-k=8

[dmin-1/2] or less= 1 or less

1. How many correctable error pattern?

2. What is the weight that can be corrected?

3. Find the decoding table?

4. Given that v=1001011 is transmitted and

then r=100111 is received, decode r using

the lookup table

s=v*HT=(011) and hence e=0000100 then

v*=100011

Hardware Implementation

• Let r = r0r1r2r3..rn

• By the H matrix , find s =s0s1….sn-k

• From the table of syndromes, a truth

table can be constructed and hence a

combinational logic circuit with s

as input and e as outputs can be

designed.

• Then, added (XOR) to the r

49

Example 3.9: Draw the decoding circuit of the code has the

following decoding table

50

Hardware Implementation

Standard Array Decoding ()• For an (n,k) linear code, standard array decoding is

able to correct exactly 2n-k error patterns, including

the all-zero error pattern.

• Illustration 1: The (7,4) Hamming code

# of correctable error patterns = 23 = 8

# of single-error patterns = 7

Therefore, all single-error patterns, and only single-

error patterns can be corrected.

EELE 6338 51

Probability of undetected error • Two new formulas will be presented with extra advantages.

• Let 𝐴0, 𝐴1, … , 𝐴𝑛 to be the weight distribution of the code (n, k)

and 𝐵0, 𝐵1, … , 𝐵𝑛 to be the weight distribution of the dual code 𝐶𝑑.

• Weight distribution polynomials (weight enumerator of C and Cd)

𝐴 𝑧 = 𝐴0 + 𝐴1𝑧 +⋯+ 𝐴𝑛𝑧𝑛, 𝐵 𝑧 = 𝐵0 + 𝐵1𝑧 +⋯+ 𝐵𝑛𝑧

𝑛

• Then, A(z) and B(z) are related by the following identity

(McWilliams identity) 𝐴 𝑧 = 2− 𝑛−𝑘 1 + 𝑧 𝑛𝐵 1−𝑧

1+𝑧

• Previously was shown that the probability of undetected error 𝑃 𝐸

= σ𝑖=1𝑛 𝐴𝑖𝑝

𝑖 1 − 𝑝 𝑛−𝑖 = 1 − 𝑝 𝑛 σ𝑖=1𝑛 𝐴𝑖

𝑝

1−𝑝

𝑖

• Substituting z = Τ𝑝 1 − 𝑝 in A(z) and noting that 𝐴0 = 1

𝐴𝑝

1 − 𝑝− 1 =

𝑖=1

𝑛

𝐴𝑖𝑝

1−𝑝

𝑖(3.34)

52

Cont.

• Then, by combining this result to 𝑃𝑢 𝐸 leads that 𝑃𝑢 𝐸 = (1

53

(7,4) C has (24) but Cd of (7,3) has (23) codewords

54

Simple Upper Bound on Pu

• Theoretically, the weight distribution can be

determined by examining its 2k or 2n-k (Cd) codewords.

• For large n, k, n-k, this would be impractical. Actually,

the weight distribution is unknown for many known

linear codes and hence is difficult (if not impossible)

to compute their Pu(E)

• However, the upper bound on average probability of

undetected errors can be determined:

• Derivation :

)(22

12 kn

n

k

uP

55

Linear Code Examples

Single-Parity-Check (SPC) codes:

1. (k+1,k) or (n,n-1) linear block code

2. With

3. Because all codewords have

even weight, SPC called even parity check codes.

4. SPC codes often used for simpler error detection

Repetition codes:

1. (n,1) linear block code

2. Only two codewords (all zeros or all ones).

3. It is Cd of SPC, show?

56

N.B. :

1. SPC and repetition codes are often used as components for constructing

long, powerful codes.

2. A linear block codes C that is equal to its dual code Cd is called self-dual

code

Cont.Self-Dual code:

• n must be even and k must equal n/2 (mean R=1/2)

• G.GT=0 (assuming G is systematic then P.PT=In/2)

Next lecture

• We will cover several classes of important

linear block codes.

58