Phd Defence

Construction and Analysis of Non SystematicCodes on Graphs for Redundant Data

Amira ALLOUM

Ecole Nationale Superieure des TelecommunicationsTelecom Paris Tech

September 5th, 2008

Presentation Outline

Introduction and Motivations

Part I: Non Systematic LDPC Codes Constructions

Part II: Density Evolution Analysis for Split-LDPC Codes

Part III: Exit Chart Analysis for Split-LDPC Codes

Part IV : EM for Joint Source-Channel Estimation

Conclusions and Future Work

Introduction

The Non Uniform Assumption

Source Encoder Channel Encoder Channel Channel Decoder Source Decoder

10

0.5

P (s = 1) = 0.5

The Uniform Assumption is not valid anymore When

1 It is not worth to compress (bad channel conditions)2 Using sub-optimal compression (highly redundant sources)

Amira ALLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 1 / 35

Introduction


Channel Encoder Channel Channel Decoder0

1

Source Encoder Source Decoder

0.9

0.1

P (s = 1) = µ

The Uniform Assumption is not valid anymore When1 It is not worth to compress (bad channel conditions)

2 Using sub-optimal compression (highly redundant sources)


Introduction


Channel Encoder Channel Channel Decoder0

1

Source Encoder

Source Encoder Source Decoder

Source Decoder

0.9

0.1

P (s = 1) = µ

The Uniform Assumption is not valid anymore When1 It is not worth to compress (bad channel conditions)2 Using sub-optimal compression (highly redundant sources)


Introduction

Channel Coding Strategies for Non Uniform Sources

Source Channel Sink

01

Channel DecoderChannel Encoder

0.9

P (s = 1) = µ

0.1

Shannon has intuited in his 1948 ParadigmAny Redundancy in the source will usually help if it is utilized at the receiveing...This redundancy will help to combat noise

Channel Coding for Redundant data follows the following strategies

1 Source Controlled Channel Coding (Hagenauer 1995).2 Non Systematic Encoding Structures (Shamai and Verdu 1997)


Introduction


Source Channel Sink

01


SCCD

0.9

0.1

µ

P (s = 1) = µ


Channel Coding for Redundant data follows the following strategies1 Source Controlled Channel Coding (Hagenauer 1995).

2 Non Systematic Encoding Structures (Shamai and Verdu 1997)


Introduction


Source Channel Sink

01


Systematic Coding

Parity

Non−Systematic Coding

Parity

Info

Best Constructions

0.9

P (s = 1) = µ

0.1

µ µ


Channel Coding for Redundant data follows the following strategies1 Source Controlled Channel Coding (Hagenauer 1995).2 Non Systematic Encoding Structures (Shamai and Verdu 1997)


Introduction

Information Theoretical Limits wih Redandancy: AWGN Channel

Capacity limit Versus Source Entropy, Coding Rate = 0.5, AWGN Channel

-12

-10

-8

-6

-4

-2

0

2

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Min

imum

Ach

ieva

ble

Eb/

N0

(dB

)

Source Entropy (bits)

Systematic code, BPSK inputNon-Systematic code, BPSK input

Gaussian input


Introduction

Information Theoretical Limits wih Redandancy: AWGN Channel

Capacity limit Versus Coding Rate , Source Entropy= 0.5, AWGN Channel

-5

-4

-3

-2

-1

0

1

2

3

4

5

6

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Min

imum

Ach

ieva

ble

Eb/

N0

(dB

)

Coding Rate

Systematic codes, BPSK inputNon-Systematic code, BPSK input

Gaussian input


Introduction

Achieving the Theoretical Limits 1

In the presence of RedundancyThe Theoretical Limits of Information Theory are mooving to betterregions.

To Attain these Challenging limits :1 Building Non Systematic Capacity Achieving Encoding Structures In the

Codes On graphs Family.

2 Using Source Controlled Channel Decoding with Iterative Algorithms Inthe Sum-Product Algorithms Family.


Introduction

Achieving the Theoretical Limits 1

In the presence of RedundancyThe Theoretical Limits of Information Theory are mooving to betterregions.

To Attain these Challenging limits :1 Building Non Systematic Capacity Achieving Encoding Structures In the

Codes On graphs Family.

2 Using Source Controlled Channel Decoding with Iterative Algorithms Inthe Sum-Product Algorithms Family.


Introduction

Related Work and Motivation

Codes On Graph

Turbo Codes LDPC

Non−Systematic LDPC Codes

Non−Systematic

MN Codes for BSC Alajaji Codes et.al.


Introduction

Main System Assumptions

Source Channel Sink

01


0.9

P (s = 1) = µ

0.1

Lossless source coding or no source coding.

Binary i.i.d. source with entropy Hs = H2(µ), where µ = P (si = 1).Source sequence s = (s1, s2, ..., sK) encoded by a binary channel code ofrate Rc = K/N , dimension K, and length N .

x = (x1, ..., xN ) denotes the codeword (channel input).

Transmitted information rate R = Hs ×Rc bits per channel use.

Any symmetric binary-input channel can be considered, mainly BEC,BSC, and AWGN.


Part I

Non Systematic LDPC Constructions

Encoding Structures Decoding Strategy Simulation Results

Scrambling-LDPC Encoding Structure

u

LDPC

c(u,v)s

Scramble

G

c = G×Cs × s

Cs

(1−R).N

R.N

R.N R.N

Scrambler

LDPC

(1−R).N

dc

v

u

s

s

v

s

α

β

β

db

db

α

u

α

ds

ds

Cs

Cs: sparse matrix of dimensionK ×K. In the regular case row andcolumn weight are ds

u: systematic bits for the innerLDPC.

v: parity bits for the inner LDPC.

s: source bits.



Splitting-LDPC Encoding Structure

u

LDPC Splitter

c(u,v)sG

c = G × C−1s × s

C−1s

R.N

(1−R).N

R.N

(1−R).N

R.N

Splitter

LDPC

v

u

s

db

s

u

v

s α

α

α

β

β

ds

ds

dc

Cs

Cs: full rank sparse matrix ofdimension K ×K. In the regularcase row and column weight are ds

u: systematic bits for the innerLDPC.

v: parity bits for the inner LDPC.

s: source bits.



Source Controlled Sum-Product Decoding

The bitnode Rule:

LLRci→pcei =LLRtype+∑

pcej∈S∗ci

LLRpcej→ci

LLRtype={

LLR0 if bitnode ∈ {u, ϑ}LLRs if bitnode ∈ {s}

LLR0 is the channel observation LLR.

LLRs = log(1− µ

µ) is the source LLR .

LLR0+ Extrinsic Information

LLRs+ Extrinsic Information

v

u

s

s

u

v

s α

α

α

β

β




The bitnode Rule:


pcej∈S∗ci

LLRpcej→ci

LLRtype={


The checknode Rule:

LLRpcei→ci = 2 tanh−1∏

cj∈S∗pcei

tanh(LLRcj→pcei

2)

Extrinsic Information

v

u

s

s

u

v

s α

α

α

β

β




The bitnode Rule:


pcej∈S∗ci

LLRpcej→ci

LLRtype={


The checknode Rule:

LLRpcei→ci = 2 tanh−1∏

cj∈S∗pcei

tanh(LLRcj→pcei

2)

LLR0+ Extrinsic Information

LLRs+ Extrinsic Information

Extrinsic Information

v

u

s

s

u

v

s α

α

α

β

β

v

u

s

s

u

v

s α

α

α

β

β



Scrambling or Splitting: Information Theoretical Comparaison

Mutual information vs. Eb/N0 for Hs = 0.5 and coding rates Rc = 0.5 (AWGN Channel).

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

-10 -5 0 5 10

Mut

ual i

nfor

mat

ion

Eb/N0(dB)

Gaussian inputNon-Systematic code, BPSK input

Scrambled ds=5Scrambled ds=3

Systematic codes, BPSK input



Finite-length Performance

1E-05

1E-04

1E-03

1E-02

1E-01

-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5

Bit E

rror

Rate

SNR

(3,6) systematic LDPC without SCCD(3,6) systematic LDPC with SCCD

scrambler ds=3MN code db=3

splitter ds=4

1E-04

1E-03

1E-02

1E-01

1E+00

-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5

Fra

me E

rror

Rate

SNR

(3,6) systematic LDPC without SCCD(3,6) systematic LDPC with SCCD

scrambler ds=3 MN code db=3

splitter ds=4

Bit (left) and word (right) error probabilities vs. signal to noise ratio for codes with rateRc = 1/2 length N = 2000 and non uniform source distribution µ = 0.1 : systematic (3,6)LDPC with and without SCCD, split-LDPC with ds = 4, scramble-LDPC ds = 3 and MNCodes db = 3.



Finite-length Performance (2/2)

1E-04

1E-03

1E-02

1E-01

1E+00

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 4

Fra

me E

rror

Rate

SNR

(3,30) systematic LDPC with SCCDscrambler ds=5

splitter ds=7

Word error probabilities vs. signal to noise ratio Eb/N0 for codes with rate Rc = 0.9 lengthN = 2000 and non uniform source distribution µ = 0.1 : systematic (3,30) LDPC with SCCD,scramble-LDPC with ds = 5 and split-LDPC with ds = 7.


Part II

DE Analysis for Split-LDPC Codes

DE Motivation and Assumption DE statement Simulation Results Stability Analysis

Motivation and Contribution

The problem:How close do Split-LDPC structures approach the Challengingasymptotical limits ?

Exploring the Split-LDPC asymptotical convergence behaviour.

The proposal: Density Evolution Analysis1 Deriving a Density Evolution algorithm for Split-LDPC codes.

2 Investigating the stability issues related to the decoder convergence.



Motivation and Contribution

The problem:How close do Split-LDPC structures approach the Challengingasymptotical limits ?

Exploring the Split-LDPC asymptotical convergence behaviour.

The proposal: Density Evolution Analysis1 Deriving a Density Evolution algorithm for Split-LDPC codes.

2 Investigating the stability issues related to the decoder convergence.



Density Evolution Assumptions

1 Concentration and the local tree Assumption.

2 Symmetry Conditions (Channel, variable nodes, Checknodes).

3 The all-zero codeword restriction.

3 types of message distribution =⇒ Local tree assumption over 3 types of trees

Message oriented Density Evolution.

12

3

Distributions Messages

s

u

dcϑ

dc

db

ds

1 1

β

αs

u

db

ds

βϑ

p2(x)

p3(x)

p1(x)

α

0 0 0 00

ATypical Set

BC[A]=2k

C(B)=2K×Hs

/∈B









12

3


s

u

dcϑ

dc

db

ds

1 1

β

αs

u

db

ds

βϑ

p2(x)

p3(x)

p1(x)

α

0 0 0 00

ATypical Set

BC[A]=2k

C(B)=2K×Hs

/∈B









12

3


s

u

dcϑ

dc

db

ds

1 1

β

αs

u

db

ds

βϑ

p2(x)

p3(x)

p1(x)

α

0 0 0 00

ATypical Set

BC[A]=2k

C(B)=2K×Hs

/∈B









12

3


s

u

dcϑ

dc

db

ds

1 1

β

αs

u

db

ds

βϑ

p2(x)

p3(x)

p1(x)

α

0 0 0 00

ATypical Set

BC[A]=2k

C(B)=2K×Hs

/∈B









12

3


s

u

dcϑ

dc

db

ds

1 1

β

αs

u

db

ds

βϑ

p2(x)

p3(x)

p1(x)

α

0 0 0 00

ATypical Set

BC[A]=2k

C(B)=2K×Hs

/∈B



The DE statement: the Checknode level

0

1

0

0 01 0

0 0

s 0

00

s Averaging

{{

ds − 1

u u

p1(x)

u

u

α

pmα (x) = Rc

(ps(x), (1 − µ) qm

1 (x) + µ qm1 (−x)

)where:

ps(x) = δ(x− log1− µ

µ) = δ(x− s)

qm1 (x) = ρα(pm

1 (x))




dc − 1

β

1

2u + 1

2ϑ 1

2u + 1

2ϑ

R p2(x) + (1 − R) p3(x)

pmβ (x) = ρ

(Rc pm

2 (x) + (1 − Rc) pm3 (x)

)




s

dc − 1

β

1

2u + 1

2ϑ 1

2u + 1

2ϑ

ds − 1

u u

α R p2(x) + (1 − R) p3(x)p1(x)

pmα (x) = Rc

(ps(x), (1 − µ) qm

1 (x) + µ qm1 (−x)

)

pmβ (x) = ρ

(Rc pm

2 (x) + (1 − Rc) pm3 (x)

)



The DE statement: the variable-node level

dc − 1

βα β

ds − 1

u u

α

12u + 1

2ϑ

ds − 1 db

12u + 1

2ϑ

α

R p2 + (1 − R) p3p1

p1

xλ(x)η(x)

u

p1(x) at the (m + 1)th iteration

pm+11 (x) = p0(x) ⊗ λ1α

(pm

α (x))

⊗ λ1

(pm

β (x))




dc − 1

βα β

ds − 1

u u

α

β

db − 1ds

1

2u + 1

2ϑ 1

2u + 1

2ϑ

xη(x)

p1 R p2 + (1 − R)p3

p2

u

λ(x)


pm+12 (x) = p0(x) ⊗ λ2α

(pm

α (x))

⊗ λ(pm

β (x))




ϑ

β

β

dc − 1

db − 1

1

2u + 1

2ϑ 1

2u + 1

2ϑ

p3

β βR p2 + (1 − R)p3


pm+13 (x) = p0(x) ⊗ λ

(pm

β (x))



-12

-10

-8

-6

-4

-2

0

2

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Min

imum

Achie

vable

Eb/N

0 (

dB

)

Source Entropy (bits)

lambda(x)=0.32660x+0.11960x^2+0.18393x^3+0.36988x^4

rho(x)=0.78555x^5+0.21445x^6

Systematic code, BPSK inputNon-Systematic code, BPSK input

Gaussian inputsplit-LDPC code, DE thresholds

Minimum achievable Eb/N0 versus source entropy Hs for a regular ds = 3 splitterconcatenated to an irregular LDPC code of rate Rc = 1/2 over a BIAWGN channel.



The asymptotical behaviour of the decoder in the neighborhood of δ∞

BSC

BSC

BSC

R

1−R

CHANNEL

µ

µ

µ

s

s

s

u

ϑ β

P0

ϑ

u

ds

(µ δ−s + (1 − µ) δs)⊗ ds

Pu0

Pu0 = p0(x) ⊗ (µ δ−s + (1− µ) δs)⊗ dS

Proposition 1In the neighborhood of δ∞, the message density given by the splitter to the core LDPC(from node α to node u) is equivalent to the initial message density of ds parallelconcatenated BSC with a crossover probability µ.




BSC

BSC

BSC

R

1−R

CHANNEL

µ

µ

µ

s

s

s

u

ϑ β

P0

ϑ

u

ds

(µ δ−s + (1 − µ) δs)⊗ ds

Pu0

Pu0 = p0(x) ⊗ (µ δ−s + (1− µ) δs)⊗ dS

Proposition 2For non-uniform sources,type-1 message distribution (from node u to node α) shows apermanent stability around the fixed point δ∞.




BSC

BSC

BSC

R

1−R

CHANNEL

µ

µ

µ

s

s

s

u

ϑ β

P0

ϑ

u

ds

(µ δ−s + (1 − µ) δs)⊗ ds

Pu0

Pu0 = p0(x) ⊗ (µ δ−s + (1− µ) δs)⊗ dS

Proposition 3When close to zero error rate, stability of the LDPC constituent is not disturbed by thesplitter.




BSC

BSC

BSC

R

1−R

CHANNEL

µ

µ

µ

s

s

s

u

ϑ β

P0

ϑ

u

ds

(µ δ−s + (1 − µ) δs)⊗ ds

Pu0

Pu0 = p0(x) ⊗ (µ δ−s + (1− µ) δs)⊗ dS

If the embedded LDPC is stable the Split-LDPC would be so.The inverse is not true !



Stability Condition for Split-LDPC

Systematic LDPC CHANNEL

p0(x)

The General Stability Condition for a systematicLDPC:

B(p0)λ′(0) ρ′(1) ≤ 1

where B(p0) =∫ +∞

−∞p0(x)e−x/2 dx is the Bhattacharyya constant of the channel.



Stability Condition for Split-LDPC

CHANNEL

CHANNELEQUIVALENT

Averaging

1−R

R

GLOBAL EQUIVALENT CHANNEL Systematic CORE−LDPC

u

ϑ βϑ

u

Pu0 = p0(x) ⊗ (µ δ−s + (1 − µ) δs)⊗ ds

p0(x)

Peq = R pu0 + (1 − R) p0(x)

The General Stability Condition for Split-LDPC:

B(Peq)λ′(0) ρ′(1) ≤ 1

where Peq = Rc Puo(x) + (1−Rc) p0(x) is the initial message density of the globalequivalent channel; and B(Peq) is the Bhattacharyya constant of the global equivalentchannel.



The Splitter Asymptotical Properties:

Proposition 4For uniform sources, the threshold and the stability condition of thesplit-LDPC code are the same for the CORE-LDPC





ExampleSplit-LDPC Stability Condition for BEC Channel:

λ′(0) ρ′(1) <1

ε×

1

[(1 − Rc) + Rc (2√

µ(1 − µ))ds]





ExampleSplit-LDPC Stability Condition for BSC Channel:

λ′(0) ρ′(1) <1

2√

λ(1 − λ)×

1

[(1 − Rc) + Rc (2√

µ(1 − µ))ds]





ExampleSplit-LDPC Stability Condition for AWGN Channel::

λ′(0) ρ′(1) < e1

2σ2 ×1

[(1 − Rc) + Rc (2√

µ(1 − µ))ds]


Part III

EXIT Chart Analysis for Split-LDPC Codes

Motivation and Assumptions Split-LDPC EXIT Chart Statement Design of Irregular Split-LDPC Results

Motivation and Proposal

The problemHigh computational complexity for Density EvolutionLow Complexity Approaches based on Gaussian Approximation

Lower Complexity and one-dimensionalMore InsightfulLess Accurate

How to build capacity achieving Split-LDPC ?

The Proposal:Message oriented Bi-dimensional low complexity approach based on a moreaccurate Exit Chart method.



Assumptions and Notations:

Message-oriented local tree assumption.

Consistency Condition is realized on all types of message distribution.

Gaussian approximation is applied at the bitnodes output based on equalmutual information.

We display the error probability as a measure of knowledge U .

dc − 1

βα β

ds − 1

u u

α

12u + 1

2ϑ

ds − 1 db

12u + 1

2ϑ

α

R p2 + (1 − R) p3p1

p1

xλ(x)η(x)

u




Message-oriented local tree assumption.

Consistency Condition is realized on all types of message distribution.

Gaussian approximation is applied at the bitnodes output based on equalmutual information.


f(x) = f(−x) ex for all x ∈ R+




Message-oriented local tree assumption.Consistency Condition is realized on all types of message distribution.Gaussian approximation is applied at the bitnodes output based on equalmutual information.


Uin

Uin

Uout

U0

Uout

U0

f

f−1




Message-oriented local tree assumption.Consistency Condition is realized on all types of message distribution.Gaussian approximation is applied at the bitnodes output based on equalmutual information.We display the error probability as a measure of knowledge U .

Uin

Uin

Uout

U0

Uout

U0

f

f−1

Perr = Q(σ

2) =

1√

2π

∫ +∞

σ/2

e− t2

2 dt



The EXIT Chart Statement for Split-LDPC : Message Combining

12u + 1

2ϑ12u + 1

2ϑ

dc − 1

α β

ds − 1

u u

α

β

db − 1ds

12u + 1

2ϑ

xη(x)

u

λ(x)

β

P∗

2 = (1 − R)P3 + RP2

ϑ

β

β

dc − 1

db − 1

12u + 1

2ϑ

β β

P3P2

P ∗

in1

P ∗

in2

P ∗out2 = G(P ∗

in1, P ∗in2)




12u + 1

2ϑ12u + 1

2ϑ

dc − 1

α β

ds − 1

u u

α

β

db − 1ds

12u + 1

2ϑ

xη(x)

u

λ(x)

β

P∗

2 = (1 − R)P3 + RP2

ϑ

β

β

dc − 1

db − 1

12u + 1

2ϑ

β β

P3P2

P ∗

in1

P ∗

in2

G(x, y) = Rc ×db∑

j=2

λj fds+1,j(x, y) + (1 − Rc) ×db∑

j=2

λj gj(y)




dc − 1

βα β

ds − 1

u u

α

1

2u + 1

2ϑ

ds − 1 db

1

2u + 1

2ϑ

α

xλ(x)η(x)

u

P ∗

in2P ∗

in1

P∗

1

P ∗out1 = F (P ∗

in1, P ∗in2)




dc − 1

βα β

ds − 1

u u

α

1

2u + 1

2ϑ

ds − 1 db

1

2u + 1

2ϑ

α

xλ(x)η(x)

u

P ∗

in2P ∗

in1

P∗

1

F (x, y) =db∑

j=2

λj fds,j+1(x, y)



The validity Range for the Bidimensional Dynamical System

The mixtures of Gaussian densities associated to inputs P ∗in1 and P ∗in2 mustsatisfy the two equalities:

P ∗in1 =db∑

j=2

λj12erfc

(12

√m0 + (ds − 1)mα + jmβ

)

P ∗in2 = Rc ×db∑

j=2

λj12erfc

(12

√m0 + dsmα + (j − 1)mβ

)

+ (1−Rc)×db∑

j=2

λj12erfc

(12

√m0 + (j − 1)mβ

)

m0: Mean of messages from the channel.mα: Mean of messages from the splitter checknodes α.mβ : Mean of messages from the LDPC checknodes β.



Trajectory of error probability near the code threshold

Right to the threshold:Es

N0= −5.58dB, Threshold=−5.68dB. Final fixed point is 0.

0.00

0.05

0.10

0.15

0.20

0.00

0.05

0.10

0.15

0.20

0.050.10

0.150.20

0.25

0.00

0.05

0.10

0.15

0.20

0.25

Pout1

Transfer Function F(x,y)Trajectory of Pout1

Pin1

Pin2

Pout1

Illustration for an irregular Split-LDPC code.



Trajectory of error probability beyond the code threshold (2)

Left to the threshold:Es

N0= −5.78dB, Threshold=−5.68dB. Final fixed point is non-zero.

0.00

0.05

0.10

0.15

0.20

0.00

0.05

0.10

0.15

0.20

0.050.10

0.150.20

0.25

0.00

0.05

0.10

0.15

0.20

0.25

Pout1

Transfer Function F(x,y)Trajectory of Pout1

Pin1

Pin2

Pout1

Illustration for an irregular Split-LDPC code.



Open Tunnel for the Bi-dimensional EXIT Chart

Open tunnel obtained by plotting the trajectory of error probability and its z = x plane

reflection atEs

N0= −5.58dB .

0.000.020.040.060.080.100.120.140.160.18

0.00 0.05 0.10 0.15 0.20 0.050.100.150.200.25

0.00

0.05

0.10

0.15

0.20

0.25

Pout1Transfer Function F(x,y)

Trajectory of Pout1Inverse of Trajectory of Pout1

Plan z=x

Pin1Pin2

Pout1



Design Irregular Split-LDPC codes

The Design linear Program:

1 Maximize∑i≥2

λi/i

2 subject to : λi ≥ 0,∑i≥2

λi = 1

3 and ∀(P ∗1in, P ∗2in) ∈ T (S)∑db

j=2 λj fds,j+1(P ∗1in, P2in∗) < P ∗1in∑db

j=2 λj [Rc × fds+1,j(P ∗1in, P ∗2in) + (1−Rc)× gj(P ∗2in)] < P ∗2in




The Design Process:Select a regular code with the desired coding rate.

Use the regular code degree sequence for mapping (P ∗1in, P ∗2in) to theappropriate input Gaussian mixture densities.

Find the EXIT charts surfaces for different variable degrees.

Find a linear combination with an open EXIT chart that maximizes therate and meets all the required design criterion.

Best Approach to Shannon limits is within 0.1 dB !




The Design Process:Select a regular code with the desired coding rate.

Use the regular code degree sequence for mapping (P ∗1in, P ∗2in) to theappropriate input Gaussian mixture densities.

Find the EXIT charts surfaces for different variable degrees.

Find a linear combination with an open EXIT chart that maximizes therate and meets all the required design criterion.

Best Approach to Shannon limits is within 0.1 dB !



Bi-dimensional EXIT Chart Accuracy

Error in approximation of threshold (dB) using EXIT chart analysis for various split-LDPC

codes of Rate one-half with ds = 3 , Hs = 0.5. ∆ is the log-ratio quantization step.

Eb/N0∗ (dB) Eb/N0

∗ (dB) Error ∆Eb/N0∗

db dc Rate DE ∆ = 0.005 EC ∆ = 0.005 for ∆ = 0.0053 6 0.5 −2.22 −2.199 0.0204 8 0.5 −1.12 −1.129 0.0095 10 0.5 −0.32 −0.369 0.049


Part IV

EM Source-Channel Estimation

Motivation and Proposal Expectation-Maximization Complex BIAWGN Channel Simultations Results


MotivationFull utilization of systematic and non-systematic LDPC codes requires:

1 Knowledge of the source probability distribution (SSI) at the decoder side.2 Knowledge of channel parameters (CSI) at the decoder side.

ProposalJoint Source-Channel Iterative Estimation and Decoding for non-uniform

sources based on the Expectation Maximization Algorithm (EM)

→ No performance loss → Negligible estimation complexity




MotivationFull utilization of systematic and non-systematic LDPC codes requires:

1 Knowledge of the source probability distribution (SSI) at the decoder side.2 Knowledge of channel parameters (CSI) at the decoder side.

ProposalJoint Source-Channel Iterative Estimation and Decoding for non-uniform

sources based on the Expectation Maximization Algorithm (EM)

→ No performance loss → Negligible estimation complexity



Brief statement of the EM algorithm

Channel Encoder

y: incomplete data (observed)x:missing data

yx

Source Channel

CSI

Channel Decoder Sink

01

set of parameters to be estimated, SSI+CSI

κ : complete data, κ = (x, y)

Θ

SSI = µ

0.9

p(s = 1) = µ0.1

Θ=SSI+CSI

E-step: Compute the Auxiliary function Q:

Q(θ|θi) = E[log p(x, y|θ)|y, θi]

=∑

x

log[P (y|x, θ) P (x|θ)]APPi(x)




Channel Encoder


yx

Source Channel

CSI


01



Θ

SSI = µ

0.9

p(s = 1) = µ0.1

Θ=SSI+CSI



=∑

x


The SSI part in the auxiliary function is:

P (x|θ) ≡ P (s|θ) = µωH(s) (1 − µ)K−ωH(s)




Channel Encoder


yx

Source Channel

CSI


01



Θ

SSI = µ

0.9

p(s = 1) = µ0.1

Θ=SSI+CSI



=∑

x


M-step:

θi+1 = arg maxθ

Q(θ|θi)



Joint Source-Channel Estimation on Complex BIAWGN

The Complex BIAWGN channel is defined as

yi = Ae=φxi + ηi with = =√−1

where the three CSI parameters are :

The amplitude A , which is real positive

The phase ambiguity φ, which is uniformly distributed between 0 and 2π.

The Gaussian noise variance σ2.






E-step: Auxiliary Function

Q(θ|θi) = log[µ

1− µ]s + K log[(1− µ)]−N log[2πσ2]

− 12σ2

N∑j=1

∣∣yj

∣∣2 − A2

2σ2

N∑j=1

∣∣xj

∣∣2 +A

σ2

N∑j=1

R{xj∗e−=φyj

}






M-step: Maximizing the auxiliary function for SSI

µi+1 =

∑Kj=1 sj

K






M-step: Maximizing the auxiliary function for CSI

Ai+1 =

∑Nj=1R

{xjy

∗j e=φi}∑N

j=1

∑xj

APPi(xj)∣∣xj

∣∣2σ2 (i+1) =

12N

N∑j=1

˜∣∣yj −Aie=φixj

∣∣2φi+1 = −Arg

N∑j=1

xj y∗j



10-4

10-3

10-2

10-1

100

-2.4 -2 -1.6 -1.2 -0.8 -0.4 0 0.4 0.8 1.2 1.6 2 2.4

Fra

me

Err

or R

ate

Eb/N0 (dB)

Systematic + UniformSystematic + Non Uniform (Perfect)

Systematic + Non Uniform (EM observation)Systematic + Non Uniform (EM APP)

Non Systematic + Non Uniform (Perfect)Non Systematic + Non Uniform (EM APP)

Systematic and non-systematic LDPC codes on AWGN, Rc = 1/2.


Conclusions and Open Issues

Conclusions

The theoretical limits for redundant data are mooving to better regions.

Split-LDPC codes are the best non systematic LDPC constructions.

Split-LDPC codes have good asymptotical performance and properties.

Designed irregular Split-LDPC codes are capacity achieving.

The 2-D EXIT Chart formalism is lower complexity and as accurate as theDE.

The performance using EM is as good as the perfect knowledge case.



Conclusions









Conclusions









Conclusions









Conclusions









Conclusions









Open Issues

Investigating more complex source and channel models.

Extending our study to irregular splitting and scrambling LDPC.

Performing Complexity and Finite-length Optimization.

Consider lossy sources assumption .

Studying and incorporating Split-LDPC in practical communicationsystems.



Open Issues








Open Issues








Open Issues








Open Issues







Documents

Phd Defence