Information Capacity of the BSC Permutation Channel

Anuran Makur

EECS Department, Massachusetts Institute of Technology

Allerton Conference 2018

A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 1 / 21

Outline

1 IntroductionMotivation: Coding for Communication NetworksThe Permutation Channel ModelCapacity of the BSC Permutation Channel

2 Achievability

3 Converse

4 Conclusion

Motivation: Point-to-point Communication in Networks

NETWORK

SENDER RECEIVER

Model communication network as a channel:

Alphabet symbols = all possible L-bit packetsmultipath routed networkpackets are impaired

NETWORK

SENDER RECEIVER

Model communication network as a channel

Alphabet symbols = all possible L-bit packetsmultipath routed networkpackets are impaired

NETWORK

SENDER RECEIVER

Alphabet symbols = all possible L-bit packets ⇒ 2L input symbols

multipath routed networkpackets are impaired

NETWORK

SENDER RECEIVER

Alphabet symbols = all possible L-bit packetsmultipath routed network or evolving network topology

packets are impaired

NETWORK

SENDER RECEIVER

Alphabet symbols = all possible L-bit packetsmultipath routed network ⇒ packets received with transpositions

packets are impaired

NETWORK

SENDER RECEIVER

Alphabet symbols = all possible L-bit packetsmultipath routed network ⇒ packets received with transpositionspackets are impaired (e.g. deletions, substitutions)

NETWORK

SENDER RECEIVER

Alphabet symbols = all possible L-bit packetsmultipath routed network ⇒ packets received with transpositionspackets are impaired ⇒ model using channel probabilities

Example: Coding for Random Deletion Network

Consider a communication network where packets can be dropped:

NETWORK

SENDER RECEIVER

Abstraction:

n-length codeword = sequence of n packets:Random permutation block: Randomly permute packets of codeword

How do you code in such channelswithout increasing alphabet size?

NETWORK

SENDER RECEIVER

RANDOM DELETION

RANDOM PERMUTATION

Abstraction:

n-length codeword = sequence of n packets

Random permutation block: Randomly permute packets of codeword

NETWORK

SENDER RECEIVER

RANDOM DELETION

RANDOM PERMUTATION

Abstraction:

n-length codeword = sequence of n packetsRandom deletion channel: Delete each symbol/packet of codewordindependently with probability p ∈ (0, 1)

NETWORK

SENDER RECEIVER

RANDOM DELETION

RANDOM PERMUTATION

Abstraction:

n-length codeword = sequence of n packetsRandom deletion channel: Delete each symbol/packet of codewordindependently with probability p ∈ (0, 1)

NETWORK

SENDER RECEIVER

RANDOM DELETION

RANDOM PERMUTATION

Abstraction:

n-length codeword = sequence of n packetsRandom deletion channel: Delete each symbol/packet of codewordindependently with probability p ∈ (0, 1)Random permutation block: Randomly permute packets of codeword

NETWORK

SENDER RECEIVER

RANDOM DELETION

RANDOM PERMUTATION

Abstraction:

n-length codeword = sequence of n packetsRandom deletion channel: Delete each symbol/packet of codewordindependently with probability p ∈ (0, 1)Random permutation block: Randomly permute packets of codeword

NETWORK

SENDER RECEIVER

ERASURE CHANNEL

RANDOM PERMUTATION?

Abstraction:

n-length codeword = sequence of n packetsEquivalent Erasure channel: Erase each symbol/packet of codewordindependently with probability p ∈ (0, 1)Random permutation block: Randomly permute packets of codeword

NETWORK

SENDER RECEIVER

ERASURE CHANNEL

RANDOM PERMUTATION?

?1 2 33

Abstraction:

n-length codeword = sequence of n packetsErasure channel: Erase each symbol/packet of codewordindependently with probability p ∈ (0, 1)Random permutation block: Randomly permute packets of codewordCoding: Add sequence numbers (packet size = L + log(n) bits,alphabet size = n 2L)

More refined coding techniques simulate sequence numbers,e.g. [Mitzenmacher 2006], [Metzner 2009]

NETWORK

SENDER RECEIVER

ERASURE CHANNEL

RANDOM PERMUTATION?

?1 2 33

Abstraction:

n-length codeword = sequence of n packetsErasure channel: Erase each symbol/packet of codewordindependently with probability p ∈ (0, 1)Random permutation block: Randomly permute packets of codewordCoding: Add sequence numbers and use standard coding techniques

More refined coding techniques simulate sequence numbers,e.g. [Mitzenmacher 2006], [Metzner 2009]

NETWORK

SENDER RECEIVER

ERASURE CHANNEL

RANDOM PERMUTATION?

?1 2 33

Abstraction:

n-length codeword = sequence of n packetsErasure channel: Erase each symbol/packet of codewordindependently with probability p ∈ (0, 1)Random permutation block: Randomly permute packets of codewordCoding: Add sequence numbers and use standard coding techniquesMore refined coding techniques simulate sequence numbers,e.g. [Mitzenmacher 2006], [Metzner 2009]

NETWORK

SENDER RECEIVER

ERASURE CHANNEL

RANDOM PERMUTATION?

Abstraction:

n-length codeword = sequence of n packetsErasure channel: Erase each symbol/packet of codewordindependently with probability p ∈ (0, 1)Random permutation block: Randomly permute packets of codeword

The Permutation Channel Model

ENCODER CHANNEL RANDOM PERMUTATION DECODER

𝑀 𝑋 𝑍 𝑌 𝑀

Sender sends message M ∼ Uniform(M)

Possibly randomized encoder fn :M→ X n produces codewordX n1 = (X1, . . . ,Xn) = fn(M) (with block-length n)

Discrete memoryless channel PZ |X with input and output alphabetsX and Y produces Zn

PZn1 |X n

1(zn1 |xn1 ) =

n∏i=1

PZ |X (zi |xi )

Random permutation generates Y n1 from Zn

Possibly randomized decoder gn : Yn →M produces estimateM = gn(Y n

1 ) at receiver

PZn1 |X n

1(zn1 |xn1 ) =

n∏i=1

PZ |X (zi |xi )

1 ) at receiver

PZn1 |X n

1(zn1 |xn1 ) =

n∏i=1

PZ |X (zi |xi )

1 ) at receiver

PZn1 |X n

1(zn1 |xn1 ) =

n∏i=1

PZ |X (zi |xi )

1 ) at receiver

PZn1 |X n

1(zn1 |xn1 ) =

n∏i=1

PZ |X (zi |xi )

1 ) at receiver

Coding for the Permutation Channel

General Principle:“Encode the information in an object that is invariant under the[permutation] transformation.” [Kovacevic-Vukobratovic 2013]

Multiset codes are studied in [Kovacevic-Vukobratovic 2013],[Kovacevic-Vukobratovic 2015], and [Kovacevic-Tan 2018]

What about the information theoreticaspects of this model?

Information Capacity of the Permutation Channel

Average probability of error Pnerror , P(M 6= M)

“Rate” of encoder-decoder pair (fn, gn):

R ,log(|M|)

log(n)

|M| = nR

Rate R ≥ 0 is achievable ⇔ ∃{(fn, gn)}n∈N such that limn→∞

Pnerror = 0

Definition (Permutation Channel Capacity)

Cperm(PZ |X ) , sup{R ≥ 0 : R is achievable}

R ,log(|M|)

log(n)

|M| = nR

Pnerror = 0

R ,log(|M|)

log(n)

|M| = nR

Pnerror = 0

R ,log(|M|)

log(n)

|M| = nR because number of empirical distributions of Y n1 is poly(n)

Pnerror = 0

R ,log(|M|)

log(n)

|M| = nR

Pnerror = 0

R ,log(|M|)

log(n)

|M| = nR

Pnerror = 0

Capacity of the BSC Permutation Channel

ENCODER BSC 𝒑 RANDOM PERMUTATION DECODER

Channel is binary symmetric channel, denoted BSC(p):

∀z , x ∈ {0, 1}, PZ |X (z |x) =

{1− p, for z = x

p, for z 6= x

Alphabets are X = Y = {0, 1}Assume crossover probability p ∈ (0, 1) and p 6= 1

Main Question

What is the permutation channel capacity of the BSC?

∀z , x ∈ {0, 1}, PZ |X (z |x) =

{1− p, for z = x

p, for z 6= x

Alphabets are X = Y = {0, 1}

Assume crossover probability p ∈ (0, 1) and p 6= 12

Main Question

∀z , x ∈ {0, 1}, PZ |X (z |x) =

{1− p, for z = x

p, for z 6= x

Main Question

∀z , x ∈ {0, 1}, PZ |X (z |x) =

{1− p, for z = x

p, for z 6= x

Main Question

Outline

1 Introduction

2 AchievabilityEncoder and DecoderTesting between Converging HypothesesIntuition via Central Limit TheoremSecond Moment Method for TV Distance

3 Converse

4 Conclusion

Warm-up: Sending Two Messages

Fix a message m ∈ {0, 1}

, and encode m as fn(m) = X n1

i.i.d.∼ Ber(qm)

𝑞13

𝑞23

Memoryless BSC(p) outputs Zn1

i.i.d.∼ Ber(p ∗ qm), wherep ∗ qm , p(1− qm) + qm(1− p) is the convolution of p and qm

Random permutation generates Y n1

i.i.d.∼ Ber(p ∗ qm)

Maximum Likelihood (ML) decoder: M = 1{1n

∑ni=1 Yi ≥ 1

∑ni=1 Yi → p ∗ qm in probability as n→∞ [WLLN]

⇒ limn→∞

Pnerror = 0 as p ∗ q0 6= p ∗ q1

Fix a message m ∈ {0, 1}, and encode m as fn(m) = X n1

i.i.d.∼ Ber(qm)

𝑞13

𝑞23

∑ni=1 Yi ≥ 1

⇒ limn→∞

i.i.d.∼ Ber(qm)

𝑞13

𝑞23

∑ni=1 Yi ≥ 1

⇒ limn→∞

i.i.d.∼ Ber(qm)

𝑞13

𝑞23

∑ni=1 Yi ≥ 1

⇒ limn→∞

i.i.d.∼ Ber(qm)

𝑞13

𝑞23

∑ni=1 Yi ≥ 1

⇒ limn→∞

i.i.d.∼ Ber(qm)

𝑞13

𝑞23

∑ni=1 Yi ≥ 1

⇒ limn→∞

Encoder and Decoder

Suppose M = {1, . . . , nR} for some R > 0

Randomized encoder: Given m ∈M, fn(m) = X n1

i.i.d.∼ Ber( m

Given m ∈M, Y n1

i.i.d.∼ Ber(p ∗ m

)ML decoder: For yn1 ∈ {0, 1}n, gn(yn1 ) = arg max

m∈MPY n

1 |M(yn1 |m)

Challenge: Although 1n

∑ni=1 Yi → p ∗ m

nRin probability as n→∞,

consecutive messages become indistinguishable i.e. mnR− m+1

nR→ 0

Fact: Consecutive messages distinguishable ⇒ limn→∞

Pnerror = 0

What is the largest R such that two consecutive messagescan be distinguished?

Encoder and Decoder

i.i.d.∼ Ber( m

Given m ∈M, Y n1

m∈MPY n

1 |M(yn1 |m)

∑ni=1 Yi → p ∗ m

nR→ 0

Pnerror = 0

Encoder and Decoder

i.i.d.∼ Ber( m

Given m ∈M, Y n1

)(as before)

ML decoder: For yn1 ∈ {0, 1}n, gn(yn1 ) = arg maxm∈M

PY n1 |M(yn1 |m)

∑ni=1 Yi → p ∗ m

nR→ 0

Pnerror = 0

Encoder and Decoder

i.i.d.∼ Ber( m

Given m ∈M, Y n1

m∈MPY n

1 |M(yn1 |m)

∑ni=1 Yi → p ∗ m

nR→ 0

Pnerror = 0

Encoder and Decoder

i.i.d.∼ Ber( m

Given m ∈M, Y n1

m∈MPY n

1 |M(yn1 |m)

∑ni=1 Yi → p ∗ m

nR→ 0

Pnerror = 0

Encoder and Decoder

i.i.d.∼ Ber( m

Given m ∈M, Y n1

m∈MPY n

1 |M(yn1 |m)

∑ni=1 Yi → p ∗ m

nR→ 0

Pnerror = 0

Encoder and Decoder

i.i.d.∼ Ber( m

Given m ∈M, Y n1

m∈MPY n

1 |M(yn1 |m)

∑ni=1 Yi → p ∗ m

nR→ 0

Pnerror = 0

Testing between Converging Hypotheses

Binary Hypothesis Testing:

Consider hypothesis H ∼ Ber(12

)with uniform prior

For any n ∈ N, q ∈ (0, 1), and R > 0, consider likelihoods:

Given H = 0 : X n1

i.i.d.∼ PX |H=0 = Ber(q)

Given H = 1 : X n1

i.i.d.∼ PX |H=1 = Ber

)Define the zero-mean sufficient statistic of X n

1 for H:

n∑i=1

Xi − q − 1

Let HnML(Tn) denote the ML decoder for H based on Tn with

minimum probability of error PnML , P(Hn

ML(Tn) 6= H)

Want: Largest R > 0 such that limn→∞

PnML = 0?

)with uniform prior

Given H = 0 : X n1

Given H = 1 : X n1

Define the zero-mean sufficient statistic of X n1 for H:

n∑i=1

Xi − q − 1

ML(Tn) 6= H)

PnML = 0?

)with uniform prior

Given H = 0 : X n1

Given H = 1 : X n1

1 for H:

n∑i=1

Xi − q − 1

ML(Tn) 6= H)

PnML = 0?

)with uniform prior

Given H = 0 : X n1

Given H = 1 : X n1

1 for H:

n∑i=1

Xi − q − 1

ML(Tn) 6= H)

PnML = 0?

)with uniform prior

Given H = 0 : X n1

Given H = 1 : X n1

1 for H:

n∑i=1

Xi − q − 1

ML(Tn) 6= H)

PnML = 0?

Intuition via Central Limit Theorem

For large n, PTn|H(·|0) and PTn|H(·|1) are Gaussian distributions [CLT]

|E[Tn|H = 0]− E[Tn|H = 1]| = 1/nR

Standard deviations are Θ(1/√n)

Figure:

𝑃 | 𝑡|0 𝑃 | 𝑡|1

12𝑛

|E[Tn|H = 0]− E[Tn|H = 1]| = 1/nR

Figure:

𝑃 | 𝑡|0 𝑃 | 𝑡|1

𝛳1𝑛

|E[Tn|H = 0]− E[Tn|H = 1]| = 1/nR

Figure:

𝑃 | 𝑡|0 𝑃 | 𝑡|1

𝛳1𝑛

|E[Tn|H = 0]− E[Tn|H = 1]| = 1/nR

Case R < 12 :

𝑃 | 𝑡|0 𝑃 | 𝑡|1

𝛳1𝑛

|E[Tn|H = 0]− E[Tn|H = 1]| = 1/nR

Case R < 12 : Decoding is possible ,

𝑃 | 𝑡|0 𝑃 | 𝑡|1

𝛳1𝑛

|E[Tn|H = 0]− E[Tn|H = 1]| = 1/nR

Case R > 12 :

𝑃 | 𝑡|0 𝑃 | 𝑡|1

𝛳1𝑛

|E[Tn|H = 0]− E[Tn|H = 1]| = 1/nR

Case R > 12 : Decoding is impossible /

𝑃 | 𝑡|0 𝑃 | 𝑡|1

𝛳1𝑛

Second Moment Method for TV Distance

Lemma (2nd Moment Method [Evans-Kenyon-Peres-Schulman 2000])∥∥PTn|H=1 − PTn|H=0

∥∥TV≥ (E[Tn|H = 1]− E[Tn|H = 0])2

4VAR(Tn)

where ‖P − Q‖TV = 12 ‖P − Q‖`1 is the total variation (TV) distance

between the distributions P and Q.

Proof:

∥∥TV≥ (E[Tn|H = 1]− E[Tn|H = 0])2

4VAR(Tn)

Proof: Let T+n ∼ PTn|H=1 and T−n ∼ PTn|H=0(

]− E

[T−n])2

t(PTn|H(t|1)− PTn|H(t|0)

∥∥TV≥ (E[Tn|H = 1]− E[Tn|H = 0])2

4VAR(Tn)

Proof: Let T+n ∼ PTn|H=1 and T−n ∼ PTn|H=0(

]− E

[T−n])2

t√PTn(t)

(PTn|H(t|1)− PTn|H(t|0)

)√PTn(t)

∥∥TV≥ (E[Tn|H = 1]− E[Tn|H = 0])2

4VAR(Tn)

Proof: Cauchy-Schwarz inequality

(E[T+n

]− E

[T−n])2

PTn(t)

)√PTn(t)

t2PTn(t)

)(∑t

)2PTn(t)

∥∥TV≥ (E[Tn|H = 1]− E[Tn|H = 0])2

4VAR(Tn)

Proof: Recall that Tn is zero-mean(E[T+n

]− E

[T−n])2

PTn(t)

)√PTn(t)

≤ VAR(Tn)

)2PTn(t)

∥∥TV≥ (E[Tn|H = 1]− E[Tn|H = 0])2

4VAR(Tn)

Proof: Hammersley-Chapman-Robbins bound

(E[T+n

]− E

[T−n])2

PTn(t)

)√PTn(t)

≤ 4VAR(Tn)

)2PTn(t)

)︸︷︷︸

Vincze-Le Cam distance

∥∥TV≥ (E[Tn|H = 1]− E[Tn|H = 0])2

4VAR(Tn)

Proof:(E[T+n

]− E

[T−n])2

PTn(t)

)√PTn(t)

≤ 4VAR(Tn)

)2PTn(t)

)≤ 4VAR(Tn)

∥∥PTn|H=1 − PTn|H=0

∥∥TV

�A. Makur (MIT) Capacity of BSC Permutation Channel 5 October 2018 14 / 21

Achievability Proof

Theorem (Achievability)

For any 0 < R < 1/2, consider the binary hypothesis testing problem with

H ∼ Ber(12

), and X n

1i.i.d.∼ Ber

(q + h

)given H = h ∈ {0, 1}.

Then, limn→∞

PnML = 0. This implies that:

Cperm(BSC(p)) ≥ 1

Proof: Start with Le Cam’s relation

PnML =

∥∥TV

Achievability Proof

H ∼ Ber(12

), and X n

1i.i.d.∼ Ber

(q + h

)given H = h ∈ {0, 1}.

Then, limn→∞

Cperm(BSC(p)) ≥ 1

Proof: Apply second moment method lemma

PnML =

∥∥TV

)≤ 1

(1− (E[Tn|H = 1]− E[Tn|H = 0])2

4VAR(Tn)

Achievability Proof

H ∼ Ber(12

), and X n

1i.i.d.∼ Ber

(q + h

)given H = h ∈ {0, 1}.

Then, limn→∞

Cperm(BSC(p)) ≥ 1

Proof: After explicit computation and simplification...

PnML =

∥∥TV

)≤ 1

(1− (E[Tn|H = 1]− E[Tn|H = 0])2

4VAR(Tn)

Achievability Proof

H ∼ Ber(12

), and X n

1i.i.d.∼ Ber

(q + h

)given H = h ∈ {0, 1}.

Then, limn→∞

Cperm(BSC(p)) ≥ 1

Proof: For any 0 < R < 12 ,

PnML =

∥∥TV

)≤ 1

(1− (E[Tn|H = 1]− E[Tn|H = 0])2

4VAR(Tn)

)≤ 3

2n1−2R

Achievability Proof

H ∼ Ber(12

), and X n

1i.i.d.∼ Ber

(q + h

)given H = h ∈ {0, 1}.

Then, limn→∞

PnML = 0.

This implies that:

Cperm(BSC(p)) ≥ 1

PnML =

∥∥TV

)≤ 1

(1− (E[Tn|H = 1]− E[Tn|H = 0])2

4VAR(Tn)

)≤ 3

2n1−2R→ 0 as n→∞

Achievability Proof

H ∼ Ber(12

), and X n

1i.i.d.∼ Ber

(q + h

)given H = h ∈ {0, 1}.

Then, limn→∞

Cperm(BSC(p)) ≥ 1

PnML =

∥∥TV

)≤ 1

(1− (E[Tn|H = 1]− E[Tn|H = 0])2

4VAR(Tn)

)≤ 3

2n1−2R→ 0 as n→∞

Outline

1 Introduction

2 Achievability

3 ConverseFano’s Inequality ArgumentCLT Approximation

4 Conclusion

Converse: Fano’s Inequality Argument

Consider the Markov chain M → X n1 → Zn

1 → Y n1 → Sn ,

∑ni=1 Yi ,

and a sequence of encoder-decoder pairs {(fn, gn)}n∈N such that|M| = nR and lim

n→∞Pnerror = 0

Standard argument, cf. [Cover-Thomas 2006]:

R log(n)

= H(M|M) + I (M; M)

≤ 1 + PnerrorR log(n) + I (M;Y n

= 1 + PnerrorR log(n) + I (M;Sn)

≤ 1 + PnerrorR log(n) + I (X n

1 ;Sn)

Divide by log(n)

and let n→∞:

R ≤ I (X n1 ;Sn)

log(n)

1 → Y n1 → Sn ,

∑ni=1 Yi ,

n→∞Pnerror = 0

Standard argument, cf. [Cover-Thomas 2006]: M is uniform

R log(n) = H(M)

= H(M|M) + I (M; M)

1 ;Sn)

Divide by log(n)

and let n→∞:

R ≤ I (X n1 ;Sn)

log(n)

1 → Y n1 → Sn ,

∑ni=1 Yi ,

n→∞Pnerror = 0

Standard argument, cf. [Cover-Thomas 2006]: Fano’s inequality, DPI

R log(n) = H(M|M) + I (M; M)

1 ;Sn)

Divide by log(n)

and let n→∞:

R ≤ I (X n1 ;Sn)

log(n)

1 → Y n1 → Sn ,

∑ni=1 Yi ,

n→∞Pnerror = 0

Standard argument, cf. [Cover-Thomas 2006]: sufficiency

R log(n) = H(M|M) + I (M; M)

= 1 + PnerrorR log(n) + I (M; Sn)

1 ;Sn)

Divide by log(n)

and let n→∞:

R ≤ I (X n1 ;Sn)

log(n)

1 → Y n1 → Sn ,

∑ni=1 Yi ,

n→∞Pnerror = 0

Standard argument, cf. [Cover-Thomas 2006]: DPI

R log(n) = H(M|M) + I (M; M)

1 ;Sn)

Divide by log(n)

and let n→∞:

R ≤ I (X n1 ;Sn)

log(n)

1 → Y n1 → Sn ,

∑ni=1 Yi ,

n→∞Pnerror = 0

R log(n) = H(M|M) + I (M; M)

1 ;Sn)

Divide by log(n)

and let n→∞:

R ≤ 1

log(n)+ Pn

errorR +I (X n

1 ;Sn)

log(n)

1 → Y n1 → Sn ,

∑ni=1 Yi ,

n→∞Pnerror = 0

R log(n) = H(M|M) + I (M; M)

1 ;Sn)

Divide by log(n) and let n→∞:

R ≤ limn→∞

I (X n1 ; Sn)

log(n)

Converse: CLT Approximation

Upper bound on I (X n1 ;Sn):

I (X n1 ;Sn) = H(Sn)− H(Sn|X n

≤ log(n + 1)−∑

xn1∈{0,1}nPX n

1(xn1 )H(bin(k , 1− p) + bin(n − k , p))

≤ log(n + 1)−∑

xn1∈{0,1}nPX n

1(xn1 )H

(bin(n

2, p))

= log(n + 1)− 1

2log(πep(1− p)n) + O

)Hence, we have:

R ≤ limn→∞

I (X n1 ;Sn)

log(n)=

Theorem (Converse)

Cperm(BSC(p)) ≤ 1

Since Sn ∈ {0, . . . , n},I (X n

1 ;Sn) = H(Sn)− H(Sn|X n1 )

≤ log(n + 1)−∑

xn1∈{0,1}nPX n

1(xn1 )H(Sn|X n

1 = xn1 )

≤ log(n + 1)−∑

xn1∈{0,1}nPX n

1(xn1 )H(bin(k , 1− p) + bin(n − k , p))

≤ log(n + 1)−∑

xn1∈{0,1}nPX n

1(xn1 )H

(bin(n

2, p))

= log(n + 1)− 1

)Hence, we have:

R ≤ limn→∞

I (X n1 ;Sn)

log(n)=

Theorem (Converse)

Cperm(BSC(p)) ≤ 1

Given X n1 = xn1 with

∑ni=1 xi = k , Sn = bin(k , 1− p) + bin(n − k , p):

≤ log(n + 1)−∑

xn1∈{0,1}nPX n

1(xn1 )H(bin(k , 1− p) + bin(n − k , p))

≤ log(n + 1)−∑

xn1∈{0,1}nPX n

1(xn1 )H

(bin(n

2, p))

= log(n + 1)− 1

)Hence, we have:

R ≤ limn→∞

I (X n1 ;Sn)

log(n)=

Theorem (Converse)

Cperm(BSC(p)) ≤ 1

Using Problem 2.14 in [Cover-Thomas 2006],

≤ log(n + 1)−∑

xn1∈{0,1}nPX n

1(xn1 )H(bin(k , 1− p) + bin(n − k , p))

≤ log(n + 1)−∑

xn1∈{0,1}nPX n

1(xn1 )H

(bin(n

2, p))

= log(n + 1)− 1

)Hence, we have:

R ≤ limn→∞

I (X n1 ;Sn)

log(n)=

Theorem (Converse)

Cperm(BSC(p)) ≤ 1

Approximate binomial entropy using CLT, cf. [Adell-Lekuona-Yu 2010]:

≤ log(n + 1)−∑

xn1∈{0,1}nPX n

1(xn1 )H(bin(k , 1− p) + bin(n − k , p))

≤ log(n + 1)−∑

xn1∈{0,1}nPX n

1(xn1 )H

(bin(n

2, p))

= log(n + 1)−∑

xn1∈{0,1}nPX n

1(xn1 )

= log(n + 1)− 1

)Hence, we have:

R ≤ limn→∞

I (X n1 ;Sn)

log(n)=

Theorem (Converse)

Cperm(BSC(p)) ≤ 1

≤ log(n + 1)−∑

xn1∈{0,1}nPX n

1(xn1 )H(bin(k , 1− p) + bin(n − k , p))

≤ log(n + 1)−∑

xn1∈{0,1}nPX n

1(xn1 )H

(bin(n

2, p))

= log(n + 1)− 1

Hence, we have:

R ≤ limn→∞

I (X n1 ;Sn)

log(n)=

Theorem (Converse)

Cperm(BSC(p)) ≤ 1

≤ log(n + 1)−∑

xn1∈{0,1}nPX n

1(xn1 )H(bin(k , 1− p) + bin(n − k , p))

≤ log(n + 1)−∑

xn1∈{0,1}nPX n

1(xn1 )H

(bin(n

2, p))

= log(n + 1)− 1

)Hence, we have:

R ≤ limn→∞

I (X n1 ; Sn)

log(n)=

Theorem (Converse)

Cperm(BSC(p)) ≤ 1

≤ log(n + 1)−∑

xn1∈{0,1}nPX n

1(xn1 )H(bin(k , 1− p) + bin(n − k , p))

≤ log(n + 1)−∑

xn1∈{0,1}nPX n

1(xn1 )H

(bin(n

2, p))

= log(n + 1)− 1

)Hence, we have:

R ≤ limn→∞

I (X n1 ; Sn)

log(n)=

Theorem (Converse)

Cperm(BSC(p)) ≤ 1

Outline

1 Introduction

2 Achievability

3 Converse

4 Conclusion

Conclusion

Theorem (Pemutation Channel Capacity of BSC)

Cperm(BSC(p)) =

1, for p = 0, 112 , for p ∈

(0, 12)∪(12 , 1)

0, for p = 12

𝐶perm BSC 𝑝

Remarks:

Cperm(·) is discontinuousand non-convex

Cperm(·) is generally agnosticto parameters of channel

Computationally tractablecoding scheme in proof

Proof technique yields moregeneral results

Conclusion

Cperm(BSC(p)) =

1, for p = 0, 112 , for p ∈

(0, 12)∪(12 , 1)

0, for p = 12

𝐶perm BSC 𝑝

Remarks:

Conclusion

Cperm(BSC(p)) =

1, for p = 0, 112 , for p ∈

(0, 12)∪(12 , 1)

0, for p = 12

𝐶perm BSC 𝑝

Remarks:

Conclusion

Cperm(BSC(p)) =

1, for p = 0, 112 , for p ∈

(0, 12)∪(12 , 1)

0, for p = 12

𝐶perm BSC 𝑝

Remarks:

Conclusion

Cperm(BSC(p)) =

1, for p = 0, 112 , for p ∈

(0, 12)∪(12 , 1)

0, for p = 12

𝐶perm BSC 𝑝

Remarks:

Thank You!

Information Capacity of the BSC Permutation Channel · 2021. 5. 27. · Information Capacity of the...

Documents

Permutation Groups

Permutation Groups:

PERMUTATION & COMBINATION - nucleoniitjeekota.com

Permutation Tests Hal Whitehead BIOL4062/5062. Introduction to permutation tests Exact and randomized permutation tests Permutation tests using standard

Permutation and combinattions

Permutation Patterns

Permutation Groups - Christian Brothers Universityfacstaff.cbu.edu/~wschrein/media/M402 Notes/M402C5.pdf · Permutation Groups Definition (Permutation of A, Permutation Group of A)

Circular permutation

Stack Sortable Permutation

Permutation Methods - win.tue.nl

NEIU Permutation Algebra

Permutation Generation Methods

Permutation & COmbination

Combinators and Permutation

§3.6 Permutation Groups

PERMUTATION CIRCUITS

Permutation graphsandapplications

ISMAA Permutation Algebra

Permutation Groups - Christian Brothers Universityfacstaff.cbu.edu/wschrein/media/M402 Notes/M402C5.pdf · CHAPTER 5 Permutation Groups Definition (Permutation of A, Permutation Group

Permutation and Combination