Broadcast Channels - Quantumquantum.phys.cmu.edu/QIP/broadcast_channels.pdf · Introduce channels: correlations between two random variables. Source symbols from some ﬁnite alphabet

Broadcast Channels

Vlad Gheorghiu

Department of PhysicsCarnegie Mellon University

Pittsburgh, PA 15213, U.S.A.

1 October 2009

Vlad Gheorghiu (CMU) Broadcast Channels 1 October 2009 1 / 20

Outline

1 References

2 Brief introduction to classical information theoryClassical ChannelsCodes and The Fundamental Channel Coding Theorem

3 Broadcast channelsThe problemMinimax schemesTime-sharing schemesCapacity region diagramsSuperimposing of information: Switch-and-talk channelsSuperimposing of information: BSC channels

4 Extension to the quantum regimeMocking a classical channel in the quantum regimeQuantum incompatibility

This talk is available online at http://quantum.phys.cmu.edu/QIP


References

1 Thomas M. Cover and Joy A. Thomas, Elements of InformationTheory, Wiley-Interscience (2005)

2 Thomas M. Cover, Broadcast channels, IEEE Trans. Inf. Th., vol.18, no. 1, Jan. (1972)


Brief introduction to classical information theory Classical Channels

Classical Channels

What is information?



Classical Channels


Information is physical.



Classical Channels



Correlations between two physical systems.



Classical Channels




Letters in a newspapers get correlated to your visual receptors.



Classical Channels





Can one easily describe information from a mathematical point ofview?



Classical Channels





Can one easily describe information from a mathematical point ofview?

If you are Shannon, then yes.



Introduce channels: correlations between two random variables.




Source symbols from some finite alphabet are mapped into somesequence of channel symbols, which then produces the outputsequence of the channel.





Each of the possible input sequences induces a probability distributionon the output sequence.






Information theory is a probabilistic theory!







The goal is to reconstruct the input from the output with a negligibleprobability of error.







The goal is to reconstruct the input from the output with a negligibleprobability of error.

The maximum rate at which this can be done is called the capacity ofthe channel.



Discrete channels

A discrete channel is a system consisting of an input alphabet X andoutput alphabet Y and a probability transition matrix p(y |x) thatexpresses the probability of observing the output symbol y given that wesend the symbol x .



Discrete channels


The channel is said to be memoryless if the probability distribution ofthe output depends only on the input at that time and is conditionalindependent of previous channel inputs or outputs.



Discrete channels



Usual notation: (X , p(y |x),Y).



Discrete channels



Usual notation: (X , p(y |x),Y).

Channel capacity of a discrete memoryless channel (Shannon)

The channel capacity is given by

C = maxp(x)I (X ;Y ),

where the maximum is taken over all possible input distributions p(x) andI (X ;Y ) is the mutual information between the random variables X and Y .



Examples of channel capacities

Example 1: Noiseless Binary Channel.




Example 1: Noiseless Binary Channel. C = 1 bit.





Example 2: Noisy Channel with Nonoverlapping Outputs.





Example 2: Noisy Channel with Nonoverlapping Outputs. C = 1 bit.






Binary Symmetric Channel (BSC).






Binary Symmetric Channel (BSC). C = 1 − H(p) bits.


Brief introduction to classical information theory Codes and The Fundamental Channel Coding Theorem

Codes

Can one saturate (at least in theory) the channel capacity?



Codes


Yes, always, provided one is smart enough.



Codes



First, must introduce codes (a fancy name for redundancy).



Codes




Codes

An (M, n) code for a discrete memoryless channel (X , p(y |x),Y) consistsof the following:

1 An index set {1, 2, . . . , M}.



Codes




Codes


1 An index set {1, 2, . . . , M}.

2 An encoding function X n : {1, 2, . . . , M} → X n, yielding codewordsxn(1), xn(2), . . . , xn(M). The set of codewords is called the codebook.



Codes




Codes


1 An index set {1, 2, . . . , M}.


3 A decoding function g : Yn → {1, 2, . . . , M}, which is a deterministic rule

that assigns a guess to each possible received vector.



Codes




Codes


1 An index set {1, 2, . . . , M}.




The repetition code: 0 → 000, 1 → 111.



Codes




Codes


1 An index set {1, 2, . . . , M}.




The repetition code: 0 → 000, 1 → 111. (M = 21,N = 3) code.



The Fundamental Channel Coding Theorem

Channel Coding Theorem (Shannon)

For a discrete memoryless channel, all rates below capacity C areachievable. Specifically, for every rate R < C , there exists a sequence of(2nR , n) codes with negligible maximum probability of error. Conversely,any sequence (2nR , n) codes with negligible probability of error must haveR ≤ C .






Although the existence of good codes is proved, there is (yet) nomethod for constructing them for arbitrary channels. Mainly becausethe existence proof is based on the idea of random coding!






Although the existence of good codes is proved, there is (yet) nomethod for constructing them for arbitrary channels. Mainly becausethe existence proof is based on the idea of random coding!

This is one of the main research topics in Classical InformationTheory.


Broadcast channels The problem

Broadcast channels: Basic problem

One source, k > 1 receivers.





The goal is to send information with negligible probability of error toall receivers.






More formally, one wants to find the set of simultaneously achievablerates (R1,R2, . . . Rk), or the capacity region.







Why is it interesting? One can easily guess...








Broadcasting TV information, giving a lecture to a group of disparatebackgrounds and aptitudes etc.








Broadcasting TV information, giving a lecture to a group of disparatebackgrounds and aptitudes etc.

The broadcast problem belongs to the still open research area ofNetwork Information Theory.


Broadcast channels Minimax schemes

Naive approach: Minimax schemes

Suppose the transmission channels to the receiver have respectivechannel capacities C1,C2, . . . ,Ck bits per second.





Send at rate Cmin =min(C1,C2, . . . ,Ck).






Even this is only possible when the channels are “compatible”(un-correlated, or orthogonal).







The transmission rate is limited by the worst channel.








At the other extreme, one may try to send at rate R = Cmax, withresulting rates Ri = 0 for all but the best channel (say the k-th one),and Rk = Cmax for the best channel.








At the other extreme, one may try to send at rate R = Cmax, withresulting rates Ri = 0 for all but the best channel (say the k-th one),and Rk = Cmax for the best channel.

Definitely these schemes do not look optimal.


Broadcast channels Time-sharing schemes

Less naive approach: Time-sharing schemes

Next ideea: time sharing.





Allocate portions of time λ1, λ2, . . . , λk , λi ≥ 0,∑

λi = 1, to sendingat rates C1,C2, . . . ,Ck .







Assuming compatibility of the channels and assumingC1 ≤ C2 ≤ · · · ≤ Ck , one finds that the rate of transmission throughthe i -th channel is

Ri =∑

j≤i

λjCj , i = 1, 2, . . . , k.







Assuming compatibility of the channels and assumingC1 ≤ C2 ≤ · · · ≤ Ck , one finds that the rate of transmission throughthe i -th channel is

Ri =∑

j≤i

λjCj , i = 1, 2, . . . , k.


Broadcast channels Capacity region diagrams

Examples of capacity region (or rate trade-off) diagrams

Rate trade-off examples for 1 → 2 broadcast channels, withX = {1, 2, 3, 4}, Y1 = {1, 2}, Y2 = {1, 2}.





Orthogonal channels: P1 =

1 01 00 10 1

, P2 =

1 00 11 00 1

.






1 01 00 10 1

, P2 =

1 00 11 00 1

.

Completely incompatible channels: P1 =

1 00 112

12

12

12

, P2 =

12

12

12

12

1 00 1

.






1 01 00 10 1

, P2 =

1 00 11 00 1

.


1 00 112

12

12

12

, P2 =

12

12

12

12

1 00 1

.

The Switch-and-talk channel. Analogy with a speaker fluent inSpanish and English must speak simultaneously to two listeners, oneof whom understands only Spanish and the other only English.






1 01 00 10 1

, P2 =

1 00 11 00 1

.


1 00 112

12

12

12

, P2 =

12

12

12

12

1 00 1

.


Conjecture: the time-sharing scheme is optimal.






1 01 00 10 1

, P2 =

1 00 11 00 1

.


1 00 112

12

12

12

, P2 =

12

12

12

12

1 00 1

.


Conjecture: the time-sharing scheme is optimal. This conjecture isFALSE!


Broadcast channels Superimposing of information: Switch-and-talk channels

Smart approach: superimposing of information

Can one do better than time-sharing for the Switch-and-talk channel?




Can one do better than time-sharing for the Switch-and-talk channel?YES!





“Super-impose” the information.





“Super-impose” the information. Make use of the fact that theEnglish receiver does not understand Spanish and vice-versa but bothrealize when the sender does not broadcast in their language, to sendextra information, common to both parties!






If channel 1 is used in proportion α of the time, then αC1

bits/transmission are received by Y1 and (1 − α)C2 by Y2.






If channel 1 is used in proportion α of the time, then αC1

bits/transmission are received by Y1 and (1 − α)C2 by Y2.

However, H(α) additional common bits/transmission may betransmitted, by choosing the ordering in which the channels are used(constrained by α).



In other words, modulation of the switch-to-talk button, subject tothe time-proportion constraint α, allows the perfect transmission of2nH(α) additional messages to both Y1 and Y2!



In other words, modulation of the switch-to-talk button, subject tothe time-proportion constraint α, allows the perfect transmission of2nH(α) additional messages to both Y1 and Y2!

Thus all rates (R1,R2) of the form

(R1,R2) = (αC1 + H(α), (1 − α)C2 + H(α))

can be achieved by choosing the subset of n transmissions devoted to

the use of channel 1 in one of the

(

n

αn

)

≈ 2nH(α) possible ways.


Broadcast channels Superimposing of information: BSC channels

Another example: Broadcast through BSC

Consider a broadcast scheme with two BSC, X = {1, 2}, Y1 = {1, 2},

Y2 = {1, 2} and P1 =

[

1 00 1

]

, P2 =

[

1 − p p

p 1 − p

]

(channel 1

is noiseless and channel 2 is a BSC with error probability p).





Y2 = {1, 2} and P1 =

[

1 00 1

]

, P2 =

[

1 − p p

p 1 − p

]

(channel 1


Naive rate trade-off diagram (on the board).





Y2 = {1, 2} and P1 =

[

1 00 1

]

, P2 =

[

1 − p p

p 1 − p

]

(channel 1



How to improve?





Y2 = {1, 2} and P1 =

[

1 00 1

]

, P2 =

[

1 − p p

p 1 − p

]

(channel 1



How to improve?

Use a code designed for a noisier channel, a cascade of 2 BSC ofparameter p and α.





Y2 = {1, 2} and P1 =

[

1 00 1

]

, P2 =

[

1 − p p

p 1 − p

]

(channel 1



How to improve?


Fewer codewords, 2nC(αp+αp), but larger tolerated noise.





Y2 = {1, 2} and P1 =

[

1 00 1

]

, P2 =

[

1 − p p

p 1 − p

]

(channel 1



How to improve?


Fewer codewords, 2nC(αp+αp), but larger tolerated noise.

Take advantage of this tolerance by packing in some extra messageinformation intended solely for the perfect receiver Y1.



With each codeword x ∈ {0, 1}n we associate the set of all codewordsat Hamming distance equal to [αn] (see the picture on the board).




This code structure allows the transmission of an arbitrary integerr ∈ {1, 2, . . . , 2n(C(αp+αp))} to both receivers 1 and 2.





In addition, an arbitrary integer

s ∈

{

1, 2, . . . ,

(

n

αn

)}

can be sent only to receiver 1.






s ∈

{

1, 2, . . . ,

(

n

αn

)}


Comments on the “cloud structure” and the decoding strategy.






s ∈

{

1, 2, . . . ,

(

n

αn

)}



Since there are

(

n

αn

)

≈ 2nH(α) points per cloud, the transmission

rate for channel 1 is

R1 = C (αp + αp) + H(α).






s ∈

{

1, 2, . . . ,

(

n

αn

)}



Since there are

(

n

αn

)

≈ 2nH(α) points per cloud, the transmission

rate for channel 1 is

R1 = C (αp + αp) + H(α).

Channel 2 has the rate of the cascaded BSC, e.g.

R2 = C (αp + αp).



This is better that the time-sharing scheme!



This is better that the time-sharing scheme!

Conclusion: superimpose high-rate information on low-rateinformation!


Extension to the quantum regime Mocking a classical channel in the quantum regime

Classical channels in the quantum regime

Introduce an isometry from an input Hilbert space Hin to amulti-partite Hilbert space HA ⊗HB ⊗ · · · ⊗ HZ . For our purposes,let’s restrict to a bipartite splitting.





Let T = {Pj} be a decomposition of the identity on the input Hilbertspace, in terms of a set of orthogonal projectors, I =

∑

Pj .






∑

Pj .

We call T a type of information.






∑

Pj .


This scenario “simulates” a classical broadcast channel, and theprojectors play the role of the input alphabet (one can regard them assymbols).






∑

Pj .



Distinguishablity of traced-down encoded projectors impliesinformation transfer.






∑

Pj .




Given the isometry and the type of information, one can easily findthe transition matrix.






∑

Pj .




Given the isometry and the type of information, one can easily findthe transition matrix.

In principle, the capacity diagram can be calculated.


Extension to the quantum regime Quantum incompatibility

Incompatible types induce different classical channels onthe same isometry

Two types of information are compatible if and only if all projectors inone commute with all projectors of the other one.





Otherwise, they are called incompatible.






Example: σX and σZ Pauli spin operators define two incompatibletypes through their spectral decomposition, i.e. {|+〉〈+|, |−〉〈−|} isincompatible with respect to {|0〉〈0|, |1〉〈1|}.







Main question: given some isometry and the capacity diagram forsome type of information T , what are the restrictions (if any) imposedon the capacity diagram for some other incompatible type Q?








The notion of incompatibility is quantum-mechanical. There is noclassical analog!









This is a hard question, for which we do not have (yet) an answer.









This is a hard question, for which we do not have (yet) an answer.

An answer may be of use in the study of quantum channel capacities.


Documents

Broadcast Channels - Quantumquantum.phys.cmu.edu/QIP/broadcast_channels.pdf · Introduce channels: correlations between two random variables. Source symbols from some ﬁnite alphabet