Lecture Multiple access channels

(Reading: NIT ., ., ., .)

∙ Discrete memoryless multiple access channel∙ Simple bounds on the capacity region∙ Time sharing∙ Capacity region∙ Gaussian multiple access channel∙ Extension to more than two senders

© Copyright – Abbas El Gamal and Young-Han Kim

Multiple access communication system

M

M

Xn

Xn

Encoder

Encoder

Decoderp(y|x , x) Yn M , M

∙ DMmultiple access channel (MAC) (X × X , p(y|x , x),Y)∙ A (nR , nR , n) code:é Message sets: [ : nR ] and [ : nR ]é Encoder j = , : xnj (mj)é Decoder: (m(yn), m(yn))∙ (M ,M) ∼ Unif([ : nR ] × [ : nR ]): xn (M) and xn (M) independent∙ Average probability of error: P(n)e = P{(M , M) = (M ,M)}∙ (R , R) achievable if ∃ (nR , nR , n) codes such that limn→∞ P(n)

e = ∙ Capacity region C : Closure of the set of achievable rate pairs (R , R) /

Simple bounds on the capacity region

C

C

C C

R

R

Time-divisioninner bound

Outer bound?

∙ Maximum achievable individual rates:

C = maxx , p(x) I(X ; Y |X = x), C = max

x , p(x) I(X ; Y |X = x)∙ Upper bound on the sum-rate:

R + R ≤ C = maxp(x)p(x) I(X ,X ; Y)

/

Examples∙ Binary multiplier MAC: X ,X ∈ {, }, Y = X ⋅ X ∈ {, }X

X

Y

R

R∙ Binary erasure MAC: X ,X ∈ {, }, Y = X + X ∈ {, , }X

X

Y

//

R

R

/

Time sharing

Proposition .

If (R� , R

�), (R��

, R�� ) ∈ C , then (R , R) = (αR�

+ αR�� , αR

� + αR��

) ∈ C for α ∈ [, ]∙ The capacity region C is convex

R

R

(R� , R

�)(R��

, R�� )(R , R)

/

Time sharing

Proposition .

If (R� , R

�), (R��

, R�� ) ∈ C , then (R , R) = (αR�

+ αR�� , αR

� + αR��

) ∈ C for α ∈ [, ]∙ The capacity region C is convex∙ Proof: Time sharing argumenté Let C�k be a sequence of (kR� , kR� , k) codes with P(k)

e → é Let C��k be a sequence of (kR�� , kR�� , k) codes with P(k)

e → é Construct a new sequence of codes by using C�αn for i ∈ [ : αn] and C��

αn for i ∈ [αn+ : n]αnαn

C�αn C��

αné By the union of events bound, P(n)e ≤ P(αn)

e + P(αn)e → ∙ Remarks:é Time division is a special case of time sharing (between (R , ) and (, R))é The capacity region of any (synchronous) channel is convex

/

R(X,X) region∙ For (X ,X) ∼ p(x)p(x), let R(X ,X) be the set of (R , R) such that

R ≤ I(X ; Y |X),R ≤ I(X ; Y |X),

R + R ≤ I(X ,X ; Y)

I(X ; Y|X)I(X ; Y|X)

I(X ; Y)I(X ; Y)R

R

∙ This region is always a pentagon since

max{I(X ; Y |X), I(X ; Y |X)} ≤ I(X ,X ; Y) ≤ I(X ; Y |X) + I(X ; Y |X) /

Capacity region (Ahlswede , Liao )

Theorem ..C is the convex closure of the set⋃p(x)p(x) R(X ,X)

C

C

C

C

R

R

∙ For binary erasure MAC example, outer bound is tight

/

Proof of achievability∙ We show that R(X ,X) is achievable for every (X ,X) ∼ p(x)p(x)∙ The rest of the proof follows by time sharing∙ Codebook generation:é Independently generate nR sequences xn (m) ∼ ∏ni= pX

(xi),m ∈ [ : nR ]é Independently generate nR sequences xn (m) ∼ ∏ni= pX

(xi),m ∈ [ : nR ]∙ Encoding:é To send messagem , encoder transmits xn (m)é To send messagem , encoder transmits xn (m)∙ Decoding:é Successive cancellation decodingé Simultaneous decoding

/

Successive cancellation decoding

I(X ; Y|X)I(X ; Y|X)

I(X ; Y)I(X ; Y)R

R

∙ Find unique m such that (xn (m), yn) ∈ T (n)є∙ If such m is found, find unique m such that (xn (m), xn (m), yn) ∈ T (n)

є

/

“Little” packing lemma∙ Let (X, Y) ∼ p(x, y)∙ Let Yn ∼ ∏ni= pY(yi)∙ Let Xn(m) ∼ ∏n

i= pX(xi),m ∈ A, |A| ≤ nR , be pairwise independent of Yn

“Little” packing lemmaThere exists δ(є) → as є → such that

limn→∞ P�(Xn(m), Yn) ∈ T (n)

є for somem ∈ A} = , if R < I(X; Y) − δ(є)Xn()Xn(m)

X n Yn T (n)є (Y)

Yn

/

Analysis of the probability of error∙ Consider P(E) conditioned on (M ,M) = (, )∙ Error events:

E = �(Xn (),Xn

(), Yn) ∉ T (n)є �,

E = �(Xn (m), Yn) ∈ T (n)

є for somem = �,E = �(Xn

(),Xn (m), Yn) ∈ T (n)

є for somem = �Thus, by the union of events bound

P(E) ≤ P(E) + P(E) + P(E)∙ By the LLN, P(E) → ∙ By the “little” packing lemma (|A| = nR − , X ← X),

P(E) → if R < I(X ; Y) − δ(є)∙ By the “little” packing lemma (|A| = nR − , X ← X , Y ← (X , Y)),P(E) → if R < I(X ; Y ,X) − δ(є) = I(X ; Y|X) −δ(є)

/

Simultaneous decoding

I(X ; Y|X)I(X ; Y|X)

I(X ; Y)I(X ; Y)R

R

∙ Find unique (m , m) such that (xn (m), xn (m), yn) ∈ T (n)є

/

Analysis of the probability of error∙ Consider P(E) conditioned on (M ,M) = (, )m m Joint pmf

p(xn )p(xn )p(yn|xn , xn )∗ p(xn )p(xn )p(yn|xn ) ∗ p(xn )p(xn )p(yn|xn )∗ ∗ p(xn )p(xn )p(yn)∙ Error events:

E = �(Xn (),Xn

(), Yn) ∉ T (n)є �,

E = �(Xn (m),Xn

(), Yn) ∈ T (n)є for somem = �,

E = �(Xn (),Xn

(m), Yn) ∈ T (n)є for somem = �,

E = �(Xn (m),Xn

(m), Yn) ∈ T (n)є for somem = ,m = �

/

Analysis of the probability of error∙ Error events:

E = �(Xn (),Xn

(), Yn) ∉ T (n)є �,

E = �(Xn (m),Xn

(), Yn) ∈ T (n)є for somem = �,

E = �(Xn (),Xn

(m), Yn) ∈ T (n)є for somem = �,

E = �(Xn (m),Xn

(m), Yn) ∈ T (n)є for somem = ,m = �∙ By the LLN, P(E) → ∙ By the “little” packing lemma,

P(E) → if R < I(X ; Y |X) − δ(є),P(E) → if R < I(X ; Y |X) − δ(є),P(E) → if R + R < I(X ,X ; Y) − δ(є)

/

Proof of the converse (Slepian–Wolf )∙ Show that: For any sequence of (nR , nR , n) codes with P(n)e → , (R , R) ∈ C∙ Code induces empirical pmf(M ,M ,X

n ,X

n , Y

n) ∼ −n(R+R)p(xn |m)p(xn |m) nIi= pY|X ,X

(yi |xi , xi)∙ By Fano’s inequality,

H(M ,M |Yn) ≤ n(R + R)P(n)e + = nєn∙ Bounds on rates:

R ≤ n

nHi= I(Xi ; Yi |Xi) + єn ,

R ≤ n

nHi= I(Xi ; Yi |Xi) + єn ,

R + R ≤ n

nHi= I(Xi ,Xi ; Yi) + єn

/

Proof of the converse∙ Time-sharing random variable: Q ∼ Unif[ : n] be independent of (Xn ,X

n , Y

n)∙ Define X = XQ , X = XQ , Y = YQ , then p(yq|xq , xq) = pY|X ,X(yq|xq , xq)∙ The bounds can be expressed as

R ≤ I(X ; Y |X ,Q) + єn ,R ≤ I(X ; Y |X ,Q) + єn ,

R + R ≤ I(X ,X ; Y |Q) + єn

for some joint pmf p(x|q)p(x|q)∙ Thus (R , R)must be in the closure of the set of (R , R) such that

R ≤ I(X ; Y |X ,Q),R ≤ I(X ; Y |X ,Q),

R + R ≤ I(X ,X ; Y |Q)for some joint pmf p(q)p(x|q)p(x|q)∙ Denote this region by C �

/

Proof of the converse∙ Clearly the capacity region C ⊆ C �∙ Can show that C � ⊆ C∙ But neither C nor C � seems computable—cardinality ofQ?∙ Can show: |Q| ≤ ; obtain the equivalent characterization of the capacity region

Theorem .The capacity region C is the set of (R , R) such that

R ≤ I(X ; Y |X ,Q),R ≤ I(X ; Y |X ,Q),

R + R ≤ I(X ,X ; Y |Q)for some pmf p(q)p(x|q)p(x|q)with |Q| ≤ ∙ Can we achieve the region in Theorem . directly?

/

Proof of achievability of the alternative characterization∙ Key idea: Coded time sharing (Han–Kobayashi )∙ Codebook generation:é Fix p(q)p(x|q)p(x|q)é Randomly generate a time-sharing sequence qn ∼ ∏ni= pQ(qi)é Conditionally independently generate xn (m) ∼ ∏n

i= pX |Q(xi|qi),m ∈ [ : nR ]é Conditionally independently generate xn (m) ∼ ∏ni= pX |Q(xi|qi),m ∈ [ : nR ]∙ Encoding:é To send messagem , encoder transmits xn (m)é To send messagem , encoder transmits xn (m)∙ Decoding:é Find the unique (m , m) such that (qn , xn (m), xn (m), yn) ∈ T (n)

є

/

Analysis of the probability of error∙ Consider P(E) conditioned on (M ,M) = (, )m m Joint pmf

p(qn)p(xn |qn)p(xn |qn)p(yn|xn , xn )∗ p(qn)p(xn |qn)p(xn |qn)p(yn|xn , qn) ∗ p(qn)p(xn |qn)p(xn |qn)p(yn|xn , qn)∗ ∗ p(qn)p(xn |qn)p(xn |qn)p(yn|qn)∙ Error events:

E = �(Qn ,Xn (),Xn

(), Yn) ∉ T (n)є �,

E = �(Qn ,Xn (m),Xn

(), Yn) ∈ T (n)є for somem = �,

E = �(Qn ,Xn (),Xn

(m), Yn) ∈ T (n)є for somem = �,

E = �(Qn ,Xn (m),Xn

(m), Yn) ∈ T (n)є for somem = ,m = �

/

Packing lemma∙ Let (U ,X, Y) ∼ p(u, x, y)∙ Let (Un , Yn) ∼ p(un , yn) be arbitrarily distributed∙ Let Xn(m) ∼ ∏ni= pX|U(xi|ui),m ∈ A, |A| ≤ nR ,

be pairwise conditionally independent of Yn given Un

Packing lemmaThere exists δ(є) → as є → such that

limn→∞ P�(Un ,Xn(m), Yn) ∈ T (n)

є for somem ∈ A} = ,

if R < I(X; Y|U) − δ(є)∙ Generalizes “little” packing lemma in two says:é Yn conditionally independent of Xn(m) given Uné (Un , Yn) can have an arbitrary pmf

/

Analysis of the probability of error∙ Error events:

E = �(Qn ,Xn (),Xn

(), Yn) ∉ T (n)є �,

E = �(Qn ,Xn (m),Xn

(), Yn) ∈ T (n)є for somem = �,

E = �(Qn ,Xn (),Xn

(m), Yn) ∈ T (n)є for somem = �,

E = �(Qn ,Xn (m),Xn

(m), Yn) ∈ T (n)є for somem = ,m = �∙ By the LLN, P(E) → ∙ By the packing lemma,

P(E) → if R < I(X ; Y |X ,Q) − δ(є),P(E) → if R < I(X ; Y |X ,Q) − δ(є),P(E) → if R + R < I(X ,X ; Y |Q) − δ(є)

/

Gaussian multiple access channel

X

X

g

g

Z

Y = gX + gX + Z∙ g , g : channel gains∙ Z ∼ N(, )∙ Average power constraints: ∑ni= xji(mj) ≤ nP,mj ∈ [ : nRj ], j = , ∙ Let Sj = gj P, j = ,

/

Capacity region (Cover , Wyner )

Theorem .The capacity region of the Gaussian MAC is the set of (R , R) such that

R ≤ C(S),R ≤ C(S),

R + R ≤ C(S + S)

C(S)C(S)

C(S/(S + ))C(S/(S + ))R

R

/

Proof sketch∙ Let X ∼ N(, P) and X ∼ N(, P) be independent∙ Then R(X ,X) is the set of (R , R) such that

R ≤ C(S),R ≤ C(S),

R + R ≤ C(S + S)∙ Thus, C = R(X ,X) and no time sharing is needed∙ Proof of achievability:é Achievability for the DM-MAC with input costs (see NIT Problem .)é Discretization procedure (NIT ..)∙ Proof of the converse: By the maximum differential entropy lemma,

h(Y |X ,Q) ≤ (/) log(πe(S + )),h(Y |X ,Q) ≤ (/) log(πe(S + )),

h(Y |Q) ≤ (/) log(πe(S + S + )) /

Comparison to point-to-point coding schemes∙ Use point-to-point Gaussian codes∙ Treating other codeword as noise:

R < C(S/(S + )),R < C(S/(S + ))∙ Time division:

R < α C(S),R < α C(S)∙ Time division with power control:

R < α C �S/α� ,R < α C �S/α�

C(S)

C(S)R

R

α = S/(S + S)

/

Successive cancellation decoding

m

m −g+ M-decoder

Encoder M-decoderyn xn (m) yn

R

R

C(S)

C(S) Successive cancellation

Successive cancellation

Time sharing

/

Extension to more than two senders

Theorem .The capacity region of the k-sender DM-MAC is the set of (R , . . . , Rk) such thatH

j∈J Rj ≤ I(X(J ); Y |X(J c),Q) for every J ⊆ [ : k]for some pmf p(q)∏k

j= p(xj|q)with |Q| ≤ k∙ For k = , the capacity region is the set of (R , R , R) such that

R ≤ I(X ; Y |X ,X ,Q),R ≤ I(X ; Y |X ,X ,Q),R ≤ I(X ; Y |X ,X ,Q),

R + R ≤ I(X ,X ; Y |X ,Q),R + R ≤ I(X ,X ; Y |X ,Q),R + R ≤ I(X ,X ; Y |X ,Q),

R + R + R ≤ I(X ,X ,X ; Y |Q)for some p(q)p(x|q)p(x|q)p(x|q)

/

Extension to more than two senders∙ The capacity region of the k-sender G-MAC is the set of (R , . . . , Rk) such thatHj∈J Rj ≤ C�H

j∈J Sj� for every J ⊆ [ : k]∙ For k = , the capacity region is the set of (R , R , R) such that

R ≤ C(S),R ≤ C(S),R ≤ C(S),

R + R ≤ C(S + S),R + R ≤ C(S + S),R + R ≤ C(S + S),

R + R + R ≤ C(S + S + S)∙ When S = S = ⋅ ⋅ ⋅ = Sk , the symmetric capacity Csym = O(log(k)/k) /

Summary∙ Discrete memoryless multiple access channel (DM-MAC)∙ Capacity region∙ Time sharing∙ Successive cancellation decoding∙ Simultaneous decoding is more powerful than successive cancellation∙ Time-sharing random variable∙ Coded time sharing is more powerful than time sharing∙ Packing lemma∙ Gaussian multiple access channelé Time division with power control achieves the sum-capacityé Capacity region achieved via ptp codes, successive cancellation decoding, time sharing

/

References

Ahlswede, R. (). Multiway communication channels. In Proc. nd Int. Symp. Inf. Theory, Tsahkadsor,Armenian SSR, pp. –.

Cover, T. M. (). Some advances in broadcast channels. In A. J. Viterbi (ed.) Advances in CommunicationSystems, vol. , pp. –. Academic Press, San Francisco.

Han, T. S. and Kobayashi, K. (). A new achievable rate region for the interference channel. IEEE Trans.Inf. Theory, (), –.

Liao, H. H. J. (). Multiple access channels. Ph.D. thesis, University of Hawaii, Honolulu, HI.

Slepian, D. and Wolf, J. K. (). A coding theorem for multiple access channels with correlated sources.Bell Syst. Tech. J., (), –.

Wyner, A. D. (). Recent results in the Shannon theory. IEEE Trans. Inf. Theory, (), –.

/