The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,

$Page 1: The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,$
Cover’s Open Problem:“The Capacity of the Relay Channel”

Ayfer Ozgur

Stanford University

Advanced Networks Colloquia SeriesUniversity of Maryland, March 2017

Joint work with Xiugang Wu and Leighton Pate Barnes.

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 1 / 26

Father of the Information Age

Claude Shannon (1916-2001)


The Bell System Technical JournalVol. XXVII J Illy, 1948 No.3

A Mathematical Theory of CommunicationBy c. E. SHANNON

IXTRODUCTION

T HE recent development of various methods of modulation such as reMand PPM which exchange bandwidth for signal-to-noise ratio has in-

tensified the interest in a general theory of communication. A basis forsuch a theory is contained in the important papers of Nyquist! and Hartley"on this subject. In the present paper we will extend the theory to include anumber of new factors, in particular the effect of noise in the channel, andthe savings possible due to the sta tistiral structure of the original messageand due to the nature of the final destination of the information.

The fundamental problem of communication is that of reproducing atone point either exactly or approximately a message selected at anotherpoint. Frequently the messages have meaning; that is they refer to or arecorrelated according to some system with certain physical or conceptualentities. These semantic aspects of communication are irrelevant to theengineering problem. The significant aspect is that the actual message isone selected from a set of possible messages. The system must be designedto operate for each possible selection, not just the one which will actuallybe chosen since this is unknown at the time of design.

If the number of messages in the set is finite then this number or anymonotonic function of this number can be regarded as a measure of the in-formation produced when one message is chosen from the set, all choicesbeing equally likely. As was pointed out by Hartley the most naturalchoice is the logarithmic function. Although this definition must be gen-eralized considerably when we consider the influence of the statistics of themessage and when we have a continuous range of messages, we will in allcases use an essentially logarithmic measure.

The logarithmic measure is more convenient for various reasons:1. It is practically more useful. Parameters of engineering importance

1 Nyquist, H., "Certain Factors Affecting Telegraph Speed," Belt System Tectmical J OUT-

nal, April 1924, p, 324; "Certain Topics in Telegraph Transmission Theory," A. I. E. E.TI aIlS., v. 47, April 1928, p. 617.

2 Hartley. R. V. L.. "Transmission oi Information.' Belt System Technical Journal, July1928, p. .'US.

379


“A method is developed for representing any communication systemgeometrically...”


AWGN Channel

Transmitter Receiver

P

Capacity

C = log

(1 +

P

N

)


Converse: Sphere Packing

pnN

pn(P + N)

pnN

pnN

Y sphere

0

Xn(1)

Xn(2)

Xn(3)

noise sphere

# of X n ≤

∣∣∣Sphere(√

n(P + N))∣∣∣

∣∣∣Sphere(√

nN)∣∣∣

.=

2n2log 2πe(P+N)

2n2log 2πeN

= 2n2log(1+ P

N )


Achievability: Geometric Random Coding

Pr(∃ false X n) ≤ |Lens|∣∣∣Sphere(√

nP)∣∣∣× 2nR

.=

2n2log 2πe PN

P+N

2n2log 2πeP

× 2nR

= 2−n2log(1+ P

N ) × 2nR


The story goes...


Cover’s Open Problem


Gaussian case

In = f(Zn)

Source

Relay

Destination

W1 ⇠ N (0, N)

W2 ⇠ N (0, N)

Xn Y n

Zn

C0

C (∞) =1

2log

(1 +

2P

N

)

Achievability: C ∗0 =∞.

Cutset-bound (Cover and El Gamal’79):

C ∗0 ≥1

2log

(1 +

2P

N

)− 1

2log

(1 +

P

N

).

Potentially, C ∗0 → 0 as P/N → 0.


Main Result

In = f(Zn)

Source

Relay

Destination

W1 ⇠ N (0, N)

W2 ⇠ N (0, N)

Xn Y n

Zn

C0

Theorem

C ∗0 =∞


Upper Bound on the Capacity

0 0.1 0.2 0.3 0.7 0.8 0.9 12.5

2.6

2.7

2.8

2.9

3

3.1SNR = 15 dB

Cut-set boundC-FOld boundNew bound

0.4 0.5 0.6 C0 (bit/channel use)


Cutset Bound

In = f(Zn)

Source

Relay

Destination

W1 ⇠ N (0, N)

W2 ⇠ N (0, N)

Xn Y n

Zn

C0

nR ≤ I (X n;Y n, In) + nεn

= I (X n;Y n) + I (X n; In|Y n) + nεn

= I (X n;Y n) + H(In|Y n)︸︷︷︸≤nC0

−H(In|Y n,X n)︸︷︷︸≥0

+nεn

≤ n(I (X ;Y ) + C0 + εn)


Cutset Bound

In = f(Zn)

Source

Relay

Destination

W1 ⇠ N (0, N)

W2 ⇠ N (0, N)

Xn Y n

Zn

C0

nR ≤ I (X n;Y n, In) + nεn

= I (X n;Y n) + I (X n; In|Y n) + nεn

= I (X n;Y n) + H(In|Y n)︸︷︷︸≤nC0

−H(In|X n)︸︷︷︸≥0

+nεn

≤ n(I (X ;Y ) + C0 + εn)


pnN

Xn

Typical set of Zn/Y n

✓ In-th bin

Zn

Y n

If H(In|X n) = 0, then H(In|Y n) = 0.


pnN

Xn


✓ In-th bin

Zn

Y n

If H(In|X n) = 0,

then H(In|Y n) = 0.


pnN

Xn


✓ In-th bin

Zn

Y n

If H(In|X n) = 0, then H(In|Y n) = 0.


In = f(Zn)

Source

Relay

Destination

W1 ⇠ N (0, N)

W2 ⇠ N (0, N)

Xn Y n

Zn

C0

R ≤ I (X n;Y n) + H(In|Y n)︸︷︷︸≤?

− H(In|X n)︸︷︷︸=−n log sin θn

+nεn

Goal:

In = f (Zn)− Zn − X n

︸︷︷︸H(In|X n)=−n log sin θn

−Y n

︸︷︷︸H(In|Y n)≤?


H(In|X n) 6= 0

Multiple bins

Xn

pnN

Typical set of Zn

# of bins =? P(each bin) =?


From n- to nB- Dimensional Space

X,Y,Z, I : B-length i.i.d. from {(X n(b),Y n(b),Zn(b), In(b))}Bb=1.

If H(In|X n) = −n log sin θn, then for any typical (x, i)

p(i|x).

= 2nB log sin θn ,

P(Z ∈ A(i)|x).

= 2nB log sin θn

pnBN

x

Typical set of Z/Y

Pr.= 2nB log sin ✓n

ith bin Ax(i)

|Ax(i)| .= 2nB( 1

2 log 2⇡eN sin2 ✓n)


Isoperimetric Inequalities

Isoperimetric Inequality in the Plane (Steiner 1838)

Among all closed curves in the plane with a given enclosed area, thecircle has the smallest perimeter.


Isoperimetric Inequalities

Isoperimetric Inequality on the Sphere (Levy 1919)

Among all sets on the sphere with a given volume, the spherical caphas the smallest boundary, or the smallest volume of ω-neighborhoodfor any ω > 0.


Blowing-up Lemma

|Ax(i)| .= 2nB( 1

2 log 2⇡eN sin2 ✓n)

Isoperimetric Inequality on the Sphere + Measure Concentration:

P(Z ∈ blow-up of Ax(i)|x) ≈ 1.

⇓P(Y ∈ blow-up of Ax(i)|x) ≈ 1.


Geometry of Typical Sets

n-dimensional space:Almost all (X n,Y n,Zn, In)

pnN

XnY n

Zn ! In

nB-dimensional space:Almost all (x, y, i)

pnBN

z ! i

xy

⇡

2� ✓n

Information Inequality: (Wu and Ozgur, 2015)

H(In|Y n) ≤ n(2 log sin θn +√

2 log sin θn ln 2 log e).


A new approach

Ax(i)

Y

Ax(i)

Y

Control the intersection of a sphere drawn around a randomly chosen Yand Ax(i).


Easy if Ax(i) is a spherical cap

pnBN

z0

y0x

✓n

Cap(z0, ✓n)

Cap(y0,!n)

Cap(z0, ✓n) \ Cap(y0,!n)

!n

|Cap(z0, θn) ∩ Cap(Y, ωn)| .= 2nB( 12log 2πeN(sin2θn−cos2 ωn))


Strengthening of the Isoperimetric Inequality

Strengthening of the Isoperimetric Inequality:

Among all sets on the sphere with a given volume, the spherical cap hasminimal intersection volume at distance ω for almost all points on the spherefor any ω > π/2− θ.

Proof: builds on the Riesz rearrangement inequality.


A Packing Argument

Given Y,# of I ≤

∣∣∣Sphere(Y,√nBN4sin2 ωn

2

)∣∣∣

2nB( 12log 2πeN(sin2θn−cos2 ωn))


Summary

Solved an problem posed by Cover and named “The Capacity ofRelay Channel” in Open Problems in Communication andComputation, Springer-Verlag, 1987.

Developed a converse technique that significantly deviates fromstandard converse techniques based on single-letterization and hassome new ingredients:

I TypicalityI Measure ConcentrationI Isoperimetric Inequality


Documents

The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and named \The Capacity of Relay Channel" in Open Problems in Communication and Computation,