Bernd Girod. Joint Source-Network Coding for Real-time Media 1

Joint Source-Network Coding for Real-time Media

Bernd Girod

Information Systems LaboratoryStanford University

Bernd Girod. Joint Source-Network Coding for Real-time Media 2

Message from the President of the IEEE Information Theory Society

“Dear Source Coders,

Don’t compress too much and consequently requireerror-free transmission. It makes the task for us channel coders very difficult. We mighthave to add more redundancy for error protection than you removed!”

Professor Joachim Hagenauer, 1996

Bernd Girod. Joint Source-Network Coding for Real-time Media 3

Outline

Joint source-channel coding– Why it’s good idea– Why it’s bad idea

Joint source-network coding– FEC and Smart Prefetching– Why joint source-channel coding is a good idea after all– Adaptive media playout– Packet Path Diversity

Bernd Girod. Joint Source-Network Coding for Real-time Media 4

Optimal Trade-Off Source Coding vs. Channel Coding

0 10 20 30 40 5010

20

30

40

50

60

70

80

90

100

25 % 25 %

25 %

25 %

25 %

9 %

9 % 9 %

9 %

9 %

6 %

6 % 6 %

6 %

6 %

2.5 %

2.5 %2.5 %

2.5 %

2.5 %

0.7 %

0.7 %0.7 %

0.7 %

0.7 %

Intra Rate [%]

Co

de

Ra

te [%

]

Burst Length:Memoryless

2 4

8

16

32

<< less more >>Source coding redundancy

<<

mo

re

les

s >

>C

ha

nn

el c

od

ing

re

du

nd

an

cy

K. Stuhlmüller, N. Färber, M. Link, and B. Girod, "Analysis of Video Transmission over Lossy Channels," IEEE Journal on Selected Areas in Communications, vol. 18, no. 6, pp. 1012-1032, June 2000.

Bernd Girod. Joint Source-Network Coding for Real-time Media 5

Shannon and the Internet

Appl. #1

TransportLayer

InternetworkingLayer

Appl. #2

Appl. #n • • •

Internet Open Architecture: 2000+

NetworkAccess

#1

NetworkAccess

#2

NetworkAccess

#m• • •Channel

coderChannel

coder

ChanneldecoderChanneldecoder

Channel

SourcecoderSourcecoder

SourcedecoderSourcedecoder

Separation PrincipleShannon, 1948

Joint Source-Channel CodingContradicts Open Architecture? Joint Source-Channel Coding

Contradicts Open Architecture?

How to Build a System that Works, When Shannon’s Separation Principle Doesn’t? I hope I have shown you that it is beneficial to consider source and channel coding jointly, and pay particular attention to source decoding also.

How to Build a System that Works, When Shannon’s Separation Principle Doesn’t? I hope I have shown you that it is beneficial to consider source and channel coding jointly, and pay particular attention to source decoding also.

Bernd Girod. Joint Source-Network Coding for Real-time Media 6

“Joint Source-Network Coding”“Joint Source-Network Coding”

From Joint Source-Channel Coding to Joint Source-Network Coding

Internet– Many different channels with link-layer error control– Application (= source coder) shielded from and unaware of physical

layer– Application “sees” late or lost packets, not bit-errors

What can be done nevertheless– Careful network adaptation layer design (Avoid “chop and ship”)– Application specific error control (for packets erasures, not bit

errors)– Jointly design and optimize

– Source coding/decoding, transcoding & error concealment– Forward error correction, packet scheduling and retransmission,

flow control, cacheing, routing

Bernd Girod. Joint Source-Network Coding for Real-time Media 7

FEC across Packets

information symbols

redundancy symbols

3packet

bitstream

code

wor

d K

N-K

1 01 110 010 ...

block of packets

network

Reed-Solomon Codes RS(N,K) with erasure decoding Any K of N packets can used to reconstruct the information packets.

U. Horn, K. Stuhlmüller, M. Link, and B. Girod, "Robust Internet Video Transmission Based on Scalable Coding and Unequal Error Protection," Image Communication, vol. 15, no. 1-2, pp. 77-94, Sept. 1999.

Bernd Girod. Joint Source-Network Coding for Real-time Media 8

Internet Media Streaming

Workstation

PC

DSL

56K modem

Media Server

Internet

GPRS

Receiver buffer & ARQ

Today: >10 sec delay

Bernd Girod. Joint Source-Network Coding for Real-time Media 9

Media Streaming with Smart Prefetching

Basic idea:

Retransmission protocol employing prefetching of more important information

Server ClientInternetInternet

Request stream

Request stream

Rate-distortionpreamble

Rate-distortionpreamblePrefetch

times

Prefetchtimes

Media dataMedia data

Repeatrequest

RepeatrequestRepeatrequest

RepeatrequestRepeatrequest

Repeatrequest

Bernd Girod. Joint Source-Network Coding for Real-time Media 10

Smart Prefetching 2-Layer Video Results

Single Layer Scalable: Optimal Scalable: Equal Prefetch

0 0.2 0.4 0.6 0.8 10

100

200

300

400

500

600

700

Packet loss rate

MS

E d

isto

rtio

n at

dec

oder

Singlelayer Two

layers,equal

prefetchTwo

layers,optimal

prefetch

Bernd Girod. Joint Source-Network Coding for Real-time Media 11

Adaptive Media Playout

State-of-the-Art Improved scheme

Idea: reduce latency and packet loss simultaneously by continuously adapting playout deadline to network conditions

Idea: reduce latency and packet loss simultaneously by continuously adapting playout deadline to network conditions

5% packet loss2 sec average receiver buffer

5% packet loss2 sec average receiver buffer

E. Steinbach, N. Färber and B. Girod, "Adaptive Playout for Low-Latency Video Streaming," Proc. International Conference on Image Processing, ICIP-2001, Thessaloniki, Greece, October 2001.

Bernd Girod. Joint Source-Network Coding for Real-time Media 12

Adaptive Media Playout

Initialdelay

Initialdelay

Start of playoutStart of playout

Throughput

Receiver buffer level

t

t

Target level

Target rate

Reduced playout speedReduced playout speed

buffer underflowavoided

buffer underflowavoided

Bernd Girod. Joint Source-Network Coding for Real-time Media 13

Reduced Pre-roll Time for Stored Streams

Probabilityof buffer

underflow< 1%

Probabilityof buffer

underflow< 1%

M. Kalman, E. Steinbach, and B. Girod, "Adaptive Media Playout for Low Delay Video Streaming over Error-Prone Channels," IEEE Transactions on Circuits and Systems for Video Technology, Special Issue on Wireless Video, submitted August 2000.

Bernd Girod. Joint Source-Network Coding for Real-time Media 14

Packet Path Diversity

Idea: set up multiple connections along different paths

Improved congestion resiliency– lower combined latency– better loss characteristics

Realization in current Internet, e.g., through relay server

D

S

RelayRelay1 2

crosstraffic

crosstrafficMedia traffic

Bernd Girod. Joint Source-Network Coding for Real-time Media 15

Multipath Voice Streaming Experiment

Exodus Comm.

Exodus Comm.

BBN PlanetBBN Planet

Netergy Networks

192.84.16.176

MIT18.184.0.50

(5 ms) (45 ms)(5 ms)

Source

Destination

QwestQwest

Harvard140.247.62.110

(40 ms) (5 ms)

Relay

Bernd Girod. Joint Source-Network Coding for Real-time Media 16

Adaptive playout with two streams

Yi J. Liang, E. Steinbach and B. Girod, "Multi-stream Voice Transmission over the Internet Using Path Diversity," Proc. ACM Multimedia 2001, Ottawa, Canada, Sept./Oct. 2001.

Bernd Girod. Joint Source-Network Coding for Real-time Media 17

Multiple Description Coding for VoIP

EStream 1

Stream 2

O E O E O

O E O E O E

E O

packet i i+1 i+2 i+3

Even samples: 8 bit Odd samples: 4 bit

Odd samples: 8 bitEven samples: 4 bit

O E

Bernd Girod. Joint Source-Network Coding for Real-time Media 18

Packet Path Diversity: Loss vs. Delay

>45%

Bernd Girod. Joint Source-Network Coding for Real-time Media 19

Conclusions

Joint source-channel coding– Improved performance, if Separation Principle does not hold– Non-obvious trade-off between source coding and channel coding

Joint source-network coding– Jointly optimize source coding/decoding and packet transport– Adaptive media playout: “real-time” more flexible than we thought– FEC across packets and smart prefetching can provide virtual priority

mechanisms– Packet Path Diversity circumvents congestion, reduces latency

Many good & relevant research problems!Many good & relevant research problems!