1NUS.SOC.CS5248OOI WEI TSANG

Playout Buffer Management

NUS.SOC.CS5248OOI WEI TSANG 2

You are Here

Network

Encoder

Sender

Middlebox

Receiver

Decoder

NUS.SOC.CS5248OOI WEI TSANG 3

How to recv and play?open socketwhile not done

if socket is readable read packet from socketremove RTP headerdecodeplay back

NUS.SOC.CS5248OOI WEI TSANG 4

What’s Wrong?packet orderingpacket lossnext packet arrive in-time?

Especially bad for audio applications

5NUS.SOC.CS5248OOI WEI TSANG

Detour: A Brief Intro to Audio Conferencing

NUS.SOC.CS5248OOI WEI TSANG 6

Audio CompressionNormally uncompressed

Telephone quality: 8-bit audio 8Khz

20-30ms per packet

NUS.SOC.CS5248OOI WEI TSANG 7

Common TechniqueSilence Detection

No need to send if there is no sound at the input

Talkspurt Sequence of consecutive audio

packets (in between silence)

NUS.SOC.CS5248OOI WEI TSANG 8

Detecting Talkspurt in RTPMarker bit

Not reliable as packet with marker bit could be lost

Deduce from timestamp and sequence number

9NUS.SOC.CS5248OOI WEI TSANG

Delay Jitter

NUS.SOC.CS5248OOI WEI TSANG 10

What causes Jitter?Network delay =

Propagation Delay (fixed) +Queueing Delay (variable)

Delay jitter is caused by variable queueing delay

NUS.SOC.CS5248OOI WEI TSANG 11

Delay Jitter

Time

TransitTime

small jitter

large jitter

NUS.SOC.CS5248OOI WEI TSANG 12

Spike

Time

TransitTime

13NUS.SOC.CS5248OOI WEI TSANG

Buffer: The Jitter Absorber

NUS.SOC.CS5248OOI WEI TSANG 14

If Zero Jitter

Time

NUS.SOC.CS5248OOI WEI TSANG 15

With Jitter

TimeLOSS

NUS.SOC.CS5248OOI WEI TSANG 16

With Jitter + Buffer

Time

Buffer Size

Playout Delay

17NUS.SOC.CS5248OOI WEI TSANG

Q: How to set playout delay?

NUS.SOC.CS5248OOI WEI TSANG 18

Adapting Playout DelayWhen jitter is low, reduce delayWhen jitter is high, increase delay

NUS.SOC.CS5248OOI WEI TSANG 19

Fundamental Trade-off

Latency vs Packet Loss

NUS.SOC.CS5248OOI WEI TSANG 20

Playout DelayOnce you set the playout delay,

cannot change!

NOT true: can change at beginning of talkspurt

NUS.SOC.CS5248OOI WEI TSANG 21

Adapting Playout Delay

SEND

RECV

PLAY

22NUS.SOC.CS5248OOI WEI TSANG

Adaptation Algorithm

NUS.SOC.CS5248OOI WEI TSANG 23

Variables and Notations

Tsend(i)

Tplay(i)

Tbuffer(i)

Tarrive(i)

Tdelay(i)

Tnet(i)

NUS.SOC.CS5248OOI WEI TSANG 24

First Packet in the TalkspurtTdelay(i) = Enet(i) + 4 Vnet(i)

Tplay(i) = Tsend(i) + Tdelay(i)

Tsend(i)

Tplay(i)

Tbuffer(i)

Tarrive(i)

Tdelay(i)

Tnet(i)

NUS.SOC.CS5248OOI WEI TSANG 25

Subsequent Packets

Tplay(j) = Tplay(i) + Tsend(j) – Tsend(i)

Tsend(i)

Tplay(i)

Tbuffer(i)

Tarrive(i)

Tdelay(i)

Tnet(i)

26NUS.SOC.CS5248OOI WEI TSANG

How to estimate Enet(i)

NUS.SOC.CS5248OOI WEI TSANG 27

Algorithm 1 (Jacobson’s)Enet(i) = aEnet(i-1) + (1-a)Tnet(i)

NUS.SOC.CS5248OOI WEI TSANG 28

Algorithm 2

if Tnet(i) > Enet(i)

Enet(i) = bEnet(i-1) + (1-b)Tnet(i)

else

Enet(i) = aEnet(i-1) + (1-a)Tnet(i)

NUS.SOC.CS5248OOI WEI TSANG 29

Algorithm 3

Enet(i) = min { Tnet(j) }

(over all packets in previous talkspurt)

NUS.SOC.CS5248OOI WEI TSANG 30

Algorithm 4 (Ramjee’s)Ramjee’s Proposal

Observation: Algorithm 1-3 take too long to react to spike.

NUS.SOC.CS5248OOI WEI TSANG 31

Spike

NUS.SOC.CS5248OOI WEI TSANG 32

Ramjee’s Idea

Works in 2 modes

SPIKE NORMAL

NUS.SOC.CS5248OOI WEI TSANG 33

Three QuestionsWhen to switch from normal to

spike mode?

When to switch from spike back to normal?

How to estimate during spike mode?

NUS.SOC.CS5248OOI WEI TSANG 34

Normal to Spike if difference in delays is large

NUS.SOC.CS5248OOI WEI TSANG 35

Normal to Spike if difference in delays is large

difference in delays:Tnet(i) – Tnet(i-1)

large:800 + 2Vnet(i)

NUS.SOC.CS5248OOI WEI TSANG 36

Another View of Spike (ZOOM)

Tnet(i)

Tarrive(i)

NUS.SOC.CS5248OOI WEI TSANG 37

Spike to Normal slope = slope/2 +

|2Tnet(i) – Tnet(i-1) – Tnet(i-2)|/8

if slope < 64 switch to normal

NUS.SOC.CS5248OOI WEI TSANG 38

Estimation in Spike ModeEnet(i) = Enet(i-1) + Tnet(i) – Tnet(i-1)

NUS.SOC.CS5248OOI WEI TSANG 39

First Packet in the TalkspurtTdelay(i) = Enet(i) + 4 Vnet(i)

Tplay(i) = Tsend(i) + Tdelay(i)

NUS.SOC.CS5248OOI WEI TSANG 40

First Packet in the TalkspurtTdelay(i) = Enet(i) + 4 Vnet(i)

Tplay(i) = Tsend(i) + Tdelay(i)

Tsend(i)

Tprop(i)

Tplay(i)

Tbuffer(i)

Tarrive(i)

Tq(i)

Tdelay(i)

Tnet(i)

41NUS.SOC.CS5248OOI WEI TSANG

How to estimate Vnet(i)

NUS.SOC.CS5248OOI WEI TSANG 42

Variation of DelayVnet(i) =

aVnet(i-1) + (1-a)|Enet(i) – Tnet(i)|

43NUS.SOC.CS5248OOI WEI TSANG

Comparisons of 4 Algorithms

NUS.SOC.CS5248OOI WEI TSANG 44

Playout Delay vs Loss Rate

Tdelay(i)

Loss Rate

45NUS.SOC.CS5248OOI WEI TSANG

Algorithm 5 (Moon’s)

NUS.SOC.CS5248OOI WEI TSANG 46

Problems of Existing Algorithms Jacobson’s react too slowlyRamjee’s follow the delay too

closely

NUS.SOC.CS5248OOI WEI TSANG 47

Moon’s IdeaCollect statistics on packets that

have arrived

Find t such that q% of last w packets has Tnet(i) < t

Tdelay(i) = t

NUS.SOC.CS5248OOI WEI TSANG 48

Example (w =50, q = 90%)

num of packets

delay

1 2 3 4 5 6 7 8 9 10 11 12

NUS.SOC.CS5248OOI WEI TSANG 49

Spike ModeTdelay(i) = Enet(1)

SPIKE NORMAL

NUS.SOC.CS5248OOI WEI TSANG 50

Spike Mode

NUS.SOC.CS5248OOI WEI TSANG 51

Switching ModeNormal to Spike

if (Tnet(i) > k*Tdelay(i))

Spike to Normal

if (Tnet(i) < k’ * OLDTdelay(i))

NUS.SOC.CS5248OOI WEI TSANG 52

Loss Rate vs. DelayTdelay(i)

Loss Rate

53NUS.SOC.CS5248OOI WEI TSANG

Theoretical Lower Bound

NUS.SOC.CS1102OOI WEI TSANG

54

Dynamic Programming

NUS.SOC.CS1102OOI WEI TSANG

55

Fibonacci Numbers

0 1 …

0 1 2

…

i-2 i-1 i

otherwise

i

i

FF

F

ii

i 1

0

1

0

21

+

NUS.SOC.CS1102OOI WEI TSANG

56

Fibonacci Numbers

fib(n)x[0] = 0x[1] = 1for (i = 2; i < n; i++)

x[i] = x[i-1] + x[i-2]return x[i]

NUS.SOC.CS1102OOI WEI TSANG

57

Binomial Coefficient

nor 0 if 1

1

1

1

kk

n

k

n

k

n 1

1 1

1 1

1 1

1

1

k

n

NUS.SOC.CS1102OOI WEI TSANG

58

Change-Making Problem [Weiss] 7.6

For a currency with coin C1, C2, .. Cn (cents), what is the min number of coins needed to make K cents of change?

NUS.SOC.CS1102OOI WEI TSANG

59

Example C = {1, 5, 10, 20, 50} K = 76 cents Give 4 coins 50 + 20 + 5 + 1 = 76

NUS.SOC.CS1102OOI WEI TSANG

60

Formulation To make a change of K cents, either

make a change of (K-50) cents, or make a change of (K-20) cents, or make a change of (K-10) cents, or make a change of (K-5) cents, or make a change of (K-1) cents

Number of coins for K = 1 + minimum of all the above choices

NUS.SOC.CS1102OOI WEI TSANG

61

Dynamic Programming)}({min1)( i

iCKcoinUsedKcoinUsed

… …

min +1

KK-5K-10K-20

62NUS.SOC.CS5248OOI WEI TSANG

Theoretical Lower Bound

NUS.SOC.CS5248OOI WEI TSANG 63

Goal Input:

A packet trace A number of packet losses

What is the minimum average playout delay Tdelay(.) ?

NUS.SOC.CS5248OOI WEI TSANG 64

Trace

k

M talkspurts

1,k 2,k 3,k j,k nk,k

NUS.SOC.CS5248OOI WEI TSANG 65

New NotationsM:

Number of TalkspurtNpacket(k) or nk

Number of packets in talkspurt kNtotal

Total number of packets

NUS.SOC.CS5248OOI WEI TSANG 66

Define..

d(k, i)minimum average playout delay for choosing i packets to be played out from k-th talkspurt

k.. M..

1,k 2,k 3,k j,k nk,k

NUS.SOC.CS5248OOI WEI TSANG 67

Define..

D(k, i)minimum average playout delay for choosing i packets to be played out from k-th to M-th talkspurt

k.. M..

NUS.SOC.CS5248OOI WEI TSANG 68

Dynamic Programming

1

2

3

M

0 1 2 N

NUS.SOC.CS5248OOI WEI TSANG 69

Formulation for Base Caseif i = 0 then

D(k,i) = 0if k = M

if i ≤ Npacket(M) then

D(k,i) = d(k,i)else

D(k,i) = ∞

NUS.SOC.CS5248OOI WEI TSANG 70

Formulation for Recursive Case

k.. M..

i

jkjdjikDjiikD

ij

),(),1()(min),(0

NUS.SOC.CS5248OOI WEI TSANG 71

Loss Rate vs. DelayTdelay(i)

Loss Rate1%

72NUS.SOC.CS5248OOI WEI TSANG

Video Playout

NUS.SOC.CS5248OOI WEI TSANG 73

Two Methods I-Policy

Display at fixed playout delay Drop late frames

E-Policy Display late frame at next period Playout delay increase

NUS.SOC.CS5248OOI WEI TSANG 74

I-Policy

a b c d e f g h

1 2 3 4 5 6 7 8 9 10

a c e f h

NUS.SOC.CS5248OOI WEI TSANG 75

E-Policy

a b c d e f g h

1 2 3 4 5 6 7 8 9 10 11

a b d f gc e h

NUS.SOC.CS5248OOI WEI TSANG 76

I-Policy

a b c d e f g h

1 2 3 4 5 6 7 8 9 10

a b e f hg

NUS.SOC.CS5248OOI WEI TSANG 77

E-Policy

a b c d e f g h

1 2 3 4 5 6 7 8 9 10

a b c d fe

NUS.SOC.CS5248OOI WEI TSANG 78

Pro and Cons I-Policy: Fixed latencyE-Policy: No Loss Frame or Gap

NUS.SOC.CS5248OOI WEI TSANG 79

Idea

if queue length > L for T secondsdrop incoming frames

long queue -> small Tshort queue -> large T

NUS.SOC.CS5248OOI WEI TSANG 80

Queue Monitoring

3

2

3

3

3

4

2

5

2

6

2

7

2

8

0 0 0 0 0 0 0

queue length

threshold

waiting time

NUS.SOC.CS5248OOI WEI TSANG 81

Queue Monitoring

3

2

3

3

3

4

2

5

2

6

2

7

2

8

0 0 0 0 0 0 0

queue length

threshold

waiting time

NUS.SOC.CS5248OOI WEI TSANG 82

Queue Monitoring

3

2

3

3

3

4

2

5

2

6

2

7

2

8

1 0 0 0 0 0 0

queue length

threshold

waiting time

NUS.SOC.CS5248OOI WEI TSANG 83

Queue Monitoring

3

2

3

3

3

4

2

5

2

6

2

7

2

8

2 1 0 0 0 0 0

queue length

threshold

waiting time

NUS.SOC.CS5248OOI WEI TSANG 84

Queue Monitoring

3

2

3

3

3

4

2

5

2

6

2

7

2

8

3 2 1 1 0 0 0

queue length

threshold

waiting time

NUS.SOC.CS5248OOI WEI TSANG 85

SummaryPreliminary work by Stone and

Jeffay on Video Playout

Ramjee’s and Moon’s adaptive audio playout algorithm

86NUS.SOC.CS5248OOI WEI TSANG

Possible Survey and Project Topic

NUS.SOC.CS5248OOI WEI TSANG 87

Adapting PlaybackBesides adapting buffering time,

we can adapt playback time

Estimating and eliminating clock drift