NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

1

Adaptive Playout

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

2

You are Here

Network

Encoder

Sender

Middlebox

Receiver

Decoder

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

3

How to recv and play?

open socketwhile not done

if socket is readable read packet from socketremove RTP headerdecodeplay back

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

4

What’s Wrong?

packet orderingpacket lossnext packet arrive in-time?

Especially bad for audio applications

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

5

Overview

Network

RTP Classifier

Decode

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

7

Packet Buffer

Sorted by sequence numberWhen ADU is complete, send to

decoder

RTP Classifier

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

8

Playout Buffer

Stored decoded data in playout order

Post-processing/Mixing may happen

Decode

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

9

Why Buffer?

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

10

Sending Packets

Time

Packet

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

11

Receiving Packets

Time

Packet

t1t0

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

12

With Jitter

Time

Packet

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

13

With Jitter

Time

Packet

t2t0

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

14

What causes Jitter?

Network delay = Transmission Delay (fixed) +

Propagation Delay (fixed) +Queuing Delay (variable)

Delay jitter is caused by variable queuing delay

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

15

Delay Jitter

Time

TransitTime

small jitter

large jitter

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

16

Spike

Time

TransitTime

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

17

Today’s Question

How big is the playout buffer?When to play back?

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

18

Types of Applications

Non-interactive Buffer can be large

Interactive As small as possible

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

19

Types of Applications

Video Frames are discrete (easier

problem)

Audio Samples are “continuous”

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

20

Naive Answer

How big is a buffer? Fixed at a small value, to reduce

latency

When to playback? Playback as soon as possible, to

reduce latency

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

21

Client Buffer Management

Bandwidth Smoothing for Non-interactive Applications

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

22

Multi-Threshold Flow Control

R. Zimmermann, K. Fu, M. Jahangiri, C. ShahabiMTAP 2006

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

23

Video-on-Demand (VoD)

High-quality playback requiredBuffer can be largeEncoding may be VBR for high

visual qualityPlayback time may be very long

(2+ hours)

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

24

Two Approaches (1)

Server-controlled Pre-compute transmission schedule Piece-wise linear approximation:

compute a number of constant-rate segments

Different optimization criteria Minimize: # of rate changes, #

utilization of client buffer, peak rate, etc.

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

25

Two Approaches (2)

Client-controlled Client adaptively informs server Advantages:

No rate history necessary (VBR) Can adjust to network conditions Works easily with interactive

commands such as FF, FR, Pause

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

26

Robust Stream Delivery

Smoothing of VBR media traffic has the following quality benefits: Better resource utilization (less

bursty) More streams with the same

network capacity

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

27

Objectives

Multi-Threshold Flow Control(MTFC) algorithm objectives: Online operation Content independence Minimizing feedback

control signaling Rate smoothing

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

28

Example

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

29

MTFC Buffer Management

Multiple Thresholds: goal is middle of buffer

Send rateadjustcommandto serverwheneverthresholdis crossed

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

30

How to Set Thresholds?

Simple: equi-distant m: number of thresholds Overflow and underflow thresholds

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

31

How to Calculate Sending Rate?

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

32

Threshold Spacing StrategiesLinear vs. arithmetic vs. geometric

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

33

Threshold Spacing StrategiesGeometric and arithmetic spacing:

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

34

Sending Rate Computation

Relevant factors: Target buffer level (i.e., 50%): THN

Current buffer level: bwobsv

Predicted duration to reach target level Data consumed during predicted duration

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

35

MTFC Results

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

36

MTFC Results: # of Thresholds

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

37

MTFC Results: # of Thresholds

NumberOf

RateChanges

8MB 16MB 32MB

0

1020

30

40

50

60

708090

100

3 5 9 17

Number of Thresholds

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

38

A Brief Introduction to Audio Conferencing

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

39

Audio Conferencing

Live, interactive application Latency is important

Typically each packet contains20-30ms of audio

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

40

Silence Suppression

Silence Detection If no sound, no need to send

Talkspurt Consecutive audio packets

(between silence) Hundreds of ms

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

41

Recall: RTP Header

marker bit: depends on payload e.g. beginning of frame

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

42

RTP and Talkspurt

First packet of a talkspurt will have marker bit set to 1

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

43

RTP and Talkspurt

Deduce talkspurt from sequence number and timestamp

240

120

360

5190

SeqNoTimeStamp

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

44

Consequences of Talkspurt

Opportunity to adjust playout delay if jitter is large, increase delay if jitter is small, decrease delay

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

45

Fixed Playout Delay

SEND

RECV

PLAY

Talkspurt 1Talkspurt 2

Time

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

46

Adaptive Playout Delay

SEND

RECV

PLAY

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

47

Adaptive Playout Delay

SEND

RECV

PLAY

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

48

Trade-Off

Latency vs. Packet Loss

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

49

Latency vs Loss-Rate

LossRate

Latency

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

50

Latency vs Loss-Rate

LossRate

Latency

• Note: Late packets are considered lost

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

51

Adaptive Playout Mechanisms for Packetized Audio Applications in WAN

R. Ramjee, J. Kurose, D. Towsley, H. SchulzrinneINFOCOM 1994

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

52

Variables and Notations

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

53

Variables and Notations

Tsend(i)

Tplay(i)

Tbuf(i)

Tarrive(i)

Tdelay(i)

Tnet(i)

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

54

1st Packet in Talkspurt

We can estimate as

Mean and variation of the end-to-end delay.

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

55

How to estimate Vnet(i)

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

56

How to estimate Enet(i)

Method 1: Jacobson’s Method

(Delay estimate as in RFC 793, TCP re-transmit timer.)

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

57

Spike

Arrival Time

Tnet

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

58

Problems

Described algorithm does not react to spike fast enough.

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

59

How to estimate Enet(i)

Ramjee’s Method:

SPIKE NORMAL

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

60

Ramjee’s Idea

SPIKE NORMAL

if Tnet(i) suddenly increases

if “slope” is small enough

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

61

In Spike ModeSPIKE

i

Tnet

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

62

In Normal ModeNORMAL

i

Tnet

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

63

Evaluations

Delay

LossRate

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

64

Evaluations

Delay

LossRate

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

65

Problems with Ramjee’s Method

Time

TransitTime

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

66

Packet Audio Playout Delay Adjustment: Performance Bounds and Algorithms

S. Moon, J. Kurose, D. TowsleyMultimedia Systems 1998

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

67

Recall Previous Methods

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

68

How to Set Tdelay(i)

Moon’s Method

Collect statistics on packets that have arrived.

Find t such that q% of last w packets have Tnet(i) < t.

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

69

Example (w =50, q = 90%)num of packets

delay

1 2 3 4 5 6 7 8 9 10 11 12

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

70

Setting Tdelay(i)

NORMAL SPIKE

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

71

Setting Tdelay(i)

Time

TransitTime

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

72

Performance Bound

Given a trace of packets, and a loss rate, find the minimum average playout delay.

Use Dynamic Programming

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

73

A Packet Trace

k

M talkspurts

1,k 2,k 3,k j,k nk,k

Tnet 13 15 10 …

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

74

More Notations

M: Number of Talkspurt

Npacket(k) or nk

Number of packets in talkspurt kNtotal

Total number of packets

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

75

Definition

minimum average playout delay for choosing i packets to be played out from k-th talkspurt

k

M talkspurts

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

76

How to find

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

77

Definition

minimum average playout delay for choosing i packets to be played out from k-th to M-th talkspurt

k

M talkspurts

M..

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

78

Base Case

D(k, 0) =

D(M, i) =

minimum average playout delay forchoosing i packets to be played out from k-th to M-th talkspurt

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

79

Recursive Case

k.. M..

1,k 2,k 3,k nk,k..

j

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

80

Performance Bound

Given a trace of M talkspurts and n packets, and a loss rate e, find the minimum average playout delay.

Answer: Minimum possible average playout delay is D(1, (1-e)n)

minimum average playout delay forchoosing i packets to be played out from k-th to M-th talkspurt

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

81

Evaluations

LossRate

Delay

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

82

Summary

Playout Adjustment for Audio Conferencing

Weighted Average Methods vs. Statistical Methods

An Analysis of Minimum Playout Delay

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

83

Practical Complications

Clock DriftsRoute Change

NUS.SOC.CS5248-2015Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

84

Advanced Techniques

Speed-up Playback