Packet audio Packet audio playout delay playout delay adjustmentadjustment

Performance bounds and Performance bounds and algorithmsalgorithms

Moon, Kurose, TowsleyMoon, Kurose, Towsley

Overall IdeaOverall Idea Because of packet jitter/delay Because of packet jitter/delay

changes, we need a changes, we need a playout bufferplayout buffer The bigger, the betterThe bigger, the better But, a large buffer hinders But, a large buffer hinders

responsive transmission of audioresponsive transmission of audio 400ms/5% loss for voice conversation400ms/5% loss for voice conversation Interactive media/video conferencing Interactive media/video conferencing

needs the smallest buffers possibleneeds the smallest buffers possible

A solution, and an A solution, and an approximationapproximation

In the first part of the paper, they give In the first part of the paper, they give bounds on the size of playout buffer bounds on the size of playout buffer needed under certain lossesneeded under certain losses Not an online algorithm, computationally Not an online algorithm, computationally

expensive (the idea is to focus on expensive (the idea is to focus on percentages)percentages)

Inelastic mediumInelastic medium In the second part of the paper, they In the second part of the paper, they

present an on-line algorithm that is present an on-line algorithm that is computationally feasible to adjust computationally feasible to adjust talkspurt playout delaytalkspurt playout delay

Related WorkRelated Work Playout delay adjustmentsPlayout delay adjustments

Per-packet and per-talkspurt Per-packet and per-talkspurt (assumptions… speech, or music?)(assumptions… speech, or music?)

Network level observationsNetwork level observations Three graphs, probe compressionThree graphs, probe compression

Baseline doesn’t change much—real Baseline doesn’t change much—real advantages in adjusting delay playout advantages in adjusting delay playout occurs in multi-talkburst delay spikesoccurs in multi-talkburst delay spikes

Problem StatementProblem Statement For a given set of losses at the For a given set of losses at the

receiver, we get to set the playout receiver, we get to set the playout delays of each talkspurt anyway we delays of each talkspurt anyway we wantwant Which assignment is the best?Which assignment is the best? For 1 packet lost? 2? 3? 134?For 1 packet lost? 2? 3? 134?

First, let’s fix some notationFirst, let’s fix some notation

NotationNotation ttkk

i – i – sender timestamp of sender timestamp of iith packet of th packet of kkth talkspurtth talkspurt aakk

ii – receiver timestamp of – receiver timestamp of iith packet of th packet of kkth th talkspurttalkspurt

nnkk – num packets in – num packets in kkth talkspurt (received)th talkspurt (received) N – total number of packets in trace (N – total number of packets in trace (ΣΣkknnkk)) ppkk

ii(A) – playout time under algorithm A(A) – playout time under algorithm A Delay: pDelay: pkk

ii(A) – t(A) – tkkii, loss if p, loss if pkk

ii(A) < a(A) < akkii

Indicator if packet is played:Indicator if packet is played: rrkk

ii(A)(A)

Notation, (con’t)Notation, (con’t) Total # packets played under ATotal # packets played under A

N(A) = N(A) = ΣΣkkM M ΣΣii

nknk r rkkii(A)(A)

Average playout delay:Average playout delay: 1/N(A) 1/N(A) ΣΣkk

M M ΣΣiinknk r rkk

ii(A)(p(A)(pkkii(A) – t(A) – tkk

ii)) Loss rate:Loss rate:

l = (N – N(A)) / N * 100l = (N – N(A)) / N * 100

Notation (con’t)Notation (con’t) d’d’kk

ii: delay between sending and : delay between sending and receivingreceiving

d’: min (d’d’: min (d’kkii))

ddkkii: normalized delay = d’: normalized delay = d’kk

ii – d’ – d’ ddkk

(i)(i): ith smallest normalized delay: ith smallest normalized delay

Off-line solution w/o Off-line solution w/o collisionscollisions

To play To play i i packets from the packets from the kkth th talkspurt, the playout delay must be at talkspurt, the playout delay must be at least (the unknowable) dleast (the unknowable) dkk

(i)(i)

Remember that if algorithm A uses a Remember that if algorithm A uses a large playout delay for one talkspurt, it large playout delay for one talkspurt, it could delay subsequent talkspurts could delay subsequent talkspurts (collisions)(collisions)

Let’s ignore them for nowLet’s ignore them for now Time: O(MNTime: O(MN22) Space: O(MN)) Space: O(MN)

Off-line solution w/o Off-line solution w/o collisionscollisions

We assume percentages of loss, not actual We assume percentages of loss, not actual loss patterns (to simplify the complexity)loss patterns (to simplify the complexity)

D(k,i) is min playout delay for i packets lostD(k,i) is min playout delay for i packets lost D(k,i) =D(k,i) =

0 if i = 00 if i = 0 ddkk

(i)(i) if k = M and i <= n if k = M and i <= nMM inf if k = M and i > ninf if k = M and i > nMM min (((i-j)D(k+1,i-j) + jdmin (((i-j)D(k+1,i-j) + jdkk

(j(j))/i)))/i) Proof by contradictionProof by contradiction

Offline algorithm with Offline algorithm with collisionscollisions

We might have to adjust the playout times of We might have to adjust the playout times of some of the talkspurts due to collisions, so D some of the talkspurts due to collisions, so D must now take those into accountmust now take those into account

We define a vector S (captures length of silence)We define a vector S (captures length of silence) We can capture the sum of the increasesWe can capture the sum of the increases Now D includes C as well (C tracks packets Now D includes C as well (C tracks packets

played out at every step of the computation)played out at every step of the computation) D now differs from the old D only in the extra D now differs from the old D only in the extra

delays incurred by the collisionsdelays incurred by the collisions The new D does not capture the optimal, though The new D does not capture the optimal, though

(why?)(why?) Time: O(MTime: O(M22NN22) Space: O(M) Space: O(M22NN22))

An online algorithmAn online algorithm Algorithm 1: LinearAlgorithm 1: Linear

Slow to catch up, good at maintaining a solid Slow to catch up, good at maintaining a solid valuevalue

Algorithm 2: Depends on spike detectionAlgorithm 2: Depends on spike detection Quick at catching up, but sometimes Quick at catching up, but sometimes

overzealousoverzealous Algorithm 3: Two ModesAlgorithm 3: Two Modes

Track spikes when they are detectedTrack spikes when they are detected Otherwise update delay and delay varience (q)Otherwise update delay and delay varience (q) Switch when you have a multiple of the delaySwitch when you have a multiple of the delay

Evaluation / ConclusionEvaluation / Conclusion They instrument the senders and the They instrument the senders and the

receiversreceivers Plot average playout delay vs packet loss Plot average playout delay vs packet loss

raterate Results seem to show that Algorithm 3 gets Results seem to show that Algorithm 3 gets

very close to the optimalvery close to the optimal However, the results are very close much of However, the results are very close much of

the timethe time Sometimes 1 is much worse, sometimes 2, Sometimes 1 is much worse, sometimes 2,

but 3 seems to always be pretty stablebut 3 seems to always be pretty stable

Queue Queue MonitoringMonitoring

A Delay Jitter Management A Delay Jitter Management PolicyPolicy

Stone, JeffayStone, Jeffay

Display and e2e JitterDisplay and e2e Jitter Recall the steps for transmitting video:Recall the steps for transmitting video:

Acquire, digitize, compress, transmit, Acquire, digitize, compress, transmit, decompressed, buffer, displaydecompressed, buffer, display

Display LatencyDisplay Latency is acquire to display is acquire to display e2e latencye2e latency is acquire to buffer is acquire to buffer

What problems can affect this process?What problems can affect this process? Delay Jitter (variance in e2e latency)Delay Jitter (variance in e2e latency)

Can we ensure constant e2e latency?Can we ensure constant e2e latency? Even with Isochronous service models?Even with Isochronous service models?

We’re going to adjust the display latency We’re going to adjust the display latency insteadinstead

Audio vs videoAudio vs video Recall the audio applicationRecall the audio application

Talkspurts vs Silence PeriodsTalkspurts vs Silence Periods Analog for video?Analog for video?

Are gaps ok during the transmission?Are gaps ok during the transmission? Display perceptionDisplay perception Network congestionNetwork congestion

Video as a datatypeVideo as a datatype Can we repeat frames, leave black spaces, Can we repeat frames, leave black spaces,

etc?etc?

Late policiesLate policies I-policy:I-policy:

DiscardDiscard All frames now have the same display latencyAll frames now have the same display latency StaticStatic

E-policy:E-policy: Play at earliest conveniencePlay at earliest convenience Increases latency for subsequent framesIncreases latency for subsequent frames Keeps getting higher than observed e2e delayKeeps getting higher than observed e2e delay

Example 1Example 1

I-Policy

0

0.5

1

1.5

2

2.5

3

1 2 3 4 5 6 7 8 9 10

Queue Length Display Latency

1 2 3 4 5 6 7 8 9 10

1 2 3 4 5 6 7 8 9 10 E-Policy

0

0.5

1

1.5

2

2.5

3

1 2 3 4 5 6 7 8 9 10

Queue Length Display Latency

Example 2Example 2

I-Policy

0

0.5

1

1.5

2

2.5

3

1 2 3 4 5 6 7 8 9 10

Queue Length Display Latency

1 2 3 4 5 6 7 8 9 10

1 2 3 4 5 6 7 8 9 10 E-Policy

0

0.5

1

1.5

2

2.5

3

1 2 3 4 5 6 7 8 9 10

Queue Length Display Latency

I-vs-EI-vs-E I policy’s advantageI policy’s advantage

Low jitter and burstsLow jitter and bursts E policy’s advantageE policy’s advantage

Good during high latency and low Good during high latency and low latency, but not good after burstslatency, but not good after bursts

Hybrid approach: Hybrid approach: Queue MonitoringQueue Monitoring

Queue MonitoringQueue Monitoring When displaying a frameWhen displaying a frame

Thresholding operationThresholding operation If qlen is If qlen is mm, then counters 1 through , then counters 1 through mm-1 are -1 are

incrementedincremented All others are resetAll others are reset

When the counter exceeds a value, the oldest When the counter exceeds a value, the oldest frame is discardedframe is discarded

If the queue has contained more than If the queue has contained more than nn frames, frames, then we can reduce the latency (the jitter is then we can reduce the latency (the jitter is stable)stable)

Large variations occur infrequently and smaller Large variations occur infrequently and smaller variations occur more frequently (still true today)?variations occur more frequently (still true today)?

EvaluationEvaluation The inherent difficultyThe inherent difficulty

Gaps vs display latencyGaps vs display latency Lexocographic ordering for two axesLexocographic ordering for two axes

Average gap rateAverage gap rate Average display latencyAverage display latency

Experimental DesignExperimental Design ““academic computer science” networkacademic computer science” network

Time of day, workload seenTime of day, workload seen

Evaluation ResultsEvaluation Results Comparison between I2, I3, and EComparison between I2, I3, and E Usually the same or betterUsually the same or better

Except for incomparable resultsExcept for incomparable results In comparison to the E-policy, it seems to In comparison to the E-policy, it seems to

be workload/network dependentbe workload/network dependent Instantaneous gap rate, delay policy Instantaneous gap rate, delay policy

would be better (perhaps)would be better (perhaps) More adaptive I-policyMore adaptive I-policy More tests, of courseMore tests, of course Addressing ad-hoc quality measuresAddressing ad-hoc quality measures