Upload
kelley-carter
View
218
Download
0
Embed Size (px)
DESCRIPTION
AofA Stream Merging Eager, Vernon, Zahorjan (1999) Goal to minimize cost = total server bandwidth Uses multicast –Many clients receive the same transmission simultaneously. Clients receive multiple channels –Clients receive different parts of the media simultaneously. We focus on receive-2. Clients use local buffers –Some data received is played back immediately while other data is stored for future play back.
Citation preview
The Analysis of Optimal Stream Merging Solutions for Media-on-
Demand Amotz Bar-Noy
CUNY and Brooklyn College
Richard LadnerUniversity of Washington
AofA 2004 2
Media-On-Demand
• Type of Media– Two hour movie – 5 minute song– 2 minute news video clip
• Demand on Media– Broadcast for very popular media– Unicast for the heavy tail
AofA 2004 3
Stream MergingEager, Vernon, Zahorjan (1999)
• Goal to minimize cost = total server bandwidth
• Uses multicast– Many clients receive the same transmission
simultaneously.• Clients receive multiple channels
– Clients receive different parts of the media simultaneously. We focus on receive-2.
• Clients use local buffers– Some data received is played back immediately
while other data is stored for future play back.
AofA 2004 4
Stream Merging System -Receive Two
Server
Client
BufferPlayer
MulticastChannels
old stream
new stream
AofA 2004 5
Stream Merging System
Server
BufferPlayer
MulticastChannels
Client
AofA 2004 6
Stream Merging System
Server
BufferPlayer
MulticastChannels
Client
AofA 2004 7
Stream Merging System
Server
BufferPlayer
MulticastChannels
Client
AofA 2004 8
Stream Merging System
Server
BufferPlayer
MulticastChannels
Client
AofA 2004 9
Batching
clientstartup delay guaranteestream
65
4
7
1
2
3
65
4
7
1
2
3
Cost = 14
AofA 2004 10
Stream Merging with Batchingclientstartup delay guaranteestream
A B
1
2
3
65
4
7
1
2
3Cost = 10
AofA 2004 11
Delay vs. Bandwidth Tradeoff2 hour movie 720 arrivals per movie
AofA 2004 12
Off-line vs. On-line
• Off-line– Arrival sequence is known in advance– Used as a basis to compare with on-line algorithms
• On-line– Future arrivals not known– Server must add new streams and lengthen old
streams to accommodate new arrivals
AofA 2004 13
Outline
• Model – Merge cost vs. Full cost
• Optimal merge cost• Optimal full cost• Fully loaded arrivals
AofA 2004 14
Merge Cost vs. Full Cost
• Optimal Merge cost– Minimum total bandwidth (sum of the lengths of
the streams) required to merge to a full stream– M(t1,…,tn)
• Optimal Full cost – Minimum total bandwidth required for a media of
length L.– F(L,t1,…,tn)
AofA 2004 15
Length of Streams
• The length of streams z(x))2(xxp(x)length(x)
A B C D
x = B
z(x) = D
p(x) = A
AofA 2004 16
Optimality Principle for Merge Cost
• Given arrivals t1,…,tn there is k such that in optimal stream merging– tk is the last stream to merge to t1
– Streams t1,…,tk-1 optimally merge eventually to t1
– Streams tk+1,…,tm optimally merge eventually to tk
AofA 2004 17
Dynamic Program for Optimal Merge Cost
• Setup– Arrivals t1,t2, … ,tn
– M(i,j) = minimum merge cost of a merge tree for the arrivals ti,t2, … ,tj
– M(1,n) is the optimal merge cost• Recurrence
– M(i,i) = 0– M(i,j) = mini < k < j{M(i,k-1) + M(k,j) + (tk - ti) + 2(tj - tk)}
• O(n3) time, O(n2) storage– Aggarwal, Wolf, Yu (1996), Eager,Vernon, Zahorjan (1999)
AofA 2004 18
O(n2) Optimal Algorithm
• SetupArrivals t1,t2, … ,tn
r(i,j) = the right most stream that merges to the root of an optimal merge tree for the arrivals ti,…,tj.
• Monotonicity (Knuth ‘71, F. Yao ’80, Borchers, Gupta ‘94)
r(i,j - 1) < r(i,j) < r(i + 1,j)
optimal
M(i,j) = minr(i,j-1) < k < r(i+1,j){M(i,k-1) + M(k,j) + (tk - ti) + 2(tj - tk)}
AofA 2004 19
New Algorithm Behavior
O(n2) time
M(i,j)
r(i,j-1)
r(i+1,j)
r(i,j-1) r(i+1,j)
AofA 2004 20
New Algorithm Behavior
O(n2) time
AofA 2004 21
Optimal Full Cost
L = 8
Optimal Full CostOptimal MergeCost + L
AofA 2004 22
Optimal Full Cost Algorithm
• Setup– Arrivals t1,t2, … ,tn
– G(i) = the optimal full cost for the last n-i+1 arrivals ti,ti+1, … ,tn
– G(1) is the optimal full cost• Recurrence
– G(n+1) = 0– G(i) = L + min{ M(i,k-1) + G(k) : i < k < mi }
where mi = maximum m such that tm-1 - ti < L-1.
AofA 2004 23
Complexity of Full Cost Algorithm
Phase 1: Precompute M(i,i) to M(i,mi)for all i.
Phase 2: Compute G(n+1) to G(1)
n
1i
n
1iii nm/n)mn(m
time using monotonicity
n
1ii )mO(
time from the recurrence for G
n
1ii )mO(
where m is the average number of arrivals in an interval of length L that starts with an arrival
AofA 2004 24
Fully Loaded Arrivals Merge Cost
Streams at 0,1,2,3,4,…,n-1
AofA 2004 25
Recurrence
• Merge recurrence for fully loaded arrivals
• Examples
2}j2nj)M(n{M(j)minM(n)0M(1)
1nj1
n 1 2 3 4 5 6 7 8 9 10 11 12 13
M(n) 0 1 3 6 9 13 17 21 26 31 36 41 46
AofA 2004 26
Fibonacci Rules!• Recurrence
• Fibonacci numbers– 0,1,1, 2, 3, 5, 8,13, 21, 34, 55, 89, …– F0 = 0, F1 = 1, Fk = Fk-1 + Fk-2
• Solution
1kk2k FnF2,F1)n(kM(n)
2}j2nj)M(n{M(j)minM(n)0M(1)
1nj1
AofA 2004 27
Induction Hypothesis Design
21 {13}, 22 {13, 14}, 23 {13, 14, 15}, 24 {13, 14, 15, 16}, 25 {13, 14, 15, 16, 17}, 26 {13, 14, 15, 16, 17, 18}, 27 {14, 15, 16, 17, 18, 19}, 28 {15, 16, 17, 18, 19, 20}, 29 {16, 17, 18, 19, 20, 21}, 30 {17, 18, 19, 20, 21}, 31 {18, 19, 20, 21}, 32 {19, 20, 21}, 33 {20, 21}, 34 {21}, 35 {21, 22}, 36 {21, 22, 23}, 37 {21, 22, 23, 24}, 38 {21, 22, 23, 24, 25}, 39 {21, 22, 23, 24, 25, 26}, 40 {21, 22, 23, 24, 25, 26, 27}, 41 {21, 22, 23, 24, 25, 26, 27, 28}, 42 {21, 22, 23, 24, 25, 26, 27, 28, 29}, 43 {22, 23, 24, 25, 26, 27, 28, 29, 30}, 44 {23, 24, 25, 26, 27, 28, 29, 30, 31}, 45 {24, 25, 26, 27, 28, 29, 30, 31, 32}, 46 {25, 26, 27, 28, 29, 30, 31, 32, 33}, 47 {26, 27, 28, 29, 30, 31, 32, 33, 34}, 48 {27, 28, 29, 30, 31, 32, 33, 34}, 49 {28, 29, 30, 31, 32, 33, 34}, 50 {29, 30, 31, 32, 33, 34}, 51 {30, 31, 32, 33, 34}, 52 {31, 32, 33, 34}, 53 {32, 33, 34}, 54 {33, 34},
2 {1}, 3 {2}, 4 {2, 3}, 5 {3}, 6 {3, 4}, 7 {4, 5}, 8 {5}, 9 {5, 6}, 10 {5, 6, 7}, 11 {6, 7, 8}, 12 {7, 8}, 13 {8}, 14 {8, 9}, 15 {8, 9, 10}, 16 {8, 9, 10, 11}, 17 {9, 10, 11, 12}, 18 {10, 11, 12, 13}, 19 {11, 12, 13}, 20 {12, 13},
55 {34}, 56 {34, 35}, 57 {34, 35, 36}, 58 {34, 35, 36, 37}, 59 {34, 35, 36, 37, 38}, 60 {34, 35, 36, 37, 38, 39}, 61 {34, 35, 36, 37, 38, 39, 40}, 62 {34, 35, 36, 37, 38, 39, 40, 41}, 63 {34, 35, 36, 37, 38, 39, 40, 41, 42}, 64 {34, 35, 36, 37, 38, 39, 40, 41, 42, 43}, 65 {34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44}, 66 {34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45}, 67 {34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46}, 68 {34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47}, 69 {35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48}, 70 {36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49}, 71 {37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50}, 72 {38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51}, 73 {39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52}, 74 {40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53}, 75 {41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54}, 76 {42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55}, 77 {43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55}, 78 {44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55}, 79 {45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55}, 80 {46, 47, 48, 49, 50, 51, 52, 53, 54, 55}, 81 {47, 48, 49, 50, 51, 52, 53, 54, 55}, 82 {48, 49, 50, 51, 52, 53, 54, 55}, 83 {49, 50, 51, 52, 53, 54, 55}, 84 {50, 51, 52, 53, 54, 55}, 85 {51, 52, 53, 54, 55}, 86 {52, 53, 54, 55}, 87 {53, 54, 55},88 {54, 55}, 89 {55},
2}j2nj)M(n{M(j)argminRM(n)1nj1
3
3
2
5
5
3
8
8
5
13
13
8
AofA 2004 28
Optimal Full Cost
• L = 24 and n = 39• Cost = 210
Full steam
merged steam
AofA 2004 29
Optimal Full Cost
• Define F(L,n,s) to be the optimal full cost when there are exactly s full streams. sopt minimizes F(L,n,s)
F(L,n) = F(L,n, sopt )
• TheoremF(L,n,s) = sL + rM(p+1) + (s-r)M(p) where n = ps + r
AofA 2004 30
Shape of F(L,n,s)
sopt = 182
AofA 2004 31
Convexity of F(L,n,s)
• Define Fk(s) < n/s < Fk(s)+1
• Theorem– Fk(s+1)+2 < L + 2 implies F(L,n,s) > F(L,n,s+1)
– Fk(s)+2 > L + 2 implies F(L,n,s) < F(L,n,s+1)
AofA 2004 32
Finding sopt
• Direct approach– Compute h, such that Fh+1 < L+2 < Fh+2
– sopt = n / Fh or n / Fh + 1
• Example– n = 10,000 and L = 100– h = 10 and Fh = 55
– n / Fh = 181
AofA 2004 33
Conclusions
• Optimal stream merging can be solved efficiently.
• Optimal fully loaded stream merging has an elegant closed form solution.