Upload
havily
View
37
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths. Mugurel Ionut Andreica Polytechnic University of Bucharest Computer Science Department. Summary. Motivation – Scheduling File Transfers Offline Scheduling - PowerPoint PPT Presentation
Citation preview
Optimal Scheduling of File Optimal Scheduling of File Transfers with Divisible Sizes on Transfers with Divisible Sizes on
Multiple Disjoint PathsMultiple Disjoint Paths
Mugurel Ionut AndreicaMugurel Ionut AndreicaPolytechnic University of BucharestPolytechnic University of Bucharest
Computer Science DepartmentComputer Science Department
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 22
SummarySummary
Motivation – Scheduling File TransfersMotivation – Scheduling File Transfers Offline SchedulingOffline Scheduling
maximum profit scheduling (files with divisible maximum profit scheduling (files with divisible sizes) – heuristic algorithmsizes) – heuristic algorithm
minimum cost scheduling – optimal algorithmminimum cost scheduling – optimal algorithm
Online Resource ManagementOnline Resource Management algorithmic framework for the block partitioning algorithmic framework for the block partitioning
method (=data structure)method (=data structure)
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 33
MotivationMotivation
world-wide development and world-wide development and deployment of distributed systems, deployment of distributed systems, services and applicationsservices and applications
competition for bottleneck resources competition for bottleneck resources => poor performance=> poor performance
inefficient usage of available inefficient usage of available resources => poor performanceresources => poor performance
scheduling techniques => scheduling techniques => performance improvementsperformance improvements
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 44
Maximum Profit SchedulingMaximum Profit Scheduling
n files (divisible sizes)n files (divisible sizes) szszii – size of the i – size of the ithth file file szsz11≤sz≤sz22 ≤... ≤ sz ≤... ≤ sznn szszii=q=qii··szszi-1i-1, q, qii≥1 is an integer≥1 is an integer ppii=profit of the i=profit of the ithth file (transfer request) file (transfer request)
k pathsk paths path j – available in the interval [0,Tpath j – available in the interval [0,Tjj]] unit speedunit speed
schedule the requests non-preemptively + schedule the requests non-preemptively + at most one request per path per time unitat most one request per path per time unit
equivalent to multiple knapsack with equivalent to multiple knapsack with divisible item sizesdivisible item sizes
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 55
Multiple Knapsack with Multiple Knapsack with Divisible Item SizesDivisible Item Sizes
NP-hardNP-hard can be solved in can be solved in O(n·max{TO(n·max{Tjj}}kk)) (extension from (extension from
single knapsack problem)single knapsack problem) heuristicsheuristics
• Greedy1MultipleKnapsack(n,k)Greedy1MultipleKnapsack(n,k) pack the items optimally in the first knapsackpack the items optimally in the first knapsack remove the q packed items and the knapsackremove the q packed items and the knapsack call Greedy1MultipleKnapsack(n-q,k-1) with the call Greedy1MultipleKnapsack(n-q,k-1) with the
remaining items and knapsacksremaining items and knapsacks• Greedy2Greedy2
sort the items according to some criterion (e.g. sort the items according to some criterion (e.g. Profit/size)Profit/size)
insert the items into knapsack using the First Fit insert the items into knapsack using the First Fit heuristicheuristic
new heuristic algorithm – this papernew heuristic algorithm – this paper
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 66
New Heuristic Algorithm (1/2)New Heuristic Algorithm (1/2) (Multiple Knapsack with (Multiple Knapsack with
Divisible Item Sizes)Divisible Item Sizes) split the items into groupssplit the items into groups
all the items in the same group (i) have the same size all the items in the same group (i) have the same size (sg(sgii))
sgsg11>sg>sg22>...>sg>...>sgGG (G=total # of groups) (G=total # of groups) within a group, sort the items in decreasing order within a group, sort the items in decreasing order
of their profitsof their profits prpri,1i,1≥pr≥pri,2i,2≥…≥pr≥…≥pri,nii,ni (n (nii=the # of items in group i)=the # of items in group i)
insert the items into knapsacks using the insert the items into knapsacks using the First FitFirst Fit heuristicheuristic
traverse the items in inreasing order of group number; traverse the items in inreasing order of group number; within a group, in increasing order of item numberwithin a group, in increasing order of item number
choice of knapsack for an item (i,j) – not important => choice of knapsack for an item (i,j) – not important => because of divisible item sizes, we insert the same set because of divisible item sizes, we insert the same set of items of items
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 77
New Heuristic Algorithm (2/2)New Heuristic Algorithm (2/2) (Multiple Knapsack with (Multiple Knapsack with
Divisible Item Sizes)Divisible Item Sizes) repeatedly improve the initial solutionrepeatedly improve the initial solution
replace an item (i,j) in the knapsack with a replace an item (i,j) in the knapsack with a subset Q of items outside the knapsack, s.t. subset Q of items outside the knapsack, s.t. pr(i,j)<profit(Q) && profit(Q)-pr(i,j)=maximumpr(i,j)<profit(Q) && profit(Q)-pr(i,j)=maximum
ignore item (i,j) from now onignore item (i,j) from now on
time complexitytime complexity O(n·S·min{n,S·log(S)})O(n·S·min{n,S·log(S)}) S=maximum size of an itemS=maximum size of an item
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 88
Performance EvaluationPerformance Evaluation(Multiple Knapsack with (Multiple Knapsack with
Divisible Item Sizes)Divisible Item Sizes) tested algorithmstested algorithms
new heuristicnew heuristic Greedy1MultipleKnapsackGreedy1MultipleKnapsack Greedy2 (with multiple criteria)Greedy2 (with multiple criteria) direct solution (extension from single knapsack)direct solution (extension from single knapsack)
resultsresults most of the cases – solved optimally by the new most of the cases – solved optimally by the new
heuristicheuristic in terms of quality of solution and running time in terms of quality of solution and running time
=> the new heuristic = clear winner=> the new heuristic = clear winner• followed by Greedy1MultipleKnapsackfollowed by Greedy1MultipleKnapsack
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 99
Minimum Cost Scheduling Minimum Cost Scheduling (1/3)(1/3)
sequence of n identical filessequence of n identical files to be sent consecutively from the same source to be sent consecutively from the same source
to the same destinationto the same destination sending time per file=1 time unitsending time per file=1 time unit
k data transfer providersk data transfer providers provider jprovider j
• cost: Ccost: Cj j per time unitper time unit• lease at most ONE time interval of duration at most lease at most ONE time interval of duration at most
TTmax,imax,i which includes [T which includes [T1,j1,j, T, T2,j2,j]]
default network link: cost Ldefault network link: cost L ii for transferring for transferring the ithe ithth file file
minimize the total costminimize the total cost
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 1010
Minimum Cost Scheduling Minimum Cost Scheduling (2/3)(2/3)
O(nO(n··k) dynamic programmingk) dynamic programming sort the data transfer providers in increasing sort the data transfer providers in increasing
order of Torder of T2,j2,j: : TT2,12,1≤T≤T2,22,2≤…≤T≤…≤T2,n2,n Cmin[i,j]Cmin[i,j]=the minimum total cost for sending =the minimum total cost for sending
the first j files using a subset of the first i the first j files using a subset of the first i providers (in the sorted order)providers (in the sorted order)
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 1111
Minimum Cost Scheduling Minimum Cost Scheduling (3/3)(3/3)
compute an auxiliary array minpcompute an auxiliary array minpi i in O(n) timein O(n) time
for for j in [Tj in [T2,i2,i, T, T1,i1,i+T+Tmax,imax,i]:]: Cmin[i,j]=min{Cmin[i-1,j], Cmin[i,j-1]+LCmin[i,j]=min{Cmin[i-1,j], Cmin[i,j-1]+L jj, , (j-T(j-T1,i1,i)·C)·Cii+minp+minpii[j-T[j-Tmax,imax,i]}]}
Cmin[k,n] – the answerCmin[k,n] – the answer O(nO(n··k) overallk) overall
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 1212
Online Resource ManagementOnline Resource Management(1/4)(1/4)
scenarioscenario a resource manager receives resource a resource manager receives resource
allocation and reservation requestsallocation and reservation requests request=amount of resource (bandwidth) + request=amount of resource (bandwidth) +
time constraints (fixed duration, earliest start time constraints (fixed duration, earliest start time, latest finish time)time, latest finish time)
modelmodel time – divided into equally-sized time slotstime – divided into equally-sized time slots many requests simultaneously => we need many requests simultaneously => we need
low response times => efficient data structurelow response times => efficient data structure
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 1313
Online Resource ManagementOnline Resource Management(2/4)(2/4)
algorithmic frameworkalgorithmic framework block partitioning methodblock partitioning method
array of n cellsarray of n cells each cell – value veach cell – value vii divide the cells: n/k blocks of size kdivide the cells: n/k blocks of size k update and query functions: O(k+n/k)update and query functions: O(k+n/k) k=sqrt(n) => O(k+n/k)=O(sqrt(n))k=sqrt(n) => O(k+n/k)=O(sqrt(n))
point and range queriespoint and range queries vvii=?=? qFunc(vqFunc(vaa, v, va+1a+1, ..., v, ..., vbb)=?)=?
point and range updatespoint and range updates vvii=u=u vvii=uFunc(u, v=uFunc(u, vii), a≤i≤ b), a≤i≤ b
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 1414
Online Resource ManagementOnline Resource Management(3/4)(3/4)
update functionsupdate functions BPpointUpdateBPpointUpdate BPrangeUpdateBPrangeUpdate BPrangeUpdatePointsBPrangeUpdatePoints BPrangeUpdatePartialBlockBPrangeUpdatePartialBlock BPrangeUpdateFullBlockBPrangeUpdateFullBlock
query functionsquery functions BPpointQueryBPpointQuery BPrangeQueryBPrangeQuery BPrangeQueryPointsBPrangeQueryPoints BPrangeQueryPartialBlockBPrangeQueryPartialBlock BPrangeQueryFullBlockBPrangeQueryFullBlock
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 1515
Online Resource ManagementOnline Resource Management(4/4)(4/4)
solves efficiently many problemssolves efficiently many problems range addition update+range min queryrange addition update+range min query range set update+range maximum sum range set update+range maximum sum
segment querysegment query ... many more... many more
typical resource reservation requeststypical resource reservation requests range addition update (reserve bandwidth for range addition update (reserve bandwidth for
a full time interval)a full time interval) range minimum query (check to see if enough range minimum query (check to see if enough
bandwidth is available in every time slot)bandwidth is available in every time slot)
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 1616
ConclusionsConclusions
offline scheduling problemsoffline scheduling problems maximum profit schedulingmaximum profit scheduling
• multiple knapsack with divisible item sizes – multiple knapsack with divisible item sizes – efficient heuristicefficient heuristic
minimum cost schedulingminimum cost scheduling• optimal dynamic programming algorithmoptimal dynamic programming algorithm
online resource managementonline resource management algorithmic framework (new) for the algorithmic framework (new) for the
block partitioning method (well known)block partitioning method (well known)
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 1717
Thank You !Thank You !