View
214
Download
0
Tags:
Embed Size (px)
Citation preview
I think your suggestion is, Can we do two things at once? Well, we’re of the view that
we can walk and chew gum at the same time.
—Richard Armitage, deputy secretary of state, June 2, 2004
To do two things at once is to do neither.
—Publilius Syrus, Roman slave, first century B.C.
Therefore…
Multitask a few…
..let the rest queue.
Computer systems are not different…
Multi-Programming Level(# concurrent tasks/threads)
Efficiency
(useful work per second)
thrashing
Therefore… ServerActivetasks
Tasks not-yet-started
Q: Max number of tasks allowed to share server?Common Soln: K*
AdmissionControl
K*
Presenting
How to Multitask Smartly
4
A Queueing-theoretic abstraction
server
# jobs at server (n)
Totalspeed(n) K*
(known)
5
A Queueing-theoretic abstraction
PSserver
1
2
K
K = MPL
# jobs at server (n)
Totalspeed(n) K*
FIFO buffer
The G/G/PS-MPL model
• General arrival process for jobs• Job sizes i.i.d. ~ X
• Sizes unknown, distribution of X known
GOAL: Find MPL (i.e. K) to minimize mean response time
response time
n
jobs
each job gets rate
(n)n
:
(known)
Optimal MPL= K* ?
0.25
0.5
0.75
1
1.25
0 5 10 15 20 25
# jobs at server (n)
Totalspeed
(jobs/sec) K*=5
Exponential job sizes (C2 = 1)
Weibull job sizes (C2 = 19)
3
4
5
6
7
4 6 8 10 12 14 16
2.75
3
3.25
4 6 8 10 12 14 16
Mea
n re
spon
se ti
me
MPL
MPL
Mea
n re
spon
se ti
me
K*
K*
OPT MPL
OPT MPL
Example
Poisson(0.8) arrival process
45%improvement
Optimal MPL= K* ?
0.25
0.5
0.75
1
1.25
0 5 10 15 20 25
# jobs at server (n)
Totalspeed
(jobs/sec) K*=5
Exponential job sizes (C2 = 1)
Weibull job sizes (C2 = 19)
3
4
5
6
7
4 6 8 10 12 14 16
2.75
3
3.25
4 6 8 10 12 14 16
Mea
n re
spon
se ti
me
MPL
MPL
Mea
n re
spon
se ti
me
K*
K*
OPT MPL
OPT MPL
Example
Poisson(0.8) arrival process
DEPENDS!
45%improvement
8
Intuition for the effect of MPL
FIFO
server
ProcessorSharing (PS)
9
Intuition for the effect of MPL
FIFO
server
ProcessorSharing (PS)
High job-size variability(C2)
High arrival rate
0.25
0.5
0.75
1
1.25
0 5 10 15 20 25
# jobs at server (n)
Totalspeed(n) K*
MPL
10
Intuition for the effect of MPL
server
FIFO ProcessorSharing (PS)
High job-size variability(C2)
High arrival rate
Optimal MPL ↑C2 ↑
Arrival rate ↑ Optimal MPL → K*
MPL
11
Why is finding the optimal MPL hard?
- No analysis even for the case of Poisson arrivals (M/G/PS-MPL)
Hurdle 1:State dependent
service rates
Hurdle 2:Finite Multi-
Programming-Limit
Hurdle 3:General (high C2) job
size distribution
Mean response time only depends on E[X]
[Kelly 79]
Transform of response time known for Exponential job size distribution
[Rege Sengupta 85]
Heavy traffic diffusion approximation
[Zhang Zwart 09]
Our work
12
Where we are…
GOAL: Optimal MPL for M/G/PS-MPL
SUBGOAL: Approximation for M/G/PS-MPL
What if the traffic rate is unknown?What if the traffic is not Poisson?
GOAL #2: Traffic-oblivious dynamic MPL control policies
?
13
Where we are…
GOAL: Optimal MPL for M/G/PS-MPL
SUBGOAL: Approximation for M/G/PS-MPL
What if the traffic rate is unknown?What if the traffic is not Poisson?
GOAL #2: Traffic-oblivious dynamic MPL control policies
?
14
RESULT: M/G/PS-MPL approximation
PSServer(n)
1
2
KFIFO buffer
Job sizes ~ X
Poisson()
Easy for X Exp (C2=1)
15
RESULT: M/G/PS-MPL approximation
• Involves only the first two moments of job size distribution
• Generalizes-Lee and Longton approx. for M/G/K multi-server system: (n)= n·-Zhang and Zwart approx. for the case (n)=
THEOREM: The above approx. is exact for the following degenerate hyper-exponential job size distribution:
- 2 degrees of freedom- Can match any E[X] and C2 ≥ 1
16
Where we are…
GOAL: Optimal MPL for M/G/PS-MPL
SUBGOAL: Approximation for M/G/PS-MPL
What if the traffic rate is unknown?What if the traffic is not Poisson?
GOAL #2: Traffic-oblivious dynamic MPL control policies
?
17
Where we are…
GOAL: Optimal MPL for M/G/PS-MPL
SUBGOAL: Approximation for M/G/PS-MPL
What if the traffic rate is unknown?What if the traffic is not Poisson?
GOAL #2: Traffic-oblivious dynamic MPL control policies
?
18
Static MPL policy for M/G/PS-MPL
Set MPL = MPL*, where:
19
3
4
5
6
7
8
4 6 8 10 12 14 16 18
7
9
11
13
4 6 8 10 12 14 16
• Approx accurately predicts the behavior of the curve as a function of MPL
• MPL* decreases as arrival rate increases
(Poisson(0.8) arrivals)
(Poisson(0.9) arrivals)
0.25
0.5
0.75
1
1.25
0 5 10 15 20 25
# jobs at server (n)
Totalspeed(n)
MPL
MPL
Mea
n Re
sp. T
ime
Mea
n Re
sp. T
ime
Our approx.
Weibull (C2=19)
Weibull (C2=19)
K*
K*
MPL*
MPL*
K* gives 25% worse performance than MPL*
K* gives 45% worse performance than MPL*
Our approx.
20
Where we are…
GOAL: Optimal MPL for M/G/PS-MPL
SUBGOAL: Approximation for M/G/PS-MPL
What if the traffic rate is unknown?What if the traffic is not Poisson?
GOAL #2: Traffic-oblivious dynamic MPL control policies
?
21
Where we are…
GOAL: Optimal MPL for M/G/PS-MPL
SUBGOAL: Approximation for M/G/PS-MPL
What if the traffic rate is unknown?What if the traffic is not Poisson?
GOAL #2: Traffic-oblivious dynamic MPL control policies
?
22
Traffic-oblivious policies
• Straw man proposal 1: Choose a “robust” static MPL– Must choose MPL=K* : but suboptimal in light/moderate traffic
• Straw man proposal 2: Learn the arrival rate– Can’t adapt to changes on small time scale/correlations
WE DEMONSTRATE: Dynamic MPL control policies which are1. Traffic-oblivious: self-adapt to variations in the arrival process 2. Light-weight: make decisions based only on current queue length, Q(t), and current MPL, K(t)
I don’t know the arrival rate!!
My arrivals are not Poisson!!
23
Traffic-oblivious policies(KEY STEPS)
GOAL: A robust MPL-control policy πOPT
STEP 1: Approximate the orig. job size distribution
X ≈ _____
STEP 2: Construct candidate policies
π1 π2 π3 … πN
STEP 3: Pick the “best” candidate policy
24
Traffic-oblivious policies(KEY STEPS)
GOAL: A robust MPL-control policy πOPT
STEP 1: Approximate the orig. job size distribution
X ≈ _____
STEP 2: Construct candidate policies
π1 π2 π3 … πN
STEP 3: Pick the “best” candidate policy
25
GOAL: Approximate the job size distribution by a tractable distribution
IDEA: The H* distribution:
• Models many small vs. few large jobs• Tractable due to Exponential branch
Traffic-oblivious policies(KEY STEPS)
For age/size independent policies, H* provides a good approximation for mean response time
p and γ chosen to match
E[X] and C2
26
Traffic-oblivious policies(KEY STEPS)
GOAL: A robust MPL-control policy πOPT
STEP 1: Approximate the orig. job size distribution
X ≈ H*
STEP 2: Construct candidate policies
π1 π2 π3 … πN
STEP 3: Pick the “best” candidate policy
27
Traffic-oblivious policies(KEY STEPS)
GOAL: A robust MPL-control policy πOPT
STEP 1: Approximate the orig. job size distribution
X ≈ H*
STEP 2: Construct candidate policies
π1 π2 π3 … πN
STEP 3: Pick the “best” candidate policy
28
GOAL: A family of candidate dynamic policies
IDEA: 1. Assume arrival ~ Poisson( )2. Solve for
3. Obtain the family by varying
Traffic-oblivious policies(KEY STEPS)
Optimal Dynamic MPL policy to minimize mean response time under H* job sizes
Intuitively :What should this look like?
29
Structure of
0
5
10
15
20
25
0 10 20 30 40 50
Current Queue Length
CurrentMPL
MPL ↓
MPL ↑K*=
obtained by combining policy iteration with some new tricks to handle infinite state space
“Stochastic” component “Fluid”
= Target MPL
30
Traffic-oblivious policies(KEY STEPS)
GOAL: A robust MPL-control policy πOPT
STEP 1: Approximate the orig. job size distribution
X ≈ H*
STEP 2: Construct candidate policies
π(λ1) π(λ2) … π(λN)
STEP 3: Pick the “best” candidate policy
31
Traffic-oblivious policies(KEY STEPS)
GOAL: A robust MPL-control policy πOPT
STEP 1: Approximate the orig. job size distribution
X ≈ H*
STEP 2: Construct candidate policies
π(λ1) π(λ2) … π(λN)
STEP 3: Pick the “best” candidate policy
32
GOAL: Robustly choose a member from family
IDEA: Minimize the worst-case relative deviation in an adversarial setting
Traffic-oblivious policies(KEY STEPS)
a
perf. ratio(,) Mean resp. time of policy under Poisson()
Mean resp. time of OPT static MPL under Poisson()
a* = argmin max perf. ratio((a),) - You pick - Adversary sees , picks - You lose perf. ratio(,)- Optimize against worst adversary
33
Traffic-oblivious policies(KEY STEPS)
GOAL: A robust MPL-control policy πOPT
STEP 1: Approximate the orig. job size distribution
X ≈ H*
STEP 2: Construct candidate policies
π(λ1) π(λ2) … π(λN)
STEP 3: Pick the “best” candidate policy min max OPT
34
Back to picking optimal MPL…
Scenario #1Arrival process ~ Poisson(?)
Scenario #2Arrival process ~ ?(?)
(results for Batch Poisson arrivals)
Worst Perf. Loss(vs. OPT Static MPL)
K*Our
Dynamic Policy
K*Our
Dynamic Policy
- Dynamic policies remain “robust” to non-Poisson arrival process- Static policies become worse
45% 20%
14%50%
35
SUMMARY
• Efficiency ≠ Min. response time
• Static MPL policies via the first approximation for M/G/PS-MPL with state dependent service rates
• First traffic-oblivious MPL control policies for resource sharing systems
See paper formore results
MPL*
MPL ↓
MPL ↑