Fundamental Characteristics of Queues with Fluctuating Load VARUN GUPTA Joint with: Mor...

Fundamental Characteristics of Queues with Fluctuating Load

VARUN GUPTA

Joint with:

Mor Harchol-Balter

Carnegie Mellon Univ.

Alan Scheller-Wolf

Carnegie Mellon Univ.

Uri Yechiali

Tel Aviv Univ.

2

Motivation

ClientsServer Farm

Requests

3

Motivation

ClientsServer Farm

Requests

4

Motivation

ClientsServer Farm

Requests

5

Motivation

ClientsServer Farm

Requests

6

Motivation

ClientsServer Farm

Requests

7

Motivation

ClientsServer Farm

Requests

8

Motivation

ClientsServer Farm

Requests

9

Motivation

ClientsServer Farm

RequestsReal

World Fluctuating arrival

and service intensities

10

A Simple Model

HL

exp(H)

exp(L)

HighLoad

LowLoad

11

• Poisson Arrivals• Exponential Job Size Distribution• H/H > L/L

• H>H possible, only need stability

A Simple Model

HighLoad

LowLoad

H,H

L,L

exp()

exp()

HH

LL

12

The Markov ChainP

has

e

Number of jobs

L

H

H

H

0 1

0 1

2

2

L

L

H

H

L

L

. . .

. . .

Solving the Markov chain provides no behavioral insight

13

HH

LL

• N = Number of jobs in the fluctuating load system

• Lets try approximating N using (simpler) non-fluctuating systems

14

HH

LL

Method 1

Nmix

15

HH

LL

Q: Is Nmix ≈ N?

A: Only when 0

Method 1

Nmix

½

½

+

,

16

HH

LL

Method 2

17

avg(H,L)avg(H,L)

Method 2

≡ Navg

Q: Is Navg ≈ N?

A: When ,

18

Example

H=1, H=0.99

L=1, L=0.01

E[Nmix] ≈ 49.5 E[Navg] = 1

0

19

Observations

• Fluctuating system can be worse than non-fluctuating

0 and asymptotes can be very far apart

E[Nmix] > E[Navg]

E[Nmix] E[Navg]

20

Questions

• Is fluctuation always bad?

• Is E[N] monotonic in ?

• Is there a simple closed form approximation for E[N] for intermediate ’s?

• How do queue lengths during High Load and Low Load phase compare? How do they compare with Navg?

More than 40 years of research has not

addressed such fundamental questions!

21

Outline

Is E[Nmix] ≥ E[Navg], always?

Is E[N] monotonic in ?

Simple closed form approximation for E[N]

Application: Capacity Planning

Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase Not covered in this talk

Please read paper.

22

Prior Work

Fluid/DiffusionApproximations

Transforms Matrix Analytic& Spectral Analysis

- P. Harrison- Adan and Kulkarni

Numerical ApproachesInvolves solution of cubic

- Clarke- Neuts- Yechiali and Naor

Involves solution of cubic

- Massey- Newell- Abate, Choudhary, Whitt

Limiting Behavior

But cubic equations have a close form solution…

?

23

Good luck understanding this!

24

Asymptotics for E[N] (H<H)

E[Navg]

E[Nmix]

E[N]

(switching rate)Highfluctuation

H=1, H=0.99

L=1, L=0.01

E[Nmix] > E[Navg]

Lowfluctuation

25

Asymptotics for E[N] (H<H)

E[N]

E[Nmix]

E[Navg]

• Agrees with our example (H = L)

• Ross’s conjecture for systems with constant service rate:

“Fluctuation increases mean delay”

Q: Is this behavior possible?

A: Yes

E[N]

E[Navg]

E[Nmix]

26

Our Results

E[N]

(H-H) > (L-L)(H-H) = (L-L)(H-H) < (L-L)

• Define the slacks during L and H as• sL = L - L

• sH = H - H

E[N]

E[N]

27

Our Results

• Define the slacks during L and H as• sL = L - L

• sH = H - H

• Not load but slacks determine the response times!

sH > sLsH = sLsH < sL

KEY IDEA

E[N]

E[N]

E[N]

28

Outline

Is E[Nmix] ≥ E[Navg], always?

Is E[N] monotonic in ?

Simple closed form approximation for E[N]

Application: Capacity Planning

Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase Not covered in this talk

Please read paper.

29

Outline

Is E[Nmix] ≥ E[Navg], always? No

Is E[N] monotonic in ?

Simple closed form approximation for E[N]

Application: Capacity Planning

Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase Not covered in this talk

Please read paper.

30

Notation

• NH: Number of jobs in system during H phase

• NL: Number of jobs in system during L phase

• N = (NH+NL)/2

H,H

L,L

exp()

exp()NH NL

31

Analysis of E[N]

First steps:

– Note that it suffices to look at switching points

– Express

• NL = f(NH)

• NH = g(NL)

– The problem reduces to finding Pr{NH=0} and Pr{NL=0}

H,H

L,L

NH NL

NL=f(g(NL))

fg

32

– Find the root of a cubic (the characteristic matrix polynomial in the Spectral Expansion method)

– Express E[N] in terms of

E[N] =

The simple way forward…

H,H

L,L

fg

A

A-A

H(L -L)0H+ L(H-H)0

L - (L -L)(H-H)

2 (A -A)+

Where 0L = 0

H = (A-A)

L(-1)(H-H)

(A-A)

H(-1)(L-L)

NH NL Difficult to even prove the monotonicity of E[N] wrt

using this!

33

Our approach (contd.)

• Express the first moment as

E[N] = f1()r+f0()(1-r)

– r is the root of a (different) cubic– r1 as 0 and r0 as

KEY IDEA

34

Monotonicity of E[N]

• E[N] = f1()r+f0()(1-r)

• r is monotonic in E[N] is monotonic in • The cubic for r has maximum power of as 2

1

0

r

35

Monotonicity of E[N]

• E[N] = f1()r+f0()(1-r)

• r is monotonic in E[N] is monotonic in • The cubic for r has maximum power of as 2

1

0

r

Need at least 3 roots for when r=c1

but has at most 2 roots

c1

36

Monotonicity of E[N]

• E[N] = f1()r+f0()(1-r)

• r is monotonic in E[N] is monotonic in • The cubic for r has maximum power of as 2

1

0

r

Need at least 2 positive roots for when r=c2

but for r>1 product of roots is negative

c2

37

Monotonicity of E[N]

• E[N] = f1()r+f0()(1-r)

• r is monotonic in E[N] is monotonic in • The cubic for r has maximum power of as 2

1

0

r

E[N] is monotonic in !

38

Outline

Is E[Nmix] ≥ E[Navg], always? No

Is E[N] monotonic in ?

Simple closed form approximation for E[N]

Application: Capacity Planning

Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase Not covered in this talk

Please read paper.

39

Outline

Is E[Nmix] ≥ E[Navg], always? No

Is E[N] monotonic in ? Yes

Simple closed form approximation for E[N]

Application: Capacity Planning

Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase Not covered in this talk

Please read paper.

40

Approximating E[N]

• Express the first moment as

E[N] = f1()r+f0()(1-r)

– r is the root of a (different) cubic– r1 as 0 and r0 as

• Approximate r by the root of a quadratic

KEY IDEA

KEY IDEA

41

Approximating E[N]

1

3

5

7

9

1.E-05 1.E-04 1.E-03 1.E-02 1.E-01 1.E+00 1.E+0110-5 10-4 10-3 10-2 10-1 100 101

3

5

7

9

E[N]

ExactApprox.

H=L=1, H=0.95, L=0.2

42

Approximating E[N]

1

3

5

7

9

1.E-05 1.E-04 1.E-03 1.E-02 1.E-01 1.E+00 1.E+0110-5 10-4 10-3 10-2 10-1 100 101

3

5

7

9

E[N]

ExactApprox.

H=L=1, H=0.95, L=0.2

43

Approximating E[N]

2

6

10

14

18

1.E-02 1.E-01 1.E+00 1.E+0110-2 10-1 100 10

ExactApprox.

H=L=1, H=1.2, L=0.2

2

6

10

14

18

E[N]

44

Approximating E[N]

2

6

10

14

18

1.E-02 1.E-01 1.E+00 1.E+0110-2 10-1 100 10

ExactApprox.

H=L=1, H=1.2, L=0.2

2

6

10

14

18

E[N]

45

Outline

Is E[Nmix] ≥ E[Navg], always? No

Is E[N] monotonic in ? Yes

Simple closed form approximation for E[N]

Application: Capacity Planning

Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase Not covered in this talk

Please read paper.

46

Outline

Is E[Nmix] ≥ E[Navg], always? No

Is E[N] monotonic in ? Yes

Simple closed form approximation for E[N]

Application: Capacity Planning

Stochastic orderings for the number of jobs seen by an arrival into an H phase, L phase Not covered in this talk

Please read paper.

47

Scenario

Application: Capacity Provisioning

HH

LL

2HH

2LL

Aim: To keep the mean response times same

48

Scenario

Application: Capacity Provisioning

HH

LL

2H2H

2L2L

Question: What is the effect of doubling the arrival and service rates on the mean response time?

49

What happens to the mean response time when , are doubled in the fluctuating load queue?

Halves

Remains almost the sameReduces by less than half

Reduces by more than halfA:

D:C:

B:

50

What happens to the mean response time when , are doubled in the fluctuating load queue?

Halves

Remains almost the sameReduces by less than half

Reduces by more than halfA:

D:C:

B:

51

What happens to the mean response time when , are doubled in the fluctuating load queue?

Halves

Remains almost the sameReduces by less than half

Reduces by more than halfA:

D:C:

B:

Look at slacks!

A: sH = sL

B: sH > sL

C: sH < sL

D: sH < 0, 0

reduces by half more than half less than half remains same

52

Our Contributions

• Give a simple characterization of the behavior of E[N] vs.

• Provide simple (and tight) quadratic approximations for E[N]

• Prove the first stochastic ordering results for the fluctuating load model (see paper)

53

Bon Appetit!

54

Direction for future research

• Analysis of higher moments of response time

• Analysis of bursty arrival process

• General phase type distributions for phase lengths

• Analysis of alternating traffic streams – look at the workload process instead of number of jobs in system

• Conjecture: NH increases stochastically as switching rates decrease

55

Comparison of NL vs. NH

Jackpot!

Honey, I think we chose the wrong time to go out!

56

Stochastic Ordering refresher

• Random variable X stochastically dominates (is stochastically larger than) Y if:

Pr{Xi} Pr{Yi}

for all i.

• If X stY then E[f(X)] E[f(Y)] for all increasing f– E[Xk] E[Yk] for all k 0.

57

Comparison of NL vs NH

• NL ≥st NM/M/1/L

• NH ≤st NM/M/1/H

• NH ≥st NL

• NH ≥st Navg

• NL st Navg

58

Why do slacks matter?

• Fact: The mean response time in an M/M/1 queue is (-)-1

– Higher slacks Lower mean response times

• What is the fraction of customers departing during H

H,H

L,L

exp()

exp()

when ?

H,H

L,L

exp()

exp()

when 0?

H

H + L

H

H + L?

59

Why do slacks matter?

when ?

H,H

L,L

exp()

exp()

when 0?

H

H + L

H

H + L?

• Fact: The mean response time in an M/M/1 queue is (-)-1

– Higher slacks Lower mean response times

• What is the fraction of customers departing during H

H,H

L,L

exp()

exp()

60

Why do slacks matter?

when ?

H,H

L,L

exp()

exp()

when 0?

A H?

• Fact: The mean response time in an M/M/1 queue is (-)-1

– Higher slacks Lower mean response times

• What is the fraction of customers departing during H

As switching rates decrease, larger fraction of customers experience lower mean response times when sH>sL

H,H

L,L

exp()

exp()

61

Q: What happens to E[N] when we double ’s and ’s?

A:System A: , ,

System B: 2, 2,

?

62

Q: What happens to E[N] when we double ’s and ’s?

A:System A: , ,

System B: 2, 2,

System C: 2, 2, 2

E[N] remains same in going from A to C

A) sL = sH : remains same

B) sL > sH : increases, but by less than twice

C) sL < sH : decreases

D) 0, H>1 : queue lengths become twice as switching rates halve, E[N] doubles

63

Example

H=1.9, H=0.99

L=0.1, L=0.01

E[Nmix] ≈ 0.6 E[Navg] = 1

0