Page 1Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Things I Thought I Knew About Queueing Theory, but was Wrong About: Part 1, Multiserver Queues

Alan Scheller-WolfTepper School of Business Carnegie Mellon University

Joint work withKarl Sigman, Rein Vesilo, and Michele Dufalla

Page 2Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

A Great Day

As a graduate student, one of the greatest things that can happen is the following:Student: “I can prove that XXX is true!”Advisor: “That can’t be right, everyone knows YYY is true.”Student: “I know, but I can prove it.”Advisor: “There must be a bug, show me.”Time passes...Advisor: “Huh, you are right, XXX is true!”

Page 3Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

My Goals

• Take you through “my great day” as a student• Help you understand why everyone believed YYY• Help you understand, mathematically, why XXX is

true• Help you see why, intuitively, XXX “has to be

true”• Discuss what general insights this gives

Page 4Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Mean Delay in GI/GI/1 Queues

i.i.d. arrivals ~ T

rate l

i.i.d. service ~ S

rate m

• Jobs are served in a FIFO (FCFS) fashion• Delay: Time customers spend in queue• Question: What are conditions for mean

(stationary) delay to be finite?• Kiefer and Wolfowitz (K&W, 1956)

r =l/m < 1E[D] finite iff E[S2] finite

Single Server Queue

S should be “heavy tailed” -- think Pareto distribution

Page 5Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Why is E[S2] Necessary (for Single-Server Queues)?

• The inspection paradoxA randomly arriving job sees an equilibrium service

time Se (or the excess of a service time)Length biasing causes the tail of Se [P(Se>x)] to behave

as the integrated tail of S – you are more likely to “see” longer service times

E[Se] is finite iff E[S2] is finite

• The upshot: If an arriving job “sees” a job in service, this job has mean remaining size on the order of E[S2]. The arriving job can’t enter service until this job is done.

Page 6Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Why is E[S2] Sufficient (for Single-Server Queues)? (I)

• The Lindley Equation (Lindley, 1952)Dn+1 = (Dn+Sn-Tn)+, where Sn is the service time of

customer n and Tn is the interarrival time between customers n and n+1

Why?Let’s say customer n arrives at time tThen customer n leaves service at time t+Dn+Sn

Customer n+1 arrives at time t + Tn

Customer n+1 enters service at time min{t+Dn+Sn,t + Tn}

Customer n+1’s delay is min{t+Dn+Sn,t + Tn} – (t + Tn)This is a reflected random walk

Page 7Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Why is E[S2] Sufficient (for Single-Server Queues)? (II)

Reflected Random Walks:• For Zn+1 = (Zn+Xn)+, E[Z2] is finite if:

E[(X+)2] is finite E[Xn] < -d < 0 for all n and on every sample path

• For our GI/GI/1 queue:E[(X+)2] = E[S2] is finite E[Xn] = E[S] - E[T] < -d < 0 ; for all n and on every

sample path, since r<1

Page 8Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Mean Delay in GI/GI/k Queues

i.i.d. arrivals ~ T

rate l

i.i.d. service ~ S

rate m

• Question: What are conditions for mean stationary delay to be finite?

• K&W, 1956 r =l/ km < 1E[D] finite if E[S2] finite

Multi-Server Queue

Page 9Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Why is E[S2] Sufficient?

• Split the GI/GI/k into k GI/GI/1 queuesThis performs no better than the original system

• Route arriving jobs to queues in cyclic (or round robin) fashion

• Each GI/GI/1 queue has finite mean delay• Therefore the original GI/GI/k queue has finite

mean delay as well

Page 10Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

What about Necessity?

“Everyone knew” that r =l/ km < 1

E[S2] finite were necessary as well.

This had even appeared in the literature.

Page 11Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Pictorially:

Requirement for finite mean delay

1

2

Moments of S

Servers1 2 3 4

∞

Page 12Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Back to the GI/GI/1

• Why is finite E[S2] Necessary?• The inspection paradox

If an arriving job “sees” a job in service, this job has mean remaining size on the order of E[S2]. The arriving job can’t enter service until this job is done.

• Does this logic hold for the GI/GI/k?NO!

• Can we capture this effect mathematically?

Page 13Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Lindley Equation for GI/GI/1

• Our logic for GI/GI/1Let’s say customer n arrives at time tThen customer n leaves service at time t+Dn+Sn

Customer n+1 arrives at time t + Tn

Customer n+1 enters service at time min{t+Dn+Sn,t + Tn}

Customer n+1’s delay is min{t+Dn+Sn,t + Tn} – (t + Tn)

• Dn+1 = (Dn+Sn-Tn)+

Page 14Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Lindley Equation for GI/GI/k

• For GI/GI/kLet’s say customer n arrives at time tThen customer n leaves service at time t+Dn+Sn

Customer n+1 arrives at time t + Tn

Customer n+1 enters service at time min{t+Dn+Sn,t + Tn}

NO!

Page 15Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

When Does Customer n+1 Enter Service? (I)

• Customer n arrives at time t• Customer n leaves service at time t+Dn+Sn

nn+1

j

Page 16Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

When Does Customer n+1 Enter Service? (II)

• When does customer j leave service?Let’s say that when customer n enters service, she

looks over at the other server, and observes the amount of time customer j has left

Let’s call this amount of time Un

Then customer j leaves service at time t+Dn+Un

nn+1

j

If there were multiple other servers, Un would

be the minimum remaining time

Page 17Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

When Does Customer n+1 Enter Service? (III)

nn+1

j

• Customer n leaves service at time t+Dn+Sn

• Customer j leaves service at time t+Dn+Un

• So the next server comes free at time t+Dn+min{Sn,Un} = t+Dn+Pn

Page 18Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Lindley Equation for GI/GI/k

• For GI/GI/k:Dn+1 = (Dn+Pn-Tn)+

How big of difference could one letter make?

We have replaced Sn with Pn

Page 19Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Back to Reflected Random Walks

• For our GI/GI/k queue, for finite mean delay it is sufficient that:E[(X+)2] = E[P2] is finite E[Xn] = E[Pn] - E[Tn] < -d < 0 ; for all n and on every

sample path

How does this help? We don’t know the first thing about P!

Page 20Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

When is E[P2] Finite?

E[P2] = E[min{S,U}2]= E[min{S,U}3/2min{S,U}1/2]< E[S3/2U1/2] = E[S3/2] E[U1/2]

And E[U1/2] < E[W1/2]And E[W1/2] is finite iff E[S3/2] (K&W, 1956)

So for a GI/GI/2 queue, E[P2] is finite if E[S3/2] is finite!

S is independent of U

W is the total work in system

Page 21Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

What about Negative Drift?

E[Xn] = E[Pn] - E[Tn] = E[min{Sn,Un}] - E[Tn] < E[Sn] - E[Tn] = E[S] - E[T]

This seems like a reallyweak bound, but is in fact

the best we can do, since this must hold on all sample paths

and for all n; Un could bearbitrarily large on a sample

path

I have negative drift for all n on every sample path if E[S] < E[T] (r < ½)

but not otherwise

Page 22Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Finite Mean Delay in GI/GI/2 (I)

So it appears that:• Finite E[S3/2] is sufficient for finite mean delay in

GI/GI/2 queues if E[S] < E[T]

• Finite E[S3/2] is not sufficient for finite mean delay in GI/GI/2 queues if E[T] < E[S] < 2E[T]

Does this make sense?

YES!

Page 23Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Finite Mean Delay in GI/GI/2 (II)

• The inspection paradoxIf an arriving job “sees” a job in service, this

job has mean remaining size on the order of E[S2]

The mean time this job has been in service also is on the order of E[S2]

The key question is what has been going on while this long job has been in service

Page 24Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Finite Mean Delay in GI/GI/2 (III)

• If E[T] < E[S] < 2E[T]: While a single extremely large job is in service

the remaining server cannot keep up with the workload, and the system “crashes”

• If E[S] < E[T]:A single extremely large job is not typically

enough to crash the system – the remaining server can keep up with the workload

It takes two extremely large jobs to crash the system

Page 25Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Redefining r

• For a k-server queue r is traditionally defined as r =l/ km

This is WRONG!

• If instead r =l/m E[S] < E[T] means r<1, and we have a “spare server”

to absorb the workload if a long job arrivesE[T] < E[S] < 2E[T] means 1<r<2, and we have no

spare capacity

r =l/m denotes the number of servers needed to handle the mean load on the system

Page 26Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

What about GI/GI/3?

• If 2<r<3 (no spare servers):E[D] is finite if E[S2] is finite

• If 1<r<2 (one spare server):E[D] is finite if E[S3/2] is finite

• If 0<r<1 (two spare servers):E[D] is finite if E[S4/3] is finite

Page 27Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

What about GI/GI/k?

• If k-1<r<k (no spare servers):E[D] is finite if E[S2] is finite

• If k-2<r<k-1 (one spare server):E[D] is finite if E[S3/2] is finite

• If j<r<j+1 (k-j-1 spare servers):E[D] is finite if E[S(k+1-j)/(k-j)] is finite

Page 28Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Pictorially: Fixed Number of Spare Servers

Requirement for finite mean delay

1

2

Moments of S

Servers1 2 3 4

∞

No spare servers

One spare serverTwo spare servers

3/24/3

Diminishing benefits

For moment conditions, 20-server queue with load 18.5 is the same as a 2-server queue

with load 0.5

Pictorially:

Page 29Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Pictorially: As Spare Servers Grow

Requirement for finite mean delay

1

2

Moments of S

Servers1 2 3 4

∞

3/24/3

As the number of spare servers approaches infinity, the moment conditions approach those of an

infinite-server queue, as they should

Page 30Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Other Results (I)

• Higher Moments: If j<r<j+1 (k-j-1 spare servers):

E[Dr] is finite if E[S(k+r-j)/(k-j)] is finite(r has replaced 1 in the previous formula)

If you have just two spare servers, E[D2] is finite if E[S5/3] is finite – you get finite variance of delay without finite variance of service time

Page 31Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Other Results (II)

• Necessary Conditions: The sufficient conditions above are also

provably necessary for all but a small set of service time distributions

Page 32Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Other Results (III)

• Integral Load: Drift has been crucial in our analysis – if r<1

E[S3/2] is sufficient for finite mean delay; if r>1 we require E[S2]

What if r=1? We have been able to show that E[S3/2] is not

sufficient, but have not been able to get close to showing E[S2] is necessary

We are still working

Page 33Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Insights• Delay moments in multi-server queues with heavy-

tailed service times depend on the moments of S and the number of spare serversr is most usefully defined as l/m for multi-

server queues • Adding spare serves helps with delay moments,

but in diminishing fashion• This works because under heavy-tailed service

times queues build up due to large jobs blocking servers

Page 34Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Open Problems

1. Dependencies:Everything we showed was for GI/GI/k; what if

interarrival and/or service times are not i.i.d?

Our “Lindley” equation for the GI/GI/k:Dn+1 = (Dn+Pn-Tn)+

also holds for a special type of GI/G/1 queue. (It actually defines a GI/G/1 queue.) So there must be a relationship/duality here.

Page 35Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Open Problems (II)

2. Bounds: As of now, the best analytical bounds on multi-

server delay under heavy-tailed service time are in general very poor

These cannot, in general, be easily simulated See work by Daley, Scheller-Wolf, Vesilo

Page 36Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Open Problems (III)• Tail Asymptotics

Moment conditions for delay are really just a proxy for what we really want to know: How the tail of the delay distribution behaves

See work by Boxma, Foss, Korshunov, Whitt, Zwart

Page 37Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

ReferencesScheller-Wolf, A. (2003). Necessary and Sufficient Conditions for Delay Moments in FIFO Multiserver Queues: Why s Slow Servers are Better than One Fast Server for Heavy-Tailed Systems. Operations Research 51, No. 5, 748-758.

Scheller-Wolf, A. and R. Vesilo (2006). Structural Interpretation and Derivation of Necessary and Sufficient Conditions for Delay Moments in FIFO Multiserver Queues. Queueing Systems, 54, No. 3 221-232.

 

Page 38Alan Scheller-Wolf Lunteren, The

Netherlands

January 15, 2013

Dank U!

Any Questions?