1

On Control of Queueing Networks and The Asymptotic Variance Rate

of Outputs

Ph.d Summary Talk

Yoni NazarathySupervised by Prof. Gideon Weiss

Haifa Statistics Seminar,November 19, 2008

2

PLANTOUTPUT

The Problem Domain

Finite Horizon [0,T]

Desired:

1. Low Holding Costs

2. Low Resource Idleness

3. Low Output Variability

3

Queues and NetworksA Brief Survey

4

Mean File Size

1 1 1

Phenomena of Queues

5

Key Phenomena• Stability / instability

• Congestion increases with utilization

• Variability of primitives causes larger queues

• Steady state

• Little’s law

• Flashlight principle

• State space collapse

…

6

Queueing Networks

7

Multi-Class

=2

8

Infinite Inputs

9

Miracles

10

PLANTOUTPUT

The Problem Domain

Finite Horizon [0,T]

Desired:

1. Low Holding Costs

2. Low Resource Idleness

3. Low Output Variability

11

Sta

cked

Que

ue L

evel

s

time T

Q1

Q2Q3

Trajectory of a single job

Finished Jobs

Server 1Server 2

1

23

3

10

( )T

kk

Q t dtAttempt to minimize:

Near Optimal Finite Horizon Control

12

1 2 3

0

1 1 1 1

0

2 2 1 1 2 2

0 0

3 3 2 2 3 3

0 0

1 3

2

min ( ) ( ) ( )

( ) (0) ( )

( ) (0) ( ) ( )

( ) (0) ( ) ( )

( ) ( ) 1

( ) 1

( ), ( ) 0

T

t

t t

t t

q t q t q t dt

q t q u s ds

q t q u s ds u s ds

q t q u s ds u s ds

u t u t

u t

u t q t

s.t.

Separated Continuous Linear Program (SCLP)

Fluid RelaxationServer 1Server 2

1

23

13

• SCLP – Bellman, Anderson, Pullan, Weiss• Piecewise linear solution• Simplex based algorithm, finite time (Weiss)• Optimal Solution:

0 10 20 30 40

0

5

10

15

203 3

2 2

1 1

1 3

2

(0) (0) 15

(0) (0) 1

(0) (0) 8

1.0

0.25

40

Q q

Q q

Q q

T

3( )q t

2 ( )q t

1( )q t

Fluid Solution

14

3

1

2

3

1

2

3

1

2

3

1

2

0 10 20 30 40

5

10

15

20

25

30

31 1 10 0 1 0 14 4 4 4

{1,2,3} {1,2,3} {1,3} {1}nK

0 { | ( ) 0, }nk nk q t t

{ | ( ) 0, }nk nk q t t

Fluid Tracking1 2 3 4

15

0 10 20 30 400

500

1000

1500

2000

0 10 20 30 400

500

1000

1500

2000

0 10 20 30 400

500

1000

1500

2000

0 10 20 30 400

500

1000

1500

2000

0 10 20 30 400

50

100

150

200

0 10 20 30 400

50

100

150

200

0 10 20 30 400

50

100

150

200

0 10 20 30 400

50

100

150

200

0 10 20 30 400

5

10

15

20

0 10 20 30 400

5

10

15

20

0 10 20 30 400

5

10

15

20

0 10 20 30 400

5

10

15

20

1N

10N

100N

seed 1 seed 2 seed 3 seed 4

Asymptotic Optimality

16

PLANTOUTPUT

The Problem Domain

Finite Horizon [0,T]

Desired:

1. Low Holding Costs

2. Low Resource Idleness

3. Low Output Variability

17 2 ( )Q t

4 ( )Q t

1S

2S

• 2 job streams, 4 steps

• Queues at 2 and 4

• Infinite job supply at 1 and 3

• 2 servers

The Push-Pull Network

1 2

34

1S 2S

2 4( ), ( )Q t Q t• Control choice based on

• No idling, FULL UTILIZATION

• Preemptive resume

Push

Push

Pull

Pull

Push

Push

Pull

Pull

2Q

4Q

18

Configurations• Inherently stable network

• Inherently unstable network

Assumptions

(A1) SLLN

(A2) I.I.D. + Technical assumptions

(A3) Second moment

Processing Times

Previous Work (Kopzon et. al.):

{ , 1,2,...}, 1, 2,3,4jk k j k

1 2

34

1 1lim , a.s. 1, 2,3, 4

nj

kj

nk

kn

2 1 2Var( ) , 1, 2,3,4k k kc k

1 ~ exp( ), 1, 2,3,4k k k

1 2

4 3

1 2

4 3

19

Policies

1 2

4 3

Inherently stable

Inherently unstable

Policy: Pull priority (LBFS)

Policy: Linear thresholds

1 2

4 3

1 2

34

TypicalBehavior:

2 ( )Q t

4 ( )Q t

2,4

1S 2S

3

4

2 1

1,3

TypicalBehavior:

5 0 1 0 0 1 5 0 2 0 0 2 5 0 3 0 0

5

1 0

2 2 4Q Q

4 1 2Q Q

Server: “don’t let opposite queue go below threshold”

1S

2S

Push

Pull

Pull

Push

1,3

20

KSRS

1 2

34

21

Push pull vs. KSRS

Push Pull

KSRS with“Good” policy

22

Stability Result

( ) Q(t), U(t)X t

1 2

34

Queue Residual

is strong Markov with state space

( )X t

Theorem: Under assumptions (A1) and (A2), X(t) is positive Harris recurrent.

Proof follows framework of Jim Dai (1995)

2 Things to Prove:

1. Stability of fluid limit model

2. Compact sets are petite

Positive Harris Recurrence:

23

PLANTOUTPUT

The Problem Domain

Finite Horizon [0,T]

Desired:

1. Low Holding Costs

2. Low Resource Idleness

3. Low Output Variability

24

Example 1: Stationary stable M/M/1, D(t) is PoissonProcess:( )

Example 2: Stationary M/M/1/1 with . D(t) is RenewalProcess(Erlang(2, )):

Variability of OutputsVariability of Outputs(1)Vt B o

Asymptotic Variance Rate

of Outputs

t

1( , )D t

3( , )D t

t1( , )X t

3( , )X t 2( , )X t

2( , )D t

Var( ( ))D t

V

21 1 1Var( ( ))

4 8 8tD t t e

Var( ( ))D t t

2

3V

m

For Renewal Processes:

25Taken from Baris Tan, ANOR, 2000.

Previous Work: NumericalPrevious Work: Numerical

26

**

* *

VV

V V

BRAVO Effect

27

0 .2 0 .4 0 .6 0 .8 1 .0 1 .2

0 .2

0 .4

0 .6

0 .8

BRAVO Effect: A Phenomena

Using a “renewal-reward” method for regenerative simulation for .V

Queues with Restricted Accessibility (Perry et. al.)

V

28

Summary of ResultsQueueing System Without Losses Finite Capacity Birth Death Queue

Push Pull Queueing Network Infinite Supply Re-Entrant Line

1*

0

K

ii

V v

stable

BRAVO (?) critical

instable

arrivals

service

V

V

V

1 2

Explicit Expressions

for , V V1

1

2

3

kk C

kk C

V

m

V

Diffusion LimitsDiffusion Limits

Matrix Analytic MethodsSimple

29

Infinite Supply Re-entrant Line

4

2

1C

1 3

56

78

10 9

( )D t

2C 3C

4C

2

13

1

: For any stable policy (e.g. LBFS): .k

k C

mkk C

Thm V

1

1Infinite QueuesSupply

1

1

2 21

1

1 {2,..., } ... ,

1 .

Means: ,...,

Variances: ,...,

1, i=2,...,Ii

I

k

k

kk C

i kk C

K C C

C

m m

m

m

30

“Renewal Like”

4

2

1C

1 3

56

78

10 9

2C 3C

4C1

1

2

3

kk C

kk C

V

m

1C

1

6

8

10

Renewal Output1 1 1 1 2 2 2 2 3 3 3 31 6 8 10 1 6 8 10 1 6 8 10

Job 1 Job 2 Job 3

, , , , , , , , , , , ,....x x x x x x x x x x x x

1 1 1 1 2 2 2 3 3 3 31 6 8 10 6 8 1 1 6 8 10

201, , , , , , , , , , , , ,...x x x x x x x x x x xx

31

A Future Direction

32

Finite QRate

1Infinite Q

Rate2

α

α

1

Steady State Total Mean Queue

Sizes

An Implication of BRAVO?

?

IT DOESN’T “WORK

”

Finite QRate1/4

Rate1/4

Finite Q

Finite Q Infinite QRate

2

Rate1/2

Infinite Q

Poisson(α)

Overflow

Overflows Priority

Infinite QRate

1

α

Steady State Mean Queue

Sizes

11/4

When rate exceeds ¼

overflows of first queue cause the second server to

mostly give priority to the fast

stream.

Non Monotonic Networks

?

34

Now Lets Do!לחיים