Lecture 10:Lecture 10:

Queueing TheoryQueueing Theory

Queueing Analysis

Jobs serviced by the system resources

Jobs wait in a queue to use a busy server

queue server

Queueing Network Models

Open Network Model

Jobs arrive in the system from an external resource and depart the system

Closed Network Model

Has a fixed number of jobs that circulate among various queues and servers

Queueing Analysis s average service time average service rate (1/s) average arrival rate ((number of jobs arrive in time T) / T) traffic intensity (/) r average response time (waiting time + service time) w average waiting time in queue q average number of jobs in the queue n number of jobs in the system (waiting & being serviced) U utilization (busy time / total time) a number of arrivals within interval T d number of departures within interval T

Operational Analysis

Average arrival rate = (requests/s)

Utilization U =

U = s Utilization Law

U = = Traffic intensity

Computer system

Job arrival rate ( Job service rate (

Operational Analysis

Ex: A single disk system

Average time to service each request is 600 s.

What is the maximum possible request rate it can tolerate?

Operational Analysis

Utilization U = s

U = = Traffic intensity

> 1 arrival rate > service rate

number of jobs in the queue increase without bound

< 1 arrival rate < service rate

stable system

Operational Analysis

Little’s Law

D = w T

D = q T

q = w Little’s Law

q = w U = s n = r

Operational Analysis

queue server

Operational Analysis

Ex:

Average response time of a web server is 11 ms.

Request rate is 450 requests/s

What is the average number of requests that are buffered in the system?

Stochastic Analysis

Kendall’s Notation

A/S/c

A – arrival process

most common arrival process is Poisson process:

interarrival times follow exponential distribution

exponential distribution is memoryless

S – service process

Poisson service process is the most common

c – number of servers

Stochastic Analysis

Interarrival time: Time between successive jobs arriving at the queue

Ex:

Single queue

Exponential interarrival time

Exponential service time

M/M/1queue server

M/M/1

Markov Chain

State transition diagram if a birth-death process:

M/M/1

Flow Equations

Pi – probability of being in state i

Flow from state n-1 to state n: Pn-1

Flow from state n to state n-1: Pn

Pn-1 = Pn

Pn = Pn-1 = n P0

M/M/1

P0 = 1/n

< 1, 1/n converges to 1-

Pn = n ()

M/M/1

Average number of jobs in the system:

E[n] = nPn = /(1-

Response time: r = =

Average number of jobs waiting in the queue: q = 2/(1-)

Average waiting time in the queue: w = /(1-)

Probability of finding k or more jobs in the system:

Pr(K≥k) = k

M/M/1

Ex:

In a web server, the time between requests follow exponential distribution with mean time between requests of 8 ms.

Time required by the server to process each request is also exponential distribution with average service time 5 ms.

a. What is the average response time observed?

b. How much faster must the server process requests to halve this average response time?

c. How many queue entries are necessary to buffer requests so that no more than one request per billion is lost?

M/M/c

M/M/c

c – number of servers

– job arrival rate

n – jobs in the system (both on server & in queue)

– service rate for each server

queue server

. . .

M/M/c

If all servers are busy, mean service rate is: c

If some servers are idle, mean service rate is: n(n<c

M/M/c

Probability of finding n jobs in the system:

(c)nP0 / n! n=1,2,… c

Pn =

(c)nP0 / c! cn-c n>c

Total traffic intensity: = c (utilization of each server)

Probability of all servers being busy:

= Pr(K≥c) = (c)cP0 / c!(1-)

M/M/c

Average number of jobs in the system:

E[n] = c + /(1-)

Average number of jobs waiting in the queue:

q = () / (1-)

Little’s law:

r = 1/c

Average time jobs wait in the queue:

w = c

M/M/c

Ex:

How does the response time in M/M/1 queue example change if the number of servers in the system is 4?