Upload
daisy-houston
View
219
Download
0
Embed Size (px)
Citation preview
Eurecom, Sophia-AntipolisThrasyvoulos Spyropoulos / [email protected]
Introduction to Continuous-Time Markov Chains and Queueing Theory
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
From DTMC to CTMC
DTMC1. Transitions at discrete time steps
n=0,1,2,…
2. Past doesn’t matter (Markov property) P(Xn+1|Xn,Xn-1,…X1) = P(Xn+1|Xn)
3. P(Xn+1 = j|Xn = i) = pij (stationarity)
2
0
p00 p01
1 2p10 p21
p12
… kk-1
pk,k-1
pk-1,k
k+1
pk,k-1
pk-1,k
…
CTMC1. Transitions between states can
happen any time
2. Past still doesn’t matter
3. Transition still independent of time t
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
CTMC: A Generic Definition
A CTMC is continuous time stochastic process {X(t), t ≥ 0}, s.t. for all s,t ≥ 0, and any i,j,x(u) P{X(t+s) = j|X(s) = i,X(u) = x(u), 0≤ u≤ s}= P{X(t+s) = j|X(s) = i} (Markov property)= P{X(t) = j| X(0) = i} (stationarity)
τi : time until CTMC leaves state i, if currently in i
Above def. implies: P{τi > t+s | τi > s} = P{τi > t} Q: What does this imply?
A: τi is exponentially distributed!
3
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
CTMC: Viewpoint 1
Step 1: When at state i, wait an exponentially distributed amount of time denote rate out of: vi
Step 2: When the process leaves i, it will enter j with probability pij
4
i
exp(vi)
l
k
pil
j
pik
pij
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
CTMC: Viewpoint 2 (more common)
When at state i: move to state j with rate λij (independently)
Q: Time to leave state i? first move to any state first phone rings
A: Exponential with rate (vi = λi1 + λi2 +…+λin = Σj λij)Q: Probability to go to state k, after leaving i?A:
5
0
3
2
λ03
1
λ02
λ01exp(λ01)
exp(λ02)
exp(λ03)
j ij
ijij λλpView 2 => View 1
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
A Single Server Example
6
Poisson (λ)e.g. 3 jobs/sec
service: exp(μ)e.g. 5 jobs/sec
0
λ
1 2
μ
…
λ
μ
λ
μ
Let πj = limt∞Pij(t) : limit probability of being in state j
Q: How do we determine πj ? how do we “solve” the chain?
A: Assume an equivalent DTMC “half-rigorous” argument
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
CTMC DTMC with time unit δ
Divide time into very small intervals δ Prob. of an arrival in the next interval δ : λδ + o(δ) Prob. of a departure in the next interval δ : μδ + o(δ) P{ii+1} = λδ (1-μδ) = λδ + o(δ) (1 arrival + 0
departure)
P{ii-1} = μδ (1- λδ) = μδ + o(δ) (0 arrival + 1 departure)
P{ii} = 1- λδ -μδ +o(δ) (no event with almost prob 1) we can ignore self-loops in the balance equations
7
0
λ
1 2
μ
…
λ
μ
λ
μ
0
λδ+o(δ)1 2
μδ+o(δ)…
λδ+o(δ)μδ+o(δ)
CTMC DTMC (time step δ 0)
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
Solving the Balance Equations for a CTMC Methodology
Step 1) Write balance equations
Step 2) Take limit as δ 0 For state 0 rate out = rate in
8
0
λδ+o(δ)1 2
μδ+o(δ)…
λδ+o(δ)μδ+o(δ)
ο(δ)μδ πο(δ)λδπ 10
δο(δ)μπlim δο(δ)λπlim 10δ00δ
δο(δ)μ πδο(δ)λπ 10
0110 πμλπμ πλπ
For state 1 rate out = rate in
ο(δ)μδπο(δ)λδ πο(δ)μδο(δ)λδπ 201
δο(δ)μπδο(δ)λ πδο(δ)μλπ 201
)01 πμλ(π
201 μπλ πμλπ
02200 πμλπμπλ πμλπμλ 2
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
Solving a CTMC – Our first Queueing Result! Lesson from previous example: can simply solve the
balance equations, rate in = rate out See book (MHB, CH.13) for generalization Can also try local balance equations Need to check for irreducibility (for πi to exist)
- What about periodicity??
Back to our queue)
Define ρ = λ\μ (utilization) (we derived earlier)
(normalize to get π0)9
Poisson (λ) service: exp(μ)
0n0n
n πρπμλπ
1
0n nπ ρ)ρπ nn 1(
Our first continuous time queueing result!!
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
Introduction to Queueing: M/M/1
The example so far was an M/M/1 system “M” stands for memoryless --- both arrivals and departures “1” for single server πn : number of customers in system (in service or queue)
Q: What is the probability that the server is empty?A: π0=1-ρQ: What is the expected number of customers?
A:
10
Poisson (λ) service: exp(μ)
d
d 1( 1( 1(
0000
nnnn
n1-nnn ρρρ)ρρnρ)ρρ)ρnπn E[N]ρ)ρρ)ρ)ρρρρ)ρ 2
1(
E[N] 1(
1 1(
1
1
d
d 1(
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
M/M/1 continued
Q1: What happens when ρ 1 ? (i.e. system very busy)A: (queue goes to infinity!)
Q2: What happens to the Markov Chain when ρ ≥ 1 ?A: All states become transient πi = 0 (for all i)
Null-recurrent for ρ = 1 But this means we cannot apply balance equations (no
balance!) Could we have guessed it from the MC?? (homework!)
Q3: We know E[N]; What is the expected delay E[T]?A: Little’s Law will tell us that E[T] = E[N]/ λ
11
E[N] ρ1ρE[N] 1ρ
Eurecom, Sophia-AntipolisThrasyvoulos Spyropoulos / [email protected]
Introduction to Queueing Theory
12
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
Queueing Systems: Kendall’s Notation A queuing system is described by A/D/n/B/S A: the arrival process
M: for Memoryless Poisson arrivals D: Deterministic arrivals at fixed intervals G: generic distribution
D: service process M: Memoryless duration of service is exponentially
distributed D: Deterministic each service last the same fixed amount of
t G: generic distribution (e.g. uniform, pareto, gamma, etc.)
n: number of servers B: size of queue/buffer
If queue is full additional arrivals are rejected No B in notation infinite queue (e.g. M/M/1, M/G/1, G/G/4)
S: Scheduling policy FCFS (default assumed), preemptive, random, etc.
13
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
Queue Stability
For the system to be stable: λ < μQ: What happens if λ > μ?A: Queue size goes to infinity (and so does delay!)
Q: Can you give the intuition?A: Consider an interval [0,t] (large t)
Number in system = (arrivals in [0,t]) – (departures in [0,t]) Þ Number in system = λt – μt = (λ- μ)tÞ t -> ∞ queue size -> ∞
14
Poisson (λ) service: exp(μ)
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
Throughput of a Queueing System
Q: Which of the two systems has a higher throughput?A: The throughput is the same for both: X = λ = 1/6 Consider a busy period of the system
Def: from time an arrival finds system empty, until it empties again
Assume A arrivals during a busy periodQ: How many departures during that busy period?A: Exactly A!
All arrived jobs should complete for the system to empty
15
Throughput X: jobs/min leaving the system
Open (stable) system: Throughput X = Input Rate
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
Little’s Law (or the most famous Queueing
Result)
E[N] : number of customers in the system (queue + server)
E[T]: time a customer spends in the system (queue+server)
Little’s Law: E[N] = λ E[T]
M/M/1:
(some) Intuition behind Little’s Law Customer X arrives at the end of a queue on average stays E[T] Q: how many customers have arrived while X was in the system? A: λ E[T] => on average a departure leaves E[N] = λ E[T] customers in the
system
16
Poisson (λ) service: exp(μ)M/M/1:
λμ1λ1ρ1ρλE[N]E[T]
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
Proof of Little’s Law
DEF: A is the area inside boxes A(t): num of arrivals by t D(t): num of departures by t
1. Sum times horizontally
2. Sum times vertically:
17
A(t)
1i iD(t)
1i i TAT(s)dsNA t
0
t TtN(s)dst TTN(s)dsT A(t)1i it0D(t)1i iA(t)1i it
0D(t)1i i
λE[T]E[N] λTNXT tA(t)A(t)TtN(s)dstD(t)D(t)T ergodicitytA(t)1i it
0D(t)1i i
area A
N(t1)=2
t1
N(t2)=3
t2
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
Generality of Little’s Law
Q: Did we assume anything about the number of servers?
A: No!
Q: Did we assume that scheduling is first-come first-serve (FCFS)?
A: No! In fact, in the previous example 3 finishes before 2!
Little’s Law applies to ANY system and ANY scheduling Many servers, many queues, queueing network, scheduling
policy
18
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
Little’s Law: Additional Results
Q: What about E[TQ]? number of jobs in queue – not server(s)
A: λE[TQ] = E[NQ] (similar proof)
Q: What about ρ (number of customers in service)?A: λE[S] = ρ ρ = λ/μ Q: What is ρ?A: System utilization -> fraction of time server(s) is busy
(time average)A: Ergodicity Prob{server is busy at a random time}
(ensemble average)
19
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
M/M/k/k loss system
Consider a call center with k total lines E.g. an actual call center, or switch with k circuits, or base
station with k channels
If all servers/lines busy call/job is rejected (Key Question): What is the probability of “blocking”?
20
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
(continuous-time) Markov Chain for M/M/k/k
21
Q: How many states do we need?A: k states
Q:What is the rate from i to i+1?A: λQ:What is the rate from i to i-1?A: kμ
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
Local balance equations for M/M/k/k
Q: Is the chain time-reversible (to apply local balance)? i.e. is rate from 12 = rate from 21?
A: Yes, it is so, we can apply local balance
22
0110 πμλ π μπλπ-
02
221 π2!1μλ π 2μπλπ-
0i
ii1-i πi!1μλ π iμπλπ-
k
i 0
11
i!1μλ π π i0i i
k
i 0 i!1μλk!1μλ πP i
kkblock
Erlang B
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
What about general service (M/G/k/k)?
Q: Job size/duration is not exponential?A: Turns out that the state probabilities πi is as in M/M/k/k
The block probability is the same and given by Erlang B
This is an important insensitivity result Result is insensitive to service distribution (only depends on
mean μ) Insensitivity results often arise when there is no queueing
23
k
i 0 i!1μλk!1μλ πP i
kkblock
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
M/M/k
24
Dispatch to first available server
queue if all busy
A very important model for server farms Amazon, Google, CNN, pretty much all major web
sites Cloud computing
Exponential service with rate μ
Poisson (λ)
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
CTMC for M/M/k
Q: How many states?A: Infinite states
Q: What is the rate from i i+1A: λQ: What is the rate from i i-1 (i ≤ k)A: iμQ: What is the rate from i i-1 (i > k)A: kμ
25
…
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
Local Balance Equations for M/M/k
26
…
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
M/M/k: Utilization and Queueing
Q: What is the expected number of busy servers?A: λ /μ- Definition: ρ = λ /kμ (utilization of a single server)
Q: What is the probability (PQ) an arriving job must queue?
A: PQ = P{arrival finds all servers busy}= P{arrival sees ≥ k jobs in system}(from PASTA property)
27
ki iπ ki
ki0k kμλπk!1μλ
kiki0
kk ρπk!kρ
i
i0kk ρπk!kρ
0
11k 0iki
00k
Q ρ)(1k!kρi!kρ πwhere ,πρ)-(1k!kρP
Erlang C:
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
M/M/k Queueing vs. M/M/k/k Blocking
Q: For which system (M/M/k and M/M/k/k) is the probability that all k server are busy higher?
A: For the M/M/k
Q: Why?A: In M/M/k jobs can arrive when all k servers are busy
these do not disappear but queue up more work for later In M/M/k/k these jobs are simply rejected Can confirm by comparing Erlang B and C (work the math as
HW)
28
kρρ1ρPE[N] Q
μ1λkμ1PλE[N]E[T] Q
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
(Back to) Comparing Channel Allocations
…or server organizations…or supermarket lines…
Q: Which is better in terms of E[T], FDM or M/M/1 (statistical multiplexing)
A: M/M/1 E[TFDM] = k/(kμ-λ) but E[TM/M/1] = 1/(kμ-λ) 29
Thrasyvoulos Spyropoulos / [email protected] Eurecom, Sophia-Antipolis
(Back to) Comparing Channel Allocations (cont’d)
…or server organizations…or supermarket lines…
Q: What about E[TM/M/1] and E[TM/M/k]?
A:
30
ρ)k(1P]E[T ]E[T M/M/kQM/M/1M/M/k
ρ ≈ 0ρ ≈ 1k]E[T ]E[T
M/M/1M/M/k
1]E[T ]E[TM/M/1M/M/k