Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
1. Consider a Markov process with states 0, 1 and 2 and with the following transition ratematrix Q:
Q =
!
"#!! ! 0µ !!! µ !µ 0 !µ
$
%&
where ! > 0 and µ > 0.
a. Derive the parameters vi and Pij for this Markov process.
b. Determine the expected time to go from state 1 to state 0.
2. Exercise 6.1.3. Exercise 6.3.4. Exercise 6.5.5. Exercise 6.6 (a), (b).
1
1.
a. We havev0 = !, v1 = ! + µ, v2 = µ,
andP01 = P20 = 1, P10 = 1! P12 =
µ
! + µ.
b. Let Ti(i = 1, 2) denote the time to go from state i to 0. Then E[T2] = 1/µ and
E[T1] =1
! + µ+
!
! + µ· E[T2] =
1
µ.
2. Let us assume that the state is (n, m). Male i mates at a rate ! with female j, andtherefore it mates at a rate !m. Since there are n males, mating occurs at a rate !nm.Therefore
vn,m = !nm.
Since any mating is equally likely to result in a female as in a male, we have
P(n,m);(n+1,m) = P(n,m);(n+1,m) =1
2.
3. This is not a birth and death process since we need more information than just thenumber working. We must also know which machine is working. We can analyze it byletting the states be
b : both machines are working1 : 1 is working, 2 is down2 : 2 is working, 1 is downd1 : both are down, 1 is being repairedd2 : both are down, 2 is being repaired.
Thenvb = µ1 + µ2, v1 = µ1 + µ, v2 = µ2 + µ, vd1 = vd2 = µ,
and
Pb,1 = 1! Pb,2 =µ2
µ1 + µ2,
P1,b = 1! P1,d2 =µ
µ1 + µ,
P2,b = 1! P2,d1 =µ
µ + µ2,
Pd1,1 = Pd2,2 = 1.
1
4.
a. Yes.
b. It is a pure birth process.
c. If there are i infected individuals then since a contact will involve an infected andan uninfected individual with probability i(n! i)/
!n2
", it follows that the birth rates
are !i = !i(n! i)/!
n2
", i = 1, . . . , n. Hence,
E[time all infected] =n(n! 1)
2!
n#
i=1
1
i(n! i).
5. Starting with E[T0] = 1!0
= 1! , employ the identity
E[Ti] =1
!i+
µi
!iE[Ti!1]
to succesively compute E[Ti] for i = 1, 2, 3, 4.
a. E[T0] + · · · + E[T3].
b. E[T2] + E[T3] + E[T4].
2
1. Consider the following queueing model: customers arrive at a service station accordingto a Poisson process with rate !. There are c servers; the service times are exponential withrate µ. If an arriving customer finds c servers busy, then he leaves the system immediately.
a. Model this system as a birth and death process.
b. Suppose now that there are infinitely many servers (c =!). Again model this systemas a birth and death process.
2. In Example 6.11 it is shown, using the backward equations, that
P !00(t) = µ" (µ + !)P00(t).
a. Derive this result using the forward equations.
b. Derive a di!erential equation for P11(t) in two ways: using the forward and backwardequations.
c. Suppose the machine is working at time 0. What is the probability that the machineis also working at time t?
3. Exercise 6.8.
4. Exercise 6.10 (but you do not have to verify that the transition probabilities satisfy theforward and backward equations).
1
1.
a. The state space is {0, 1, . . . , c! 1, c}, and the birth rates are
!n = !, n = 0, 1, . . . , c! 1,
and the death ratesµn = nµ, n = 1, 2, . . . , c.
Hence the transition rate matrix Q is given by
Q =
!
"""""""""""#
!! ! 0 0 0 · · · 0µ !(! + µ) ! 0 0 · · · 00 2µ !(! + 2µ) ! 0 · · · 0...
. . . . . . . . . . . . . . ....
0 · · · 0 (c! 2)µ !(! + (c! 2)µ) ! 00 · · · 0 0 (c! 1)µ !(! + (c! 1)µ) !0 · · · 0 0 0 cµ !cµ
$
%%%%%%%%%%%&
b. Now the state space does not stop at c, i.e., it is {0, 1, . . .} and the birth rates are
!n = !, n = 0, 1, . . . ,
and the death ratesµn = nµ, n = 1, 2, . . . .
2.
a. The forward equations are
P!
00(t) = µP01(t)! !P00(t)
P!
01(t) = !P00(t)! µP01(t).
Using P01(t) = 1! P00(t), we obtain the following di!erential equation,
P!
00(t) = µ! (µ + !)P00(t).
b. The backward equations are
P!
11(t) = µP01(t)! µP11(t)
P!
01(t) = !P11(t)! !P01(t).
Multiplying these equations by ! and µ, respectively, and then adding them, weobtain
!P!
11(t) + µP!
01(t) = 0.
1
Hence, for some c,!P11(t) + µP01(t) = c.
Substituting t = 0 yields c = !. Using the above equation, we get
P!
11(t) = !! (! + µ)P11(t).
The forward equations are
P!
11(t) = !P10(t)! µP11(t)
P!
10(t) = µP11(t)! !P10(t).
By substituting P10(t) = 1!P11(t), we obtain the same di!erential equation as before.
c. Solving the di!erential equation for P00(t) with initial condition P00(0) = 1 yields
P00(t) =µ
! + µ+
!
! + µe!(!+µ)t, t " 0.
3. The number of failed machines is a birth and death process with
!0 = 2!, !1 = !, !n = 0, n > 1, µ1 = µ2 = µ, µn = 0, n #= 1, 2.
Now substitute into the backward equations.
4. Let
Ij(t) =
!0, if machine j is working at time t,1, otherwise.
Also, let the state be (I1(t), I2(t)). This is clearly a continuous-time Markov chain with
v(0,0) = !1 + !2, !(0,0);(0,1) = !2, !(0,0);(1,0) = !1,
v(0,1) = !1 + µ2, !(0,1);(0,0) = µ2, !(0,1);(1,1) = !1,
v(1,0) = µ1 + !2, !(1,0);(0,0) = µ1, !(1,0);(1,1) = !2,
v(1,1) = µ1 + µ2, !(1,1);(0,1) = µ1, !(1,1);(1,0) = µ2.
By the independence assumption we have
P(i,j),(k,l)(t) = Pi,k(t)Qj,l(t), (1)
where Pi,k(t) is the probability that the first machine is in state k at time t given that itwas at state i at time 0; Qj,l(t) is defined similarly for the second machine. By example4.11 we have
P0,0(t) =µ1
!1 + µ1+
!1
!1 + µ1e!(!1+µ1)t,
P1,0(t) =µ1
!1 + µ1! µ1
!1 + µ1e!(!1+µ1)t,
2
and by the same argument,
P1,1(t) =!1
!1 + µ1+
µ1
!1 + µ1e!(!1+µ1)t,
P0,1(t) =!1
!1 + µ1! !1
!1 + µ1e!(!1+µ1)t.
Of course, similar expressions for the second machine are obtained by replacing (!1, µ1) by(!2, µ2). We then get P(i,j),(k,l)(t) by formula (??). For instance,
P(0,0),(0,0)(t) = P0,0(t)Q0,0(t) =µ1 + !1e!(!1+µ1)t
!1 + µ1· µ2 + !2e!(!2+µ2)t
!2 + µ2.
3
1.
a. If the state is the number of individuals at time t, we get a birth and death processswith
!n = n! + ", n < N,
!n = n! n ! N,
µn = nµ.
b. Let Pi be the long-run probability that the system is in state i. Since this is also theproportion of time the system is in state i, we are looking for
!!
i=3
Pi.
We havePkµk = Pk"1!k"1, k = 1, 2, . . . .
This yields
P1 ="
µP0 =
1
2P0,
P2 =" + !
2µP1 =
1
2P1 =
1
4P0,
P3 =" + 2!
3µP2 =
1
2P2 =
1
8P0,
and for k ! 3,
Pk =(k " 1)!
kµPk"1 =
k " 1
k
1
2Pk"1 = · · · =
3
k
"1
2
#k"3
P3.
Hence!!
k=3
Pk = P3
!!
k=3
3
k
"1
2
#k"3
= 24P3
!!
k=3
1
k
"1
2
#k
.
Since !!
k=1
1
kxk = " log(1" x),
we get!!
k=3
Pk = P3(24 log 2" 15).
Because all probabilities add up to 1, we have
P0
"1 +
1
2+
1
4+
1
8(24 log 2" 15)
#= 1.
1
So
P!10 = 3 log 2! 1
8,
and thus finally,"!
k=3
Pk =24 log 2! 15
24 log 2! 1" 0.105.
2. With the number of customers in the shop as the state, we get a birth and death processwith
!0 = !1 = 3, µ1 = µ2 = 4.
Therefore
P1 =3
4P0, P2 =
3
4P1 =
"3
4
#2
P0.
And since P0 + P1 + P2 = 1, we get
P0 =16
37.
a. The average number of customers in the shop is
P1 + 2P2 =30
37.
b. The proportion of the customers that enter the shop is
!(1! P2)
!= 1! P2 =
28
37.
c. Now µ = 8, so
P0 =64
97.
So the proportion of the customers that enter the shop is
1! P2 =88
97.
The rate of added customers is therefore
!"
88
97
#! !
"28
37
#" 0.45.
The business he does would improve by 0.45 customers per hour.
3. With the number of customers in the system as the state, we get a birth and deathprocess with
!0 = !1 = !2 = 3, !i = 0, i # 4, µ1 = 2, µ2 = µ3 = 4.
2
Therefore the balance equations reduce to
P1 =3
2P0, P2 =
3
4P1 =
9
8P0, P3 =
3
4P2 =
27
32P0.
And therefore,
P0 =!1 +
3
2+
9
8+
27
32
"!1
=32
143.
a. The fraction of potential custoemrs that enter the system is
!(1! P3)
!= 1! P3 =
116
143.
b. With a server working twice as fast we would get
P1 =3
4P0, P2 =
3
4P1 =
!3
4
"2
P0, P3 =!
3
4
"3
P0,
and
P0 =
#
1 +3
4+
!3
4
"2
+!
3
4
"3$!1
=64
175.
So that now
1! P3 =148
175.
4. Say the state is 0 if the machine is up, say it is in state i when it is down due to a typei failure, i = 1, 2. The balance equations for the limiting probabilities are as follows.
!P0 = µ1P1 + µ2P2,
µ1P1 = !pP0,
µ2P2 = !(1! p)P0,
P0 + P1 + P2 = 1.
These equations are easily solved to give the results
P0 = (1 + !p/µ1 + !(1! p)/µ2)!1 , P1 = !pP0/µ1, P2 = !(1! p)P0/µ2.
3
1. There are k + 1 states: state 0 means the machine is working, state i means that it isin repair phase i, i = 1, . . . , k. The balance equations for the limiting probabilities are
!P0 = µkPk,
µ1P1 = !P0,
µiPi = µi!1Pi!1, i = 2, . . . , k,
and the normalization equation is
P0 + · · · + Pk = 1.
To solve, note thatµiPi = µi!1Pi!1 = µi!2Pi!2 = · · · = !P0.
Hence,Pi = (!/µi)P0,
and, upon summing,
1 = P0
!
1 +k"
i=1
!
µi
#
.
Therefore,
P0 =
!
1 +k"
i=1
!
µi
#!1
, Pi = (!/µi)P0, i = 1, . . . , k.
a. Pi.
b. P0.
2. The number in the system is a birth and death process with parameters
!n = !/(n + 1), n ! 0, µn = µ, n ! 1.
From the equation above (6.20),
1/P0 = 1 +""
n=1
(!/µ)n/n! = e!/µ
and from (6.20),
Pn = P0(!/µ)n/n! = e!!/µ(!/µ)n/n!, n ! 0.
3. Let the state denote the number of machines that are down. This yields a birth anddeath process with
!0 =3
10, !1 =
2
10, !2 =
1
10, !i = 0, i ! 3,
1
and
µ1 =1
8, µ2 =
2
8, µ3 =
2
8.
The balance equations reduce to
P1 =3/10
1/8P0 =
12
5P0,
P2 =2/10
2/8P1 =
4
5P1 =
48
25P0,
P3 =1/10
2/8P2 =
4
10P2 =
192
250P0.
Hence, using P0 + P1 + P2 + P3 = 1, yields
P0 =!1 +
12
5+
48
25+
192
250
"!1
=250
1522.
a. Average number not in use is
P1 + 2P2 + 3P3 =2136
1522=
1068
761.
b. Proportion of time both repairmen are busy is
P2 + P3 =336
761.
4.
a. !0 = 13 .
b. !0 = 1.
c. !0 = (!
3" 1)/!
2.
2