View
223
Download
1
Embed Size (px)
Citation preview
Problems 10/3
1. Ehrenfast’s diffusion model:
P(Xn+1 =k−1Xn =k) =k2a
, k≥1
P(Xn+1 =k + 1Xn =k) =1−k2a
, k < 2a
P =
0 1 0 0 L 0 012a 0 1−1
2a 0 L 0 00 2
2a 0 1−22a L 0 0
L L L L L L L
0 0 0 0 L 1 0
⎛
⎝
⎜⎜⎜⎜⎜
⎞
⎠
⎟⎟⎟⎟⎟
Problems, cont.
2.
Discrete uniform on {0,...,n}
PX|U=u(s) =(1−u+us)n
PX (s) = PX|U=u(s)du0
1
∫ = (1+u(s −1))ndu0
1
∫=
(1+ u(s − 1))n+1
(n + 1)(s − 1)
⎡
⎣⎢
⎤
⎦⎥
u=0
1
=1
n + 1
sn+1 − 1
s − 1
=
1
n + 11+ s + s2 +L + sn
( )
Problems, cont.
3.
where
k=0?
P( Sn+1 =k + 1 Sn =k)
=P(Sn+1 =k + 1Sn =k or Sn =−k)
=P(Xn+1 = 1)P(Sn = k) + P(Xn+1 = −1)P(Sn = −k)
P(Sn = k) + P(Sn = −k)
=ppk
(n) + qp−k(n)
pk(n) + p−k
(n)
pk(n) =P(Sn =k) =
nn+k2
⎛
⎝⎜⎞
⎠⎟pn+k2 q
n−k2
ppk(n) +qp−k
(n)
pk(n) +p−k
(n) =pk+1 +qk+1
pk +qk
Classification of states
A state i for a Markov chain Xk is called persistent if
and transient otherwise.
Let
and .
j is persistent iff fjj=1.
Let
P(Xk =i for somek≥1X0 =i) =1
fij(n) =P(X1 ≠j,X2 ≠j, ...,Xn−1 ≠j,Xn =j X0 =i)
fij = fij(n)
n=1
∞
∑
Pij (s) = pij(n)sn
n=0
∞
∑ Fij (s) = fij(n)sn
n=0
∞
∑
Some results
Theorem:
(a)Pii(s)=1+Fii(s)Pii(s)
(b)Pij(s)=Fij(s)Pij(s) for i ≠ j.Proof:As for the random walk case we deduce from the Markov property that
Multiply both sides by sm, sum over m≥1 to get
pij(m) = fij
(r)pjj(m−r)
r=1
m
∑
Pij (s)−1(i=j) =Fij (s)Pij (s)
Some results, cont.
Corollary:
(a) State j is persistent if and then for all i.
(b) State j is transient if and then for all i.
Proof:
SInce we see that
But (by Abel’s thm)
p jj(n) =∞∑
pij(n) =∞∑
p jj(n) < ∞∑
pij(n) < ∞∑
Pjj (s) =1
1−Fjj (s)
Pjj (s)→ ∞ iff Fjj (1) =fjj =1
lims↑1
Pjj (s) = pjj(n)∑
A final consequence
If i is transient, then
Why?
Example: Branching process
What states are persistent? Transient?
State 0 is called absorbing, since once the process reaches 0, it never leaves again.
p jj(n) → 0
Mean recurrence time
Let Ti = min{n>0: Xn = i} and i = E(Ti|X0=i).
For a transient state i = ∞.
For a persistent state
We call a recurrent state positive persistent if i < ∞, null persistent otherwise.
Example: Simple random walk
positive recurrent = non-null persistent
i = nfii(n)∑
A model forradiation damage
Initial damage from radiation can either heal or get worse until it is visible.
0 is a healthy organism (absorbing)
3 visible damage (absorbing)
1 initial damage
2 amplified damage
P =
1 0 0 023 0 1
3 00 1
3 0 23
0 0 0 1
⎛
⎝
⎜⎜⎜⎜
⎞
⎠
⎟⎟⎟⎟
Radiation damage, cont.
Recovery probability 0 is probability of reaching 0 before 3.
Last step must go
Thus
1→ 00 = p10 p11
(n)
n=0
∞
∑
p11(2n) =( 13 ×1
3)n
0 =2
3
1
9⎛⎝⎜
⎞⎠⎟
n
=2
3
89n=0
∞
∑ =3
4
Communication
Two states i and j communicate, , if for some m.
i and j intercommunicate, , if and .
Theorem: is an equivalence relation.
What do we need to prove?
i→ jpij
(m) > 0
i ↔ j i→ jj→ i
↔
Equivalence classes of states
Theorem: If then
(a) i is transient iff j is transient
(b) i is persistent iff j is persistent
Proof of (a): Since there are m,n with
By Chapman-Kolmogorov
so summing over r we get
i ↔ j
pii(m+r+n) = pij
(m)pjj(r)pji
(n)
r=1
n
∑ ≥pjj(r)pij
(m)pji(n)
>01 24 34
i ↔ jpij
(m)p ji(n) > 0
pii(r ) < ∞ if pjj
(r) < ∞∑∑
Closed and irreducible sets
A set C of states is closed if pij=0 for all i in C, j not in C
C is irreducible if for all i,j in C.
Theorem:
S=T+C1+C2+...
where T are all transient, and the Ci are irreducible disjoint closed sets of persistent states
Note: The Ci are the equivalence classes for
i ↔ j
↔
Example
S={0,1,2,3,4,5}
{0,1},{4,5} closed irreducible persistent
{2,3} transient. Why?
P =
12
12 0 0 0 0
14
34 0 0 0 0
14
14
14
14 0 0
14 0 1
414 0 1
4
0 0 0 0 12
12
0 0 0 0 12
12
⎛
⎝
⎜⎜⎜⎜⎜⎜⎜
⎞
⎠
⎟⎟⎟⎟⎟⎟⎟
f11(n) =
p11 =34 if n=1
p10 (p00 )n−2p01 = 1
812( )
n−2if n > 1
⎧⎨⎪
⎩⎪
1 = nf11(n) =∑ 3
4 + 18 n 1
4( )n−2
=3
2n=2
∞
∑
Long-term behavior
Recall from the 0-1 process that
When does this not depend on n?
(a) p01 = p11
(b)
(c)
1(n) =
p01
1− (p11 − p01)+ μ1
(0) −p01
1− (p11 − p01)
⎛
⎝⎜⎞
⎠⎟(p11 − p01)
n
1(0) =
p01
1− (p11 − p01)
n→ ∞
Stationary distribution
Case (b) is the general one. Here is the idea: Recall that (n)= (0)Pn. In order to get the same distribution for all n, we use (0)=where solves P =
(1) = P = P2 = P = ...
Pn =
Snoqualmie Falls
so
or
P =.602 .398.166 .834⎛⎝⎜
⎞⎠⎟
0 = .602π0 + .166π1
π1 = .398π0 + .834π1
π0 + π1 = 1
0 =.166
1− .602 + .166= .295
1 = .705
P5 =.305 .695.290 .710⎛⎝⎜
⎞⎠⎟