Embed Size (px)
Citation preview
COPYRIGHT © 2006 by LAVON B. PAGE
Regular MarkovChains — steady-state probabilitydistributions
COPYRIGHT © 2006 by LAVON B. PAGE
Example
T =.8 .2
.4 .6
!
" # #
$
% & &
T2
=.72 .28
.56 .44
!
" # #
$
% & &
COPYRIGHT © 2006 by LAVON B. PAGE
Example
T =.8 .2
.4 .6
!
" # #
$
% & &
T2
=.72 .28
.56 .44
!
" # #
$
% & &
T6
=.668 .332
.664 .336
!
" # #
$
% & &
T6 !
2 /3 1/3
2 /3 1/3
"
# $ $
%
& ' '
COPYRIGHT © 2006 by LAVON B. PAGE
As n gets larger and larger, Tn gets
closer and closer to the matrix
Example
T =.8 .2
.4 .6
!
" # #
$
% & &
T2
=.72 .28
.56 .44
!
" # #
$
% & &
T6
=.668 .332
.664 .336
!
" # #
$
% & &
T6 !
2 /3 1/3
2 /3 1/3
"
# $ $
%
& ' '
2 /3 1/3
2 /3 1/3
!
" # #
$
% & &
COPYRIGHT © 2006 by LAVON B. PAGE
Notice
1 0[ ] 2 /3 1/3
2 /3 1/3
!
" # #
$
% & &
= 2 /3 1/3[ ]
COPYRIGHT © 2006 by LAVON B. PAGE
Notice
1 0[ ] 2 /3 1/3
2 /3 1/3
!
" # #
$
% & &
= 2 /3 1/3[ ]
0 1[ ] 2 /3 1/3
2 /3 1/3
!
" # #
$
% & &
= 2 /3 1/3[ ]
COPYRIGHT © 2006 by LAVON B. PAGE
Notice
1 0[ ] 2 /3 1/3
2 /3 1/3
!
" # #
$
% & &
= 2 /3 1/3[ ]
0 1[ ] 2 /3 1/3
2 /3 1/3
!
" # #
$
% & &
= 2 /3 1/3[ ]
1 /2 1/2[ ] 2 /3 1/3
2 /3 1/3
!
" # #
$
% & &
= 2 /3 1/3[ ]
COPYRIGHT © 2006 by LAVON B. PAGE
no matter what a and b are.
Notice
1 0[ ] 2 /3 1/3
2 /3 1/3
!
" # #
$
% & &
= 2 /3 1/3[ ]
0 1[ ] 2 /3 1/3
2 /3 1/3
!
" # #
$
% & &
= 2 /3 1/3[ ]
1 /2 1/2[ ] 2 /3 1/3
2 /3 1/3
!
" # #
$
% & &
= 2 /3 1/3[ ]
a b[ ] 2 /3 1/3
2 /3 1/3
!
" # #
$
% & &
= 2 /3 1/3[ ]
COPYRIGHT © 2006 by LAVON B. PAGE
Moral of the story: No matter whatassumptions you make about the initialprobability distribution, after a largenumber of steps have been taken theprobability distribution is approximately
(2/3 1/3)
Remember T n !
when n is large
2 /3 1/3
2 /3 1/3
!
" # #
$
% & &
COPYRIGHT © 2006 by LAVON B. PAGE
Moral of the story: No matter whatassumptions you make about the initialprobability distribution, after a largenumber of steps have been taken theprobability distribution is approximately
(2/3 1/3)
Question: How could we determinethis without computing large powersof T and estimating the limitingmatrix?
COPYRIGHT © 2006 by LAVON B. PAGE
Clue: Notice that the probabilitydistribution (2/3 1/3) has the propertythat
2 /3 1/3[ ] .8 .2
.4 .6
!
" # #
$
% & &
= 2 /3 1/3[ ]
COPYRIGHT © 2006 by LAVON B. PAGE
is x = 2/3, y = 1/3
In other words, a solution of the matrixequation
Clue: Notice that the probabilitydistribution (2/3 1/3) has the propertythat
2 /3 1/3[ ] .8 .2
.4 .6
!
" # #
$
% & &
= 2 /3 1/3[ ]
x y[ ] .8 .2
.4 .6
!
" # #
$
% & &
= x y[ ]
COPYRIGHT © 2006 by LAVON B. PAGE
We can find the steady state probabilitydistribution [ (2/3 1/3) in this example] bysolving for x and y below:
(Remember also that x + y =1)
Idea!!!
x y[ ] .8 .2
.4 .6
!
" # #
$
% & &
= x y[ ]
COPYRIGHT © 2006 by LAVON B. PAGE
We can find the steady state probabilitydistribution [ (2/3 1/3) in this example] bysolving for x and y below:
(Remember also that x + y =1)
x y[ ] .8 .2
.4 .6
!
" # #
$
% & &
= x y[ ]
.8x + .4y = x
.2x + .6y = y
x + y = 1
COPYRIGHT © 2006 by LAVON B. PAGE
We can find the steady state probabilitydistribution [ (2/3 1/3) in this example] bysolving for x and y below:
(Remember also that x + y =1)
x = 2y
x y[ ] .8 .2
.4 .6
!
" # #
$
% & &
= x y[ ]
.8x + .4y = x
.2x + .6y = y
x + y = 1
.4y = .2x
.2x = .4y
COPYRIGHT © 2006 by LAVON B. PAGE
We can find the steady state probabilitydistribution [ (2/3 1/3) in this example] bysolving for x and y below:
(Remember also that x + y =1)
x = 2y
2y + y = 1
x y[ ] .8 .2
.4 .6
!
" # #
$
% & &
= x y[ ]
.8x + .4y = x
.2x + .6y = y
x + y = 1
.4y = .2x
.2x = .4y
COPYRIGHT © 2006 by LAVON B. PAGE
We can find the steady state probabilitydistribution [ (2/3 1/3) in this example] bysolving for x and y below:
(Remember also that x + y =1)
x = 2y
2y + y = 1y = 1/3x = 2/3
x y[ ] .8 .2
.4 .6
!
" # #
$
% & &
= x y[ ]
.8x + .4y = x
.2x + .6y = y
x + y = 1
.4y = .2x
.2x = .4y
COPYRIGHT © 2006 by LAVON B. PAGE
Example: Bob, Alice and Carol are playingFrisbee. Bob always throws to Alice andAlice always throws to Carol. Carol throwsto Bob 2/3 of the time and to Alice 1/3 ofthe time. In the long run what percentageof the time do each of the players have theFrisbee?
COPYRIGHT © 2006 by LAVON B. PAGE
Example: Bob, Alice and Carol are playingFrisbee. Bob always throws to Alice andAlice always throws to Carol. Carol throwsto Bob 2/3 of the time and to Alice 1/3 ofthe time. In the long run what percentageof the time do each of the players have theFrisbee?
B CAA
B
C
T =
0 0 1
1 0 0
1/3 2 /3 0
!
"
# # # #
$
%
& & & &
COPYRIGHT © 2006 by LAVON B. PAGE
B CAA
B
C
We must solve the matrix equation
and use the fact that x + y + z = 1
T =
0 0 1
1 0 0
1/3 2 /3 0
!
"
# # # #
$
%
& & & &
x y z[ ]0 0 1
1 0 0
1/3 2 /3 0
!
"
# # # #
$
%
& & & &
= x y z[ ]
COPYRIGHT © 2006 by LAVON B. PAGE
We multiply out the left side and equateto the right side:
x y z[ ]0 0 1
1 0 0
1/3 2 /3 0
!
"
# # # #
$
%
& & & &
= x y z[ ]
COPYRIGHT © 2006 by LAVON B. PAGE
We multiply out the left side and equateto the right side:
y + 1/3 z = x
2/3 z = y
x = z
x y z[ ]0 0 1
1 0 0
1/3 2 /3 0
!
"
# # # #
$
%
& & & &
= x y z[ ]
COPYRIGHT © 2006 by LAVON B. PAGE
We multiply out the left side and equateto the right side:
y + 1/3 z = x
2/3 z = y
x = z
Remember also: x + y + z = 1
x y z[ ]0 0 1
1 0 0
1/3 2 /3 0
!
"
# # # #
$
%
& & & &
= x y z[ ]
COPYRIGHT © 2006 by LAVON B. PAGE
x + y + z = 1
y + 1/3 z = x
2/3 z = y
x = z
Equations to solve:
COPYRIGHT © 2006 by LAVON B. PAGE
x + y + z = 1
y + 1/3 z = x
2/3 z = y
x = z
Solution: x + y + z = 1 can be rewritten as
z + 2/3 z + z = 1
Equations to solve:
COPYRIGHT © 2006 by LAVON B. PAGE
x + y + z = 1
y + 1/3 z = x
2/3 z = y
x = z
Equations to solve:
Solution: x + y + z = 1 can be rewritten as
z + 2/3 z + z = 1so z = 3/8
Therefore x = 3/8 and y = 1/4
COPYRIGHT © 2006 by LAVON B. PAGE
x = 3/8, y = 1/4, z = 3/8
Alice — 3/8 probability
Bob — 1/4 probability
Carol — 3/8 probability
COPYRIGHT © 2006 by LAVON B. PAGE
In the truck rental problem we had thefollowing transition matrix:
NC SC VA
NC
SC
VA
What fraction of the time does a truckspend in each of the 3 states?
.5 .2 .3
.4 .4 .2
.4 .1 .5
!
"
# # # #
$
%
& & & &
COPYRIGHT © 2006 by LAVON B. PAGE
remembering that x + y + z = 1.
We have to solve for
[x y z]
.5 .2 .3
.4 .4 .2
.4 .1 .5
!
"
# # # #
$
%
& & & &
= [x y z]
COPYRIGHT © 2006 by LAVON B. PAGE
remembering that x + y + z = 1.So the system of equations is
.5x + .4y + .4z = x
.2x + .4y + .1z = y
.3x + .2y + .5z = z
x + y + z =1
We have to solve for
[x y z]
.5 .2 .3
.4 .4 .2
.4 .1 .5
!
"
# # # #
$
%
& & & &
= [x y z]
COPYRIGHT © 2006 by LAVON B. PAGE
Rewrite the equations in standard form:
.5x + .4y + .4z = x
.2x + .4y + .1z = y
.3x + .2y + .5z = z
x + y + z =1
x + y + z = 1
!.5x + .4y + .4z = 0
.2x + !.6y + .1z = 0
.3x + .2y + !.5z = 0
COPYRIGHT © 2006 by LAVON B. PAGE
Rewrite the equations in standard form:
This system ofequations could besolved using theaugmented matrixmethod of Chapter 1.
.5x + .4y + .4z = x
.2x + .4y + .1z = y
.3x + .2y + .5z = z
x + y + z =1
x + y + z = 1
!.5x + .4y + .4z = 0
.2x + !.6y + .1z = 0
.3x + .2y + !.5z = 0
COPYRIGHT © 2006 by LAVON B. PAGE
The methods we have learned in thissection work only with regular Markovchains.
COPYRIGHT © 2006 by LAVON B. PAGE
The methods we have learned in thissection work only with regular Markovchains.
A regular Markov Chain is one where forsome positive integer n, the matrix T
n
has no 0 entries.
COPYRIGHT © 2006 by LAVON B. PAGE
"
#
$$$$$$$
%
&
'''''''
0 0 1
23
013
12
14
14
"
#
$$$$$$$$$
%
&
'''''''''
16
34
112
13
24
3
16
13
48
4396
2164
43192
T = T 2 =
T 3 =
"
#
$$$$$$$$$
%
&
'''''''''
23
013
16
34
112
724
916
748
COPYRIGHT © 2006 by LAVON B. PAGE
T is regularbecause T 3
contains no 0entries.
"
#
$$$$$$$
%
&
'''''''
0 0 1
23
013
12
14
14
"
#
$$$$$$$$$
%
&
'''''''''
16
34
112
13
24
3
16
13
48
4396
2164
43192
T = T 2 =
T 3 =
"
#
$$$$$$$$$
%
&
'''''''''
23
013
16
34
112
724
916
748
