Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
Random Walk
1
2
3
4
5
β¦
A random walk is a mathematical object, known as a stochastic or random process, that describes a path that consists of a succession of random steps on some mathematical space such as the integers.
XC 2020
Markov Chain
1
2
3
4
5
XC 2020
Specifying a Markov Chain
Β§ We describe a Markov chain as follows: We have a set of states, π = π !, π ", β― , π # .
Β§ The process starts in one of these states and moves successively from one state to another. Each move is called a step. π !
π "
π #
π $
π %
XC 2020
Β§ If the chain is currently in state π !, then it moves to state π " at the next step with a probability denoted by π!" .
Β§ The probability π!" does not depend upon which states the chain was in before the current state.
Β§ These probabilities are called transition probabilities.
π !
π "
π #
π $
π %
π!"π$"
π%&
π&" π"&
XC 2020
Β§ The process can remain in the state it is in, and this occurs with probability π!! .
π !
π "
π #
π $
π %
π!!
π%%
XC 2020
Β§ An initial probability distribution, defined on π, specifies the starting state. Usually this is done by specifying a particular state as the starting state.
π’!
π #
π’"
π $
π’#
π %
π’$
π &
π’%
π '
π’= π’# π’$ π’% π’& π’'
0 1 0 0 0
(!(#
)
π’! = 1
XC 2020
THE LAND OF OZ
Β§ the Land of Oz is blessed by many things, but not by good weather.
Β§ They never have two nice days in a row. If they have a nice day, they are just as likely to have snow as rain the next day.
Β§ If they have snow or rain, they have an even chance of having the same the next day.
Β§ If there is change from snow or rain, only half of the time is this a change to a nice day.
XC 2020
Β§ They never have two nice days in a row. If they have a nice day, they are just as likely to have snow as rain the next day.
12
12
XC 2020
Β§ If they have snow or rain, they have an even chance of having the same the next day.
12
12
12
12
XC 2020
Β§ If there is change from snow or rain, only half of the time is this a change to a nice day.
12
12
12
12
14
14
XC 2020
Β§ If there is change from snow or rain, only half of the time is this a change to a nice day.
12
12
12
12
14
14
14
14
XC 2020
12
12
12
12
14
14
14
14
Β§
βRNS
R N S
XC 2020
12
12
12
12
14
14
14
14
Β§
βRNS
R N S)*
)+
)+
XC 2020
12
12
12
12
14
14
14
14
Β§
βRNS
R N S)*
)+
)+
)* 0 )
*
XC 2020
12
12
12
12
14
14
14
14
Β§
βRNS
R N S)*
)+
)+
)*
0 )*
)+
)+
)*
XC 2020
12
12
12
12
14
14
14
14
Β§ States: Β§ π !: rainΒ§ π ": niceΒ§ π &: snow
Β§ π =
!"
!$
!$
!"
0 !"
!$
!$
!"
XC 2020
Transition Matrix
Β§ The entries in the first row of the matrix π in the example represent the probabilities for the various kinds of weather following a rainy day.
Β§ Similarly, the entries in the second and third rows represent the probabilities for the various kinds of weather following nice and snowy days, respectively.
Β§ Such a square array is called the matrix of transition probabilities, or the transition matrix.
π =
12
14
14
12
012
14
14
12
12
12
12
12
14
14
14
14
XC 2020
π =
12
14
14
12
012
14
14
12
οΌthe probability that, given the chain is in state π today, it will be in state π tomorrow
Β§ States: Β§ π !: rainΒ§ π ": niceΒ§ π &: snow
12
12
12
12
14
14
14
14
XC 2020
π =
12
14
14
12
012
14
14
12
οΌthe probability that, given the chain is in
state π today, it will be in state πtomorrow
Β§ States: Β§ π !: rainΒ§ π ": niceΒ§ π &: snow
π'((!) = π'(
οΌthe probability that, given the chain is in state π today, it will be in state π the day
after tomorrowπ'((") = β―
12
12
12
12
14
14
14
14
XC 2020
π =
12
14
14
12
012
14
14
12
Β§ States: Β§ π !: rainΒ§ π ": niceΒ§ π &: snow
Day 0 Day 1 Day 2 β¦
οΌ β¦
π!&(") = β―
12
12
12
12
14
14
14
14
XC 2020
12
12
12
12
14
14
14
14
π =
12
14
14
12
012
14
14
12
Β§ States: Β§ π #: rainΒ§ π $: niceΒ§ π %: snow
Day 0 Day 1 Day 2 β¦
β¦π!&(")
= π!!π!& + π!"π"& + π!&π&&
π## π#%
π#$ π$%
π#% π%%
XC 2020
π =π## π#$ π#%π$# π$$ π$%π%# π%$ π%%
=
12
14
14
12
012
14
14
12
Β§ States: Β§ π #: rainΒ§ π $: niceΒ§ π %: snow
π#%($) = π##π#% + π#$π$% + π#%π%%
Day 0 Day 1 Day 2 β¦
β¦
π!! π!#
π!" π"#
π!# π##
π## π#$ π#% π#%π$%π%%
π$ =π## π#$ π#%π$# π$$ π$%π%# π%$ π%%
2π## π#$ π#%π$# π$$ π$%π%# π%$ π%%
=π#%($)
XC 2020
π#%($) = π##π#% + π#$π$% + π#%π%%
= (,(#
%
π#,π,%
Day 0 Day 1 Day 2 β¦
β¦
π!! π!#
π!" π"#
π!# π##
Day 0 Day 1 Day 2 β¦
β¦
π!! π!#
π!" π"#
π!& π&#
β¦π!β― πβ―#
π#%($) = π##π#% + π#$π$% +β―+ π#-π-%
= (,(#
-
π#,π,%
XC 2020
Day 0 Day 1 Day 2 Day β¦ Day n β¦
οΌ β¦οΌ β¦
Day 0 Day 1 Day 2 β¦
β¦
π!! π!#
π!" π"#
π!# π##
π$ =π## π#$ π#%π$# π$$ π$%π%# π%$ π%%
2π## π#$ π#%π$# π$$ π$%π%# π%$ π%%
=π#%($)
π) =π## π#$ π#%π$# π$$ π$%π%# π%$ π%%
)
=π#%())
XC 2020
Transition Matrix
Β§ Let π be the transition matrix of a Markov chain.
Β§ The ππth entry π'( of the matrix π+ gives the probability that the Markov chain, starting in state π ', will be in state π ( after π steps.
Day 0 Day 1 Day 2 Day β¦ Day n β¦
οΌ β¦οΌ β¦ π) =π## π#$ π#%π$# π$$ π$%π%# π%$ π%%
)
=π#%())
XC 2020
Β§ Starting states: Β§ rain: π’#Β§ nice: π’$Β§ snow: π’%
π’ = π’! π’" π’&
Β§ Transition matrix:
π =π!! π!" π!#π"! π"" π"#π#! π#" π##
=
12
14
14
12
012
14
14
12
π!" π!#
π##π""
π"! π#!
π"#
π#"
π’%(#) = π’#π#% + π’$π$% + π’%π%%
π’# π’$ π’%π## π#$ π#%π$# π$$ π$%π%# π%$ π%%
π’(#) = π’#(#) π’$
(#) π’%(#) = π’π
π!#
π"#
π##
Day 0 Day 1 β¦
β¦
π’!
π’"
π’#
Β§ the probability that the chain is in state π ( after π steps:
Β§ π = 3Β§ π = 1
XC 2020
π’,(#) = π’#π#, + π’$π$, + π’%π%,
π’(#) = π’#(#) π’$
(#) π’%(#) = π’π
π!#
π"#
π##
Day 0 Day 1 β¦
β¦
π’!
π’"
π’#
Β§ the probability that the chain is in state π ( after π steps:
Β§ π = 1
π’%(#) = π’#π#% + π’$π$% + π’%π%%
Β§ the probability that the chain is in state π ( after π steps:
Β§ π = 2
Day 0 Day 1 Day 2 β¦
π’!
β¦
π!! π!#
π!"π"#
π!#
π##
π’"
π’#
π!#(") = π!!π!# + π!"π"# + π!#π## = .
(+!
#
π!(π(#
π’#(") = π’!π!#
(") + π’"π"#(") + π’#π##
(")
π’((") = π’!π!(
(") + π’"π"((") + π’#π#(
(")
π’(") = π’!(") π’"
(") π’#(") = π’π"
XC 2020
Transition Matrix
Β§ Let π be the transition matrix of a Markov chain, and let π’ be the probability vector which represents the starting distribution.
Β§ Then the probability that the chain is in state π , after π steps is the πth entry in the vector
π’(+) = π’π+.
Day 0 Day 1 Day 2 Day β¦ Day n β¦
οΌ β¦οΌ β¦
π’()) = π’π)
= π’π## π#$ π#%π$# π$$ π$%π%# π%$ π%%
)
= π’#()) π’$
()) π’%())
π’!
π’"
π’#
XC 2020
Open book
Scope: Mostly Chapters 7, 8, 9, 10, and 11
§ sum of random variables§ LLN and CLT§ generating functions§ Markov chains
Materials: slides, homework, quizzes, textbook
Date & Time: 12:00 pm β 10: 00 pm, August 30
Office hours: August 27, 28
Homework due: 11:00 pm August 28
August 2020Final
26 27 28 29 30 31 01
02 03 04 05 06 07 08
09 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 2827
30 0531 01 02 0403
29
Sun Mon Tue Wed Thu Fri Sat
XC 2020