Stochastic ProcessesNancy Griffeth

January 10, 2014

Funding for this workshop was provided by the program “Computational Modeling and Analysis of Complex Systems,” an NSF Expedition in

Computing (Award Number 0926200).

Motivation

Continuous solutions depend on large numbers (e.g., ODE’s)

Stochastic solutions are better for small numbers Strange and surprising things happen!

Game

Two players each have a pile of pennies Taking turns:

Player flips a penny Heads, he gets a penny from the other

player Tails, he gives the penny to the other

player Until one player runs out of pennies

Questions

What each player’s chance of winning? What is the expected value of the game

to a player? How long will the game go on before a

player runs out of pennies? What is the probability after m moves

that player A has a pennies and player B has b pennies? Or the probability distribution over (a,b)?

Chance of winning

If A and B have the same number of coins, they’re equally likely to win.

With m coins to player A and n coins to player B A wins with probability m/(m+n) B wins with probability n/(m+n)

The player with more coins is more likely to win.

Expected Value

Definition: Exp(winnings)=Prob(winning)*value of winning

+Prob(losing)*value of losing Value of winning is m+n Value of losing is 0 So, if A starts with m coins and B with n:

E(A winnings) = m/(m+n)*(m+n) + n/(m+n)*0= m

E(B winnings) = n/(m+n)*(m+n) + m/(m+n)*0=n

Markov Chain

3,3

2,4

1,5

0,6

2,4

1,5

0,6

The probability of each transition is ½, except 0,6->0,6 and 6,0->6,0, wherethe probability is 1

How long

Relatively hard from the UCLA tutorial Sum over n of

n*[p((5,1);n)*0.5+p((1,5);n)*0.5] = Sum over n of n*p((5,1;n)

Starting from (3,3), about 8 turns

Probability Matrix

0,6 1,5 2,4 3,3 4,2 5,1 6,0

0,6 1 0 0 0 0 0 0

1,5 0.5 0 0.5 0 0 0 0

2,4 0 0.5 0 0.5 0 0 0

3,3 0 0 0.5 0 0.5 0 0

4,2 0 0 0 0.5 0 0.5 0

5,1 0 0 0 0 0.5 0 0.5

6,0 0 0 0 0 0 0 1

Two Steps

0,6 1,5 2,4 3,3 4,2 5,1 6,0

0,6 1 0 0 0 0 0 0

1,5 0.5 0.25 0 0.25 0 0 0

2,4 0.25 0 0.5 0 0.25 0 0

3,3 0 0.25 0 0.5 0 0.25 0

4,2 0 0 0.25 0 0.5 0 0.25

5,1 0 0 0 0.25 0 0.25 0.5

6,0 0 0 0 0 0 0 1

Three Steps

0,6 1,5 2,4 3,3 4,2 5,1 6,0

0,6 1 0 0 0 0 0 0

1,5 0.625 0 0.25 0 0.125 0 0

2,4 0.25 0.25 0 0.375 0 0.125 0

3,3 0.125 0 0.375 0 0.375 0 0.125

4,2 0 0.125 0 0.375 0 0.25 0.25

5,1 0 0 0.125 0 0.25 0 0.625

6,0 0 0 0 0 0 0 1

64 Steps

0,6 1,5 2,4 3,3 4,2 5,1 6,0

0,6 1 0 0 0 0 0 0

1,5 0.833 0 0 0 0 0 0.167

2,4 0.667 0 0 0 0 0 0.333

3,3 0.5 0 0 0 0 0 0.5

4,2 0.333 0 0 0 0 0 0.667

5,1 0.167 0 0 0 0 0 0.833

6,0 0 0 0 0 0 0 1

Steady States

Sooner or later, somebody wins (with probability=1)

Either party can win

The system has two stable steady states (bistable)

L 0

R 8

L.R 2

L.R.R.L 0

The “Game” with Molecules

L 2

R 10

L.R 0

L.R.R.L 0

L 1

R 9

L.R 1

L.R.R.L 0

L 0

R 8

L.R 0

L.R.R.L 1

Chemical Master Equation

A set of first-order differential equations describing the change in the probability of states with time t.

With time-independent reaction rates, the process is Markovian.

The Chemical Master Equation

We are interested in p(x; t), the probability that the chemical system will be in state x at time t.

The time evolution of p(x; t) is described by the Chemical Master Equation:

Where sμ is the stoichiometric vector for reaction μ, giving the changes in each type of molecule as a result of μ, and aμ is the propensity for reaction μ

Bistability

A steady state is a state that a system tends to stay in once it reaches it

A steady state can be stable or unstable

A system that has two stable steady states is called bistable.