Many useful applications, especially in queueing systems, inventory management, and reliability analysis. A connection between discrete time Markov chains

• Many useful applications, especially in queueing systems, inventory management, and reliability analysis.

• A connection between discrete time Markov chains and continuous time Markov chains.

Exponential distribution and the Poisson process

A continuous random variable X is said to have an

with parameter , 0, if its probability density

function is given by

0 ( )

0,

xe xf x

exponential

distribution

0

1 0 ( ) ( )

0, 0

xx

x

e xF x f y dy

x

The exponential distribution

0

0 0

( ) ( )

Integrating by parts leads to

1( )

x

x x

E X xf x dx x e dx

E X xe e dx

( )

0 0

0

22

2 3

0 0

The moment generating function ( ) is given by

( ) [ ]

, for

( ) 1[ ]

( ) 2[ ]

( )

tX tx x t x

t

t t

t

t E e e e dx e dx

tt

d tE X

dt

d tE X

dt t

2

2 22

2

1( ) [ ] [ ]Var X E X E X

A random variable is said to be memoryless if

( | ) ( ) for all , 0

( , ) ( )( )

( ) ( )

( ) ( ) ( )

P X s t X t P X s s t

P X s t X t P X s tP X s

P X t P X t

P X s t P X t P X s

The memoryless property

( )

If has the exponential distrbution, then

( ) ( ) ( )s t s t

X

P X s t e e e P X t P X s

Exponentially distributed random variables arememoryless.

It can be shown that the exponential distribution is the only distribution that has the memoryless property.

Example 1: The amount of time one spends in a bank is exponentially distributed with mean ten minutes ( = 1/10). What is the probability that a customer spends more than 15 minutes? What is the probability that the customer spends more than 15 minutes given that she is in the bank after 10 minutes?

15 1.5

5 0.5

( 15) 0.220

( 15 | 10) ( 5) 0.604

P X e e

P X X P X e e

Example 1: The amount of time one spends in a bank is exponentially distributed with mean ten minutes ( = 1/10). What is the probability that a customer spends more than 15 minutes? What is the probability that the customer spends more than 15 minutes given that she is in the bank after 10 minutes?

Example 2: Consider a branch of a bank with two agents serving customers. The time an agent takes with a customer is exponentially distributed with mean 1/. A customer enters and finds the two agents busy serving two other customers. What is the probability that the customer that just entered would be last to leave?

Suppose that X1, X2, ..., Xn are independent exponential random variables, with Xi having rate i, i=1, ..., n.What is P(min(X1, X2, ..., Xn )>x)?

The minimum of n exponentially distributed random variables

1 2

1 2 1 2

1 2

(min( , , ..., ) ) {( ), ( ), ..., ( )}

{( )} {( )} ... {( )}

...

n n

n

x x

P X X X x P X x X x X x

P X x P X x P X x

e e e

1 2( ... )

1 2

1 2

The distribution of the random variable (min( , , ..., )

1is exponentially distributed with mean .

...

N

N

x

x

n

n

e

P X X X

Suppose that X1, X2, ..., Xn are independent exponential random variables, with Xi having rate i, i=1, ..., n.What is P(min(X1, X2, ..., Xn )>x)?

The minimum of n exponentially distributed random variables

Suppose that X1 and X2 are independent exponentially distributed random variables with rates 1 and 2.What is P(X1 < X2)?

Comparing two exponentially distributed random variables

1

1 1

2 1 1 2

1 2 1 2 10

1 2 1 1 2 10 0

( )1 10 0

1

1 2

( ) ( | ) ( )

( | ) ( )

.

X

x x

x x x

P X X P X X X x f x dx

P X X X x e dx P x X e dx

e e dx e dx

Suppose that X1 and X2 are independent exponentially distributed random variables with rates 1 and 2.What is P(X1 < X2)?

Comparing two exponentially distributed random variables

Suppose that X1 and X2 are independent exponential random variables with rates 1=2 =. What is the distribution of fX1+X2 (x)?)?

The sum of 2 identical exponentially distributed random variables

1 2 1 20

( )

0

2 2

0

( ) ( ) ( )

x

X X X X

x s x s

xx x

f x f s f x s ds

e e ds

e ds xe

Suppose that X1 and X2 are independent exponential random variables with rates 1=2 =. What is the distribution of fX1+X2 (x)?

The sum of 2 identical exponentially distributed random variables

1 2

1

...

( )( )

( 1)!

n

nx

X X X

xf x e

n

Suppose that X1,..., XN are independent exponential random variables with rates i= for i =1, ..., n. What is the distribution of fX1+...

+Xn (x)?

X1,..., XN has the gamma distribution with parameters n and .

The sum of n identical exponentially distributed random variables

Suppose that X1 and X2 are independent exponential random variables with rates 1≠2. What is the distribution of fX1+X2 (x)?

The sum of 2 exponentially distributed random variables

Suppose that X1 and X2 are independent exponential random variables with rates 1≠2. What is the distribution of fX1+X2 (x)?

The sum of 2 exponentially distributed random variables

1 2 1 2

1 2

2 1 2

2 1

0

( )1 20

( )1 2 0

1 22 1

1 2 1 2

( ) ( ) ( )

x

X X X X

x s x s

xx s

x x

f x f s f x s ds

e e ds

e e ds

e e

The Poisson Process

A stochastic process {N(t), t ≥ 0} is said to be a counting process if N(t) represents the total number of “events” that occur by time t (i.e., in the time interval [0, t]).

Counting Processes


Example 1: N(t) is the number of customers that enter a store at or prior to time t. An event corresponds to a person entering the store.

Counting Processes



Example 2: N(t) is the number of individuals born at or prior to time t. An event occurs whenever a child is born.

Counting Processes



Example 2: N(t) is the number of individuals born at or prior to time t. An event occurs whenever a child is born.

Example 3: N(t) is the number of calls made to a technical help line at or prior to time t. An event occurs whenever a call is placed.

Counting Processes

A counting process satisfies the following properties.

(i) N(t) ≥ 0.(ii) N(t) is integer valued.(iii) If s < t, then N(s) ≤ N(t).(iv) For s < t, N(t) – N(s) equals the number of events that occurs in the time interval (s, t].

Properties of counting processes

A counting process is said to possess independent increments if the number of events that occur in disjoint intervals are independent.


Example 1: The number of customers N(10) that enter the store in the interval [0, 10] is independent from the number of customers N(15) – N(10) that enter the store in the interval (10, 15].


Example 1: The number of customers N(10) that enter the store in the interval [0, 10] is independent from the number of customers N(15) – N(10) that enter the store in the interval (10, 15].

Example 2: The number of individuals N(10) born in the interval [1996, 2000] is not independent from the number of individuals N(2004) – N(2000) that enter the store in the interval (2000, 2004].

A counting process is said to possess stationary increments if the distribution of the number of events that occur in an interval depend only on the length of the interval and not the starting time of the interval.

A counting process is said to possess stationary increments if the distribution of the number of events that occur in an interval depend only on the length of the interval and not the starting time of the interval.

Example 1: The number of customers N(t) – N(s) that enter the store in the interval (s, t] does not depend on s (this is true if there is not a particular time of day where more customers enter the store).

The Poisson processes

The counting process {N(t) t ≥ 0} is said to be a Poisson process having rate , > 0, if

(i) N(0) = 0.(ii) The process has independent increments.(iii) The number of events in any interval of length t is Poisson distributed with mean t. That is for all s, t ≥ 0

( ){ ( ) ( )} , for 0,1,...

!

nt t

P N t s N t e nn

The distribution of interarrival times

• Let Tn describe the time that elapses between (n-1)th event and the nth event for n > 1 and let T1 be the time of the first event.

• The sequence {Tn , n = 1, 2, ...} is called the sequence of interarrival times.


• Let Tn describe the time that elapses between (n-1)th event and the nth event for n > 1 and let T1 be the time of the first event.

• The sequence {Tn , n = 1, 2, ...} is called the sequence of interarrival times.

Example: if T1 = 5 and T2 = 10 the first event arrives at time t = 5 and event 2 occurs at time t = 15.


• P(T1 > t) = P(N(t) = 0) = e-t T1 has the exponential distribution.



• P(T2 > t) = E[P(T2 > t|T1) ]

Since P(T2 > t|T1=s) = P(0 events in (s, s+t]|T1=s) = P(0 events in (s, s+t]) = e-t Then, P(T2 > t) = E[P(T2 > t|T1) ] = e-t T2 has the exponential



• P(T2 > t) = E[P(T2 > t|T1) ]

Since P(T2 > t|T1=s) = P(0 events in (s, s+t]|T1=s) = P(0 events in (s, s+t]) = e-t Then, P(T2 > t) = E[P(T2 > t|T1) ] = e-t T2 has the exponential

•The same applies to other values of n Tn has the exponential distribution.


Let Tn denote the inter-arrival time between the (n-1)th event and the nth event of a Poisson process, then the Tn (n=1, 2, ...) are independent, identically distributed exponential random variables having mean 1/.

Documents

Many useful applications, especially in queueing systems, inventory management, and reliability analysis. A connection between discrete time Markov chains