STAT 430/510 Probability Lecture 16, 17: Compute by ...stat.wharton.upenn.edu/~pengyuan/stat430/notes/... · others. Each hunter independently hit his target with probability p. We’ve

STAT 430/510 Lecture 16-17: Conditioning

STAT 430/510 ProbabilityLecture 16, 17: Compute by Conditioning

Pengyuan (Penelope) Wang

Lecture 16 - 17

June 21, 2011


Computing Probability by Conditioning

A is an arbitrary eventIf Y is a discrete random variable,P(A) =

∑y P(A|Y = y)P(Y = y)

It is just the rule of total probability.

if Y is a continuous random variable,P(A) =

∫∞−∞ P(A|Y = y)fY (y)dy


Computing Probability by Conditioning

A is an arbitrary eventIf Y is a discrete random variable,P(A) =

∑y P(A|Y = y)P(Y = y)

It is just the rule of total probability.if Y is a continuous random variable,P(A) =

∫∞−∞ P(A|Y = y)fY (y)dy


Example

Let U be a uniform random variable on (0,1), and supposethat the conditional distribution of X , given that U = p, isbinomial with parameters (n,p). Find the probability thatX = 0.

P(X = 0) =

∫ 1

0P(X = 0|U = p)fU(p)dp

=

∫ 1

0P(X = 0|U = p)dp

=

∫ 1

0(1− p)ndp


Example

Let U be a uniform random variable on (0,1), and supposethat the conditional distribution of X , given that U = p, isbinomial with parameters (n,p). Find the probability thatX = 0.

P(X = 0) =

∫ 1

0P(X = 0|U = p)fU(p)dp

=

∫ 1

0P(X = 0|U = p)dp

=

∫ 1

0(1− p)ndp


Computing Expectations by Conditioning

The conditional expectation of X given Y = y is defined as

E(X |Y ) =∑

x

xP(X = x |Y ) (Discrete)

orE(X |Y ) =

∫x

xfX |Y (x |Y )dx (Continuous).



If Y is a discrete random variable,E [X ] =

∑y E [X |Y = y ]P(Y = y)

If Y is a continuous random variable,E [X ] =

∫y E [X |Y = y ]fY (y)dy

E [X ] = E [E [X |Y ]]



If Y is a discrete random variable,E [X ] =

∑y E [X |Y = y ]P(Y = y)

If Y is a continuous random variable,E [X ] =

∫y E [X |Y = y ]fY (y)dy

E [X ] = E [E [X |Y ]]


Example

There are some bulbs in a box. 30% of them are style Abulbs, which can last 10 hours with standard deviation 1hour; the other are style B bulbs, which can last 15 hourswith standard deviation 2 hour. If you choose on bulbrandomly, what is the expectation of its lifetime?

E(X ) = E(E(X |style)) = 10 ∗ 30% + 15 ∗ 70% = 13.5.


Example

There are some bulbs in a box. 30% of them are style Abulbs, which can last 10 hours with standard deviation 1hour; the other are style B bulbs, which can last 15 hourswith standard deviation 2 hour. If you choose on bulbrandomly, what is the expectation of its lifetime?E(X ) = E(E(X |style)) = 10 ∗ 30% + 15 ∗ 70% = 13.5.


Case 1: Expectation of a Sum of a Random Number of R.V.’s

Suppose that the number of people entering a departmentstore on a given day is a random variable with mean 50.Suppose further that the amounts of money spent by thesecustomers are independent random variables having acommon mean of 8 dollars. Finally, suppose also that theamount of money spent by a customer is also independentof the total number of customers who enter the store. Whatis the expected amount of money spent in the store on agiven day?


Example: Solution

Let N denote the number of customers that enter the storeand Xi be the amount spent by the ith customer.E [

∑Ni=1 Xi ] = E [E [

∑Ni=1 Xi |N]]

Since E [∑N

i=1 Xi |N = n] = E [∑n

i=1 Xi ] = nEX1,we have E [

∑Ni=1 Xi |N] = NEX1.

Thus E [∑N

i=1 Xi ] = E [NE [X1]] = EN × E [X1] =50× 8 = 400


Example: Solution


∑Ni=1 Xi ] = E [E [

∑Ni=1 Xi |N]]

Since E [∑N

i=1 Xi |N = n] = E [∑n



Thus E [∑N

i=1 Xi ] = E [NE [X1]] = EN × E [X1] =50× 8 = 400


Example: Solution


∑Ni=1 Xi ] = E [E [

∑Ni=1 Xi |N]]

Since E [∑N

i=1 Xi |N = n] = E [∑n



Thus E [∑N

i=1 Xi ] = E [NE [X1]] = EN × E [X1] =50× 8 = 400


Case 2: Method of Indicator: Recall the shooting duck example

10 hunters are waiting for ducks to fly by. When a flock ofducks flies overland, the hunters fire at the same time, buteach chooses his target at random, independently of theothers. Each hunter independently hit his target withprobability p. We’ve already computed the expectednumber of ducks got hit when a flock of size k fliesoverhead to be:Given k ducks, E [X ] = k [1− (1− p

k )10].

Now suppose the number of ducks in this flock follows aGeometric distribution with mean 12. What is the expectednumber of duck got hit?E [X ] = E(E(X |number of ducks)) =Σinf

k=1k [1− (1− pk )10]× P(k ducks),

where P(k ducks) = (1− 112)k−1 1

12 .




k )10].Now suppose the number of ducks in this flock follows aGeometric distribution with mean 12. What is the expectednumber of duck got hit?

E [X ] = E(E(X |number of ducks)) =Σinf

k=1k [1− (1− pk )10]× P(k ducks),


12 .




k )10].Now suppose the number of ducks in this flock follows aGeometric distribution with mean 12. What is the expectednumber of duck got hit?E [X ] = E(E(X |number of ducks)) =Σinf

k=1k [1− (1− pk )10]× P(k ducks),


12 .


Case 3: Another Example which is a little confusing

A miner is trapped in a mine containing 3 doors. The firstdoor leads to a tunnel that will take him to safety after 3hours of travel. The second door leads to a tunnel that willreturn him to the mine after 5 hours of travel. The thirdleads to a tunnel that will return him to the mine after 7hours. If we assume that the miner is at all times equallylikely to choose any one of the doors, what is the expectedlength of time until he reaches safety?


Example: Solution

Let X denote the amount of time until the miner reachessafety, and let Y denote the door he initially chooses.E [X ] = E [X |Y = 1]P(Y = 1) + E [X |Y = 2]P(Y =2) + E [X |Y = 3]P(Y = 3)

E [X |Y = 1] = 3, E [X |Y = 2] = 5 + E [X ],E [X |Y = 3] = 7 + E [X ]

P(Y = 1) = P(Y = 2) = P(Y = 3) = 1/3E [X ] = 15


Example: Solution


E [X |Y = 1] = 3, E [X |Y = 2] = 5 + E [X ],E [X |Y = 3] = 7 + E [X ]

P(Y = 1) = P(Y = 2) = P(Y = 3) = 1/3E [X ] = 15


Example: Solution


E [X |Y = 1] = 3, E [X |Y = 2] = 5 + E [X ],E [X |Y = 3] = 7 + E [X ]

P(Y = 1) = P(Y = 2) = P(Y = 3) = 1/3

E [X ] = 15


Example: Solution


E [X |Y = 1] = 3, E [X |Y = 2] = 5 + E [X ],E [X |Y = 3] = 7 + E [X ]

P(Y = 1) = P(Y = 2) = P(Y = 3) = 1/3E [X ] = 15


Computing Variance by Conditioning

The conditional variance of X given Y = y is defined as

Var(X |Y ) = E [(X − E [X |Y ])2|Y ]

But most of the time we just use the intuitive. Var(X |Y ) isjust the variance of X , but under the condition that Y isgiven.

The conditional variance formula:

Var(X ) = E [Var(X |Y )] + Var(E [X |Y ])


Computing Variance by Conditioning

The conditional variance of X given Y = y is defined as

Var(X |Y ) = E [(X − E [X |Y ])2|Y ]

But most of the time we just use the intuitive. Var(X |Y ) isjust the variance of X , but under the condition that Y isgiven.The conditional variance formula:

Var(X ) = E [Var(X |Y )] + Var(E [X |Y ])


Case 1:

There are some bulbs in a box. 30% of them are style Abulbs, which can last 10 hours with standard deviation 1hour; the other are style B bulbs, which can last 15 hourswith standard deviation 2 hour. If you choose on bulbrandomly, what is the standard deviation of its lifetime?Var(X ) = E(Var(X |style)) + Var(E(X |style)).

Var(X |style A) = 1,Var(X |style B) = 4, thusE(Var(X |style)) = 0.3× 1 + 0.7× 4 = 3.1.E(X |style A) = 10,E(X |style B) = 15, thusVar(E(X |style)) = 0.3× [10− (0.3× 10 + 0.7× 15)]2 +0.7× [15− (0.3× 10 + 0.7× 15)]2 = 5.25.SD(X ) =

√Var(X ) =

√3.1 + 5.25 = 2.89.


Case 1:

There are some bulbs in a box. 30% of them are style Abulbs, which can last 10 hours with standard deviation 1hour; the other are style B bulbs, which can last 15 hourswith standard deviation 2 hour. If you choose on bulbrandomly, what is the standard deviation of its lifetime?Var(X ) = E(Var(X |style)) + Var(E(X |style)).Var(X |style A) = 1,Var(X |style B) = 4, thusE(Var(X |style)) = 0.3× 1 + 0.7× 4 = 3.1.

E(X |style A) = 10,E(X |style B) = 15, thusVar(E(X |style)) = 0.3× [10− (0.3× 10 + 0.7× 15)]2 +0.7× [15− (0.3× 10 + 0.7× 15)]2 = 5.25.SD(X ) =

√Var(X ) =

√3.1 + 5.25 = 2.89.


Case 1:

There are some bulbs in a box. 30% of them are style Abulbs, which can last 10 hours with standard deviation 1hour; the other are style B bulbs, which can last 15 hourswith standard deviation 2 hour. If you choose on bulbrandomly, what is the standard deviation of its lifetime?Var(X ) = E(Var(X |style)) + Var(E(X |style)).Var(X |style A) = 1,Var(X |style B) = 4, thusE(Var(X |style)) = 0.3× 1 + 0.7× 4 = 3.1.E(X |style A) = 10,E(X |style B) = 15, thusVar(E(X |style)) = 0.3× [10− (0.3× 10 + 0.7× 15)]2 +0.7× [15− (0.3× 10 + 0.7× 15)]2 = 5.25.

SD(X ) =√

Var(X ) =√

3.1 + 5.25 = 2.89.


Case 1:

There are some bulbs in a box. 30% of them are style Abulbs, which can last 10 hours with standard deviation 1hour; the other are style B bulbs, which can last 15 hourswith standard deviation 2 hour. If you choose on bulbrandomly, what is the standard deviation of its lifetime?Var(X ) = E(Var(X |style)) + Var(E(X |style)).Var(X |style A) = 1,Var(X |style B) = 4, thusE(Var(X |style)) = 0.3× 1 + 0.7× 4 = 3.1.E(X |style A) = 10,E(X |style B) = 15, thusVar(E(X |style)) = 0.3× [10− (0.3× 10 + 0.7× 15)]2 +0.7× [15− (0.3× 10 + 0.7× 15)]2 = 5.25.SD(X ) =

√Var(X ) =

√3.1 + 5.25 = 2.89.


Case 2: Variance of Sum of a Random Number of R.V.’s

Suppose there are N people in household, and eachperson’s income independently and identically follows aNormal distribution with mean 30,000 dollars and standarddeviation 5,000 dollars. N follows a Geometric distributionwith mean 3. What is the expected value of the totalincome, and the variance?

Let X represent the total income. ThenE(X ) = E(E(X |N)) = E(30,000N) = 90,000 dollars.Var(X ) = E(Var(X |N)) + Var(E(X |N)) = E(5,0002N) +

Var(30,000N) = 5,0002 × 3 + 30,0002 × 1−1/3(1/3)2 .



Suppose there are N people in household, and eachperson’s income independently and identically follows aNormal distribution with mean 30,000 dollars and standarddeviation 5,000 dollars. N follows a Geometric distributionwith mean 3. What is the expected value of the totalincome, and the variance?Let X represent the total income. ThenE(X ) = E(E(X |N)) = E(30,000N) = 90,000 dollars.

Var(X ) = E(Var(X |N)) + Var(E(X |N)) = E(5,0002N) +

Var(30,000N) = 5,0002 × 3 + 30,0002 × 1−1/3(1/3)2 .



Suppose there are N people in household, and eachperson’s income independently and identically follows aNormal distribution with mean 30,000 dollars and standarddeviation 5,000 dollars. N follows a Geometric distributionwith mean 3. What is the expected value of the totalincome, and the variance?Let X represent the total income. ThenE(X ) = E(E(X |N)) = E(30,000N) = 90,000 dollars.Var(X ) = E(Var(X |N)) + Var(E(X |N)) = E(5,0002N) +

Var(30,000N) = 5,0002 × 3 + 30,0002 × 1−1/3(1/3)2 .


Variance of Sum of a Random Number of R.V.’s

Let X1,X2, · · · be a sequence of independent andidentically distributed random variables, and let N be anonnegative integer-valued random variable that isindependent of the sequence Xi , i ≥ 1. Then,

Var(N∑

i=1

Xi) = E [N]Var(X ) + (E [X ])2Var(N)


Case 3: The trap example

A miner is trapped in a mine containing 3 doors. The firstdoor leads to a tunnel that will take him to safety after 3hours of travel. The second door leads to a tunnel that willreturn him to the mine after 5 hours of travel. The thirdleads to a tunnel that will return him to the mine after 7hours. If we assume that the miner is at all times equallylikely to choose any one of the doors, what is the varianceof the length of time until he reaches safety?


Example

Var(X ) = E(Var(X |Y )) + Var(E(X |Y ))

Var(X |Y = 1) = 0; Var(X |Y = 2) = Var(X ); Var(X |Y =3) = Var(X ).SoE(Var(X |Y )) = 0 ∗ 1

3 + Var(X ) ∗ 13 + Var(X ) ∗ 1

3 = 23Var(X ).

E [X |Y = 1] = 3, E [X |Y = 2] = 5 + E [X ],E [X |Y = 3] = 7 + E [X ].We have computed that EX = 15 in a previous example.So E [X |Y = 1] = 3, E [X |Y = 2] = 20, E [X |Y = 3] = 22.Thus Var(E(X |Y )) = 218/3.Thus Var(X ) = 2

3Var(X ) + 218/3⇒ Var(X ) = 218.


Example



3 + Var(X ) ∗ 13 + Var(X ) ∗ 1

3 = 23Var(X ).


3Var(X ) + 218/3⇒ Var(X ) = 218.


Example



3 + Var(X ) ∗ 13 + Var(X ) ∗ 1

3 = 23Var(X ).

E [X |Y = 1] = 3, E [X |Y = 2] = 5 + E [X ],E [X |Y = 3] = 7 + E [X ].We have computed that EX = 15 in a previous example.So E [X |Y = 1] = 3, E [X |Y = 2] = 20, E [X |Y = 3] = 22.Thus Var(E(X |Y )) = 218/3.

Thus Var(X ) = 23Var(X ) + 218/3⇒ Var(X ) = 218.


Example



3 + Var(X ) ∗ 13 + Var(X ) ∗ 1

3 = 23Var(X ).


3Var(X ) + 218/3⇒ Var(X ) = 218.


All the above stuff can also be computed using:

Var(X ) = E(X 2)− (EX )2.E(X 2) can be computed by conditioning.The bulb example in Case 1.

Documents

STAT 430/510 Probability Lecture 16, 17: Compute by ...stat.wharton.upenn.edu/~pengyuan/stat430/notes/... · others. Each hunter independently hit his target with probability p. We’ve