30
ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum 1 ENGG2012B

ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

Embed Size (px)

Citation preview

Page 1: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

ENGG2012BLecture 16

Conditional probability

Kenneth Shum

kshum 1ENGG2012B

Page 2: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

Midterm

• 22nd Mar• One and a half hour.• Bring calculator and blank papers.• Close-book and close-note exam.• Coverage:

– Lecture 1 to 15.– Tutorial 1 to 7.– Homework 1 to 3.

kshum ENGG2012B 2

Page 3: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

Derangements of n=4 objects

• Let be the set of all 24 permutations of 1,2,3,4.• For i = 1,2,3,4, let Ai be the set of permutations

which fix i.• A1 = {1234, 1243, 1324, 1342, 1423, 1432}.• A2 = {1234, 1243, 3214, 3241, 4213, 4231}.• A3 = {1234, 1432, 2134, 2431, 4132, 4231}.• A4 = {1234, 1324, 2134, 2314, 3124, 3214}.• A permutation of {1,2,3,4} not in A1 to A4 has no

fixed point, and is called a derangement.kshum ENGG2012B 3

Page 4: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

Venn diagram

kshum ENGG2012B 4

2143 2341 2413 3142 3412 3421 4123 4312 4321

A2

A3

A4

1234

1243

2134

14324231

32141324

13421423

32414213

2431

4132

2314

3124

Page 5: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

The case n=4 (cont’d)• A1 A2 = {1234, 1243}.• A1 A3 = {1234, 1432}.• A1 A4 = {1234, 1324}.• A2 A3 = {1234, 4231}.• A2 A4 = {1234, 3214}.• A3 A4 = {1234, 2134}.• A1 A2 A3 = A1 A2 A4 = A1 A3 A4 =A2 A3 A4

= A1 A2 A3 A4 ={1,2,3,4}.• By PIE, the number of derangements for n=4 is equal to

4! – 46 + 62 – 41 + 1 = 9.• Indeed, the nine derangements for n=4 are

2143, 2341, 2413, 3142, 3412, 3421, 4123, 4312, 4321.kshum ENGG2012B 5

Page 6: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

Probability of no fixed number• For n=4, if we pick a permutation randomly,

– the probability of no fixed number is 9/24= 0.375.– the probability of at least one fixed number is 15/24= 0.625.

• For large n, it can be shown that the probability of no fixed number is approximately 1/e = 0.3679…

kshum ENGG2012B 6

n Probability that a random permutation of length n is a derangement

3 0.3333

4 0.3750

5 0.3667

6 0.3681

7 0.3679

Page 7: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

John Venn (1834-1923)

• British logician and philosopher• http://en.wikipedia.org/wiki/John_Venn

kshum ENGG2012B 7

Page 8: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

Classical definition of probability

• From Laplace’s Théorie analytique des probabilités (1812)– “The probability of an event is

the ratio of the number of cases favourable to it, to the number of all possible cases.”

kshum ENGG2012B 8

http://en.wikipedia.org/w

iki/Pierre-

Sim

on_Laplace

Page 9: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

BASIC PROBABILITY

kshum ENGG2012B 9

Page 10: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

Disjoint events

• Two events are called disjoint if the intersection is empty, i.e., if they have no overlap.

• The calculation of union of events in general is complicated when the events overlap.

• It is much easier if we want to compute the probability of a union of disjoint event. We simply add the probability of the events

kshum ENGG2012B 10

Page 11: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

Union of disjoint events

kshum ENGG2012B 11

A

B

C

Pr(A B C) = Pr(A) + Pr(B) + Pr(C).A, B, C mutually disjoint

Page 12: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

Partition of sample space

kshum ENGG2012B 12

A

B

C

D

Pr(A) + Pr(B) + Pr(C) + Pr(D)= Pr(A B C D) = 1

A B C D = , A, B, C, D mutually disjoint

Page 13: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

Law of total probability

kshum ENGG2012B 13

A

B

C

D

Pr(E) = Pr(EA)+ Pr(EB)+ Pr(EC)+ Pr(ED).

A B C D = , A, B, C, D mutually disjoint

E

Page 14: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

CONDITIONAL PROBABILITY

kshum ENGG2012B 14

Page 15: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

Example• There are four cards, two of them are red

and two of them are black.• Shuffle the four cards and lay out the cards

on the table with the front of each card facing downwards.

• What is the probability that the first card is red?

• We reveal the last card. – If it turns out that the last card is red, what is

the probability that the first card is red?– If it turns out that the last card is black, what

is the probability that the first card is red?

kshum ENGG2012B 15

Page 16: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

Conditional probability

• If we are given an addition information that the outcome is in event B, then we can update the likelihood to |AB|/ |B|.

• If Pr(B) is nonzero, then we define the conditional probability of A given B by

Pr(A|B) = Pr(AB)/Pr(B).

kshum ENGG2012B 16

A

B

At the beginning, the probability of event A is |A| / ||, assuming that all outcomes in are equally likely.

Page 17: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

The law of total probability in terms of conditional probability

kshum ENGG2012B 17

A

B

C

D

Pr(E) = Pr(EA)+ Pr(EB)+ Pr(EC)+ Pr(ED)

= Pr(E|A)Pr(A)+ Pr(E|B)Pr(B)+ Pr(E|C)Pr(C)+ Pr(E|D)Pr(D).

A B C D = , A, B, C, D mutually disjoint

E

Page 18: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

Example

• Alice rolls a fair dice two times. • Suppose Bob is told that the face value of the

first roll is 6. What is the probability the second roll is also 6?

• Suppose Carol is told that the face value of one of the roll is 6. What is the probability that the other roll is also 6?

kshum ENGG2012B 18

Page 19: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

Independent events

• If P(A|B) = P(A), then it means that the likelihood of the event A does not change after we are told that event B has occurred. In this case, event A is said to be independent of event B.

• As Pr(A|B) = Pr(AB)/Pr(B), event A is independent of B if Pr(AB) = Pr(A) Pr(B).

• Two events A and B are said to be independent, or statistically independent, if

Pr(AB) = Pr(A) Pr(B).

kshum ENGG2012B 19

Page 20: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

Example• Roll two fair dice. • Let A be the event that the face value of the first die is

even. • Let B be the event that the sum of the dice is odd.• Are events A and B independent?

kshum ENGG2012B 20

11 12 13 14 15 16

21 22 23 24 25 26

31 32 33 34 35 36

41 42 43 44 45 46

51 52 53 54 55 56

61 62 63 64 65 66

11 12 13 14 15 16

21 22 23 24 25 26

31 32 33 34 35 36

41 42 43 44 45 46

51 52 53 54 55 56

61 62 63 64 65 66

First numberis even

Sum is odd

Page 21: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

Pairwise independent does not imply Pr(ABC)=Pr(A)Pr(B)Pr(C).

• Let be the sample space {0,1,2,3}. The four outcomes are equally likely.

• Let A be the event {0,1}, B be the event {0,2}, and C be the event {0,3}.

• The three events A, B and C are pairwise independent, because– Pr(A) Pr(B) = (1/2)2 = 1/4 = Pr(AB)– Pr(B) Pr(C) = (1/2)2 = 1/4 = Pr(BC)– Pr(C) Pr(A) = (1/2)2 = 1/4 = Pr(CA)

• However, – Pr(ABC) = 1/4– Pr(A) Pr(B) Pr(C) = (1/2)3 =1/8

kshum ENGG2012B 21

1

23

0

A

BC

Page 22: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

Example• There are two boxes. • The first box contains 3 red balls and 7 blue balls.• The second box contains 16 red balls and 4 blue balls.• Perform the following random experiment

– Throw a fair die.– If the number is 1, pick a ball randomly from the first box.– If the number is 2 to 5, pick a ball randomly from the second

box.• What is the probability that a red ball is drawn?• What is the probability that a blue ball is drawn?• Given that a red ball is picked, what is the probability that

the first box is chosen?

kshum ENGG2012B 22

Page 23: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

The Bayes’ rule

• Let A and B be two events. Suppose that the probability of B is nonzero.

• Probability of A given B can be computed from the probability of B given A by

kshum ENGG2012B 23

http://en.wikipedia.org/wiki/Bayes'_theorem

Page 24: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

Thomas Bayes (1701-1761)

• English minister and mathematician• http://en.wikipedia.org/wiki/Thomas_Bayes

kshum ENGG2012B 24

Page 25: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

The Monty Hall Problem

• Quote from wikipedia– http://en.wikipedia.org/wiki/Monty_Hall_problem

Suppose you're on a game show, and you're given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what's behind the doors, opens another door, say No. 3, which has a goat. He then says to you, "Do you want to pick door No. 2?" Is it to your advantage to switch your choice?

• An explanation from youtube– http://www.youtube.com/watch?v=mhlc7peGlGg

kshum ENGG2012B 25

Page 26: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

Explanation using conditional probability

• We use – “100” to represent “prize behind the first door”.– “010” to represent “prize behind the second door”.– “001” to represent “prize behind the third door”.

• The host may need to flip a coin in order to open the door. We use “H” and “T” for the outcome of the coin toss.

• Set up a sample space of six elements = {100H, 100T, 010H, 010T, 001H, 001T}.

Each outcome is equally probable.

kshum ENGG2012B 26

Page 27: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

Before any door is opened by the host

• Suppose you choose door 1

kshum ENGG2012B 27

100

010

001

1/3

1/3

1/3

The probability thatthere is a car behind thefirst door is 1/3.

Page 28: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

After a door is opened• Suppose you choose door 1.• A door is opened by the host of the game.

kshum ENGG2012B 28

100

010

001

1/3

1/3

1/3

H

T

H

T

H

T

100

100

010

010

001

001

Door 3 is opened

Door 2 is opened

Door 3 is opened

Door 3 is opened

Door 2 is opened

Door 2 is opened

Page 29: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

Probability given that door 3 is opened

• Suppose you choose door 1.• Suppose door 3 is opened• Pr(Prize is behind door 1 | door 3 is opened) = 1/3• If you switching to door 2, you will win with probability 2/3

kshum ENGG2012B 29

100

010

001

1/3

1/3

1/3

H

T

H

T

H

T

100

100

010

010

001

001

Door 3 is opened

Door 3 is opened

Door 3 is opened

Page 30: ENGG2012B Lecture 16 Conditional probability Kenneth Shum kshum1ENGG2012B

Probability given that door 2 is opened

• Suppose you choose door 1.• Suppose door 3 is opened• Pr(Prize is behind door 1 | door 2 is opened) = 1/3• If you switching to door 3, you will win with probability 2/3

kshum ENGG2012B 30

100

010

001

1/3

1/3

1/3

H

T

H

T

H

T

100

100

010

010

001

001

Door 2 is opened

Door 2 is opened

Door 2 is opened