1

Lecture 5. Reminder + Warming-upConditional Probability.Independence

2

Reminder

B

3

A few rules:

Complement Rule: The probability that A does not occur is equal to the probability that the complement of event A occurs. P(Ac) = 1 - P(A). Difference Rule: If A is a subset of B, then the probability of B occurring but not A is P(B) - P(A) = P(B Ac). Inclusion-Exclusion Rule: The probability of either A or B (or both) occurring is P(A U B) = P(A) + P(B) - P(AB).

Solve it:

If the probability of having blew eyes is 10%, the probability of having dark hair is 75%, and the probability of being a blue-eyed dark haired person is 9%, what is the probability of •not having blue eyes? [find P(Ac)] •having blue eyes but not dark hair? [find P(A) - P(AB)] •having blue eyes and/or dark hair? [find P(A U B)]

4

Warming up

5

Consider three problems:

(a) You make a new friend and you ask if he has any children. Yes, he says, two. Any girls? Yes, he says. What is the probability that both are girls?

Answer: 1/3.

(b) You make a new friend and you ask if she has any children. Yes, she says, two – ages 6 and 9. Is the oldest a girl? You ask. Yes, she says. What is the probability that both of her children are girls?

Answer: 1/2

(c) You make a new friend and you ask if he has any children. Yes, he says, two. Any girls? Yes. Next day you meet her with a young girl. Is this your daughter? Yes, she says.

What’s the probability that both of her children are girls?

Answer: 1/2

6

Seemingly, the problem (c) is not different from ( a).

Why are the answers different?

At Home:

Please, try to figure out the nature of this apparent paradox.

7

Solution of the extra credit question (HW4) Here is a solution to the test problem 2.1 offered by one of the students.Problem 2.1 A committee of 5 is chosen from a group of 8 men and 4 women. What is the probability the group contains a majority of women?Suggested SolutionFor a group to contain a majority of women there are two possibilities:wwwwm and wwwmm.The probabilities are:P(wwwwm) = (4/12)(3/11)(2/10)(1/9)(8/8) = 0.002.P(wwwmm) = (4/12)(3/11)(2/10)(8/9)(7/8) = 0.014.P(majority) = 0.002 + 0.014 = 0.016.QuestionYou can see that this answer differs from the solution discussed in the class. At the same time, the calculation of P(wwwmm) and P(wwwmm) is apparently correct.Please, fix the solution using the same approach. 

8

Solution:

The solution correctly described two of many contributions to the event A (majority of women),

namely wwwwm and wwwmm.

The total number the contributions are C(5,1) =5 for the first group (4 women, 1 man) and C(5,2) = 10 for the second group (3 women, 2 men).

It’s easy to verify that, regardless the order the probabilities within each group are the same.

For instance, P(wwwwm) = P(wwmww) and P(wwwmm)=P(wmwwmw). As a result,

P(A) = C(5,1) P(wwwwm) + C(5,2) P(wwwmm)== 5/33.

9

Some problems involving conditioning

Here is the classical “hit and run” problem

A cab was involved in a hit and run accident at night. Two cab companies, Green Cab and Blue Cab, operate in the city. You are given the following data:

(a) 85% of cabs in the city are Green and 15% are Blue.(b) A witness identified the cab as Blue. (c) The court tested the reliability of the witness and found that he correctly identified one of the two colors 80% of the time and failed 20%.

What is the probability that the cab involved in the accident was Blue rather than Green?

Make your best guess.

10

Consider a simpler problemBill's sister Debby lives in Canada. Learning that Debby was pregnant, Bill drew a sample space diagram based on (his interpretation of ) their family history . He was eager to have a niece because he already had two nephews

Using this map, he estimated a probability of the event F= "at least one of newborns will be a girl“. What was P(F)?

G

B

BB

BG

GG

1/3

1/3

1/6

1/12

1/12

11

He found that P(F)=7/12. Later on it turned out that Debby was expecting twins. Based on this new information, Bill redrew the sample diagram, First, he noticed that the probabilities of singletons became 0. This fact turned the area marked by a green rectangle into the new sample space. The probabilities of the twins should now be rescaled, thought Bill, so that their ratios will stay the same while they should now sum up to 1. This resulted in the following distribution of probabilities:

G

B0

0

1/4 BB

BG

GG1/4

1/2

Using this diagram Bill found: P(F)=P(BG)+P(GG)=3/4. Its increase by 1/6 relative to the initial estimate made Bill even happier.

12

The probability of F under the condition (the hypothesis) that B has occurred is called a conditional probability P(F|B).

B1/3

1/3

1/12 BB

In our example, Bill was considering a probability P(F|T) that at least one of the newborns will be a girl under the condition that twins will be born.

His “rescaling” approach is correct but too clumsy.

Bill could’ve recognized that the “rescaling” was not necessary.

The same result could be derived as

P(F/T) = (1/6 + 1/12)/(1/12 + 1/6 + 1/12)= (3/12) / (1/3) =3/4.

But what is the meaning of the nominator?

This is exactly the probability of FT, intersection between the event F and the condition T

F

T

G

1/6

1/12

BG

GGFT

13

It is also obvious that the denominator is exactly the probability of T, the condition itself. In other words, Bill couild have avoided rescaling by using the universal formula

B1/3

1/3

1/12 BB

F

T

P(F|T)= P(FT) / P(T) (1)

G

1/6

1/12

BG

GGFT

The probability of F given T is the probability of the intersection between F and T divided by the probability of T.

14

P(FT) = P(F|T)P(T)= P(T|F)P(F)) (2)

The formula (1) can also be rewritten as

In this representation it is called "the multiplication rule“ for the conditioning.

15

Examples:

1. A fair coin is flipped twice. What is the conditional probability that both flips result in heads, given that the first flip results in heads?  

Solution: If E = {HH} and F = {HH, HT}, then 

   P(E|F) = P(EF)/P(F) = [P{HH}]/[P{HH, HT}] 

                                    = (1/4)/(2/4) = 1/2. 

Notice that the probability of two heads ( 1 / 4 ) is less than the probability of two heads given that the first is head.

16

2. Suppose we draw two cards out of deck of 52 without replacement. Let A =“The first card is a spade”, and B= “The second card is a spade”. What is the probability that both A and B occur.

P(AB) = P(B|A)P(A)=(12/51)*( 1/4)

Note that we computed P(B|A) by thinking about the situation that exists after A has occurred rather than using the definition (1). It’s more common to use P(A) and P(B|A) to compute P(AB) rather than using P(AB) and P(A) to compute P(B|A).

You probably noticed that we already used the conditional probability when we solved similar problems before, without calling it by name.

"Good Heavens! For more than forty years I have been speaking prose without knowing it." Jean Baptiste Moliere, “The Middle Class Gentleman “

17

3 Two fair coins are tossed and you are told that at least one coin came up heads.What is the probability that both coins came up heads?

Warning : the two “instinctive” answers, ½ and ¼ are both incorrect. Use the conditional probability formula (1)

Solution:

(1) introduce two events:B=“ At least one coin came up heads”; A= “Both coins came up heads”

(2) choose the relevant formula:P(A|B) = P(AB)/P(B).

(3) to find P(AB) and P(B), notice that B= {HT,TH,HH}; AB = {HH} (intersection);

(4) As a result, P(B) = ¾; P(AB)= ¼; P(A|B)=(1/4)/(3/4) = 1/3.

18

4. And here is one of many examples showing that the results of conditioning are often not too obvious.

A person picks 13 cards out of a deck of 52. Let A2=“He receives at least two Aces”. A1=“He receives at least one Ace”, and H=“He has the Ace of Hearts”.

Question Which probability is larger, P(A2|A1) or P(A2 |H)?

In other words, which is more likely: having two aces given that he has at least one Ace, or having two aces given that he has an Ace of hearts.

Make your best guess.

Now, try to solve this problem (extra credit)

19

5. “DNA Fingerprinting in the Courts” by  Bruce Weir,  North Carolina State University

As a part of your home assignment, please watch this video. You can find

it here . Use Ctr+F to search for the author’s name, Weir, or the title to locate the video.

It runs about an hour. You can browse the site to find a lot of other interesting lectures.

20

6. It’s time now to discuss the “hit and run” problem.

A cab was involved in a hit and run accident at night. Two cab companies, Green Cab and Blue Cab, operate in the city. You are given the following data:

(a) 85% of cabs in the city are Green and 15% are Blue.(b) A witness identified the cab as Blue. (c) The court tested the reliability of the witness and found that he correctly identified one of the two colors 80% of the time and failed 20%.

What is the probability that the cab involved in the accident was Blue rather than Green?

Let’s firs express the question in terms of a conditional probability.

What event plays a role of the condition?

Can it be (a) ? (b) ? ( c) ?

21

SB="Witness says it's blue", B="The car was Blue“. What is P(B|SB)=?Let’s make now a diagram of everything that we know.First, I drew these two rectangular areas representing the shares of the Green and Blue cabs. But then I was confused trying to figure out how to depict the A event in the same picture. Let’s try it together. First, notice that due to a possible witness error, the A event contains not only the blue area (which would be the case if the witness was absolutely reliable) but also the green area. What are their relative portions? SBGreen= 0.2*0.85=0.17. A Blue = 0.8*0.15=0.12.

A

P(B|SB)=P(BSB) / P(SB)=0.12/(0.17+0.12)=0.12/0.29=0.41

Btw, check that the total P (SB OR SG) = 1

22

A question for the self-test:

What would be the probability that the car was green What if the witness testified that it was green?

23

7. A laboratory blood test is 95% effective in detecting a certain disease when it is, in fact, present (sensitivity). However, the test also yields a "false positive" result for 1% of the healthy persons tested. (That is, if a healthy person is tested, then, with probability .01 the test result will be positive.) If .5% of the population actually has the disease (prevalence), what is the probability a person has the disease given that the test result is positive? 

  

Solution.Let D = “the tested person has the disease” and Pos = “the test result is positive”. The desired probability P(D|Pos) is given by 

P(D|Pos)=P(D Pos)/P(Pos)  =[P(Pos|D) P(D)]/[P(Pos|D) P(D)+P(Pos|DC])P(DC)]   =(.95) (.005)/[(.95) (.005) + (.01) (.995)] =.323.  

24

8. The Monty Hall problem (MHP)

This problem is named for the host of the TV show “let’s make a deal”.

Three curtains are numbered 1,2 and 3. Behind one curtain is a car, behind two others are donkeys.

You pick a curtain, say #1. Then host opens one of the two remaining curtains, say # 3, to reveal a donkey.

What is the probability that you will win given that there is a donkey behind #3? Should you switch curtains and pick #2 if you are given a chance?

25

A few historical notes ( http://www.mathpuzzle.com/MontyHall.txt, The man who loved only numbers ,

etc). Btw, does anyone want to make a special presentation, and to discuss the solution using the Mathematica modeling? I will supply

the volunteer with required materials

An earlier version of MHP, the Three Prisoner Problem, was analyzed in 1959 by Martin Gardner in the journal Scientific American. He called it "a wonderfully confusing little problem" and presciently noted that "in no other branch of mathematics is it so easy for experts to blunder as in probability theory."

26

The fierce debates around the MHP erupted after Marylin vos Savant *, the columnist for the Parade magazine, answered this brain teaser advising her correspondent to switch the door. Sticking with the first choice gives a 1/3 chance of winning, but switching doubles it to 2/3.

*Marylin von Savant was pronounced by the Guinness Book of World Records as the

“person with Highest IQ ever recorded” (228).

She sported a wedding ring of pyrolytic carbon, a special material that was invented by

her husband, Robert Jarvik, creator of the First Artificial (“Jarvik”) Heart .

27

The experts responded in force to Ms. Savant's column. Of the critical letters she received, close to 1,000 carried signatures with Ph.D.'s, and many were on letterheads of mathematics and science departments.

"Our math department had a good, self-righteous laugh at your expense," wrote Mary Jane Still, a professor at Palm Beach Junior College.

". . . I am sure you will receive many letters from high school and college students. Perhaps you should keep a few addresses for help with future columns." -W. Robert Smith, Ph.D., Georgia State University 

"You are utterly incorrect. . . How many irate mathematicians are needed to get you to change your mind? -E. Ray Bobo, Ph.D., Georgetown University 

". . . If all those Ph.D.s were wrong, the country would be in very serious trouble." -Everett Harman, Ph.D., U.S. Army Research Institute 

28

Robert Sachs, a professor of mathematics at George Mason University: "You blew it! Let me explain: If one door is shown to be a loser, that information changes the probability of either remaining choice -- neither of which has any reason to be more likely -- to 1/2. As a professional mathematician, I'm very concerned with the general public's lack of mathematical skills. Please help by confessing your error and, in the future, being more careful."

Later, Dr. Sachs, was one of the few with the grace to concede his mistake.

"I wrote her another letter," Dr. Sachs said last week, "telling her that after removing my foot from my mouth I'm now eating humble pie. I vowed as penance to answer all the people who wrote to castigate me. It's been an intense professional embarrassment."

29

Many people, including some professional mathematicians, argue that two unopened curtains are the same, so they each will contain car with probability ½ , and hence there is no point in switching. This answer is incorrect.

To compute an answer, we should first make an assumption about how the host behaves.

Suppose that he always chooses to show you a donkey, and picks at random if there are two unchosen curtains with donkeys.

Assuming you pick curtain # 1, there are three possibilities:

30

#1 #2 #3 Host’s action

Case 1 donkey donkey car Opens #2

Case 2 donkey car donkey Opens #3

Case 3 car donkey donkey Opens #2 or #3

Now, P(open door #3) = P(case 2, open door#3) + P(case 3, open door 3);

(note that “,” also means “and” or “”)

P(case 2, open door#3) =1*1/3 = 1/3; P(case 3, open door#3)= 1/3*1/2=1/6.

It gives: P(open door #3) = 1/3+1/6=1/2.

What is the probability you will win given that there is donkey behind #3 (or the probability of winning given that MH opens #3)

)3#(

)3#3()3#|3(

dooropenP

dooropencasePdooropencaseP

31

This answer is “obvious”. Initially we have 1/3 probability of picking a winning door. Opening the door with a donkey does not change this probability. It means that switching the door increases the probability from 1/3 to 2/3.

And it follows that

P(case 3|open door 3) = P(case 3,Open door 3)/P(open door #3) =1/6/1/2 = 1/3.

3/12/1

6/1

)3#(

)3#,3()3#|3(

dooropenP

doorOpencasePdooropencaseP

32

Independent events

Definition: Two events E and F are independent if

P(E|F)=P(E) and P(F|E) = P(F) (3)

The equivalent definition: The events A and B are independent if P(AB)=P(A)P(B) (4)

Assignment: prove that these definitions are equivalent

33

Two examples of the independent events

1. Flip two coins. A =“The first coin shows H”, B=“The second coin shows H”. Prove that A and B are independent.

Hint: Omega={{HH},{HT},{TH},{TT}}

2. Roll two dice. A="The first die shows 5”, B="The second die shows 2". P(A)=1/6, P(B) = 1/6, P(AB)= P({52})=1/36.

Dependent events

Roll two dice. A="The sum of the two dice is 9". B="The first die is 2". P(B)=1/6. A = {{6,3},{5,4},{4,5},{3,6}}. P(A)=4/36=1/9. P(AB)=0.

34

Suppose an unfair die has the property that the probability of number i turning up is i/21. Are the events A=“ an even number turns up” and B= “a number 4 or less turns up” independent. What is the corresponding result for a fair die?

Solution 1 (1) describe the events: A= {2,4,6}; B={1,2,3,4}. AB= {2,4},

(2) find the probabilities of these events using the probability function

P(A)= 2/21+4/21 + 6/21= 12/21; P(B)= 1/21 + 2/21 + 3/21 + 4/21=10/21;

P(AB) = 2/21 + 4/21 = 6/21.(3) find P(A)P(B) = 120/21*21=40/147 < P(AB)=6/21= 42/147

Answer: the events are not independent

Solution 2 (1) the events are the same as above

(2) find the probabilities of these events using the probability function

P(A)= 1/6+1/6 + 1/6= 1/2; P(B)= 4*1/6= 2/3. P(AB) = 2*1/6 = 1/3.(3) find P(A)P(B) = 1/3 = P(AB)

Answer: the events are independent

35

Some more examples.

It is not always self-evident if the events are independentTry to analyze the following example (group work):

Roll two dice. Let S7="The sum of the two dice is 7", F4="The first die is 4", DE="The difference of two dice is even".Are S7and F4 independent? S7 and DE? F4 and DE?

36

Solution:

(a) Describe the events and their probabilities

S7={{1,6},{2,5}{3,4},{4,3},{5,2},{6,1}}; P(S7)=1/6; P(F4)=1/6.

DE= {{1,3},{1,5},{2,4},{2,6},{3,5},{4,6},{6,4},{5,3},{6,2},{4,2},{5,1},{3,1}}; P(DE)=1/3

(b) Find the intersections and their probabilities.

S7*F4= {4,3}, P(S7*F4)= 1/36 = P(S7)*P(F4); S7 and F4 are independent.

S7*DE=, P(S7*DE)=0 (non-equal) P(S7)*P(DE): S7 and DE are not independent.They are disjoined. They can never occur together. Occurrence of S7 makes simultaneous occurrence of DE impossible.

F4*DE={4,6}, P(F4*DE)=P({42} OR {46}) =2/36 = 1/18 = P(F4)*P(DE)

Please finish this problem. We made an error in the class. Find P(DE)!

37

Definition: A set of events {A1, A2, …An} is said to be mutually independent if for any subset {Ai, Aj, … Am} of these events we have

P(Ai Aj … Am)=P(Ai)P( Aj) … P(Am) (5)

It is natural to ask: if all pairs of a set of events are independent, is the whole set mutually independent? The answer is “not necessary”.

Building various events with two dice experiments.

The sample space can be presented through the table 6×6.Different groups of cells correspond to different events. Two events are shown in the picture: Green= "Sum of two throws <5" and Grey="Sum of two throws = 9".

38

Are they independent?

Now, try to name the events shown in Figures a through d and evaluate their (in)dependence. Colored black are the intersection points between different events. In the case(d), be careful evaluating the mutual (in)dependence of three events.

39

40

11 12 13 14 15 16

21 22 23 24 25 26

31 32 33 34 35 36

41 42 42 44 45 46

51 52 53 54 55 56

61 62 63 64 65 66

Here is another example of two dependent events:S4= “Sum = 4”, X2 = “Second is 2”;

P(S4 X2)= 1/36;

P(S4)*P(X2)= 1/6*1/12=1/72.

41

Suppose we roll a green and a red die. Let A="The red die shows 2 or 5. B="The some of the two dice is at least 7". Are A and B independent?

P(A)=P(Grey+Black)=1/3;

P(B)=P(yellow+Black)=21/36. P(AB)=P(black)=7/36. The events are independent

42

Home assignment (see also WebCT).

• Watch the video (p.19).• Try to experiment with the Non-Linear Regression file:Change the positions of the detectors, position and the intensity of the source. Try

generating the data and solving the problem of locating the source based on those data.

Familiarize yourself with the functions used for the regressionanalysis. Be prepared to describe the Least Squares Approximation

Extra credit (4 problems, but you can solve only a part of them) :

• Cards problem, page 18.

• Solve the additional cab problem, p. 22

• Prove the equivalence of two definitions of the independence (p. 32).

• Finish the last problem in page 36.