Chapter 2: Conditional Probability

Definition of Conditional Probability

Definition: Conditional Probability. Suppose that we

learn that an event B has occurred and that we wish to

compute the probability of another event A taking into

account that we know that B has occurred. The new

probability of A is called the conditional probability of

the event A given that the event B has occurred and is

denoted P (A|B). If P (B) > 0, we compute this proba-bility as

P (A|B) = P (AB)P (B)

.

A B

Figure 1: The outcomes in the event B that also belong to the event A.

Example: Rolling Dice. Suppose that two dice were

rolled and it was observed that the sum T of the two

numbers was odd.We shall determine the probability that

21

T was less than 8.

If we let A be the event that T < 8 and let B be the

event that T is odd, then A B is the event that T is 3,5, or 7.

P (A B) = P (T = 3or5or7)= P (T = 3) + P (T = 5) + P (T = 7)

=2

36+

4

36+

6

36

=1

3P (B) = P (T = 3or5...or11)

= P (T = 3) + P (T = 5)... + P (T = 11)

=2

36+

4

36+ ... +

2

36

=1

2

P (A|B) = P (AB)P (B)

=1/3

1/2=

2

3.

Example: Rolling Dice Repeatedly. Suppose that two

dice are to be rolled repeatedly and the sum T of the

two numbers is to be observed for each roll. We shall

determine the probability p that the value T = 7 will be

observed before the value T = 8 is observed.

22

(1) The desired probability p could be calculated di-

rectly as follows: We could assume that the sample space

S contains all sequences of outcomes that terminate as

soon as either the sum T = 7 or the sum T = 8 is ob-

tained. Then we could find the sum of the probabilities

of all the sequences that terminate when the value T = 7

is obtained.

Let pi be the probability that the first 7 occurs on the

i th position.

p1 =6

36 5

36+

6

36 (1 5

36) 5

36+ ... =

1

6

p2 = (111

36) p1 = (1

11

36) 1

6

p3 = (111

36)2 1

6....

p = p1 + p2 + ... =1

6 1

1 (1 1136)=

6

11

(2) We can consider the simple experiment in which

two dice are rolled. If we repeat the experiment until

either the sum T = 7 or the sum T = 8 is obtained, the

effect is to restrict the outcome of the experiment to one

23

of these two values. Hence, the problem can be restated

as follows: Given that the outcome of the experiment is

either T = 7 or T = 8, determine the probability p that

the outcome is actually T = 7. If we let A be the event

that T = 7 and let B be the event that the value of T is

either 7 or 8, then A B = A and

p = P (A|B) = P (AB)P (B)

=P (A)

P (B)=

6/36

5/36 + 6/36=

6

11.

Example: A Clinical Trial. Consider 150 patients who

entered the study after an episode of depression.

ResponseTreatment group

TotalImipramine Lithium Combination Placebo

Relapse 18 13 22 24 77

No Relapse 22 25 16 10 73

Total 40 38 38 34 150

Table 1: Results of the clinical depression study

If a patient were selected at random from this study

and it were found that the patient received the placebo

treatment, what is the conditional probability that the

patient had a relapse?

Let B be the event that the patient received the placebo,

and let A be the event that the patient had a relapse.

P (B) = 34150, P (AB) =24150,

P (A|B) = P (AB)P (B) =24/15034/150 =

2434.

24

The Multiplication Rule for Conditional Probabilities

Theorem: Let A and B be events. If P (B) > 0, then

P (A B) = P (B) P (A|B)

If P (A) > 0, then

P (A B) = P (A) P (B|A)

Example: Selecting Two Balls. Suppose that two balls

are to be selected at random, without replacement, from

a box containing r red balls and b blue balls. We shall

determine the probability p that the first ball will be red

and the second ball will be blue.

Let A be the event that the first ball is red, and let B

be the event that the second ball is blue.

P (A B) = P (A) P (B|A) = rr + b

br + b 1

.

Theorem: Suppose that A1, A2, ..., An are events such

that P (A1 A2... An1) > 0. Then

P (A1A2...An) = P (A1)P (A2|A1)...P (An|A1A2...An1)

Example: Suppose that four balls are selected one at

a time, without replacement, from a box containing r red

balls and b blue balls (r 2, b 2). We shall determine

25

the probability of obtaining the sequence of outcomes red,

blue, red, blue.

Let Rj denote the event that a red ball is obtained on

the jth draw and let Bj denote the event that a blue ball

is obtained on the jth draw (j = 1, . . . , 4), then

P (R1 B2 R3 B4)= P (R1)P (B1|R1)P (R3|R1b2)P (B4|R1B2R3)

=r

r + b b

r + b 1 r 1

r + b 2 b 1

r + b 3.

Note: Conditional Probabilities Behave Just Like Prob-

abilities. For example,

P (Ac|B) = 1 P (A|B)

P (A1A2|B) = P (A1|B) P (A2|A1B)

Conditional Probability and Partitions

Definition: Partition. Let S denote the sample space

of some experiment, and consider k events B1, ..., Bk in S

such that (1) B1, ..., Bk are disjoint and (2) ki=1Bi = S.It is said that these events form a partition of S.

Theorem: Law of total probability. Suppose that the

events B1, ..., Bk form a partition of the space S and

P (Bj) > 0 for j = 1, ..., k. Then, for every event A

26

in S,

P (A) =

kj=1

P (Bj)P (A|Bj).

B1 B2

B5 B4 B3

A

Figure 2: The intersections of A with events B1, ..., B5 of a partition

Example: Suppose that one box contains 60 long bolts

and 40 short bolts, and that the other box contains 10

long bolts and 20 short bolts. Suppose also that one box

is selected at random and a bolt is then selected at ran-

dom from that box. Determine the probability that this

bolt is long.

Let B1 be the event that the first box (the one with 60

long and 40 short bolts) is selected, let B2 be the event

that the second box (the one with 10 long and 20 short

bolts) is selected, and let A be the event that a long bolt

is selected.

27

P (A) = P (B1)P (A|B1) + P (B2)P (A|B2)

=1

2 60

100+

1

2 10

30

=7

15.

Bayes Theorem

Example: (Selecting Boltscontinued) if we learn that

event A has occurred, that is, a long bolt was selected,

we can compute the conditional probabilities of the two

boxes given A.

P (B1|A) =P (B1A)

P (A)

=P (B1)P (A|B1)

P (B1)P (A|B1) + P (B2)P (A|B2)

=12

60100

12

60100 +

12

1030

=9

14.

Similarly, P (B2|A) = P (B2)P (A|B2)P (B1)P (A|B1)+P (B2)P (A|B2) =514. Or

P (B2|A) = 1 P (B1|A) = 514.Theorem: Bayes theorem. Suppose that the events

B1, ..., Bk form a partition of the space S and P (Bj) > 0

28

for j = 1, ..., k. and let A be an event such that P (A) > 0,

Then, for j = 1, ..., k,

P (Bj|A) =P (Bj)P (A|Bj)kj=1 P (Bj)P (A|Bj)

.

Independent Events

If learning that B has occurred does not change the

probability of A, then we say that A and B are indepen-

dent.

Example: Suppose that a fair coin is tossed twice. Let

event A = {H on second toss}. B = {T on first toss}.

S = {HH,HT, TH, TT}

A = {HH,TH}, A B = {TH}, B = {TH, TT}P (A) = 24 =

12, P (A|B) =

P (AB)P (B) =

1/42/4 =

12

P (A) = P (A|B)

we dont change the probability of A even after we learn

that B has occurred. In this case, we say that A and B

are independent events.

Definition: Two events A and B are independent if

P (A B) = P (A) P (B).

Independence of Two Events

29

If two events A and B are considered to be independent

because the events are physically unrelated.

Example: Suppose that two machines 1 and 2 in a fac-

tory are operated independently of each other. Let A be

the event that machine 1 will become inoperative during

a given 8-hour period, let B be the event that machine 2

will become inoperative during the same period, and sup-

pose that P (A) = 1/3 and P (B) = 1/4. Determine the

probability that at least one of the machines will become

inoperative during the given period.

P (A B) = P (A) + P (B) P (A B)= P (A) + P (B) P (A) P (B)

=1

3+

1

4 1

3 1

4

=1

2.

Example: Suppose that a balanced die is rolled. Let

A be the event that an even number is obtained, and let

B be the event that one of the numbers 1, 2, 3, or 4 is

obtained. Show that the events A and B are independent.

P (A) =1

2, P (B) =

2

3, P (AB) =

1

3

30

P (AB) = P (A) P (B).

It follows that the events A and B are independent events,

even though the occurrence of each event depends on the

same roll of a die.

Theorem: If two events A and B are independent, then

the events A and Bc, Ac and B, Ac and Bc are indepen-

dent.

Independence of Several Events

Definition: (Mutually) Independent Events. The k

events A1, ..., Ak are independent (or mutually indepen-

dent) if, for every subset Ai1, ..., Aij of j of these events

(j = 2, 3, ..., k),

P (Ai1 ... Aij) = P (Ai1)...P (Aij).

As an example, in order for three events A, B, and C

to be independent, the following four relations must be

satisfied:

P (AB) = P (A)P (B), P (AC) = P (A)P (C)

P (BC) = P (B)P (C), P (ABC) = P (A)P (B)P (C)

31

Example: Pairwise Independence. Suppose that a

fair coin is tossed twice so that the sample space S =

{HH,HT, TH, TT} is simple. Define the following threeevents:

A = {H on first toss} = {HH,HT},

B = {H on second toss} = {HH,TH},

C = {Both tosses the same} = {HH,TT}.

Then AB = AC = BC = ABC = {HH}. Hence,

P (A) = P (B) = P (C) =1

2

P (AB) = P (AC) = P (BC) = P (ABC) =1

4P (AB) = P (A)P (B), P (AC) = P (A)P (C)

P (BC) = P (B)P (C), P (ABC) 6= P (A)P (B)P (C)

These results can be summarized by saying that the

events A, B, and C are pairwise independent, but all three

events are not independent.

Theorem: Let n > 1 and let A1, ..., An be events that

are mutually exclusive. The events are also mutually in-

dependent if and only if all the events except possibly one

of them has probability 0.

32