Chapter 3

Probability

I Linda is 31 years old, single, outspoken, and very bright.She majored in philosophy. As a student, she was deeplyconcerned with issues of discrimination and social justice,and also participated in anti-nuclear demonstrations.

I Which is more probable?

I Linda is a bank teller.

I Linda is a bank teller and is active in the feministmovement.

Monty Hall problem

Suppose you’re on a game show, and you’re given the choice ofthree doors: Behind one door is a car; behind the others, goats.You pick a door, say No. 1, and the host, who knows what’sbehind the doors, opens another door, say No. 3, which has agoat. He then says to you, "Do you want to pick door No. 2?" Isit to your advantage to switch your choice?

I Assume that in a family each child is a M or F with prob .5

I You have moved to a new home. You talk to your neighborand he says he has two children

I you see a 10 year old girl going to school from the house

I What is the probability that the other child is M, 12 ,

13 ,

23?

PROBABILITY

I Sample space and Events

I Rules of Probability

I Conditional Probability and Independence

I Combinatorial Concepts

PROBABILITY

I Toss a coin once

I P(H)= 12 P(T)=1

2

I Tacitly assumed ‘fair’ coin. Two possible outcomes. Bothequally likely

I S = {H,T}- Sample space

Throw a six faced die twice(1,1) (1,2) (1,3) (1,4) (1,5) (1,6)(2,1) (2,2) (2,3) (2,4) (2,5) (2,6)(3,1) (3,2) (3,3) (3,4) (3,5) (3,6)(4,1) (4,2) (4,3) (4,4) (4,5) (4,6)(5,1) (5,2) (5,3) (5,4) (5,5) (5,6)(6,1) (6,2) (6,3) (6,4) (6,5) (6,6)

Assume all outcomes have equal probability

So the probability of any outcome is 136

(1,1) (1,2) (1,3) (1,4) (1,5) (1,6)(2,1) (2,2) (2,3) (2,4) (2,5) (2,6)(3,1) (3,2) (3,3) (3,4) (3,5) (3,6)(4,1) (4,2) (4,3) (4,4) (4,5) (4,6)(5,1) (5,2) (5,3) (5,4) (5,5) (5,6)(6,1) (6,2) (6,3) (6,4) (6,5) (6,6)

Probability that both the throws give even numbers = 936

Probability that the sum of both throws is 8 = 536

Probability that the at least one throw is even = 2736

Formal definitions

I SAMPLE SPACE

I Sample space S is the set of all possible outcomes

I EVENT

I Any collection of outcomes is called an Event. The eventoccurs if the outcome belongs to the subset.

I Let S = {s1, s2, s3, . . . , sn}

I Specify prob of each outcome P(si) = pi

for i = 1,2, . . . ,n

I pi satisfy

pi ≥ 0,n∑

i=1

pi = 1

I For any event A,

P(A) = sum of prob of all outcomes in A =∑

si in A

pi

Equally likely experiments

I A common situation is when the sample space has noutcomes and all the outcomes have the same probability

I Then P(A) is just the ratio #(A)#(S) , where #(A) is the number

of outcomes in A

I This leads to counting #(A) an #(S)

Combinatorics

Suppose we have a box with 5 tickets with labels A,B,C,D,E andwe want to choose 3 out of this, one after the other. How manyways can this be done?There are 5 ways in which the first draw can be madeWith each of the first draw there are 4 possibilities for thesecond. There are 5 x 4 = 20 ways to choose two

1

2

3

4

5

2

1

3

4

5

3

1

2

4

5

4

1

2

3

5

5

1

2

3

4

Combinatorics

Proceeding,The number of ways of choosing 3 out of 5 objects, keepingtrack of order is

5× 4× 3 =5× 4× 3× 2× 1

2× 1=

5!

(5− 3)!

In general: The number of ways of choosing r out of n objects,keeping track of order is ( number of permutations of r out of n)

nPr = n(n − 1)(n − 2) . . . (n − (r − 1) =n!

(n − r)!

Combinations

If we choose 3 out the 5 tickets how many distinct triplets arepossible? We do not keep track of order. For instance the triplet{A,B,C} can arise in 6 ways. The first can be any of the threethe second any of the remaining 2 and the last one: i.e 3!.That is 6 ordered triplets likeABC,ACB,BAC,BCA,BAC,CAB,CBA will go to the same set{A,B,C}.So we can get 5!

3!(5−3)! unordered triplets from 5 objects

Combinations

The number of ways of choosing 3 out of 5 objects, withoutkeeping track of order is

5× 4× 33!

=5× 4× 3× 2× 1

2× 1=

5!

3!(5− 3)!

In general: The number of ways of choosing r out of n objects,without keeping track of order is ( number of combinations of rout of n) (

nr

)=

n!

r !(n − r)!

Problem

A company has four departments: manufacturing,distribution,marketing and management. The number of people in eachdepartment is 55,30,21 and 13. Each department is expectedto send one representative to a meeting. How manypossibilities are there?

55× 30× 21× 13 = 450450

Problem

Nine sealed bids for oil drilling leases arrive at a regulatoryagency. In how many different orders can the nine bids beopened.

9× 8× 7× 6× 5× 4× 3× 2× 1 = 362880

Problem

fifteen locations in a given area are believed likely to have oil.An oil company can only afford to dig 8 sites, sequentiallychosen. How many possibilities are there?

15!

(15− 8)!= 15× 14× 13× 12× 11× 10× 9× 8 = 259459200

Problem

In a shipment of 14 computer parts, 3 are faulty and theremaining 11 are in working order. Three elements are chosenat random. what is the probability that all the faulty ones arechosen

1(143

)

A more easy argument is

314

213

112

Problems 3.21,3.24

(1,1) (1,2) (1,3) (1,4) (1,5) (1,6)(2,1) (2,2) (2,3) (2,4) (2,5) (2,6)(3,1) (3,2) (3,3) (3,4) (3,5) (3,6)(4,1) (4,2) (4,3) (4,4) (4,5) (4,6)(5,1) (5,2) (5,3) (5,4) (5,5) (5,6)(6,1) (6,2) (6,3) (6,4) (6,5) (6,6)

I Let A = first throw is a “6“ , P(A) = 636

I Let B = Second throw is a “6“ , P(B) = 636

I “ A and B “ : first throw is “6“ AND ‘ second throw is“6“

I ‘ all outcomes that are both in A and B ‘

I This called ‘ A intersection B ‘, written as A ∩ B.

(1,1) (1,2) (1,3) (1,4) (1,5) (1,6)(2,1) (2,2) (2,3) (2,4) (2,5) (2,6)(3,1) (3,2) (3,3) (3,4) (3,5) (3,6)(4,1) (4,2) (4,3) (4,4) (4,5) (4,6)(5,1) (5,2) (5,3) (5,4) (5,5) (5,6)(6,1) (6,2) (6,3) (6,4) (6,5) (6,6)

I Let A = first throw is a “6“ , P(A) = 636

I Let B = Second throw is a “6“ , P(B) = 636

I “ A or B “ : first throw is “6“ or ‘ second throw is“6“

I ‘ all outcomes that are at least in one of A or B ‘

I This called ‘ A Union B ‘, written as A ∪ B.

(1,1) (1,2) (1,3) (1,4) (1,5) (1,6)(2,1) (2,2) (2,3) (2,4) (2,5) (2,6)(3,1) (3,2) (3,3) (3,4) (3,5) (3,6)(4,1) (4,2) (4,3) (4,4) (4,5) (4,6)(5,1) (5,2) (5,3) (5,4) (5,5) (5,6)(6,1) (6,2) (6,3) (6,4) (6,5) (6,6)

I P(A ∪ B) = 1136

I P(A) = 636 P(B) = 6

36

I P(A ∪ B) = P(A) + P(B)− P(A ∩ B)

(1,1) (1,2) (1,3) (1,4) (1,5) (1,6)(2,1) (2,2) (2,3) (2,4) (2,5) (2,6)(3,1) (3,2) (3,3) (3,4) (3,5) (3,6)(4,1) (4,2) (4,3) (4,4) (4,5) (4,6)(5,1) (5,2) (5,3) (5,4) (5,5) (5,6)(6,1) (6,2) (6,3) (6,4) (6,5) (6,6)

I Let A = first throw is a “6“ , P(A) = 636

I C = first throw is a “4“ , P(C) = 636

I P(A ∪ C) = 1236 = P(A) + P(C)

I A and C have no outcome in common.

I A,C are said to be ‘ Disjoint‘ or ‘Mutually Exclusive‘

I Let A = first throw is a “6“ , P(A) = 636

I Probability ‘ the first throw is NOT a “6 “ = 3036

I Ac – A Complement : All outcomes not in A

I P(Ac) = 1− P(A)

We have mentionedI eventI complement of an event

I union of two events

I intersection of two events

I Next we look at these abstractly

Formal definitions

I SAMPLE SPACE

I Sample space S is the set of all possible outcomes

I EVENTI An event is a subset of the sample space. The event

occurs if the outcome belongs to the subset.

Formal definitions

I A is an event.

A (A- COMPLEMENT) stands for “Not A”. A does not

Sometimes I would denote A- complement by A occur

I A,B events. A ∪ B (read A UNION B) At least one of A, Boccurs. A or B . All outcomes in at least one of A,B

I A ∩ B (read A INTERSECTION B) Both A and B occur.All outcomes which are both in A and in B

A-Complement

A

A

A-Complement

A

A

A-Union B

A

A-Union B

A

B

A-Union B

A ∪ B

A-intersection B

A

A-intersection B

B

A

A-intersection B

A ∩ B

Mutually Exclusive events

A and B are MUTUALLY EXCLUSIVE OR DISJOINT ifA ∩ B = ∅

i.e.,A and B have no outcomes in common

Both A and B cannot occur at the same time

Mutually Exclusive

A

B

Convince yourselves thatI A ∪ B = A ∩ BI A ∩ B = A ∪ B

Probability: Formal Properties

Probability is an assignement of numbers P(A) to every eventA, such that

1. 0 ≤ P(A) ≤ 1

2. P(S) = 1

3. P(A ∪ B) = P(A) + P(B)− P(A ∩ B)

4. If A and B are mutually exclusive, since P(A ∩ B) = 0,

P(A ∪ B) = P(A) + P(B)

5. (i) and (ii) imply that P(A) = 1− P(A)

problem 3.3.33,3.41

I Events of the form A1∩A2∩A3 are important. For instance,if we take a sample and are interested in what happens tothe first sample and the second sample and so on

I If we know, probabilities of intersections they can be usedto calculate prob of unions

I probailities of events of the form A ∩ B are important

I conditional probabilities provide are helpful in calculatingprobailities of events of the form A ∩ B

Conditional probability

I

P(B|A)

I Conditional probability of B given A

I Probability of B if it is known that A has occured

Conditional probability

I Basic Rule (Product Rule)

I

P(A ∩ B) = P(A)P(B|A)

I Since A ∩ B is same as B ∩ A,

P(A ∩ B) = P(B)P(A|B)

I Warning: P(A|B) is not same as P(B|A)

Conditional probability

I How to justify the product ruleI Try out in simple examples.

I Suppose we have a box with 3 Green and 2 Red balls.Draw one at random, do not replace and draw one more.

I Let RI ,RII stand for Red in the first draw, Red in seconddraw

I Similarly GI ,GII

I Events of interest are RI ∩ RII ,RI ∩GII ,GI ∩ RII ,GI ∩GII

Second DrawFirst draw G R

G P(GI ∩GII) P(GI ∩ RII) P(GI)

R P(RI ∩GII) P(RI ∩ RII) P(RI)

P(GII) P(RII) 1

G1G2 G1G3 G1 R1 G1 R2G2G1 G2G3 G2 R1 G2 R2G3G1 G3G2 G3 R1 G3 R2R1G1 R1G2 R1G3 R1R2R2G1 R2G2 R2G3 R2R1

P(GI ∩GII) =6

20P(RI ∩GII) =

620

I

Second DrawFirst draw G R

G P(GI ∩GII) P(GI ∩ RII) P(GI)

R P(RI ∩GII) P(RI ∩ RII) P(RI)

P(GII) P(RII) 1

I

Second DrawFirst draw G R

G 620

620

1220

R 620

220) 8

20

P(GII) P(RII) 1

using product rule

I Suppose we have a box with 3 Green and 2 Red balls.Draw one at random, do not replace and draw one more.

I Interpret this asP(GI) = 3

5 ,P(RI) = 24 ,P(GII |GI) = 2

4 ,P(GII |GI) = 34

I P(G1 ∩GII) = 35

24 = 6

20

I P(R1 ∩GII) = 25

34 = 6

20I We get the same numbers as before

A study revealed that 45% of college freshmen are male andthat 18% of male freshmen earned college credits while still inhigh school. Find the probability that a randomly chosencollege freshman will be male and have earned college creditswhile still in high school.

I Two events of interestM = Person is a male,H = Person earned college credit in school

I We have P(M) = .45,P(H|M) = .18 Want P(M ∩ H)

I So P(M ∩ H) = P(M)P(H|M) = (.45)(.18)

3.50

For two events A and B, P(A) = .4,P(B) = .2,P(A|B) = .6I Find P(A ∩ B)

I = P(B)P(A|B) = .2× .6 = .12I Find P(B|A)

I P(B|A) = P(A∩B)P(A) = .12

0.4

Independence

I Two events A,B are said to be independent, if

P(A|B) = P(A)

I i.eP(A ∩ B)

P(B)= P(A)

I i.e.P(A ∩ B) = P(A)P(B)

I Note independence is symmetric , if A is independent of B,then B is also independent of A

I What does A and B are independent mean?

I A has no information about B. Occurrence of A does notaffect the probability of occurrence of B

I How do we know events are independent?

I Sometimes specified in the problem,

I Description of the experiment like successive tosses of acoin, sample with replacement etc.

I If A, B are mutually exclusive (disjoint) can they beindependent?

I Being mutually exclusive P(A ∩ B) = 0I If A, B are independent

P(A ∩ B) = P(A)P(B) = 0 !!I If mutually exclusive they cannot be independent

Summary Formulas

I P(A) = 1− P(A)

I P(A ∪ B) = P(A) + P(B)− P(A ∩ B)

I P(A ∩ B) = P(A)P(B|A)

I P(A ∩ B) = P(B)P(A|B)

I If A, B are independentP(A ∩ B) = P(A)P(B)

3.50

For two events A and B, P(A) = .4,P(B) = .2,P(A ∩ B) = .1I Find P(B|A)

I P(B|A) = P(A∩B)P(A) = 0.1

0.4

I Find P(A|B)

I P(A|B) = P(A∩B)P(B) = 0.1

0.2

I Are A and B independent?I is P(A ∩ B) = P(A)P(B) ? NoI Not independent

3.58Suppose that 15% of all full time workers exhibit arrogantbehavior on the job and that 10% of all the full time workersreceive a poor performance rating. Also, assume that 5% of allfull time workers exhibit arrogant behavior and receive poorrating. Let A be the event that full time worker exhibits arrogantbehavior and B be the event that a full time worker receives apoor performance rating.

I Are the events A and B mutually exclusive?

I Find P(B|A)

I Are A and B independent?

3.67

An ambulance station has one vehicle and two demandlocations, A and B. The probability that an ambulance cantravel to a demand location under8 minutes is .58 for location Aand .42 for location B. The probability that the ambulance isbusy at any point of time is .3.

I Find the probability that EMS can meet the demand forambulance at location A

3.67

An ambulance station has one vehicle and two demandlocations, A and B. The probability that an ambulance cantravel to a demand location under8 minutes is .58 for location Aand .42 for location B. The probability that the ambulance isbusy at any point of time is .3.

I Find the probability that EMS can meet the demand forambulance at location B

3.69A computer intrusion detection system (IDS)is designed toprovide an alarm whenever an intrusion is attempted into acomputer system. Consider a double IDS system with systemA and system B. If there is an intruder, system A sounds analarm with probability 0.9 and B sounds an alarm withprobability 0.95. If there is no intruder, system A sounds analarm ( false alarm) with probability 0.2 and system B soundsan alarm with probability 0.1. Assume under a given condition (intruder or not) system A and B operate independently.

I Using symbols express the four probabilities given in theexample

I If there is an intruder what is the probability that bothsystems sound an alarm ?

I If there is no intruder what is the probability that bothsystems sound an alarm?

I Given an intruder, what is the probability that at least oneof the systems sound an alarm?

3.69

A computer intrusion detection system (IDS)is designed toprovide an alarm whenever an intrusion is attempted into acomputer system. Consider a double IDS system with systemsA and systems B. If there is an intruder, system A sounds analarm with probability 0.9 and B sounds an alarm withprobability 0.95. If there is no intruder, system A sounds analarm ( false alarm) with probability 0.2 and system B soundsan alarm with probability 0.1. Assume under a given condition (intruder or not) system A and B operate independently.

I Using symbols express the four probabilities given in theexample

3.69

I If there is an intruder what is the probability that bothsystems sound an alarm ?

I If there is no intruder what is the probability that bothsystems sound an alarm?

3.69

I Given an intruder, what is the probability that at least oneof the systems sound an alarm?

Bayes Theorem

I What is Bayes theorem all about?I You are given P(A),P(B|A),P(B|Ac). Use these to find

P(A|B).

Bayes Theorem-Example

A study revealed that 40% of college freshmen are male andthat 15% of male freshmen earned college credits while still inhigh school. Among the 60% female freshmen, 20% earnedcollege credit while in high school. If a freshman picked atrandom has earned college credit in high school, find theprobability that the freshman is a male.

I Two events of interestM = Person is a male, Mc = person is a femaleH = Person earned college credit in school

I We have P(M) = .4,P(H|M) = .15 ,P(H|Mc) = .2. WantP(M|H)

I Think in terms of percentages. What percentage of H is M?

H Hc

M 45

Mc 55

I We have P(M) = .4,P(H|M) = .15 ,P(H|Mc) = .2. WantP(M|H)

H Hc

M .45

Mc .55

B Bc

A P(A ∩ B) P(A ∩ Bc) P(A)

= P(A)P(B|A) = P(A)P(Bc |A)

Ac P(Ac ∩ B) P(Ac ∩ Bc) P(A)

= P(Ac)P(B|Ac) = P(Ac)P(Bc |Ac)

P(B) P(Bc) 1

P(A|B) =P(A ∩ B)

P(B)=

P(A)P(B|A)

P(A)P(B|A) + P(Ac)P(B|Ac)

Bayes theorem - formal statement

If A1,A2, . . . ,An are disjoint and A1 ∪ A2, . . . ∪ An = S ,then

P(Ai |B) =P(A ∩ B)

P(B)(1)

P(Ai)P(B|Ai)∑nj=1 P(Aj)P(B|Aj)

(2)

3.88

A computer intrusion detection system (IDS)is designed toprovide an alarm whenever an intrusion is attempted into acomputer system. Consider a double IDS system with systemsA and systems B. If there is an intruder, system A sounds analarm with probability 0.9 and B sounds an alarm withprobability 0.95. If there is no intruder, system A sounds analarm ( false alarm) with probability 0.2 and system B soundsan alarm with probability 0.1.Assume under a given condition (intruder or not) system A and B operate independently. Theprobability of an intrusion is 0.4.

I Given both the system sound an alarm, what is theprobability that an intruder is detected?

I Given at least one system sounds an alarm, what is theprobability that an intruder is detected?

3.88

A stands for A sounds alarm, B stands for B sounds alarm, Ithere is an intruder

I P(I) = .4I P(A ∩ B|I) = (.9)(.95) P(A ∩ B|Ic) = (.2)(.1)

I P(Ac ∩ B|I) = (.1)(.95) P(Ac ∩ B|Ic) = (.8)(.1)

I P(A ∩ Bc |I) = (.9)(.05) P(A ∩ Bc |Ic) = (.8)(.9)

I P(Ac ∩ Bc |I) = (.1)(.05) P(Ac ∩ Bc |Ic) = (.8)(.9)

3.88

In terms of table, fill the following table

A ∩ B Ac ∩ B A ∩ Bc Ac ∩ Bc

I

Ic

problem 3.82

A company employs three sales engineers. Engineers 1,2 and3 estimate the costs of 30%, 20% and 50% respectively of alljobs bid by the company. For i = 1,2,3, let Ei b the event a jobis estimated by engineer i. The following table describes therates at which the engineers make serious errors in estimatingcosts:P( error |E1) = .01 P( error |E2) = .03 P( error |E3) =.02If a particular bid results in a serious error in estimating thecost, what is the probability that it was made by

1. Engineer 12. Engineer 23. Engineer 3