8/8/2019 2 Statprob Walpole
1/29
WalpoleWalpoleCh 02:Ch 02: 11
Spring 2007
Probability & StatisticsProbability & Statisticsfor Engineers &for Engineers &Scientists, byScientists, by
Walpole, Myers, MyersWalpole, Myers, Myers
& Ye& Ye~~
Chapter 2 NotesChapter 2 Notes
8/8/2019 2 Statprob Walpole
2/29
WalpoleWalpoleCh 02:Ch 02: 22
Spring 2007
Probab ility IntroProbab ility Intro
n The sample space S of an experiment is the set of allpossib le outcomes. We must understand the sample space in order to determine
the probability of each outcome occurring.
The sample space is a set, the domain of the probabilityfunction.
Each probability value, p, is a real number 0 p 1.n Imp ortant: The sum of the prob abilities for all
elements in the samp le space always equals 1.
Why is this important?
This fact allows us to check our answers.
n Properly enu merating the sample space is key tocorrectly calculating probabilities.
A tree diagram is sometimes useful.
8/8/2019 2 Statprob Walpole
3/29
WalpoleWalpoleCh 02:Ch 02: 33
Spring 2007
Examples of Sample Space
All 6 faces of a die: S = {1,2,3,4,5,6}
All 52 cards of a bridge deck:
8/8/2019 2 Statprob Walpole
4/29
WalpoleWalpoleCh 02:Ch 02: 44
Spring 2007
Example of Samp le SpaceExample of Samp le Space
n 3 items are selected at random from a m anu facturingprocess. Each item is inspected and classified detective,D, or non detective, N.
n The sample space is :
S={ DDD, DDN, DND, DNN, NDD, NDN, NND,
NNN }
8/8/2019 2 Statprob Walpole
5/29
8/8/2019 2 Statprob Walpole
6/29
WalpoleWalpoleCh 02:Ch 02: 66
Spring 2007
EventsEvents
nAn event is a sub set of a samp le space (E S) Note that both S and are events as well.
n Samp le spaces can be contin uous or d iscrete.
What is a continuous vs. discrete sample space?
n Example: Life in years of a component. S = ?
S = {t | t 0} => all values of t such that t 0 A = component fails before the end of the fifth year.
A = {t | t < 5}.
n Example: Flip a coin three times. S = ?
S = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}
n Even t A = 1st flip is heads.
A = {HHH, HHT, HTH, HTT}
8/8/2019 2 Statprob Walpole
7/29
WalpoleWalpoleCh 02:Ch 02: 77
Spring 2007
Visualizing EventsVisualizing Events
nContingency Tables
nTree Diagrams
Red 2 24 26
Black 2 24 26
Tota l 4 48 52
Ace Not Ace Tota l
Full Deck
of 52 Car ds
Sample
Space
Sample
Space2
24
2
24
8/8/2019 2 Statprob Walpole
8/29
WalpoleWalpoleCh 02:Ch 02: 88
Spring 2007
Event/Set Op erationsEvent/Set Op erations
n The comp lemen t of an event A?
The set of all elements of S not in A. Denoted A.
A = 1st flip is heads. A = first flip is not heads.
n The intersection of two events A and B?
The set of all elements in both A and B. Denoted A B.
B = 2nd
or 3rd
flip, but not both, are heads. B = {HHT, HTH, THT, TTH}. A B = ?
A B = {HHT, HTH}
n Two events are mu tually exclusive if?
A B =
n The un ion of two events, A and B?
The set of elements in either A or B. A B = ?
A B = {HHH, HHT, HTH, HTT , THT, TTH}.
8/8/2019 2 Statprob Walpole
9/29
WalpoleWalpoleCh 02:Ch 02: 99
Spring 2007
Venn D iagramsVenn D iagrams
nVenn Diagrams show various events graph ically, and
are sometimes helpful in un derstand ing set theoryproblems.
n Standard set theory results hold :
A = ?
A
= ? A A = ?
A A = ?
S = ?
= ?
(A) = ?
(A B) = ?
(A B) = ?
(A B)=A B, (A B)=A B
8/8/2019 2 Statprob Walpole
10/29
WalpoleWalpoleCh 02:Ch 02: 1010
Spring 2007
Some Resu lts from Defin itionsSome Resu lts from Defin itions
8/8/2019 2 Statprob Walpole
11/29
WalpoleWalpoleCh 02:Ch 02: 1111
Spring 2007
Events of Samp le Space SEvents of Samp le Space S
8/8/2019 2 Statprob Walpole
12/29
WalpoleWalpoleCh 02:Ch 02: 1212
Spring 2007
Intu itive Sample Point Coun tingIntu itive Sample Point Coun ting
n If one operation can b e performed in n 1 ways, and for
each w ay, a second can be performed in n 2 ways, thenthe two can be performed a total of n 1 n 2 ways.
For three operations?
n1 n2 n3.
How many passwords of length 5 need to be checked by a
password hacking program if only lower case letters are used? 265 = 11,881,376.
n This is called samp ling with replacemen t.
nA tree diagram can b e used to enumerate all of theoptions.
8/8/2019 2 Statprob Walpole
13/29
WalpoleWalpoleCh 02:Ch 02: 1313
Spring 2007
Permutation OrderingsPermutation Orderings
nA permutation is an ordering of a set or subset of
objects.
n The numb er of d istinct orderings of n items?
n items can go in the first position.
Once the first item is fixed, n-1 items can go in the 2nd position.
Then n-2 items in the third position, etc. Number of orderings is n (n-1) (n-2) 1, or n!
Remember that 1! = 0! = 1.
n This is called sam pling withou t rep lacemen t.
Once we use a value, it cant be used again.
8/8/2019 2 Statprob Walpole
14/29
WalpoleWalpoleCh 02:Ch 02: 1414
Spring 2007
Permutations of Partial OrderingsPermutations of Partial Orderings
n Sup pose that we will give 3 d ifferent awards to three
stud ents ou t of a class of 60 stud ents. How m any w ayscan the awards b e given.
What if the problem was slightly different and one studentcould win all three awards?
This would be with replacement.
The number is 603 = 216,000nWithout replacemen t, 60 stud ents could get the first
award , then 59 stud ents are eligible for the 2nd and 58for th e th ird , or 60*59*58 = 205,320.
n In general, the num ber of permutations of n thingstaken r at a time is written nPr = n! / (n r)!
60! / 57! = 60*59*58.
8/8/2019 2 Statprob Walpole
15/29
WalpoleWalpoleCh 02:Ch 02: 1515
Spring 2007
Other PermutationsOther Permutations
nCircu lar permu tations: n d istinct ob jects arranged in a
circle. The position of the first object could be anywhere.
If all objects moved one position clockwise, its still the samepermutation.
Fix the first object anywhere on the circle, then n-1 objects can
go to the left, n-2, next, etc. The number of circular permutations of n objects is (n - 1)!.
Another way to look at it: the set of all permutations is n!. Foreach starting value, there are n orderings that are identical(moving the same ordering around the circle). So the totalnumber of different orderings is n!/n = (n-1)!
For example, for the ordering 35142 for n = 5. There are 5 identical orderings (35142, 51423, 14235, 42351,
23514).
8/8/2019 2 Statprob Walpole
16/29
WalpoleWalpoleCh 02:Ch 02: 1616
Spring 2007
Permutations with Iden tical ObjectsPermutations with Iden tical Objects
n If some ob jects are iden tical, with n 1 of typ e 1, n 2 of
typ e 2, , n k of type k , and n = n1 + n 2 + + n k, thenumber of distinct permu tations is n ! / (n 1! n 2! n k!)
Example, 3 items of type 1, 1 of type 2 gives 4 permutations.
1112, 1121, 1211, and 2111.
For each of the four, there would be 3! orderings if the 1s were
distinct (say a, b, and c). For example, 1121 would be ab2c, ac2b, ba2c, bc2a, ca2b, and
cb2a. Or, if all items are distinct, there would be 4*3! Or 4!orderings.
Three items being identical reduces the number ofpermutations by a factor of 3!, so we divide by 3!.
8/8/2019 2 Statprob Walpole
17/29
WalpoleWalpoleCh 02:Ch 02: 1717
Spring 2007
Example of Permutation CaseExample of Permutation Case
n In a college football training session, thedefensive coordin ator need s to have 10 p layersstand ing in a row. Among these 10 p layers, there
are 1 freshm an, 2 sophomores, 4 juniors, and 3seniors. How m any d ifferent ways can they bearranged in a row if on ly their class level will bedistinguished?
nAnswer :
600,12!3!4!2!1
!10 =
8/8/2019 2 Statprob Walpole
18/29
WalpoleWalpoleCh 02:Ch 02: 1818
Spring 2007
Arranging n Objects Into r CellsArranging n Objects Into r Cells
n Partition ing n d istinct ob jects into r cells or subsets,
each of a given fixed size, where the ordering of objectswithin a cell doesn t matter.
n Example divid e 5 items in to two cells, one of size 3 andone of size 2.
{(1,2,3), (4,5)}, {(1,2,4), (3,5)}, {(1,2,5), (3,4)}, {(1,3,4), (2,5)},{(1,3,5), (2,4)}, {(1,4,5), (2,3)}, {(2,3,4), (1,5)}, {(2,3,5), (1,4)},{(2,4,5), (1,3)}, {(3,4,5), (1,2)}
n There are n! total possib le orderings, bu t n 1! In the 1stcell, and n 2! in the 2nd cell, etc., are identical.
n
In general, the number of distinct combinations of ndistinct objects into r cells, with n 1 items in the 1st cell,
n 2 in the 2nd, , and n r in the r
th cell is n! / (n 1! n 2!n r!).
8/8/2019 2 Statprob Walpole
19/29
WalpoleWalpoleCh 02:Ch 02: 1919
Spring 2007
Comb inations of n Items Taken r at a TimeComb inations of n Items Taken r at a Time
n To review, there are how m any p ermu tations of n items
taken r at a time if each ordering is distinct? n (n-1) (n-2) ... (n-r+1), or
nPr = n! / (n r)! .
For any given set of r items, there are r! possible orderings.
n So wh at if the order of the r items d oesnt matter?
Divide nPr by r! to get the number of distinct outcomes.
n The numb er of combinations of n items taken r at atime, wh ere order doesnt matter, is nCr = n! / (r!(n-r)!).
n Examp le: How many d istinct poker hand s of 5 cards
each can be dealt using a deck of 52 cards? 52C5 = 52! / (5!47!) = 52*51*50*49*48 / 5*4*3*2*1 = 2,598,960.
8/8/2019 2 Statprob Walpole
20/29
WalpoleWalpoleCh 02:Ch 02: 2020
Spring 2007
Probab ility of an EventProbab ility of an Event
n For now , we only consider discrete samp le spaces.
n Each p oint in a samp le space is assigned a weight orprobab ility value. The h igher the p robab ility, the morelikely that ou tcome is to occur.
n The probability of an event A is the sum of the
probabilities of the ind ividu al points in A. Then , 0 P(A) 1
P() = ?
P(S) = ?
If two events are mutually exclusive, (which means?)
(that they have no points in common, or A
B =
), then P(A B) = P(A) + P(B).
8/8/2019 2 Statprob Walpole
21/29
WalpoleWalpoleCh 02:Ch 02: 2121
Spring 2007
Relative Frequ ency for Probab ilityRelative Frequ ency for Probab ility
n If an experiment has N d ifferent equ ally likely
outcomes, and n outcomes correspond to even t A, thenP(A) = ?.
P(A) = n / N.
8/8/2019 2 Statprob Walpole
22/29
WalpoleWalpoleCh 02:Ch 02: 2222
Spring 2007
Add itive Probability RulesAdd itive Probability Rules
nWe already k now th at if A and B are mutually
exclusive, P(A B) = P(A) + P(B).nAlso, if more than 2 events are mu tually exclusive, the
probab ility of the union of all of those events is thesum of all of the in dividu al probab ilities.
nWhat is P(A B) if A and B are not mutually exclusive? Can use a Venn diagram to show this case.
The sample points in P(A B) are double counted.
So P(A B) = ?
P(A B) = P(A) + P(B) - P(A B).
nGiven th at we know P(A), what is P(A)? P(A) = 1 P(A).
8/8/2019 2 Statprob Walpole
23/29
WalpoleWalpoleCh 02:Ch 02: 2323
Spring 2007
Cond itional ProbabilityCond itional Probability
nCond itional probab ility, written P(B| A), is the
probab ility of B, given A, the p robab ility that Boccurs, given that we know that A has occurred.
Look at a Venn diagram of A and B.
Suppose all sample points are equally likely.
P(B) = (# of outcomes in B)/(total # of outcomes in S)
P(B|A) = ? P(B|A) = (# of outcomes in A B)/(total # of outcomes in A)
or, P(B|A) = P(A B) / P(A) (as long as P(A) > 0).
8/8/2019 2 Statprob Walpole
24/29
WalpoleWalpoleCh 02:Ch 02: 2424
Spring 2007
Conditional Probability ExampleConditional Probability Example
n Example. For the following population of 900 people:
Employed Unemployed Total Male 360 140 500
Female 240 160 400
Total 600 300 900
n If a person is selected at rand om from this group ,
P(E) = ?
P(M) = ?
P(E M) = ?
P(E|M) = ?
P(M|E) = ?
Answers: P(E)=600/900=2/3, P(M)=500/900=5/9,P(E M)=360/900=2/5, P(E|M)=360/500=18/25,P(M|E)=360/600=3/5.
8/8/2019 2 Statprob Walpole
25/29
WalpoleWalpoleCh 02:Ch 02: 2525
Spring 2007
IndependenceIndependence
nConditional probab ility helps us up date the
probability of an even t given add itional in formation.
n Sup pose P(B| A) = P(B). What d oes th is tell us?
Whether A occurs or not, the probability of B occurringdoesnt change.
n
If P(B| A) = P(B), then A and B are indepen dent. Can show that if P(B|A) = P(B) is true, then P(A|B) = P(A) is
always also true.
n From the above, and the definition of conditionalprobability, if A and B are ind epend ent,
P (A B ) = P(A) P(B)
8/8/2019 2 Statprob Walpole
26/29
WalpoleWalpoleCh 02:Ch 02: 2626
Spring 2007
Multiplicative RulesMultiplicative Rules
nRearranging the cond itional probability formula, if
both A an d B can occur, thenP(A B) = P(B| A) P(A)
Or, the p robab ility of both A and B occurring equ alsthe p robab ility of B given A times the prob ability of A.
nNote that it is also true that
P(A B) = P(A| B) P(B)n If (and only if) events A and B are indepen dent, then
from the above formu la, we haveP(A B) = P(A) P(B)
n For more than tw o ind ependent events, mu ltiply all ofthe p robab ilities together.
8/8/2019 2 Statprob Walpole
27/29
WalpoleWalpoleCh 02:Ch 02: 2727
Spring 2007
Theorem of Total Probab ilityTheorem of Total Probab ility
n Sup pose the samp le space S can be partitioned into
events A1, A2, and A3. What does this mean? A1, A2, and A3 are disjoint and between them cover all of S.
n Then the p robab ility of an event B occurring can b ecalcu lated u sing cond itional p robab ilities given thateither A1 or A2 or A3 occurred .
n P(B) = P(B| A1)P(A1) + P(B| A2)P(A2) + P(B| A3)P(A3).In w ords, this means?
The probability that B and A1 occur + the probability that Band A2 occur + the probability that B and A3 occur.
n This ru le, called the th eorem of total p robab ility, or therule of elimin ation , holds for any partitioning of S.
8/8/2019 2 Statprob Walpole
28/29
WalpoleWalpoleCh 02:Ch 02: 2828
Spring 2007
Bayes RuleBayes Rule
nHere we can calculate reverse conditional p robabilities.
nUsing an examp le from an earlier slide:
Employed Unemployed Total
Male 360 140 500
Female 240 160 400
Total 600 300 900n First we show how to use the theorem of total
probability to calculate P(M).
S can be partitioned into mutually exclusive events E and U.
Then P(M) = P(M|E)P(E) + P(M|U)P(U)
Here P(M) = (360/600)(600/900) + (140/300)(300/900) =2/5 + 7/45 = 5/9 (as expected).
Interpret 2/5 and 7/45 above.
Works with S partitioned into more than 2 events.
8/8/2019 2 Statprob Walpole
29/29
WalpoleWalpoleCh 02:Ch 02: 2929
Spring 2007
Bayes TheoremBayes Theorem
nBayes Theorem can b e used to adjust the probab ility of
a partition sub set of S given add itional information. In the example suppose we know that the person selected is a
male (M). Then what is the probability that the person selectedis employed?
As before, P(E|M) = P(E M)/P(M), by the definition ofconditional probability.
By Bayes Theorem,P(E|M) = _ P(M|E)P(E) _
P(M|E)P(E) + P(M|U)P(U)
In the example,P(E|M) = (360/600)(600/900) / (2/5 + 7/45) = 18/25
P(E) was 2/3. If we know M occurred, P(E|M) becomes 18/25.n P(E) is called the p rior (or "a p riori"), and P(E| M) is
called the posterior (or "a posteriori") probabilities.