CPSC 531: Probability Review1 CPSC 531:Probability & Statistics: Review Instructor: Anirban Mahanti Office: ICT 745 Email: [email protected] Class

CPSC 531: Probability Review 1

CPSC 531:Probability & Statistics: Review

Instructor: Anirban MahantiOffice: ICT 745Email: [email protected] Location: TRB 101Lectures: TR 15:30 – 16:45 hoursClass web page:

http://pages.cpsc.ucalgary.ca/~mahanti/teaching/F05/CPSC531

Notes derived from “Probability and Statistics” by M. DeGroot and M. Schervish, Third edition, Addison Wesley, 2002.

http://pages.cpsc.ucalgary.ca/~mahanti/teaching/F05/CPSC531


Outline Experiments, sample space, and events Review of set theory Probability: definition, property,

interpretation Finite sample space Counting methods Conditional probability Independent events Law of total probability Bayes’ Theorem


Experiments, sample space, and events Experiment – in probability theory refers to

a process whose outcome is not known in advance with certainty E.g., An experiment consists of tossing a coin 10

times. What is the probability that at least four heads are obtained in a row?

Sample space (S) – is the collection of all possible outcomes of an experiment. Each outcome is a sample point. E.g., Rolling a six-sided dice: S = {1, 2, 3, 4, 5,

6} Event – is a subset of the sample space

E.g., A is an event that an even number is rolled: A = {2, 4, 6}


Set Theory (1)

Some sets contain only finite number of elements, while others contain infinitely many elements. An infinite set can be classified as: Countable - if there is a one-to-one

correspondence between elements of the set and the set of natural numbers

Uncountable – converse of countable; e.g., set of real numbers, numbers in the interval [0, 1]

Empty set: Ø is the “null” set Union: A B = is the event containing all

outcomes that are in A or B, or both


Set Theory (2) Intersection: A B = event contained in

both A and B Complement: AC is the complement of

event A and contains all outcomes in the sample space S that do not belong to A

Idempotent laws A A = A A A = A

Associative laws (A B) C = A (B C) = (A B) C (A B) C = A (B C) = (A B) C


Set Theory (3)

Commutative laws A B = B A A B = B A

Distributive laws A (B C) = (A B) (A C) A (B C) = (A B) (A C)

Identity laws A Ø = A A U = A A U = U A Ø = Ø


Probability - definition A probability distribution, or simply a

probability, on a sample space S is a specification of numbers P(.) that satisfies the following axioms:

1) For every event A, P(A) ≥ 02) P(S) = 13) For every infinite sequence of mutually

exclusive events A1, A2, …

11 i)iP(A

iiAP


Properties of Probability

For any event A, P(AC) = 1 – P(A)

Since Ø = SC, P(Ø) = 0

If A B, P(A) ≤ P(B)

For every event A, 0 ≤ P(A) ≤ 1

Home work: Recall Venn diagrams … they can help in proving the above properties.


Example (1)

Select one ball from a box containing white (W), red (R), blue (B), and green (G) balls. Suppose that P(R) = 0.1 and P(B) = 0.5. What is the probability of selecting a white or a green ball?

Solution:W U R U B U G = SFrom axiom 3, we have P(S) = P(W) + P(R) + P(B)

+ P(G)Also, P(WUG) = P(W)+P(G) = 1 – P(R) – P(B) = 0.4


Example (2)

Rolling a fair dice: P(1) = P(2) = P(3) = P(4) = P(5) = P(6) = 1/6, where P(i) is the probability of rolling a face with i dots.

Question: P({1, 3}) = ? P({2, 4, 6}) = ?


Interpretations of probability

Classical view: Probability measures the long-term likelihood of outcomes. This is, if an experiment is repeated “many” times, event A should occur roughly P(A) fraction of the time. Can “many” be quantified?

Bayesian view: Probability represents a personal estimate of how likely an event is going to occur.

Note: the theory of probability does not depend on the interpretation. Good!


Finite Sample Spaces A common assumption for a finite sample

space S containing n outcomes s1, s2, … sn is that each outcome is equally likely.

If an event A in this sample space contains exactly m outcomes, then P(A) = m/n

Example: Toss three fair coins simultaneously. What is the probability of obtaining exactly two heads?S = {(HHH), (HHT), (HTH), (THH), (TTH), (HTT), (THT), (TTT)}P(2 H’s) = 3/8


Counting Methods

Multiplication rule

Permutations

Combinations


Multiplication rule

If an experiment has k “parts” (k ≥ 2), and the ith part has ni possible outcomes, then the total number of possible outcomes is n1 x n2 … x nn

Example: 6 coins are tossed 6 parts to the experiments: toss 1, 2, …, 6 2 possible outcomes per part (i.e., per toss) Possible outcomes = 26 = 64


Permutations (1)

Consider an experiment that involves sampling without replacement. E.g., arrange 5 different books on a shelf. How many ways can the books be arranged?

Think of the shelf as having 5 slots 5 choices for filling the first slot 4 choices for filling the second slot 3 choices for filling the third slot 2 choices for filling the fourth slot …. So, we have 5x4x3x2x1 = 120 possibilities


Permutations (2) Given n items, the number of permutations

(or arrangements) of all n items equals n! n! = n x (n-1) x (n-2) x … x 2 x 1 Definition for 0! - It equals 1 since there is only

1 way to arrange zero items. When not all items are ordered: E.g., a club

has 25 members and 2 officers (a president and a secretary). These officers are to be chosen from the members of the club. How many ways can the officers be selected? 25 ways to select the first officer and 24 ways to

select the second officer. Answer = 25x24 = 600


Permutations (3)

Generalization: The number of ordered ways k items can be selected from a pool of n items is:

Pn,k = n x (n-1) x … X (n-k+1) = n!/(n-k)! Home work problem: What is the probability

that at least two people share the same birthday? You may assume the following: 365 possible birthdays (ignore leap years) Our group consists of 2 ≤ k ≤ 365 people who are

unrelated (no twins) Each birthday is equally likely (birth rate

independent of the time of year)


Combinations (1)

Consider choosing a subset containing k items from a set of n distinct items. How many different subsets are possible? The arrangement of elements in the subset is

irrelevant. We treat each subset as an unit. Such unordered samples are called

combinations. The number of combinations possible when selecting k items from a set of n items is denoted by Cn,k.

We will use the multiplication rule to find Cn,k.


Combinations (2)

Note that one way to find Pn,k is to compute Cn,k (i.e., unordered selection of k items) and multiply by the number of ways of ordering k items (i.e., k!).

Therefore, Pn,k = Cn,k x k! That is,

)!(!

!

!,

, knk

n

k

PC kn

kn


Combinations (3) Example: A fair coin is tossed 10 times. Determine

the probability, p, of obtaining exactly 3 heads.Solution Each experiment has two possible outcome, heads or

tails. Since the coins are fair, we can assume that each outcome is equally likely. Therefore, the total number of outcomes in this experiment is = 210

The number of different arrangements possible with 3 heads and 7 tails is = C10,3

Therefore,

1172.0210

3,10 C

p


Conditional Probability (1) The conditional probability of event A given that

event B has occurred is

The conditional probability P(A|B) is undefined if P(B)=0 The frequency interpretation: If an experiment is

repeated a large number of times, then event B occurs approximately P(B) fraction of the time and the event A and B both occur approximately P(A B) portion of time. Therefore, among the repetitions in which B occurs, the proportion of events in which A also occurs is P(A B) /P(B). P(A B) also denoted as P(AB).

)(

)()|(

BP

BAPBAP


Conditional Probability (2) Example: Two balls are selected at random,

without replacement, from a bag containing r red balls and b blue balls. Determine probability p that the first selected ball is red and the second selected ball is blue.SolutionA = event that first ball is red; B = event that second ball is blueP(A) = prob. first ball is red = r/(r+b)P(B|A) = prob. blue in second draw = b/(r+b-1)P(B|A) = P(AB)/P(A) P(AB) = (r.b)/((r+b)(r+b-1))


Conditional Probability (3) P(AB) = P(A) P(A|B); extended, as shown next Suppose that A1, A2, A3, …, An are events such that

P(B) > 0 and P(A1A2A3 …An-1) > 0. Then,P(A1A2A3…An) = P(A1)P(A2|A1) P(A3|A1A2)… P(An|A1A2A3 …An-1)

Example: Draw 4 balls, one by one, without replacement, from a box containing r red balls and b blue balls. Probability of sequence red, red, blue, blue?Solution:Let Rj and Bj denote event that a red and blue ball is drawn, respectively.P(R1R2B3B4) = P(R1)P(R2|R1)P(B3|R1R2) P(B4|R1R2B3)

complete the calculations on your own …


Independent Events (1)

Independent events are those that don’t have any effect on each other. That is, knowing one of them occurs does not provide any information about the occurrence of the other event. Mathematically, A and B are independent if P(A|B) = P(A) and P(B|A) = P(B)

From conditional probability definition, we have P(AB) = P(A|B)P(B). Therefore, if A and B are independent events, P(AB) = P(A)P(B)


Independent Events (2)

Example: Determine the probability pn that exactly n tosses are required for a head to appear for the first time.

SolutionWe need to determine the probability of a sequence of (n-1) tails followed by a head. The probability of obtaining a heads or tails (for a fair coin) is ½. Therefore, pn = (½)n.


Law of Total Probability

Let B1, B2, B3, …, Bk be mutually disjoint events such that B1 U B2 U B3 U … Bk equal the sample space S. Then, for any event A in S, we have

k

jjj BAPBPAP

1)|()()(Explanation

A = (B1A)U (B2A)U… (BkA).

The (BjA)’s are disjoint events. Therefore, using laws of conditional probability we get:

B1 B2

B3B4

A

kjBP

BAPBPABPAP

j

k

jjj

k

jj

,...,1for 0)( if

,)|()()()(11


Bayes’ Theorem

Partition: The events B1, B2, B3, …, Bk form a partition of a set S if they are mutually disjoint and SBk

i i 1

Bayes’ Theorem: Suppose that B1, B2, B3, …, Bk form a partition of sample space S such that P(Bj)> 0 for j = 1, …, k. Let A be an event in S such that P(A)>0. Then, for i = 1, …, k,

kj jj

iii

BAPBP

BAPBPABP

1 )|()(

)|()()|(


Summary

A review of basic probability and statistics was presented. You probably enjoyed it!

Next, we will talk about discrete and continuous distributions.

Documents

CPSC 531: Probability Review1 CPSC 531:Probability & Statistics: Review Instructor: Anirban Mahanti Office: ICT 745 Email: [email protected] Class