Introduction to Introduction to ProbabilitiesProbabilities

Farrokh Alemi, Ph.D.Farrokh Alemi, Ph.D.Saturday, February 21, 2004Saturday, February 21, 2004

Probability can quantify Probability can quantify how uncertain we are how uncertain we are about a future eventabout a future event

Why measure uncertainty?Why measure uncertainty?

To make tradeoffs among uncertain To make tradeoffs among uncertain events  events 

To communicate about uncertaintyTo communicate about uncertainty

What is probability?What is probability?

In the Figure, where are the events that are not “A”?

How to Calculate How to Calculate Probability?Probability?

A

P(A)=

A

P(A)=

Calculus of Probabilities Calculus of Probabilities Helps Us Keep Track of Helps Us Keep Track of Uncertainty of Multiple Uncertainty of Multiple

EventsEvents

Joint probability, probability of Joint probability, probability of either event occurring, revising either event occurring, revising

probability after knew knowledge probability after knew knowledge is available, etc.is available, etc.

Probability of One or Other Probability of One or Other Event OccurringEvent Occurring

P(A or B) = P(A) + P(B) - P(A & B)

Example: Who Will Join Example: Who Will Join Proposed HMO?Proposed HMO?

P(Frail or Male) = P(Frail) - P(Frail & Male) + P(Male)

Probability of Two Probability of Two Events co-occurringEvents co-occurring

Effect of New KnowledgeEffect of New Knowledge

Conditional ProbabilityConditional Probability

Example: Hospitalization rate Example: Hospitalization rate of frail elderlyof frail elderly

Sources of DataSources of Data

Objective frequencyObjective frequency    – For example, one can see out of 100 For example, one can see out of 100

people approached about joining an people approached about joining an HMO, how many expressed an intent to HMO, how many expressed an intent to do so?   do so?  

Subjective opinionsSubjective opinions of experts  of experts  – For example, we can ask an expert to For example, we can ask an expert to

estimate the strength of their belief that estimate the strength of their belief that the event of interest might happen.  the event of interest might happen. 

Two Ways to Assess Subjective Two Ways to Assess Subjective ProbabilitiesProbabilities

Strength of Beliefs Strength of Beliefs – Do you think employees will join the Do you think employees will join the

plan?  On a scale from 0 to 1, with 1 plan?  On a scale from 0 to 1, with 1 being certain, how strongly do you feel being certain, how strongly do you feel you are right?you are right?  

Imagined Frequency Imagined Frequency – In your opinion, out of 100 employees, In your opinion, out of 100 employees,

how many will join the plan?how many will join the plan?  

Uncertainty for rare, one time events can be

measured

Axioms are always met,but that we want

them to be followed

All Calculus of Probability is All Calculus of Probability is Derived from Three AxiomsDerived from Three Axioms

1.1. The probability of an event is a The probability of an event is a positive number between 0 and 1 positive number between 0 and 1

2.2. One event will happen for sure, so One event will happen for sure, so the sum of the probabilities of all the sum of the probabilities of all events is 1events is 1

3.3. The probability of any two mutually The probability of any two mutually exclusive events is the sum of the exclusive events is the sum of the probability of each.probability of each.

Probabilities provide a Probabilities provide a context in which beliefs context in which beliefs

can be studied can be studied

Rules of probability provide a Rules of probability provide a systematic and orderly systematic and orderly

method method

Partitioning Leads to Bayes Partitioning Leads to Bayes FormulaFormula

P(Joining) = (a +b) / (a + b + c + d)  P(Joining) = (a +b) / (a + b + c + d)  P(Frail) = (a + c) / (a + b + c + d) P(Frail) = (a + c) / (a + b + c + d) P(Joining | Frail)  = a / (a + c) P(Joining | Frail)  = a / (a + c) P(Frail  |  Joining) = a / (a + b)P(Frail  |  Joining) = a / (a + b) P(Joining | Frail)  = P(Frail  | Joining)  *  P(Joining) /  P(Frail)P(Joining | Frail)  = P(Frail  | Joining)  *  P(Joining) /  P(Frail)

Frail elderly

Not frail elderly

Total

Joins the HMO a b a + bDoes not join the HMO c d c + dTotal a + c b + d a + b + c + d

Bayes Formula

Odds Form of Bayes Odds Form of Bayes FormulaFormula

Posterior odds after review of clues =Likelihood ratio associated with the clues * Prior odds

IndependenceIndependence

The occurrence of one event does not The occurrence of one event does not tell us much about the occurrence of tell us much about the occurrence of anotheranother

P(A | B) = P(A)P(A | B) = P(A) P(A&B) = P(A) * P(B) P(A&B) = P(A) * P(B)

Independence SimplifiesIndependence SimplifiesCalculation of Calculation of ProbabilitiesProbabilities

Joint probability can be Joint probability can be calculated from marginal calculated from marginal

probabilitiesprobabilities

Conditional Conditional Independence Simplifies Independence Simplifies

Bayes FormulaBayes Formula

Example of Example of DependenceDependence

P(Medication error ) P(Medication error ) ≠≠ P(Medication error| Long shift)P(Medication error| Long shift)

Conditional IndependenceConditional Independence

P(A | B, C) = P(A | C) P(A&B | C) = P(A | C) * P(B | C)

Conditional Independence Conditional Independence versus Independenceversus Independence

P(Medication error ) P(Medication error ) ≠≠ P(Medication error| Long shift) P(Medication error| Long shift)

P(Medication error | Long shift, Not fatigued) = P(Medication error| Not fatigued)

Can you come up with other examples

Example: What is the odds for Example: What is the odds for hospitalizing a female frail hospitalizing a female frail

elderly?elderly?

Posterior odds of

hospitalization=

Likelihood ratio

associated with being frail elderly

*

Likelihood ratio

associated with being

female

*Prior odds of

hospitalization

Likelihood ratio for frail elderly is 5/2Likelihood ratio for frail elderly is 5/2 Likelihood ratio for Females is 9/10. Likelihood ratio for Females is 9/10.  Prior odds for hospitalization is 1/2Prior odds for hospitalization is 1/2

Posterior odds of hospitalization=(5/2)*(9/10)*(1/2) = 1.125

Verifying IndependenceVerifying Independence

Reduce sample size and recalculateReduce sample size and recalculate Correlation analysisCorrelation analysis Directly ask expertsDirectly ask experts Separation in causal mapsSeparation in causal maps

Verifying Independence by Verifying Independence by Reducing Sample SizeReducing Sample Size

P(Error | Not fatigued) = 0.50P(Error | Not fatigued) = 0.50 P(Error | Not fatigue & Long shift) = 2/4 = P(Error | Not fatigue & Long shift) = 2/4 =

0.50 0.50

CaseMedication

error Long shift Fatigue1 No Yes No2 No Yes No3 No No No4 No No No5 Yes Yes No6 Yes No No7 Yes No No8 Yes Yes No9 No No Yes10 No No Yes11 No Yes Yes12 No No Yes13 No No Yes14 No No Yes15 No No Yes16 No No Yes17 Yes No Yes18 Yes No Yes

Verifying Conditional Verifying Conditional Independence Through Independence Through

CorrelationsCorrelations

RRabab is the correlation between A and B is the correlation between A and B

RRacac is the correlation between events A is the correlation between events A and Cand C

RRcbcb is the correlation between event C is the correlation between event C and Band B

If RIf Rabab= R= Racac R Rcb cb then A is independent of then A is independent of B given the condition CB given the condition C

Verifying Independence Verifying Independence Through CorrelationsThrough Correlations

0.91 0.91 ~~ 0.82 * 0.95  0.82 * 0.95 

Case Age BP Weight1 35 140 2002 30 130 1853 19 120 1804 20 111 1755 17 105 1706 16 103 1657 20 102 155

Rage, blood pressure  = 0.91

Rage, weight  = 0.82

R weight, blood pressure  = 0.95

Rage, blood pressure

=

0.91

0.82 * 0.95 =  R

a

g

e

,

w

e

i

g

h

t

*

  R

w

e

i

g

h

t

,

b

l

o

o

d

p

r

e

s

s

u

r

e

Verifying Independence by Verifying Independence by Asking ExpertsAsking Experts

Write each event on a 3 x 5 cardWrite each event on a 3 x 5 card   Ask experts to assume a population where Ask experts to assume a population where

condition has been met  condition has been met    Ask the expert to pair the cards if knowing the Ask the expert to pair the cards if knowing the

value of one event will make it considerably value of one event will make it considerably easier to estimate the value of the other easier to estimate the value of the other

  Repeat these steps for other populations Repeat these steps for other populations Ask experts to share their clusteringAsk experts to share their clustering Have experts discuss any areas of Have experts discuss any areas of

disagreement disagreement   Use majority rule to choose the final clustersUse majority rule to choose the final clusters

Verifying Independence by Verifying Independence by Causal MapsCausal Maps

Ask expert to draw a causal mapAsk expert to draw a causal map Conditional independence: A node that if Conditional independence: A node that if

removed would sever the flow from cause removed would sever the flow from cause to consequence to consequence

Blood pressure does not depend on age given weight

Probability of Rare EventsProbability of Rare Events

Event of interest is quite rare (less Event of interest is quite rare (less than 5%)than 5%)– Because of lack of repetition, it is difficult Because of lack of repetition, it is difficult

to assess the probability of such events to assess the probability of such events from observing historical patterns.  from observing historical patterns. 

– Because experts exaggerate small Because experts exaggerate small probabilities, it is difficult to rely on probabilities, it is difficult to rely on experts for these estimates.  experts for these estimates. 

Measure rare probabilities through Measure rare probabilities through time to the event time to the event

Examples for Calculation of Examples for Calculation of Rare ProbabilitiesRare Probabilities

Probability = 1 / (1+time to event)

ISO 17799 word Frequency of event Calculation

Rare  probability

Negligible Once in a decade =1/(1+3650) 0.0003

Very low 2-3 times every 5 years =2.5/(5*365) 0.0014

Low <= once per year =1/365 0.0027

Medium <= once per 6 months =1/(6*30) 0.0056

High <= once per month =1/30 0.0333

Very high => once per week =1/7 0.1429

Take Home LessonsTake Home Lessons

Probability calculus allow us to keep Probability calculus allow us to keep track of complex sequence of eventstrack of complex sequence of events

Conditional independence helps us Conditional independence helps us simplify taskssimplify tasks

Rare probabilities can be estimated Rare probabilities can be estimated from time to the eventfrom time to the event