Copyright 2002

David M. Hassenzahl

Using r and 2

Statistics

for

Risk Analysis

Copyright 2002

David M. Hassenzahl

Objectives

• Purpose: to compare model to data– “validate model” (or not)

• Two techniques– r (correlation coefficient) 2 (Chi-squared)

• Apply to a familiar problem (barium decay)

Copyright 2002

David M. Hassenzahl

Statistics

• Descriptive

• Comparison– Z-scores, hypotheses– Confidence levels– Evaluating models– Correlation and Chi-squared

Copyright 2002

David M. Hassenzahl

Confidence Levels

• Given 100 flips of a coin. Would you bet $1000 that the next flip will yield heads if– 50 heads?– 90 heads?– 99 heads?– 999 heads out of the last 1000 flips?

• How about for $5? For 50% of your current net worth?

Copyright 2002

David M. Hassenzahl

Statistical Significance

• Z = – (sample occurrence – number in sample

times expected probability – Divide by square root of (np(1-p)

• Student’s t• One-sided versus two sided tests!• “p values”• Confidence intervals

Copyright 2002

David M. Hassenzahl

Type I and II errors

• Type I: reject the truth! (accuracy)

• Type II: accept an untruth! (precision)

• This is important… there’s often a tradeoff here!

Copyright 2002

David M. Hassenzahl

Z-scores Intuition

• Z score will be big if– Numerator: if xbar >> OR >> xbar – s is very small– n is very big

• Bigger Z-score: confidence that xbar • Small Z-score: confidence that xbar

ns

μxZ 0

Copyright 2002

David M. Hassenzahl

From Z’s to r’s and 2

• r and 2 compare more than one estimate

• Compare – Set of model predictions to– Set of data or observations

• If r is SMALL (little correlation) the model doesn’t fit

• If 2 is SMALL then the model does fit

Copyright 2002

David M. Hassenzahl

“Goodness of fit”

• We say that r and 2 evaluate “goodness of fit”

• Note that a good fit does not mean that the model is right!

Copyright 2002

David M. Hassenzahl

Barium Decay

• Theory: barium is removed as a constant function of concentration

• “Exponential decay”

• C(T) = C(0)ekT

– k = -0.007/min– C(0) = 0.16 mgBa / liter blood

• (From SWRI page 56 – 63; hypothetical)

Copyright 2002

David M. Hassenzahl

Exponential Decay Model

Figure 2-9 from Should We Risk It?

Bloo

d ba

rium

conc

entra

tion

(mg/

l)

time

0

0.16

0.12

0.08

0.04

0 12060 180 300240 420360

Copyright 2002

David M. Hassenzahl

Sample blood at 1 hour intervals

Time (hours) Measured Concentration

0 0.16

1 0.13

2 0.087

3 0.055

4 0.040

5 0.022

6 0.009

7 0.002

8 0.001

Copyright 2002

David M. Hassenzahl

Measured and ExpectedTime (hours) Measured

ConcentrationPredicted

Concentration

0 0.16 0.16

1 0.13 0.11

2 0.087 0.070

3 0.055 0.045

4 0.040 0.030

5 0.022 0.020

6 0.009 0.013

7 0.002 0.0095

8 0.001 0.0056

Copyright 2002

David M. Hassenzahl

Graphical Comparison

After Figure 2-9 from Should We Risk It?

Bloo

d ba

rium

conc

entra

tion

(mg/

l)

time

0

0.16

0.12

0.08

0.04

0 12060 180 300240 420360

Copyright 2002

David M. Hassenzahl

How well does the model fit?

• Why do we care?– Future predictions– Is there a better model?

• Looks OK. Is that good enough?

• Try our two tools: r and 2

Copyright 2002

David M. Hassenzahl

r Conceptual

• Compares model predictions to the data

• Asks – “What if there is no relationship (or

correlation) between model and data?”– Is the model as close to the average value

of the x’s as it is to the actual x’s?

Copyright 2002

David M. Hassenzahl

r terms or components

• Predicted mean and standard deviation

• Observed mean and standard deviation

• “Covariance”– Do they go up and down together?– If independent, covariance = 0

• r = Covariance (predicted, observed)

(STDEV O) (STDEV P)

Copyright 2002

David M. Hassenzahl

Means

• Observed xobar = ( xoi) /n

• xobar = (0.16+0.13+0.087+0.055+0.040+0.022+0.009+0.002+0.000)/9 = 0.056

• Observed xpbar = ( xpi) /n = 0.051

Copyright 2002

David M. Hassenzahl

Standard Deviations

051.011 2 pipp xxns

056.011 2

oioo xxns

Copyright 2002

David M. Hassenzahl

Covariance

pipoio xxxxnpoCov 11,

Copyright 2002

David M. Hassenzahl

Calculated r

• r = Covariance (predicted, observed)

(STDEV O) (STDEV P)

99.0

s s

o , p covr

po

Copyright 2002

David M. Hassenzahl

Intuition Behind r

• If there is no relationship between observed and predicted, r = 0

• If r 0, positive correlation

• If r 0, negative correlation

Copyright 2002

David M. Hassenzahl

r Discussed

• 0.99 seems reasonably good

• Is there a better fit

• What about theory?

• Limitations: even low correlations may be okay…just a screening tool

Copyright 2002

David M. Hassenzahl

t test for r

• n-2 = 7 degrees of freedom

• Look it up in the Student’s-t table

• Accept model validity at 99% confidence level if Student’s t is greater than 2.998

21

2

r

nrt

Copyright 2002

David M. Hassenzahl

Chi-squared

• This formula “normalizes” to the size of the individual xoi

• If all xoi xip, 2 = 0

• Look up value in table (page 398)

n

i 1 pi

2pioi2

x

xxχ

Copyright 2002

David M. Hassenzahl

Chi-squared

• 9 data points• Suppose we are concerned with 99%

confidence level• We would need a chi-squared of greater

than 21.7 to reject this line• Calculating, we find that 2 = 0.06! • Note that it still might be possible to find

a better line, even with the exponential

Copyright 2002

David M. Hassenzahl

Conclusion

• Both r and Chi-squared appear to validate this model

• Suggests that our theoretical idea about the model may be valid

• Doesn’t tell us we are right, just that we may be acceptably wrong!