Decision Analysis and Risk Analysis Applications

1-25-05MS&E 290, Stanford University 1

Decision Analysis and Risk Analysis Applications

D. Warner North

MS&E 290

Tuesday, January 25, 2005


New York Times - Sunday, August 11, 2002

• If the Big One Hits the Big Easy, the Good Times May Be Over Forever

By ADAM COHEN (NYT) 1018 words Late Edition - Final , Section 4 , Page 12 , Column 1

• ABSTRACT - Adam Cohen Editorial Observer on grave threat that big hurricane could wipe out low-lying city of New Orleans; describes complacency of its people and inadequate preparations by government Dot Wilson knows how bad the Big One could be. When Hurricane Betsy hit New Orleans in 1965 with 125 m.p.h. winds, leaving 75 dead in Louisiana and South Florida, she walked more than a mile in chest-high water, holding her infant daughter overhead. But when Ms. Wilson held a hurricane-preparedness teach-in recently at the community center she heads, attendance was sparse. ''A lot of people don't see storms as serious,'' she said with a sigh. But people who have been around for a while know better, she said, adding, ''We saw the bodies.''

• New Orleans -- home to the French Quarter's iron-latticed buildings and the Garden District's stately Greek Revival mansions, to Preservation Hall jazz and Mardi Gras parades -- may be America's most architecturally distinctive and culturally rich city. But it is also a disaster waiting to happen. New Orleans is the only major American city below sea level, and it is wedged between Lake Pontchartrain and the Mississippi. If a bad hurricane hit, experts say, the city could fill up like a cereal bowl, killing tens of thousands and laying waste to the city's architectural heritage. If the Big One hit, New Orleans could disappear.


New York Times, Thursday, October 3, 2002

• Thousands Seek Safety as Hurricane Nears Gulf Coast

By JEFFREY GETTLEMAN (NYT) 1308 words Late Edition - Final , Section A , Page 24 , Column 3

• ABSTRACT - About half million people flee southern Louisiana and Texas with approach of Hurricane Lili, which is heading toward Gulf Coast with winds of more than 140 miles per hour, making it daunting Category 4 storm; map; chart of five most intense hurricanes to hit US since 1928; photos (M) Highways across southern Louisiana and Texas were solid columns of steel today as more than half a million people grabbed their valuables and fled their homes, looking for higher, safer ground before Hurricane Lili hit.

• The exodus of cars and trucks, some with furniture lashed down on their roofs, began in low-lying areas but quickly spread inland as the storm intensified and threatened to become the worst natural disaster here in decades.


October 4, 2002 — Using state-of-the-art equipment, including a radar collaboratively built by the NOAA National Severe Storms Laboratory, research scientists captured Hurricane Lili Thursday morning as she came onshore along the southern Louisiana coast. Three mobile Doppler radars, as well several instrumented towers, were strategically placed near Lafayette, La., in order to study the structure of the rainfall and wind flow around the storm. The data collected may help scientists develop better estimates of rainfall amounts, which could lead to more accurate and timely forecasts of inland flooding in the future.

Hurricane Lili

http://www.nssl.noaa.gov/


New York Times, Friday, October 4, 2002

• Hurricane Hits Gulf Coast, Weakened but Still Punishing

By JEFFREY GETTLEMAN (NYT) 1052 words Late Edition - Final , Section A , Page 18 , Column 1

• ABSTRACT - Hurricane Lili slams into Louisiana, but with winds sharply diminished as storm devolves from Category 4 to Category 2 by time it makes landfall; still, there is widespread damage, with community of Abbeville being hardest hit; map; chart of Lili's wind speeds; map; photo (M) Hurricane Lili slammed into Louisiana today with tornado-force winds that ripped trees from the ground, smashed mobile homes and caused widespread blackouts that may last for weeks.

• But it could have been worse. The storm lost significant strength overnight, dwindling from a Category 4 hurricane to Category 2, and no deaths or serious injuries were reported.

The Decision to Seed Hurricanes

Science, 176, 1191-1202, 1972

(paper on class website, pdf file)

R.A. Howard, J.E. Matheson, D.W. North


Decision Analysis of Hurricane Modification

Background:The Cost of Hurricanes

Average Annual Cost of Hurricane Damage: $400 MillionHurricane Betsy, 1965: $1.4 BillionHurricane Camille, 1969:

$1.4 Billion

The Hurricane Debbie Experiment, 1969

Reductions of 31%, 15% in Maximum Wind Speed Were Observed in the Two Seeding Experiments


Decisions:• Strategic Decision:

Should Operational Seeding by the U.S. Government be Permitted?

• Research Policy Decision:Should the Present Level of Research and Experimentation be

Changed?

• Tactical Operational Decision:Should a Particular Hurricane be Seeded? (not analyzed in SRI

study)

Decision Analysis of Hurricane Modification


The Operational Seeding Decision: Conceptual Overview

Meteorological Knowledge

Hurricane Model

Economical Model

Legal/Social Model

Seeding Decision Strategic

Storm Parameters:

Critical Storm Characteristics:

Maximum Sustained Wind

Property Damage

Decision Criteria

Government Responsibility Cost


Maximum Sustained Winds Over Time

t0

Seeding Initiated

t1

Landfall

w(t1)

w’(t1)

w(t0)

Maximum Sustained Winds, w

12 Hours

w(t)

w(t) Without Seeding

w’(t) With Seeding


The Seeding Decision: Decision Tree

Property Damage Government Responsibility

Property Damage Government ResponsibilitySeed

Do Not Seed

Decision Alternatives

Resolution of Uncertainty:

Change in Maximum Sustained Surface

WindConsequenc

es


Probabilistic Model for 12-Hour Change in Maximum Sustained

Wind, Natural Hurricane

No Predictability from Meteorological Environment Assumed.

Probability Distribution Based on:

• Observations of 12-Hour Changes in Central Pressure

• Empirical Formula Relating Central Pressure to Maximum Sustained Wind

40 70 100 130 160Maximum Sustained Wind, mph (or %)

Prob. Density Function

time t = 0 (seeding initiated)

40 70 100 130 160Maximum Sustained Wind, mph (or %)

Prob. Density Function

time t = 12 hours (landfall)

= 15.6


m =100 =15.6

m =110 =18.6

Probabilistic Model for 12-Hour Change in Maximum Sustained

Wind, Natural Hurricane

Probability Distribution Based on:

• Probability Distribution for 12-Hour Change in Natural Hurricane

• Expert Judgment on Average Effect of Seeding

• Expert Judgment on Fluctuations From Average Effect in Seeding a Particular Storm

Maximum Sustained Wind, mph (or %)40 70 100 130 160

Probabilities Assigned

to Hypothese

s:H1: Stormfury

Hypothesis:“Seeding Causes an Average Reduction in maximum Sustained Wind”

m = 85 =18.6

H2: Null Hypothesis:“Seeding Has No Effect on Maxium Sustained Winds”

H3: Pessimistic Hypothesis:“Seeding Causes an Average Increase in Maximum Sustained wind”

0.49

0.49

0.02


Probabilities for the Three Models

Judgments from Hurricane Experts: • Before Debbie, P(H1) > P(H3)

• After Debbie, P(H1) = P(H2)

Resulting Probabilities: Before the Hurricane Debbie Seeding: P(H1)= 0.15; P(H2)= 0.75; P(H3)= 0.10After Hurricane Debbie Seeding: P(H1)= 0.49; P(H2)= 0.49; P(H3)= 0.02Before and After Probabilities Related by

Bayes’ Rule

H1: Stormfury Hypothesis:“Seeding Causes an Average Reduction in maximum Sustained Wind”

H2: Null Hypothesis:“Seeding Has No Effect on Maximum Sustained Winds”

H3: Pessimistic Hypothesis:“Seeding Causes an Average Increase in Maximum Sustained wind”


Summary of Current Meteorological Information

Maximum Sustained Wind 12 Hours After Seeding Decision (%)60 70 80 90 100 110 120 130

1.0

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0.0

Unseeded Hurricane

Seeded Hurricane

Prob

abili

ty th

e W

ind

Spee

d is

Gr

eate

r Tha

n Am

ount

Sho

wn


Maximum Sustained Wind Versus Property Damage

w, Maximum Sustained Surface wind Speed (mph)50

Tota

l Equ

ival

ent

Resi

dent

ial P

rope

rty

Dam

age

(196

9 $)

60 70 80 90 100 110 120 130 140 150 160 180 2001 x 106

10 x 106

100 x 106

d = c1wc2

c2 = 4.363


Probability Distributions on Property Damage for the Seeded and

Unseeded Hurricane

Property Damage (M$)0 50 100 150 200 250 300 350

1.0

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0.0

Prob

abili

ty T

hat

Prop

erty

D

amag

e is

Gre

ater

tha

n Am

ount

Sho

wn

Unseeded

Seeded

400


The Seeding Decision for the Nominal Hurricane

+32%

+16

0

-16

-34

Seed:Expected Loss

Probabilities Assigned to Outcomes

Change in Maximum

Sustained Wind$335.8

191.1

100.0

46.7

16.3

Property Damage

Loss (M$)0.038

0.143

0.3920.2550.172

= $94.08 + $0.25= $94.33

Do Not Seed:Expected Loss

= $116.00

Cost of Seeding = $0.25

Expected Value (M$)

+32%

+16

0

-16

-34

0.054

0.206

0.4800.2060.054

$335.8

191.1

100.0

46.7

16.3

Resolution of Uncertainty:

Economic Value of Seeding = $21.67 million (18.7% reduction in loss)Decision not sensitive to specific assumptions.


Government Responsibility Costs

Public Outrage?Lawsuits?

If a SeededHurricane Intensifies

When a Hurricane Labeled “U.S. Government Seeded” it is No longer a “Natural Disaster”

Concept: What Increment of Property Damage Reduction Justifies the Assumption of Responsibility Entailed by Seeding a Hurricane?


Government Responsibility Cost Assumed for Strategic Analysis

($100 Million Storm)

12-HOUR WINDCHANGE

PROPERTYDAMANGE(millions of

dollars) GOVERNMENT RESPONSIBILITY COST% millions of dollars

SevereIntensification(+32%) 335.8 +50% 167.9

ModerateIntensification(+16%) 191.1 +30% 57.3

No change (0%) 100 5% 5

Winds Diminish:(-16%)(-34%)

46.716.3

00

00


The Seeding Decision for the Nominal Hurricane (Government Responsibility

Cost Included)

+32%

+16

0

-16

-34

Seed:Expected Loss

Probabilities Assigned to Outcomes

Change in Maximum

Sustained Wind$335.8

191.1

100.0

46.7

16.3

Property Damage

Loss (M$)0.038

0.142

0.3920.2550.172

= $110.67 + $0.25= $110.92

Do Not Seed:Expected Loss= $116.00

Cost of Seeding = $0.25

Expected Value (M$)

+32%

+16

0

-16

-34

0.054

0.206

0.4800.2060.054

$335.8

191.1

100.0

46.7

16.3Value of Seeding = $5.08 million (4.4% reduction)

Government Responsibility

Cost (% of property damage)

$503.7

248.4

105.0

46.7

16.3

Total Cost (M$)

$335.8

191.1

100.0

46.7

16.3

+50%

+30

+5

0

0-

-

-

-

-


Legal findings: No firm Legal Basis for Operational Seeding Currently

Appears to Exist

Sovereign Immunity:In conclusion, existing immunity law provides only partial and unpredictable protection at best. There are also grounds for recognizing that immunity defenses may be avoided in most cases if the plaintiff carefully chooses his remedy, his legal theory, and his forum. Only specific Congressional action offers a prospect of substantial, predictable immunity protection.- Appendix D, p. 9

Basis for Lawsuits:The common law and inverse condemnation theories appear to offer plaintiff's attorneys substantial grounds for recovering damages where they can prove that modification activities caused injury, death, or property damage. This is particularly significant in the light of recent procedural developments (especially the decline of immunity defenses as discussed in an earlier memorandum). These tentative results suggest that the project agency may wish specific Congressional authorization for its project.- Appendix E, p. 15


The Value of an Expanded Research and Experimentation

Program

Limiting Case:What is the Value of Perfect Information

Concept:How Much Should the Government be Willing to Pay a Clairvoyant to Learn Which of the Three Hypotheses,H1’ H2’ or H3’

Is Actually True Before making the Operational Seeding Decision For a Single Hurricane?


Expected Value of the Clairvoyant’s Information: Which Hypothesis Describes the Effect of Seeding?

+32 %+16

0-16-34

+32+16

0-16-34

+32+16

0-16-34

+32+16

0-16-34

+32+16

0-16-34

+32+16

0-16-34

+32+16

0-16-34

+32+16

0-16-34

Outcomes$335.8

191.1100.0

46.716.3

335.8191.1100.0

46.716.3

335.8191.1100.0

46.716.3

335.8191.1100.0

46.716.3

335.8191.1100.0

46.716.3

335.8191.1100.0

46.716.3

335.8191.1100.0

46.716.3

335.8191.1100.0

46.716.3

Property Damage Loss

(M$)$69.42

$116.00

SeedDo Not Seed

H1 True

$69.42

Seeding Decision

$116.25

$116.00

SeedDo Not Seed

H2 True

$116.00

$167.61

$116.00

SeedDo Not Seed

H3 True

$116.00

$94.33

$116.00

SeedDo Not Seed$94.33

$93.17

$93.17

$94.33

0.49

0.49

0.02Obtain

Information

Do not Obtain

Information

Results of Information

Choice of Whether to Gather

Information

Value = $1.16 property damage onlyValue = 13.63 property damage plus government responsibility cost.Value is Higher if Seeding is not Permitted with Present Information.

Expected Value (M$)


Value of A Seeding Experiment(Government Responsibility Cost

Included)

Value = $2.96 millionActual cost = $0.25 million

$103.44

+32%+16

0-16-34+32%+16

0-16-34+32%+16

0-16-34+32%+16

0-16-34

Outcomes

$503.7248.4105.0

46.716.3

503.7248.4105.0

46.716.3

503.7248.4105.0

46.716.3

503.7248.4105.0

46.716.3

Total Cost(millions of

dollars)Operational

Seeding Decision

$103.44

$116.00

SeedDo Not Seed

$110.92

$116.00

SeedDo Not Seed$110.92

$107.96

$110.92

0.038

Perform Experimen

t

Do not Perform

Experiment

Results of Experiment

Choice of Whether to Perform Experiment

Expected Value (M$)

$87.83

$116.00

$116.00

$116.00

0.143

0.392

0.255

0.172


Summary of the Value of Additional Information on the Effect of Seeding

(Values in Millions of Dollars)“Standard”

Hurricane Usedin Analysis of

Chapters III and IV

SingleHurricaneSeason*

All FutureHurricane

Seasons (discountedat 7%)

Expected PropertyDamage Without Seeding $116 (100%) $220 $3,142

Espected Value of PerfectInformation 13.6 (11.8%) 26 370

Expected Value of A FieldExperimant Consisting ofTwo Experimental Seedings 5.4 (4.7%) 10.2 146

Espected Value of FieldExperiments, Assuming thatPrior Operational Seedingis not Permitted:

With GovernmentResponsibilty costs 8.8 (7.6%) 16.6 238

With GovernmentResponsibilityCosts Equalto Zero 12.4 (10.7%) 23.5 335

Considering only the 50% of hurricanes that are assumed to be possible candidates for seeding because of tactical consideration. If all hurricanes are assumed to be candidates for operational seeding, the figures of the last two columns should be doubled.


Findings1. Current meteorological and

economic information indicates that the seeding alternative stochastically dominated the non-seeding alternative

2. No firm legal basis for operational seeding appears to exist

3. The decision to seed a particular hurricane should take into account its specific characteristics

4. Resolving meteorological uncertainty on the effect of hurricane modification is worth over $20 millions/year

Recommendations1. The present policy prohibiting

seeding any hurricane threatening the U.S. should be rescinded

2. A hurricane modification agency with authority to seed operationally should be established

3. Decision procedures supported by further analysis should be developed

4. Modification experiments should be conducted on an expanded scale to provide a more refined basis for making each operational seeding decision

Findings and Recommendations


http://www.csmonitor.com/2003/0102/p10s02-sten.html

Tinkering with cloudsResearchers say evolving technologies could allow manipulation of

major weather patterns. But should humans tamper?By Peter N. Spotts | Staff writer of The Christian Science

Monitor, Jan 2, 2003 On Sept. 11, 1992, hurricane Iniki slammed into the Hawaiian

island of Kauai, packing winds gusting up to 175 m.p.h. The storm inflicted an estimated $2 billion in damage and 105

casualties, damaged or destroyed 10,000 homes and businesses, and left once-lush tropical mountainsides looking as though they'd been mowed by a giant weed-whacker



Tinkering with clouds (2) • Over the past two decades, the idea of modifying large-scale

storms such as hurricanes has lain dormant, following 20 years of inconclusive research. Now, however, a small group of atmospheric scientists is giving the concept a fresh look.

• Researchers seeded hurricanes in a 20-year federal research project dubbed Project Storm Fury. Scientists were testing the idea that seeding could be used to take some of the punch out of hurricanes before they made landfall. But the program foundered on inconclusive results.



Tinkering with clouds (3) … during the Vietnam War, the US military seeded monsoon clouds in

Operation Popeye in an attempt to use weather to hamper troop and supply movements along the Ho Chi Minh Trail. When information about the program was declassified in the mid-1970s, the international community established the UN Convention on the Prohibition of Military or Any Other Hostile Use of Environmental Modification Techniques.

• Federal funds for weather-modification research have dried up as well. According to Colorado State University atmospheric scientist William Cotton, federal dollars for weather modification research peaked at roughly $19 million a year in the 1970s. They dropped to less than $5 million a year during the '90s, and now hover at about $500,000.

• The field has entered what Dr. Cotton calls the "dark ages," where weather-modification programs are forging ahead with little or no scientific research programs to back them.


New Methodology for Assessing the Probability of Contaminating Mars

D. W. North, B.R. Judd, and J.P. Pezier, Life Sciences and Space Research XIII, P. A.

Sneath, ed., Akademie-Verlag, Berlin, pp. 103-109, 1975.

“Limitations, Definitions, Principles, and Methods of Risk Analysis,” Risk Assessment for

Veterinary Biologicals, special issue, Office International des Epizooties, Scientific and Technical Review, Vol. 14, pp. 913-923, 1995.

(on class website)


Assessment of the Probability of Contaminating Mars

Carried out during 1972-3 for NASA Headquarters, Office of Planetary Programs, NASA Contracts 2451, 2535Background: Committee on Space Research (COSPAR) Resolution: Total probability of contaminating Mars during the

quarantine period shall be less than 10-3.

Mariner 9 photos – evidence of past liquid water Difficulties of dry-heat sterilization for 1975 Viking Lander


Mission Contamination Model


Bio-burden Submodel

Input Elements: • Pre-sterilization burden by location• Sensitivity to sterilization• Sterilization regime• Recontamination• Inflight mortality or proliferation• Contamination and subsequent amplification in biology experiment: (probability = 10-6)

Outputs: • Bio-burden estimates by Viking Project• Expected number of Viable Terrestrial Organisms (VTOs), by location on spacecraft


Release Submodel Input Elements:• Bioburden by location (from bio-burden submodel)• Probability of hard landing• Fracture ratio for hard landing• Lethality for release, given location and landing (hard versus nominal)

Outputs: • Implantation: microbes (VTOs) directly deposited in Martian soil, without UV exposure• Erosion: VTOs relaeased through aeolian erosion of spacecraft into Martian atmosphere• Vibration: VTOs fall from spacecraft onto surface of Mars due to mechanical vibration, thermal effects, etc. VTOs require shielding to survive UV exposure.


Transport Submodel Input Elements: • Expected number of VTOs released, by

mechanism • Lethality of UV radiation, in normal

atmosphere and dust storm (probability that a VTO survives transit)

• Extent of usable water – Probability it exists anywhere– Portion of surface covered

Output• Expected number of VTOs that will reach

usable water


Reproduction SubmodelInput Elements: • Fraction of VTOs that are

facultatively anaerobic and psychrophilic: (0.05)

• Probability that nutrients needed for reproduction will be present in the water microenvironment (0.10)

Output:• Expected number of VTOs that

reproduce at least once, defined as “contamination”


Mission Contamination Model Results


Mission Contamination Model Marginal Sensitivity Analysis

Probability of Contamination

Contamination Model Variables Values Units: = 10**-6

Extreme Intermed. Intermed. Extreme Nominal: 5.9

Low Low NOMINAL High High Low High

Bio-Burden Variables

1. bio External 2.2 5.5 11 22 55 5 10.7

2. bio Covered 3.2 8 16 32 80 3.1 20.2

3. bio Encapsulated 4,000 10,000 20,000 40,000 100,000 5 10.4

Release Variables

1. rel Hard Landing Probability 0.0004 0.001 0.002 0.004 0.01 5.2 9.6

3. rel Newly Exposed/Hard, Encaps 0.0001 0.0002 0.001 0.005 0.01 5.4 10.9

4. rel Implanted, Soft 0.0001 0.0002 0.001 0.005 0.01 5.7 8.7

6. rel VTO/Vibration 0.001 0.002 0.01 0.05 0.01 5.4 11.1

9. rel VTO/Erosion, Encaps 0.00001 0.00002 0.0001 0.0005 0.001 5.4 10.9


Mission Contamination Model Marginal Sensitivity Analysis -2

Probability of Contamination

Contamination Model Variables Values Units: = 10**-6

Extreme Intermed. Intermed. Extreme Nominal: 5.9

Low Low NOMINAL High High Low High

Transport Variables

1 tra Survive Transit 0.001 0.002 0.01 0.05 0.1 2.2 45.2

2 tra Find Water 0.0005 0.001 0.005 0.025 0.05 1.5 49.9

4 tra Water Deposition 0.00005 0.0001 0.0005 0.0025 0.005 5 15.2

5 tra Stay Lodged 0.1 0.2 0.5 0.8 0.9 5.5 10

Reproduction Variables

1 rep Pyschrophic, Anaerobic 0.005 0.01 0.05 0.1 0.25 0.6 29.6

2 rep Availability of Nutrients 0.01 0.02 0.1 0.2 0.5 0.6 29.6


National Academy of Sciences, Viewpoint -1992

“… it is the unanimous opinion of the task group that terrestrial organisms have almost no chance of multiplying on he surface of Mars and in fact have little chance of surviving for long periods of time, especially if they are exposed to wind and to UV radiation.”

---- Space Studies Board, National Research Council, Biological Contamination of Mars, 1992, page 49.


Retrospective• Impact of our analysis:

– Acceptance of mission risk by scientific leaders– NASA’s decision to eliminate the mid-course correction

on the Mars Orbiter • Planetary Quarantine since Viking:

– Not a major concern, perhaps excepting Mars sample return

• Example of Quantitative Risk Analysis built on highly judgmental information

Documents

Decision Analysis and Risk Analysis Applications