METEOROLGICAL FACTS OF LIFE

Dealing With Uncertainties in Weather Forecasts

(Chair, DC-AMS)

* Formerly: Program Officer, Office of Naval Research (2002-2006)

Research Meteorologist, NWS/NCEP (1975- 2002)

The Butterfly Effect: sensitive dependence on initial conditions in chaos theory where small variations of the initial condition of a nonlinear dynamical system produce increasingly large variations in the long term behavior of the system.

The single flap of a butterfly’s wings change the initial conditions of the system

enough to cause large-scale phenomena (hurricanes and such) since any variation in the initial conditions is vastly magnified with each iteration. And every flap of every butterfly wing in the world continually changes those conditions

Lorenz Model Variations in predictability can be illustrated using the Lorenz (1963) model:

X aX aY

Y XZ bX Y

Z XY cZ

Simple non-linear system. Possible atmospheric analogue: Zonal Flow Blocked Flow

More Predictable -Quasi deterministic

Predictable at first

Unpredictable

Initial state

BUTTERFLY EFFECT (CHAOS!!)

HIGHCONFIDENCE

MODERATECONFIDENCE

LOWCONFIDENCE

UNPREDICTABLE

time

“He believed in the primacy of doubt (uncertainty) not as a blemish upon our ability to know, but as the essence of knowing”Gleick (1992) on Richard Feynman’s philosophy of science.

Initial state is imperfect Problems with observations and data coverage Problems with assimilating the data

Imperfect statistical and numerical forecast methods Random (and systematic) errors

Numerical model is imperfect Limited resolution

Processes represented in model must be truncated Spatially Temporally Physically

Systematic (and random) errors

LIMITS IN WEATHER FORECASTING

Small errors (uncertainties) in initial conditions AND imperfect models can amplify rapidly => Forecasts lose skill with increasing lead time

When Docs (Weather Forecasters) Are in Doubt**:(Adapted from an article in Newsweek (http://www.newsweek.com/id/78155) by the wife of a doctor concerning the uncertainties doctors face in their profession. With only minor wordsmithing, as highlighted below, we see that our concerns parallel those of other professions, in this case, medicine.)

Few people know better than I do (author of article) that doctors (weather forecasters) are imperfect beings:

Doctors (weather forecasters) too, are prone to the occasional error; they don't always have all the answers. A few don't like to admit that. They'd rather seem omniscient or at least supremely confident, in line with the old surgical (meteorological) saying, "Sometimes wrong, never in doubt." But lately they're in the minority. More of the doctors (weather forecasters) I encounter now want to talk about their doubts (uncertainties) and mistakes (busted forecasts), not paper over them. They are willing, and even eager, to be seen as human.

They're right in tune with popular culture, which has replaced the image of the all-knowing healer (forecaster) with that of the highly educated guesser.

Doctors (forecasters) …. don't always know what's wrong with their patients (users) or what to do in response. They grapple with uncertainty. Increasingly, they're sharing that fact with their patients (users), because that's what patients (users) say they want to hear. ….

Two weeks ago I had to look into the black box myself. My mother was in the hospital (on vacation) , and no one knew exactly what was making her sick (what the weather would be). ……The docs (forecasters) did everything by the best-selling books (models) : they admitted their doubts

00 hour

5.5 day

10 day

SPAGHETTI CHARTSPAGHETTI CHART

What a mess!!

10 day

WHICH ONE IS CORRECT??WHICH ONE IS CORRECT??

SCREAMING MESSAGE:

Weather Forecasts Will AALWAYSLWAYS Be Coupled With

Varying Degrees of Uncertainty

EFFECT!!!

Varying

FORECAST PROCESS IS INHERENTLY STOCHASTIC (PROBABILISTIC) IN NATURE

INCORPORATING UNCERTAINTIES INTO FORECASTS ENHANCES THEIR VALUE

• NEED TO DO SO IN READILY COMPREHENSIBLE AND RELEVANT TERMS

YOUR FORECAST HAS A 30% YOUR FORECAST HAS A 30% CHANCE OF BEING 70% CORRECTCHANCE OF BEING 70% CORRECT

FORECAST PROCESS IS INHERENTLY STOCHASTIC (PROBABILISTIC) IN NATURE

INCORPORATING UNCERTAINTIES INTO FORECASTS ENHANCES THEIR VALUE

• NEED TO DO SO IN READILY COMPREHENSIBLE AND RELEVANT TERMS

• EDUCATION OF PROVIDERS AND USERS ESSENTIAL

T

The true state of the atmosphere exists as a single point in phase space that we never know exactly.

A point in phase space completely describes an instantaneous state of the atmosphere. (pres, temp, etc. at all points at one time.)

Nonlinear error growth and model deficiencies drive apart the forecast and true trajectories (i.e., Chaos Theory)

PHA

SE

SPACE

12hforecast 36h

forecast

24hforecast

48hforecast

T

48hverification

T

T

T

12hverification

36hverification

24hverification

An analysis produced to run an NWP model is somewhere in a cloud of likely states.

Any point in the cloud is equally likelyto be the truth.

Deterministic Forecasting Limitations

Error

AC = 0

time

ForecastForecast Uncertainty

Climatology

Initial Condition Uncertainty

Deterministic Forecast

Analysis

X

Verif

LIMITS OF PREDICTABILITY:LIMITS OF PREDICTABILITY:

PREDICTAILITY LIMIT WHEN ERROR VARIANCE PREDICTAILITY LIMIT WHEN ERROR VARIANCE CLIMO VARIANCE CLIMO VARIANCE

Same Deterministic Model with Different Convection Schemes Results In Different Precipitation Forecasts

Uncertainties arise also because models are imperfect!!

Initial uncertainty

= distribution of possible initial states

Single deterministic forecast

verification

Ensemble members

Forecast uncertainty

= distribution of possible forecast states

INFO on DISTRIBUTION of SCENARIOS: - how many scenarios - how likely is each - how sharply defined is each

F1, F2 … FN = predicted variables of interest, for example, wind speed

Fcr = user-specified “critical value”

F1

F2

F3

F4..FN

Models

P(F)

Fcr

T

Q (e.g., T< 0o, Sn>4”)

T = chance of critical value being exceeded aid in making decisions, risk analysis

0%

100%

Decision-making with probabilities

Rational decision-making depends on the user’s sensitivity – illustrate with how we respond to low probabilities:

• 5% risk that a plane will crash - would you board it?

• 5% risk of rain – would you play golf?

Decisions must be based on user’s Cost/Loss ratio

• users with low C/L should protect at low probabilities

BSSSS

USER REQUIREMENTS:PROBABILISTIC FORECAST INFORMATION IS CRITICAL

GENERAL IDEA!!!:GENERAL IDEA!!!: JAN 9, 2002: “SURPRISE” ICE STORM ISSUE: No advance warning in actuality

How to deal with uncertainty - 50/50 chance of freezing rain during morning rush ~ 12hr in advance

CONSIDER:

Cost of not taking action (staging sand trucks)

and event happens (as was the case THIS time) versus Taking action and event doesn’t occur

Prob of Frzng rain: 15 hr fcst from 21Z 01/08/02, vt 12Z 01/09/02

Prob of Frzng rain: 18 hr fcst from 21Z 01/08/02, vt 15Z 01/09/02

The Associated Press – Jan 9, 2002

"Surprise Icy Conditions Claim Teenager's Life" Washington Post Wednesday, January 9, 2002

"This morning's weather evolved without warning. ...." Weather forecasters said … had little advance notice of the rain … first indications of the light rain appeared on radar about 4AM..” (just before morning rush) WJLA Viewer Opinions and Feedback

“I think its a real shame the NWS can't prepare us for this weather. We seem to have spent thousands of $$$ on this fancy weather reporting equipment so they can forecast weather and guess what??

Jan 25-26, 2000

DC “Surprise” Snowstorm

Not Good- especially when effecting DC (just after announce-ment of new Super Computer by NWSHQ

MAJOR SNOWSTORM AMBUSHES WASHINGTON

Wash Post Interview – Jan 26, 2000

Warren Washington (NCAR) said, “It (computer forecast) wasn’t as accurate as it should have been, but forecasters can’t always account for every ‘chaos aspect’, little changes in weather that can cause major shifts. The public should not expect a perfect forecast”Reporter: “ OK, OK, WHAT THE HELL should we expect when they (NWS) just paid $35 million for the (new computer) system”

“In my next life I want to come back as a weather- man. That way I can be dead wrong 80% of the time and not get fired”

“Models? Next time, read pig entrails.” Tony Kornheiser (Wash Post)

What’s wrong with the forecast models? (Earth Observatory)

“Uccellini admits that there was some uncertainty in the minds of meteorologists as to what path the storm would take. He says the NWS could have emphasized that uncertainty more in its forecasts. Forecasters know the models aren’t perfect. There should be some way to present alternate scenarios without following the mainstream (models) and creating a panic (i.e., ‘crying wolf’)." Impetus to SREF and Winter Weather Exp

Bottom Line

CAUTION

Louis ULouis U.

“Deterministic forecasting is not healthy”!

EVEN EXPERTS CAN HAVE A BAD DAY

Figure shows the difference between the storm’s observed behavior and the forecast made 24 hours before. The storm traveled much closer to shore than was predicted, and dropped a lot more snow.

SCHEMATIC: BASIC PROBLEM WITH FORECASTS

Ensemble provides a clear “heads up” on morning of 24th for the possibility of a major snow event, especially when considered in context of independent information from satellite imagery and radar that suggested storm track closer to coast and precip further inland than available operational models were indicating

Wide range of solutions (CTL vs Best) in precip and Storm Track (next) => Deterministic (yes/no forecast) very risky!

TO THE VIDEO TAPE, PLEASE !!!

BOB’S ORIGINAL BROADCAST EVENING BEFORE

JAN 25 “SURPRISE” SNOWSTORM, FOLLOWED BY HYPOTHETICAL BROADCAST HE MIGHT HAVE GIVEN INCORPORATING INFORMATION ON UNCERTAINTIES

Bob Ryan

CASES

Dec 29-30, 2000

DC surprise NON Snowstorm

(The Millennium Snowstorm elsewhere)

NWS FOR DC/BALTNWS FOR DC/BALT9PM DEC 28 (Thurs):…WINTER STORM WATCH FOR FRIDAY NIGHT THROUGH SATURDAY...

SNOW WILL BEGIN LATE FRIDAY EVENING AND INTENSIFY TOWARD DAYBREAK. SNOWFALL MAY SNOWFALL MAY BE HEAVY AT TIMES SATURDAY MORNINGBE HEAVY AT TIMES SATURDAY MORNING INTO THE AFTERNOON. BRISK NORTH WINDS WILL CAUSE BLOWING AND DRIFTING OF SNOW. SNOWFALL ACCUMULATIONS OF 4 TO 8 INCHES ARE POSSIBLE.

10PM DEC 29 (Friday):…WINTER STORM WARNING FOR LATE TONIGHT THROUGH SAT EVENING

…….5-10” EXPECTED FROM THIS STORM.5-10” EXPECTED FROM THIS STORM …

Dec 29: "It's going to be ugly,"It's going to be ugly," … Air Lines spokesman "We're going to be taking down a significant portion of our schedule throughout the Northeast." because of predictions of heavy snowfall CNN: “In Washington, more than 300 snowplows and salt trucks were ready to go into action Saturday, according to city officials.” ___________________ Dec 30 “ …Washington got no snowWashington got no snow and flights were expected to resume at 6 p.m., an airline spokeswoman said CNN: ““The storm unexpectedly spared WashingtonThe storm unexpectedly spared Washington, hit Philadelphia less hard than expected …. Forecasters said.

WHAT DC MISSED!!WHAT DC MISSED!!

WHAT DC GOT!WHAT DC GOT!

SREF 24hr spaghetti from 12Z 29 Dec for .50” 12 hr precip ending 12 GMT 30

Ensemble indicated a 30-40% chance of signif- icant snow; thus, in this case SREF gave a “heads up” (60%) for the chance of no snowstorm

KEY POINTS

January 24/25, 2000 DC Snowstorm:

Ensembles gave “heads up” for snowstorm in face of deterministic model and official forecasts of no snow

December 30, 2000 DC Non-Snowstorm:

Ensemble gave “heads up” of no snow in face of deterministic model (Eta) predicted and official forecasts of snowstorm

010519/0000V63 SREFX-CMB; 24HR PQPF OF .25”

PROBABILITY CHARTS

Percentage of members with QPF > .25”/24h

SREF

SREF Combined or Joint Probability

Pr [MUCAPE > 2000 J/kg] XPr [ESHR > 40 kts] XPr [C03I > 0.01”]

Probability of convection in high CAPE, high shear environment

(favorable for supercells)

GLOBAL DATA PROCESSING AND FORECASTING SYSTEMS

Risk Identification: Early warnings (2)

Strike probability (within 65 nm) of

Typhoon Rusa over the next 120 hours.

Starting time of the forecast is 27

August 2002 12 UTC.

Full dots give the observed position over the period 27

August to 1 September 2002

Completing the Forecast

Board on Atmospheric Sciences and Climate

Characterizing and Communicating Characterizing and Communicating Uncertainty for Better Decisions Using Uncertainty for Better Decisions Using

Weather and Climate ForecastsWeather and Climate Forecasts

• Vision: Enterprise-wide partnership that generates and communicates forecast uncertainty information meeting Nation’s needs for informed decisions:

–protecting life and property,

–supporting national defense and homeland security,

–enhancing the economy, and

–meeting specific needs of partners, users, and customers.

• Mission: Develop Enterprise-wide goals and roadmap for providing forecast uncertainty information, building off NRC recommendations

• Deliverable: An Enterprise implementation (action) plan for forecast uncertainty that has been reviewed and coordinated with partners

• Goal: Enterprise partners put into effect

Ad Hoc Committee on Uncertaintyin Forecasts (ACUF)

Committee Overview Enterprise Plan Components

• Needs, opportunities, and benefits of providing hydrometeorological forecast uncertainty products and services to the Nation

– Why is uncertainty information important?

• Enterprise Goals for forecast uncertainty products and services

– Specifics – What will Nation get and how good will it be?

• Description of what is needed to meet goals and reach vision

– Solutions: What is needed to do this?

• Suggested roles and responsibilities of enterprise partners

– Who should do what and with who?

• Enterprise-wide Roadmap

– How will all the pieces fit together?