Judith Berner , NCAR

Judith Berner: Representing Model Error by Stochastic Parameterizations

Representing model uncertainty in weather Representing model uncertainty in weather and climate: stochastic versa multi-physics and climate: stochastic versa multi-physics

representationsrepresentations

Judith BernerJudith Berner, NCAR, NCAR


There is model error in weather and climate models from the need to parameterize subgrid-scale fluctuations

This model error leads to overconfident uncertainty estimates and possibly model bias

We need a model error representation Hierarchy of simulations where statistical output from one

level is used to inform the next (e.g., stochastic kinetic energy backscatter)

Reliability of ensemble systems with stochastic parameterizations start to become comparable to that of ensembles systems with multi-physics

Key Points

“Domino Parameterization strategy”

Higher-resolution model inform output of lower-resolution model Stochastic kinetic energy backscatter scheme provides such a

framework … But there are others, e.g. Cloud-resolving convective

parameterization or super-parameterization


10 m 100 m 1 km 10 km 100 km 1000 km 10000

km

Turbulence Cumulus

clouds

Cumulonimbus

clouds

Mesoscale

Convective

systems

Extratropical

Cyclones

Planetary

waves

Large Eddy Simulation (LES) Model

Cloud System Resolving Model (CSRM)

Numerical Weather Prediction (NWP) Model

Global Climate Model

Multiple scales of motion

1mm

Micro-

physics

The spectral gap …

(Stull)


Atmospheric

Scientists

Nastrom and Gage, 1985

459APRIL 2008AMERICAN METEOROLOGICAL SOCIETY |

TOWARD SEAMLESS PREDICTIONCalibration of ClimateChangeProjectionsUsingSeasonalForecasts

BY T.N. PALMER, F. J. DOBLAS-REY ES,A .W EISHEIMER, AND M. J.RODWELL

In aseamless prediction system,the reliabilityof coupled climatemodel forecastsmade onseasonal time scales canprovideusefulquantitativeconstraints for

improvingthetrustworthiness of regional climatechange projections.

Inweatherandclimatepredictioncircles, thephrase_seamlesspredic-tion_ isverymuchinvogue„ indeedthenotionofseamlesspredictionliesattheheart of the WorldClimateResearchProgramme_srecent

strategicframework(availableonlineathttp://wcrp.wmo.int/pdf/WC RP_strategImple_LowRes.pdf). However, what doesthisphrasemean, andwhatisitsrelevancefor thepracticeofweatherandclimateforecasting?

FIG .1.A schem a tic fig ure illustrating th at the lin k b etw een clim ate fo rcing and clim ate im pact in vo lves pro -cesses acting on d ifferent tim e scales.T he w h ole ch ain is as stro ng as its w eak est lin k.T he use ofa se am lesspredictio n system allow s probab ilistic pro jection sofclim ate change to b e constraine d by validatio nso n w e atheror seaso nalforecasttim e scale s.

.... The link between

climate forcing and

climate impact involves

processes acting on

different timescales …









NPW NPW

modelmodel

Climate modelClimate modelCloud resolving Cloud resolving

modelmodel

Resolved Resolved

microphysicsmicrophysics

Large Eddy Large Eddy

simulationsimulation

Cloud resolving Cloud resolving

modelmodel

Attempt to capture Multi-scale nature of atmospheric motion

Attempt to capture Multi-scale nature of atmospheric motion










NPW NPW

modelmodel

Climate modelClimate modelCloud resolving Cloud resolving

modelmodel

Resolved Resolved

microphysicsmicrophysics

Large Eddy Large Eddy

simulationsimulation

Related: Grabowski 1999, Shutts and

Palmer, 2007

Hierarchical Parameterization Strategy


Validity of spectral gap …


Atmospheric

Scientists

Mathematicians


Atmospheric

Scientists

M pathematicians


Spectral gap not necessary for stochastic Spectral gap not necessary for stochastic parameterizationsparameterizations


Kinetic energy spectra in 500hPaKinetic energy spectra in 500hPa

Kinetic energy spectrum is closer to that of T799 analysis !

Rotational part Rotational part


Limited vs unlimited predictabilityLimited vs unlimited predictability

Rotunno and Snyder, 2008


Stochastic parameterizations have the potential Stochastic parameterizations have the potential to reduce model errorto reduce model error

Weak noise

Multi-modal

Strong noise

Unimodal

Stochastic parameterizations can change the mean and variance of a PDF

Impacts variability of model (e.g. internal variability of the atmosphere)

Impacts systematic error (e.g. blocking, precipitation error)

Potential

PDF


OutlineOutline Parameterizations in numerical weather prediction models and

climate models A stochastic kinetic energy backscatter scheme

Impact on synoptic probabilistic weather forecasting (short/medium-range) Impact on systematic model error (seasonal to climatic time-scales)

Aime Fournier, So-young Ha, Josh Hacker, Thomas Jung, Tim Palmer, Paco Doblas-Reyes, Glenn Shutts, Chris Snyder,

Antje Weisheimer

AcknowledgementsAcknowledgements


Sensitivity to initial perturbationsSensitivity to initial perturbations

Representing initial state uncertainty by an Representing initial state uncertainty by an ensemble of statesensemble of states

2t

0t

1t

analysis

spread

RMS error

ensemble mean

Represent initial uncertainty by ensemble of states Flow-dependence:

Predictable states should have small ensemble spread Unpredictable states should have large ensemble spread

Ensemble spread should grow like RMS error True atmospheric state should be indistinguishable from ensemble

system


Buizza et al., 2004

Systems

Underdispersion of the ensemble systemUnderdispersion of the ensemble system

-------------- spread around ensemble meanspread around ensemble mean

RMS error of ensemble meanRMS error of ensemble mean

The RMS error grows faster than the spread

Ensemble is underdispersive

Ensemble forecast is overconfident

The RMS error grows faster than the spread

Ensemble is underdispersive

Ensemble forecast is overconfident

Underdispersion is a form of model error

Forecast error = initial error + model error + boundary error

Underdispersion is a form of model error

Forecast error = initial error + model error + boundary error


Manifestations of model errorManifestations of model error

In medium-range: Underdispersion of ensemble system (Overconfidence)

Can “extreme” weather events be captured?

On seasonal to climatic scales: Systematic Biases Not enough internal variability

To which degree do e.g. climate sensitivity depend on a correct estimate of internal variability?

Shortcomings in representation of physical processes:Underestimation of the frequency of blockingTropical variability, e.g. MJO, wave propagation


Representing model error in ensemble Representing model error in ensemble systemssystems

The multi-parameterization approach: each ensemble member uses a different set of parameterizations (e.g. for cumulus convection, planetary boundary layer, microphysics, short-wave/long-wave radiation, land use, land surface)

The multi-parameter approach: each ensemble member uses the control pysics, but the parameters are varied from one ensemble member to the next

Stochastic parameterizations: each ensemble member is perturbed by a stochastic forcing term that represents the statistical fluctuations in the subgrid-scale fluxes (stochastic diabatic tendencies) as well as altogether unrepresented interactions between the resolved an unresolved scale (stochastic kinetic energy backscatter)


Recent attempts at remedying model error in Recent attempts at remedying model error in NWPNWP

Using conventional parameterizations

Stochastic parameterizations (Buizza et al, 1999, Lin and Neelin, 2000)

Multi-parameterization approaches (Houtekamer, 1996, Berner et al. 2010)

Multi-parameter approaches (e.g. Murphy et al,, 2004; Stainforth et al, 2004)

Multi-models (e.g. DEMETER, ENSEMBLES, TIGGE, Krishnamurti et. al 1999)

Outside conventional parameterizations

Cloud-resolving convective parameterization (CRCP) or super-parameterization (Grabowski and Smolarkiewicz 1999, Khairoutdinov and Randall 2001)

Nonlocal parameterizations, e.g., cellular automata pattern generator (Palmer, 1997, 2001)

Stochastic kinetic energy backscatter in NWP (Shutts 2005, Berner et al. 2008,2009,…)


Stochastic kinetic energy backscatter Stochastic kinetic energy backscatter schemes schemes

Stochastic kinetic energy backscatter LES Mason and Thompon, 1992, Weinbrecht and Mason, 2008 Stochastic kinetic energy backscatter in simplified models Frederiksen and Keupert 2004 Stochastic kinetic energy backscatter in NWP

IFS ensemble system, ECMWF: Shutts and Palmer 2003, Shutts 2005, Berner et al. 2009a,b, SteinheimerMOGREPS, MetOffice Bowler et al 2008, 2009; Tennant et al 2010Canadian Ensemble System Li et al 2008, Charron et al. 2010AFWA mesoscale ensemble system, NCAR Berner et al. 2010


Forcing streamfunction spectra by coarse-Forcing streamfunction spectra by coarse-graining CRMsgraining CRMs

from Glenn Shutts


“Domino Parameterization strategy”

Higher-resolution model inform output of lower-resolution model Stochastic kinetic energy backscatter scheme provides such a

framework … But there are others, e.g. Cloud-resolving convective

parameterization or super-parameterization


Forecast error growthForecast error growth

For perfect ensemble system:

the true atmospheric state should be indistinguishable from a perturbed ensemble member

forecast error and model uncertainty (=spread) should be the same

Since IPs are reduced, forecast error is reduced for small forecast times

More kinetic energy in small scales


Model error in weather forecasting and climate models A stochastic kinetic energy backscatter scheme: SPectral

Backscatter Scheme Impact of SPBS on probabilistic weather forecasting (medium-

range) Impact of SPBS on systematic model error Impact in a mesoscale model and comparison to a multi-

physics scheme


Experimental Setup for Seasonal RunsExperimental Setup for Seasonal Runs

“Seasonal runs: Atmosphere only” Atmosphere only, observed SSTs 40 start dates between 1962 – 2001 (Nov 1) 5-month integrations One set of integrations with stochastic

backscatter, one without Model runs are compared to ERA40 reanalysis

(“truth”)

No StochasticBackscatterNo StochasticBackscatter Stochastic BackscatterStochastic Backscatter

Reduction of systematic error of z500 over Reduction of systematic error of z500 over North Pacific and North AtlanticNorth Pacific and North Atlantic


Increase in occurrence of Atlantic and Increase in occurrence of Atlantic and Pacific blockingPacific blocking

ERA40 + confidence ERA40 + confidence intervalinterval

No StochasticBackscatterNo StochasticBackscatter

Stochastic BackscatterStochastic Backscatter

Wavenumber-Frequency SpectrumWavenumber-Frequency Spectrum Symmetric part, background removed Symmetric part, background removed

(after Wheeler and Kiladis, 1999)(after Wheeler and Kiladis, 1999)

No Stochastic BackscatterNo Stochastic BackscatterObservations (NOAA)Observations (NOAA)

Improvement in Wavenumber-Frequency Improvement in Wavenumber-Frequency SpectrumSpectrum

Stochastic BackscatterStochastic BackscatterObservations (NOAA)Observations (NOAA)

Backscatter scheme reduces erroneous westward propagating modes


Model error in weather forecasting and climate models A stochastic kinetic energy backscatter scheme: SPectral

Backscatter Scheme Impact of SPBS on probabilistic weather forecasting (medium-

range) Impact of SPBS on systematic model error Impact in a mesoscale model and comparison to a multi-

physics scheme

Experiment setupExperiment setup

Ensemble A/B: 10 member ensemble with and without SPBS Ensemble C: 10 member multi-physics suite Weather Research and Forecast Model 30 cases between Nov 2008 and Feb 2009 40km horizontal resolution and 40 vertical levels Limited area model: Continuous United States (CONUS) Started from GFS initial condition (downscaled from NCEPs

Global Forecast System)


Multiple Physics packagesMultiple Physics packages


WRF short-range ensemble: 60h-forecast for WRF short-range ensemble: 60h-forecast for Oct 13, 2006: SLP and surface windOct 13, 2006: SLP and surface wind

Control Physics

Ensemble


WRF short-range ensemble: 60h-forecast for WRF short-range ensemble: 60h-forecast for Oct 13, 2006: SLP and surface windOct 13, 2006: SLP and surface wind

Stochastic Backscatter Ensemble


Spread-Error Relationship Spread-Error Relationship

BackscatterBackscatter

ControlControl

Multi-PhysicsMulti-Physics


Brier Score, U Brier Score, U


ControlControl



Scatterplots of Scatterplots of verification verification

scoresscores

Both, Stochastic backscatter and Multi-physics

are better than control

Stochastic backscatter is better than Multi-

physics is better

Their combination is even better


Multiple Physics packagesMultiple Physics packages


Brier ScoreBrier Score


ControlControl



Spread-Error Relationship Spread-Error Relationship


ControlControl



Seasonal PredicationSeasonal Predication

Multi-model

Stochastic Ensemble

Curtosy: TimPalmer

Uncalibrated Calibrated

Summary and conclusionSummary and conclusion Stochastic parameterization have the potential to reduce

model error by changing the mean state and internal variability.

It was shown that the new stochastic kinetic energy backscatter scheme (SPBS) produced a more skilful ensemble and reduced certain aspects of systematic model error Increases predictability across the scales (from

mesoscale over synoptic scale to climatic scales)Stochastic Backscatter outperforms Multi-physics Ens.

Stochastic backscatter scheme provides a framework for hierarchical parameterization strategy, where stochastic parameterization for the lower resolution model is informed by higher resolution model

Future WorkFuture Work

Understand the nature of model error better Inform more parameters from coarse-

grained high-resolution output Impact on climate sensitivity Consequences for error growth and

predictability


ChallengesChallenges

How can we incorporate the “structural uncertainty” estimated by multi-models into stochastic parameterizations?


BibliographyBibliography

Berner, J., 2005: Linking Nonlinearity and non-Gaussianity by the Fokker-Planck equation and the associated nonlinear stochastic model, J. Atmos. Sci., 62, pp. 2098-2117

Shutts, G. J., 2005: A kinetic energy backscatter algorithm for use in ensemble prediction systems. Quart. J. Roy. Meteor. Soc., 612, 3079-3102

Berner, J., F. J. Doblas-Reyes, T. N. Palmer, G. Shutts, and A. Weisheimer, 2008: Impact of a quasi-stochastic cellular automaton backscatter scheme on the systematic error and seasonal predicition skill of a global climate model, Phil. Trans. R. Soc A, 366, pp. 2561-2579, DOI: 10.1098/rsta.2008.0031.

Berner J., G. Shutts, M. Leutbecher, and T.N. Palmer, 2009: A Spectral Stochastic Kinetic Energy Backscatter Scheme and its Impact on Flow-dependent Pre- dictability in the ECMWF Ensemble Prediction System, J. Atmos. Sci.,66,pp.603-626

T.N. Palmer, F.J. Doblas-Reyes, A. Weisheimer, G.J. Shutts, J. Berner, J.M. Murphy, 2008: Towards the Probabilistic Earth-System Model, J.Clim., in preparation

Documents

Judith Berner , NCAR