Roberto Buizza and Martin Leutbecher European Centre for … · 2015. 3. 11. · Pr&In&(ARPA&Bologna,&Feb&2015)!"!!Roberto!Buizza:!The!forecastskill!horizon!!!!!1 Roberto Buizza and

Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 1

Roberto Buizza and Martin Leutbecher European Centre for Medium-Range Weather Forecasts

The forecast skill horizon


L2

Synopsis of my three lectures

§  L1 -‐ Ensembles: why? And how have we designed them at ECMWF?

Ø  A complete solu>on of the weather predic>on problem can be stated in terms of an appropriate probability density func>on (PDF)

Ø  Ensemble predic>on based on a finite number of integra>ons is the only feasible method to predict the PDF of forecast states

Ø  At ECMWF, ini>al uncertain>es are simulated using EDA-‐ and SV-‐based perturba>ons, and model uncertain>es with two stochas>c schemes

§  L2 – Ensemble forecast skill: how could we measure it?

Ø  Reliability is an important property that probabilis>c systems must have Ø  Different metrics should be used to evaluate the quality of probabilis>c

forecasts (BS and BSS, reliability diagrams, ROCA, V, EFI and its metric) to highlight different aspects of their quality

L1


Synopsis of my three lectures

§  L3 – Predictability: where is the Forecast Skill Horizon (FiSH)?

Ø  There is not a unique defini>on of forecast skill limit: the forecast skill horizon (the FiSH length) depends on the field (scale, variable, region)

Ø  The forecast skill horizon is well beyond 2 weeks even for local, instantaneous fields, thus confirming results published in literature that certain phenomena (MJO, NAO, blocking, ..) can be predicted beyond 2 weeks

L3

Pr&In (ARPA Bologna, Feb 2015) -‐ Roberto Buizza: The forecast skill horizon 4 © ECMWF

1)  Context 2)  Ensembles: why? 3)  Where is the forecast skill horizon? Why is it there? 4)  Conclusions

Outline


Context: where is the forecast skill horizon (FiSH)?

The view so far has been that local, daily values can be predicted only up to about 2 weeks. ‘… the range of predictability (is defined as) the >me interval within which the errors in predic>on do not exceed some pre-‐chosen magnitude …’ ‘.. the range of predictability is about 16.8 days ..’ ‘.. these results .. offer liEle hope for those who would extend the two-‐week goal to one month … ’

(Lorenz, 1969)


The forecast skill horizon: the view of the 1970s-‐80s

30

25

20

15

5

0

Fc day

FiSH No skill

Forecast skill horizon (@ ̴ 2 weeks)


An elongated area with some severe

thunderstorms producing localised flash floods

21 Feb 2013: flash flood in Catania.

Today we can predict severe events few days ahead


Day3 Day1

Day5 Day7

PR(TP>10mm/d): a week signal of increased chance for larger precip is present in the area even up to 7 days in advance

21 Feb 2013: flash flood in Catania.



Day7

Day5

Day3

Day1



0.2

0.3

0.4

0.5

0.6

Cor

rela

tion

2002 2003 2004 2005 2006 2007 2008 2009 2010 2011

YEAR

NAO Correlation

MJO in IC NO MJO in IC

The skill of monthly forecasts have been con>nuously improving both in the tropics for the MJO (top leg) and the extra-‐tropics for the NAO (top right). Improvements in the physics have led to beher teleconnec>on between tropics and extra-‐tropics (bohom).

5

6

7

8

9

10

11

12

13

14

15

Fore

cast

Day

2002 2003 2004 2005 2006 2007 2008 2009 2010 2011

YEAR

NAO Correlation

0.5 0.6 0.8

0

2

4

6

8

10

12

14

16

18

20

22

24

26

28

30

32

Fore

cast

Day

2002 2003 2004 2005 2006 2007 2008 2009 2010 2011

YEAR

MJO Bivariate Correlation

0.5 0.6 0.8

NH – NAO (11d)

TR – MJO (26d)

Large-‐scale events can be predicted weeks ahead

(From Frederic Vitart)


Italy's electricity prices jumped in June and power trade volumes improved as a heat wave boosted demand, Italian energy markets operator GME said on Friday (Reuters, 13 July).

2mT 9-‐15 July 2012

5 July +5-‐11 days

21 June +19-‐25 days

Large-‐scale events can be predicted weeks ahead


The tropics remain the area where seasonal predic>on has the highest skill, as indicated e.g. by the accuracy of 1-‐year forecasts of SST anomaly in the Nino3.4 area.

1 Nov ‘11 > Nov ‘12

Tropics: El-‐nino SST’ can be predicted months ahead

1 Nov ‘12 > Nov ‘13 1 Nov ‘13 > Nov ‘14


Some_mes EU-‐2mT’ also can be predicted months ahead

Est EU MJJ12. Since Feb S4 predicts 75% probability of below normal condi>ons.

S4 SPI(MJJ) – 1Feb+456m S4 SPI(MJJ) – 1May+123m

SPI(MJJ) – ERA-I

(From Fredrik WeEerhall)


30

25

20

15

5

0

Fc day

FiSH No skill

How can we conciliate these viewpoints?

Is the forecast skill horizon at 2 weeks as thought in the 1970s-‐80s? Or is it longer? How did we manage to push the limit beyond 2 weeks?


How can we conciliate these viewpoints?

We aim to address the following key ques>ons: 1.  If we consider local, instantaneous Z500 fcs, where is the forecast skill horizon (how

long is the FiSH)? 2.  Does it make sense to talk very generally about a forecast skill horizon or should we

qualify its defini>on? 3.  Can we develop a unifying framework that allows us to compare in a clear way the

skill of forecasts of different variables at different scales and over different regions?



Outline


fc0

fcj

reality

PDF(0)

PDF(t)

Forecast time

To predict the >me evolu>on of the probability density func>on of forecast states. In other words, to predict the most likely scenario and its uncertainty, expressed e.g. in terms of probabili>es of weather events, to es>mate the forecast confidence, ..

Ensembles: why?


ENS51 S451

HRES

EDA25

ORAS45

ERA 4DV

PDF(0) << 4DV+EDA25+ORAS45 PDF(0) << ERA+ORAS45 (the past) PDF(T) << HRES+ENS51/S451

System components simulate the effect of: •  Observa_on uncertain_es •  Model uncertain_es (2 stochas_c schemes)

2014: ensembles are used in analysis and forecast modes


The ECMWF IFS (2013) and the coupled ocean-‐atm ENS

ORTAS45 Real Time Ocean Analysis ~8 hours

HRESTL1279L91 (d0-10)

ENS51

TL639L62 (d 0-10)TL319L62 (d10-32)

Atmospheric model

Wave model

Ocean model (d0)

Atmospheric model

Wave model

EDA11

TL399L91

NB: ver>cal resolu>on was increased in 2013: since June EDA and HRES have been using L137 and ENS L91; EDA members also increased to 25.


Background error correla>on length scale for long(pmsl) and pmsl

km

Ensembles are now used to es_mate flow dependent stats

The EDA provides 4DV-‐HRES with flow dependent background error co-‐variances. EDA-‐based perturba>ons have been used to generate ENS ini>al perturba>ons since May 2010.

• Un-‐pert obs • EDA var/co-‐var

4DV00

• Sta>c Jb • Perturbed obs • Model err

EDA00 • Un-‐pert obs • EDA var/co-‐var

4DV12


ENS re-‐fc suite to es_mate the model-‐climate

51m ENS is run twice a week up to 32d. A 5m ENS is run for the past 20y to es>mate the M-‐climate (re-‐fc suite). ENS fcs have been bias-‐corrected, with bias computed using 500 ENS re-‐fcs [5w*(5m*20y)]. A reference 100m climatological ensemble (CLI) has been defined by 32d consecu>ve analyses (with the same IC as the ENS refc).

20y

51 T639 L62

51 T319 L62

2013

5 5 5 5

5 5 5 5

5 5

…28 6 13 20 27 March …

2012

5 5 5 5

5 5 5 5

5 5

5 5 5 5

5 5 5 5

5 5

2011

5 5 5 5

5 5 5 5

5 5

2010

1993

…..



Outline


The predic_ve skill limit: defini_on

The predic>ve skill limit is the >me when the forecast error crosses a certain threshold. As threshold, we have used m-‐2σ, where m is the average climatological error.

m-‐2σ

Forecast

Forecast steps (days)

error

CLI reference

F


CLI single fcs

ENS single fcs (control)

2w 17d

<Z500>180km over NH: local instantaneous skill

Results indicate that for local, instantaneous single fc of Z500 over NH is beyond 2-‐weeks. FiSH is @ ̴22 days!


Let’s think ensemble and generalise the problem

Ø  ENS fcs: bias-‐corrected forecasts are from the ECMWF 51-‐member ENS, the medium-‐range/monthly forecasts (32km up do d10, 64km agerwards)

Ø  Verifica_on: ERA-‐I analyses Ø  CLI fcs: 100-‐member climatological ensemble defined by ERA-‐I 32-‐d subsequent

analyses Ø  Accuracy metric: Con>nuous Ranked Probability Score (CRPS) Ø  Skill: CRPS(ENS) vs CRPS(CLI) Ø  Cases: 141 (2 per week, for 16m from 2/7/12 to 4/11/13)

ENS +5d

CLI CLI CLI

ENS +10d ENS + … d?

obs obs obs


<Z500>180km over NH: local instantaneous skill

ENS probabilis_c fcs

2w 17d

The same conclusion can be reached if we think in probabilis>c terms. FiSH is @ ̴22 days

CLI ensemble


Forecast skill depend on the spa_al-‐temporal scale

Large-‐scale, >me-‐average features can be predicted longer ahead than instantaneous, grid-‐point values. Certain (large-‐scale, low-‐frequency) phenomena can be predicted weeks and months ahead.

0.30.4 0.4

0.4

Local, instantaneous wind-‐speed

Weekly-‐mean, regional

temperature anomaly

Monthly-‐mean, con_nental-‐scale rain anomaly

10 100 1000 10000 km 0.1 1 10 100 days


MJO and NAO

•  Few days: the >me limit up to which local, instantaneous variables can be predicted

•  Few weeks: the >me limit up to which large-‐scales (NAO, MJO, ..) can be predicted

•  Few months: the >me limit up to which coupled, very large-‐scales (Nino) can be predicted

MJO over tropics

2 weeks

3-‐4 weeks

6 months



All forecasts represent average values over a space-‐>me volume: even a instantaneous, local values represents an implicit average. Large-‐scale, t-‐average features are more predictable than instantaneous, local values. Unpredictable “noise” can be removed by averaging to isolate the predictable signal. We have applied the same metric to differently averaged (in 4D) forecasts and asked: a)  Does FiSH depend on the spa:al-‐temporal average (and on the variable)? b)  Does it make sense to talk very generally about a forecast skill limit?



Consider increasingly coarser fields, defined by temporally averaged and spectrally truncated fields: •  Spa>ally: spectrally

truncated from T120 (180km) to T60 (360km), T15, T7, T3

•  Temporally: from instantaneous (H0) to 1, 2, 4 and 8 day averages (H24-‐H192)

H0 -‐ T120 (180km)

H0 -‐ T30 (720km)

H0 -‐ T15 (1500km)

H0 -‐ T7 (3000km)



Consider increasingly coarser fields, defined by temporally averaged and spectrally truncated fields: •  Spa>ally: spectrally

truncated from T120 (180km) to T60 (360km), T15, T7, T3

•  Temporally: from instantaneous (H0) to 1, 2, 4 and 8 day averages (H24-‐H192)

H0 – T120 (180km)

2d – T120 (180km)

4d – T120 (180km)

8d – T120 (180km)


<Z500>180km over NH: instantaneous, <..>48h and <..>96h

CLI ENS

ENS fc

Instantaneous

4-‐day average

8-‐day average


<Z500>180km over NH: instantaneous, <..>48h and <..>96h

CLI ENS

ENS fc

Instantaneous 1-‐day average 2-‐day … 4-‐day … 8-‐day … 16-‐day …


<Z500>180km over SH: instantaneous, <..>48h and <..>96h

CLI ENS

ENS fc

Instantaneous

4-‐day average

8-‐day average


<T850>180km NH, SH & TR: instantaneous, <..>48h and <..>96h

FiSH depends on the variable, the 4D-‐scale (i.e. 4D volume where average is taken) and the area where accuracy is computed.


The forecast skill horizon: results based on ECMWF ENS

30

25

20

15

5

0

F (T850T120,H0)

F(T850T120,H24)

F(T850T120,H96)

FiSH depends on the variable, the 4D-‐scale and the area

Fc day

FiSH


The forecast skill horizon: results based on ECMWF ENS

30

25

20

15

5

0

FiSH

Fc day

FiSH is shown here for: §  Variables: Z500, T850 and T200 §  Time-‐averages: 0, 2-‐days and 8

days §  Trunca>on: T120 §  Areas: NH, SH and TR

Values are well beyond 2 weeks even for instantaneous, local forecasts.

8d (H48) 2d (H24)

Instantaneous (H0)

FiSH depends on the variable, the 4D-‐scale and the area


Suppose that we have a good system that can simulate all scales relevant to predict phenomena with a scale (X,T), and ini>alise them properly. The skill of the phenomena depends on the compe>>on between: •  Errors propaga>ng from the smaller scales, i.e. noise destroying the signal, and •  Predic>ve signal propaga>ng from the wider, longer-‐range scales

Phenomena slave

External forcing

free

wider longer-‐_me

(XS,TS) (X,T) (XL,TL)

(from Hoskins 2012, QJRMS)

Why is FiSH so long? How can we interpret these results?


How can we interpret these results?

Errors propagate from the small to the large scales thus reducing the predic_ve skill

Predictable signals propagate from the large scales to the smallest scales

For example, looking at the atmosphere only:


-‐  The MJO can affect extra-‐tropical, low-‐frequency phenomena such as blocking -‐  Diurnal tropical convec>on influences organized convec>on and the MJO -‐  The MJO propagates interac>ng with El Nino -‐  El Nino and the MJO are affected by varia>ons in solar radia>on and greenhouse gases -‐  Blocking influences and is influenced by synop>c scales, fronts -‐  …..

Blocking

fronts

Solar radia_on Greenhouse gases

organiz convec

MJO, El Nino

convec

(XS,TS) (X,T) (XL,TL)

Free smaller scales

An example: blocking over the Euro-‐Atlan_c sector


CY31R1 (system-‐3) CY36R1 with system-‐3 convec_on CY36R1 (close to System-‐4) CY36R1 but slightly modified entrainment CY36R1 but slightly modified CAPE adjustment _me scale TRMM observa_ons

PDF of daily (24-‐hour accumulated) precipita_on

Progression in tropical precipita>on modelling as a result of convec>on improvements (entrainment / detrainment and closure formula>ons)

Changes in convec_on improves tropical precipita_on

obs

(From Peter Bechtold)


Progression in MJO modelling as a result of convec>on improvements (entrainment / detrainment and closure formula>ons).

Cy 31r1 (as in system-‐3) CY36R1 (close to system-‐4)

Beser physics > more realis_c MJO propaga_on



(From B. Wang and J.-‐Y. Lee)

ECMWF ECMWF

We have seen that the ECMWF system can predict ENSO SST up to 1 year ahead. Considering the MJO, the ECMWF system is capable to predict it up to about 25 days.

More realis_c MJO > skilful MJO fcs up to 3 weeks


The skill of d19-‐25 PR(2mT>Upp3) forecasts is higher if there is an ac>ve MJO in the Ics. (results based on 45 cases, 1989-‐2008).

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1forecast probability

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

obs f

requ

ency

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1forecast probability

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

obs f

requ

ency

MJO in ICs BSS(EU)=0.03 NO MJO in ICs BSS(EU)=-0.09

MJO in ICs BSS(NH)=0.04 NO MJO in ICs BSS(NH)=-0.06

Beser MJO predic_on > higher skill over Europe




Outline


Where is the forecast skill horizon?

Lorenz (1969): ‘… one flap of a sea gull’s wing would forever change the future course of the weather .. Such a change would be realized within about 17 days ..’

We showed that there is not a unique defini>on of forecast skill limit, and that the Forecast Skill Horizon, say the FiSH length, depends on the field (scale, variable, region). The forecast skill horizon is well beyond 2 weeks even for local, instantaneous fields, thus confirming results published in literature that certain phenomena (MJO, NAO, blocking, ..) can be predicted beyond 2 weeks using a unifying, coherent framework.


•  Reduced ini>al errors •  More complete models (coupling to land and ocean) •  BeEer models (improved moist processes, ..) •  New methods (ensembles, ..) •  Understanding of sources of predictability •  Scale analysis

2 weeks 3 weeks 4 weeks 5 weeks

1970s <(t)>180km

<.;.>180km,48h

<.;.>180km,96h

<.;.>180km,192h

Z500 over NH

FiSH length

Where is the forecast skill horizon?


In other words .. 1970s: results based on atmosphere-‐only models suggested that a sea-‐gull wing could affect the weather anywhere ager ~ 2 weeks 2010s: results based on more accurate, higher resolu>on coupled ocean-‐atmosphere models indicate that the limit is well beyond 2 weeks and that the forecast skill limit has not yet been reached

forget the sea-‐gulls .. think FiSH!!

Documents

Roberto Buizza and Martin Leutbecher European Centre for … · 2015. 3. 11. · Pr&In&(ARPA&Bologna,&Feb&2015)!"!!Roberto!Buizza:!The!forecastskill!horizon!!!!!1 Roberto Buizza and