[email protected] 9.8.2005 WWRP_WSN05, Toulouse, 5-9 September 2005 / 1 Pertti Nurmi Juha Kilpinen Sigbritt Näsman Annakaisa Sarkanen ( Finnish Meteorological

9.8.2005WWRP_WSN05, Toulouse, 5-9 September 2005 / [email protected]

Pertti Nurmi

Juha KilpinenSigbritt Näsman

Annakaisa Sarkanen( Finnish Meteorological Institute )

Probabilistic Forecasts andTheir Verification as

Decision-Making Tools forWarnings against Near-Gale Force Winds

WSN05: WWRP Symposium onNowcasting and Very Short Range Forecasting

Toulouse, 5-9 September 2005


• Develop warning criteria / Guidance methods to forecast probability of near-gale force winds in the Baltic Joint Scandinavian research undertaking

– e.g. Finland and Sweden issue near-gale & storm force wind warnings for same areas using different criteria Homogenize !

• 6 Finnish coastal stations c. 15-20 stations from Sweden, Denmark, Norway

• Probabilistic vs. deterministic approach

• HIRLAM ECMWF model input

• Different calibration methods, e.g. Kalman filtering

Goal: Common Scandinavian operational warning practice

Introduction:


• HIRLAM (limited area model)

RCR ~ 22 km version MBE ~ 9 km version Data coverage: 9.11.2004 – 31.3.2005 ~ 140 cases

• ECMWF Applied as reference, only

Data interpolated to 0.5o *0.5o Nearest grid point Data coverage: 1.10.2004 – 30.4.2005 ~ 210 cases

• Forecast lead time: +6 hrs (and beyond ECAM paper)

• Forecasts: wind speed at 10m

• Observations: 10 minute mean wind speed

Data:


Potential problems:

• with height of instrumentation ?

• with observing site surroundings and obstacles ?

– with the coast ?

– with nearby islands ?

– with barriers ?

– with installations ?

• with low-level stability ?

NE

“Statistical correction”

scheme available at FMI


(m)

55

50

45

40

35

30

25

20

15

10

5

Height of the instrumentation - Large filled dots: 6 Finnish stations being used- Yellow dot: Station_981; Results presented here


32 m

15,5 m/s

10 m

15

Wind speed dependence:Logarithmic wind profile

14 m/s

979 - Bogskär Unstable

Neutral

Stable

threshold


-20 -10 0 10 20ERROR

0

30

60

90

Coun

t0.0

0.1

0.2

0.3

0.4

Proportion per Bar

FITTED DISTRIBUTION

Methods for producing probabilistic forecasts 1:

Deterministic forecasts:

• Error distribution of original sample (~140 cases)

• Approximation of the error distribution with a Gaussian fit (, ):

”Dressing” method

1. ECMWF EPS (51 members) P (wind speed) > 14 m/s

2. Kalman filtering Various approaches No details given here

3. Deterministic forecast, “dressed” with “a posteriori” description of the observed error distribution of the past, dependent sample P (wind speed) > 14 m/s

“Simplistic reference” !


Methods for producing probabilistic forecasts 2:4. Deterministic forecast, adjusted with a Gaussian fit

to model forecasted stability( Temperature forecasts from 2 adjacent model levels )

P (wind speed) > 14 m/s “Stability” method~ Scheme used at SMHI (H. Hultberg)

5. “Uncertainty area” method (aka ”Neighborhood method”) (aka ”Probabilistic upscaling”)

Spatial (Fig.) and/or temporal neighboring grid points

Size of uncertainty area ? Size of time window ? c. 50-500 “members”

RCR: ± 3 points ~ 150*150 km2

MBE: ± 6 points ~ 120*120 km2


Relative Operating Characteristic

• To determine the ability of a forecasting system to discriminate between situations when a signal is present (here, occurrence of near-gale) from no-signal cases (“noise”)

• To test model performance ( H vs. F ) relative to a given probability threshold

• Applicable for probability forecasts and also for categorical deterministic forecasts Allows for their comparison

“R” statistical package used for ROC computation/presentation

Probabilistic FCs: ROC


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

mbe_nh1_UtoMBEProb6Px_12_6h_H

False Alarm Rate

Hit

Rat

e

0.10.20.3

0.40.5

0.6

0.70.8

0.9

1

ROC AREA 0.82 ( 0.908 )

ROCA = 0.82

ROCA fit = 0.91

0

20

40

60

80

100

120

.05 .15 .25 .35 .45 .55 .65 .75 .85 .96

ROC curve/area; Station_981; +6 hrs; No. of events ~25/130

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

mbe_dress1_UtoMBE06Hh_12DD_06h

False Alarm Rate

Hit

Rat

e

0.1

0.20.3

0.4

0.5

0.60.7

0.8

0.9

1

ROC AREA 0.928 ( 0.911 )

ROCA = 0.93

ROCA fit = 0.91

”Simple reference” (dep. sample):

HIR_MBE_”Dressing”

0

20

40

60

80

100

120

.05 .15 .25 .35 .45 .55 .65 .75 .85 .95

HIR_MBE_”Uncertainty area”

~ 120 * 120 km


0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

mbe_dress1_UtoMBE06Hh_12DD_06h

False Alarm Rate

Hit

Rat

e

0.1

0.20.3

0.4

0.5

0.60.7

0.8

0.9

1

ROC AREA 0.928 ( 0.911 )

ROCA = 0.93

ROCA fit = 0.91

0

20

40

60

80

100

120

.05 .15 .25 .35 .45 .55 .65 .75 .85 .95

ROC curve/area; Station_981; +6 hrs; No. of events ~25/130

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

mbe_hh_UtoMBE06Hh_12_06h

False Alarm Rate

Hit

Rat

e

0.1

0.2

0.30.4

0.5

0.6

0.7

0.80.91

ROC AREA 0.844 ( 0.82 )

ROCA = 0.84

ROCA fit = 0.82

0

20

40

60

80

100

120

.05 .15 .25 .35 .45 .55 .65 .75 .85 .95

HIR_MBE_”Stability”

”Simple reference” (dep. sample):

HIR_MBE_”Dressing”


Comparison of methods; Station_981; +6 hrs

HIR_MBE HIR_RCR

"Uncertainty area" method

Dr Stb S S + t L L + t S S + t L L + t

ROC A .93 .84 .82 .85 .87 .88 .82 .85 .87 .88

BSS .59 .36 .38 .50 .34 .45 .35 .47 .24 .37

No. events: ~ 25 /130

Dr - "Dressing" of dependent sample

Stb - "Stability" method

"Uncertainty area" method:

S - Smaller area

S + t - Smaller area with ± 3 hour forecast time window

L - Larger (double) area

L + t - Larger (double) area with ± 3 hour forecast time window


• We’ve only scratched the (sea) surface Need (much) more experimentation with various methods Different methods for different time/space scales ?Apply to data of other Scandinavian counterparts (here, only single station)

• Scores depend on station properties(e.g. observation height; Not dealt with here) (Statistical) adjustment of original observations required !

Finland has an operational scheme for this !

• “Dressing” of dependent sample: quality level hard to reach

• “Uncertainty area” size: a tricky issue

• Higher resolution HIRLAM version produces higher scores Not necessarily a trivial result !

Reach the goal, i.e. common operational practice !!!

Conclusions Future:

Documents

[email protected] 9.8.2005 WWRP_WSN05, Toulouse, 5-9 September 2005 / 1 Pertti Nurmi Juha Kilpinen Sigbritt Näsman Annakaisa Sarkanen ( Finnish Meteorological