Comparison of Several Methods for Probabilistic Forecasting of Locally-Heavy

Rainfall in the 0-3 Hour Timeframe

Z. Sokol1, D. Kitzmiller2, S. Guan2

 1Institute of Atmospheric Physics AS CR, Prague, Czech Republic 2Hydrology Laboratory, Office of Hydrologic Development, NOAA National Weather Service, Silver Spring, Maryland, USA

Introduction

• Description of the current operational model used by NWS (U.S.A.)

• Aims of the study• Alternative models tested on the selected

subregion• Comparison of the regression model results• Conclusions

Current Model

• Predictands: probabilities that rainfall will reach or exceed 2.5, 12.5, 25.4, and 50.8 mm during the succeeding 3-h period at boxes of a 40-km grid covering the conterminous United States

• Predictors: – Extrapolated radar reflectivity, lightning strike rate,

and cloud-top temperature by advecting the corresponding initial-time fields at the velocity of the forecasted 700-500 hPa mean wind vector

– Forecasts of humidity, stability indices, moisture divergence, and precipitation from the operational Eta (NAM) model

Current Model

• Forecasting tool: – Linear regression model for each threshold

amount and 8 daytimes (01-03 UTC, … , 22-00 UTC)

– Separate sets of equations for warm (April-September) and cool (October-March) seasons

– One model for all boxes in the conterminous United States

– Regression model derived from historical data (MOS)

Example of Outputs1800-2100  UTC, 4 June 2005

Radar/gauge precipitation estimates during verifying period.

Categorical rain amount forecast.

Probability of 25 mm (1 inch) rainfall.

Probability of 50 mm (2 inches) rainfall.

Aims of This Study

• Attempt to refine existing model for U.S.– Examine regression models not previously

considered– Consider effects of local and regional models,

rather than single general model

• Consider implications for development of a model for the Czech Republic

Tests

• Selected subregion: The northeastern United States (New York, Massachusetts, Vermont, New Hampshire, Rhode Island, and Maine) during the warm season (May-September).

This area has a summertime precipitation regime similar to that of the Czech Republic.

• Data: 4 years May-September, 1997-2000 Development of the model:

– 3 years – calibration data– 1 year – independent data

Tests

• Categorical forecast (yes/no) for given thresholds for boxes– Mean precipitation in 40x40 km region– Maximum 4x4 km precipitation within 40x40 km

region

• Transition from probabilistic to categorical forecast– Fixed threshold 0.5– Optimum threshold derived on the calibration data

• Verification measure: Equitable thread score (ETS)

Types of models:

• REG - Linear regression

• REG3 - Localized linear regression models (derived for single boxes)

• LREG - Logistic regression

• RAT - Rational regression

• NN - Neural network

(perceptron type, 1 hidden layer)

NN xa...xaxaay 22110

NN xaxaxaay

...

1)log(

22110

NN

NN

xbxbxb

xaxaxaay

...1

...

2211

22110

Predictand: 40x40 km, Precipitation 5mm (1%-3%)

 

Model 00-03 04-06 07-09 10-12 13-15 16-18 19-21 22-00 Mean

REG 0.20 0.25 0.21 0.20 0.17 0.25 0.27 0.28 0.229

LREG 0.23 0.23 0.23 0.23 0.19 0.23 0.29 0.30 0.239

RAT 0.20 0.22 0.23 0.22 0.18 0.25 0.28 0.30 0.235

NN 0.22 0.24 0.24 0.25 0.15 0.27 0.27 0.29 0.238

REGG3 0.20 0.21 0.18 0. 19 0.14 0.23 0.21 0.23 0.199

REGALL_5 0.22 0.26 0.24 0.24 0.20 0.28 0.28 0.29 0.252

Model 00-03 04-06 07-09 10-12 13-15 16-18 19-21 22-00 Mean

REG 0.04 0.09 0.09 0.04 0.02 0.10 0.07 0.14 0.073

LREG 0.12 0.08 0.11 0.09 0.03 0.13 0.17 0.18 0.115

RAT 0.07 0.10 0.10 0.11 0.04 0.13 0.13 0.17 0.107

NN 0.06 0.11 0.10 0.09 0.03 0.11 0.10 0.16 0.096

REGG3 0.04 0.11 0.09 0.07 0.05 0.12 0.07 0.14 0.087

REGALL_5 0.06 0.12 0.08 0.09 0.06 0.11 0.07 0.14 0.090

a) Yes/No Threshold = 0.5

b) Optimum Yes/No Threshold

Distribution of Forecast Probabilities

 

Example of forecasts by REG and LREGpredictand maximum 4x4km precipitation

 

2 4 6 8 10 12 14

2

4

6

8

10a) T=12.5 m m , R EG

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

2 4 6 8 10 12 14

2

4

6

8

10d) T=25.4 m m , LREG

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

2 4 6 8 10 12 14

2

4

6

8

10b) T=12.5 m m , LREG

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

2 4 6 8 10 12 14

2

4

6

8

10c) T=25.4 m m , R EG

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

2 4 6 8 10 12 14

2

4

6

8

10e) 2000071514 - observation

0.1

1

2.5

5

10

15

20

25

30

2 4 6 8 10 12 14

2

4

6

8

10f) 2000071511 - previous observation

0.1

1

2.5

5

10

15

20

25

30

Probability Forecasts for12.5 mm

Probability Forecasts for25.4 mm

Verifying precipitation amount(left) and antecedent amount(right)

Conclusions• The localized approach REGG3 did not improve the

forecasts for the northeastern U.S. • For the 0.5 yes/no threshold REG results are worse

than results of other methods. It is valid for higher precipitation thresholds.

• If optimum threshold (maximizing ETS) is used then resultant ETS of all the methods are similar.

• REG yields smoother probability fields than other methods; LREG yields smaller areas of nonzero probabilities but higher values within those areas.

• In general the best results were obtained by LREG and NN methods.

• Our experience shows that NN method should use only a limited number (10-30) a priori selected predictors, otherwise the results are worse.