Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Verifying the Accuracy ofSeasonal Climate Outlooks
Harvey Stern & Jonathan Pollock, Victorian Regional Climate Services Centre, Bureau of Meteorology, Melbourne
Background
Seasonal rainfall outlooks have been issued by the Bureau of Meteorology since the late 1980s, whilst seasonal minimum and maximum temperature outlooks have been issued from early 2000.
The purpose of this paper is to present preliminary results from an analysis of how accurate these outlooks have been.
For almost the entire period, the outlooks have been represented by a map of Australia with the probability of the parameter predicted (total rainfall, mean minimum and mean maximum temperature) exceeding the median.
2
Methodology
For each state and season, the forecast rainfall, maximum temperature, and minimum temperature, is regarded as equivalent to +1 should there be a region with >60% probability of the element exceeding the median and no region with <40% probability of the element exceeding the median.
For each state and season, the forecast element is regarded as equivalent to -1 should there be a region with <40% probability of the element exceeding the median and no region with >60% of the element exceeding the median.
For each state and season, all other forecasts are regarded as equivalent to 0.
For each state and season, the observed element is regarded as being equivalent to the element’s anomaly in mm (in the case of rainfall) or deg C (in the case of temperature).
3
Summary Graphics
The summary graphics shown on the next few slides depict the percent of the observed variance explained by the forecasts since January 2000 (FMA 2000).
The data are depicted on a ‘rolling’ season by season basis, with each season represented by the performance achieved by all forecasts including any month of that season, and also on a state by state basis.
To illustrate, the spring (SON) graphic refers to data associated with all five three-month seasons including a spring month, i.e. JAS, ASO,SON, OND, NDJ.
4
Rainfall
5
Seasonal Climate Outlooks 2000-2011 (% Variance Explained)
-10%
-5%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
1.JF
M
2.FM
A
3.M
AM
4.A
MJ
5.M
JJ
6.JJ
A
7.JA
S
8.A
SO
9.S
ON
10.O
ND
11.N
DJ
12.D
JF
ALL
Rain-NT Rain-QLD Rain-NSW Rain-VIC Rain-TAS Rain-SA Rain-WA
Rainfall
The graphic shows that rainfall outlooks have displayed accuracy predominantly during the spring half of the year.
The State with the strongest performance is the Northern Territory, with more than 25% of the variance explained in ASO, SON and OND.
This is closely followed by Queensland and New South Wales, with a much weaker but still positive performance evident in the case of rainfall outlooks for Western Australia and Victoria.
Little skill is evident in rainfall outlooks for South Australia and Tasmania.
6
Minimum Temperature
7
Seasonal Climate Outlook 2000-2011 (% Variance Explained)
-10%
-5%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
1.JF
M
2.FM
A
3.M
AM
4.AM
J
5.M
JJ
6.JJ
A
7.JA
S
8.AS
O
9.SO
N
10.O
ND
11.N
DJ
12.D
JF
ALL
Min-NT Min-QLD Min-NSW Min-VIC Min-TAS Min-SA Min-WA
Minimum Temperature
The graphic shows that minimum temperature outlooks have displayed accuracy predominantly during the autumn half of the year.
As is the case with the rainfall outlooks, the State with the strongest performance is the Northern Territory, with more than 25% of the variance explained in FMA, MAM and AMJ.
This is closely followed by Western Australia.
A much weaker but still positive performance is evident in the case of minimum temperature outlooks for New South Wales, Queensland and Western Australia.
Little skill is evident in minimum temperature outlooks for Victoria and Tasmania.
8
Maximum Temperature
9
Seasonal Climate Outlook 2000-2011 (% Variance Explained)
-10%
-5%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
1.JF
M
2.FM
A
3.M
AM
4.AM
J
5.M
JJ
6.JJ
A
7.JA
S
8.A
SO
9.S
ON
10.O
ND
11.N
DJ
12.D
JF
ALL
Max-NT Max-QLD Max-NSW Max-VIC Max-TAS Max-SA Max-WA
Maximum Temperature
The graphic shows that maximum temperature outlooks have displayed some skill at most times of the year, the exception being late winter/early spring .
The States with the strongest performances are Queensland, Northern Territory, Western Australia and New South Wales.
Little skill is displayed by the maximum temperature outlooks for the other states –Victoria, Tasmania and South Australia.
10
Annual & Monthly
The thirteen graphics following depict the percent of the observed variance explained by the forecasts since January 2000 (FMA 2000), annual plus on a ‘rolling’ season by season basis, and also on a state by state basis.
Each season is represented by the performance achieved by all forecasts including any month of that season.
To illustrate, the spring (SON) graphic refers to data associated with all five three-month seasons including a spring month, i.e. JAS, ASO,SON, OND, NDJ.
11
Annual
12
Seasonal Climate Outlooks 2000-2011 (% variance explained)
-10%
-5%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
NT QLD NSW VIC TAS SA WA ALL
State
% v
aria
nce
ALL.MAX-T ALL.MIN-T ALL.RAIN
Annual
The graphic shows that the outlooks have displayed some overall skill across the year in the case of predictions for most states.
The best rainfall outlooks are for Northern Territory, Queensland and New South Wales, the best minimum temperature outlooks are for Northern Territory and Western Australia whilst the best maximum temperature outlooks are for Queensland, Northern Territory, Western Australia and New South Wales.
13
JFM/FMA
14
Seasonal Climate Outlooks 2000-2011 (% variance explained)
-10%
-5%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
NT QLD NSW VIC TAS SA WA ALL
State
% v
aria
nce
01.MAX-T-JFM 01.MIN-T-JFM 01.RAIN-JFM
Seasonal Climate Outlooks 2000-2011 (% variance explained)
-10%
-5%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
NT QLD NSW VIC TAS SA WA ALL
State
% v
aria
nce
02.MAX-T-FMA 02.MIN-T-FMA 02.RAIN-FMA
The graphic shows that during the early part of the year, whilst the rainfall outlook have limited skill, notable skill is observed in the case of some of the temperature outlooks, especially the minimum temperature outlooks for the Northern Territory, and the maximum temperature outlooks for the Northern Territory, Queensland, New South Wales and Western Australia.
MAM/AMJ
15
As one moves through the autumn months, the skill of the rainfall outlooks all but disappears. However, the skill at predicting temperature increases, especially for the Northern Territory and Western Australia.
Seasonal Climate Outlooks 2000-2011 (% variance explained)
-10%
-5%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
NT QLD NSW VIC TAS SA WA ALL
State
% v
aria
nce
03.MAX-T-MAM 03.MIN-T-MAM 03.RAIN-MAM
Seasonal Climate Outlooks 2000-2011 (% variance explained)
-10%
-5%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
NT QLD NSW VIC TAS SA WA ALL
State
% v
aria
nce
04.MAX-T-AMJ 04.MIN-T-AMJ 04.RAIN-AMJ
MJJ/JJA
16
Through autumn and into the early part of winter, the skill of the rainfall outlooks continues to be negligible with the exception of the predictions for New South Wales. However, the skill at predicting temperature continues to be good, especially for the prediction of minimum and maximum temperature in the Northern Territory, minimum temperature in Western Australia and maximum temperature in Queensland.
Seasonal Climate Outlooks 2000-2011 (% variance explained)
-10%
-5%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
NT QLD NSW VIC TAS SA WA ALL
State
% v
aria
nce
05.MAX-T-MJJ 05.MIN-T-MJJ 05.RAIN-MJJ
Seasonal Climate Outlooks 2000-2011 (% variance explained)
-10%
-5%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
NT QLD NSW VIC TAS SA WA ALL
State
% v
aria
nce
06.MAX-T-JJA 06.MIN-T-JJA 06.RAIN-JJA
JAS/ASO
17
As we move into spring, the skill at forecasting seasonal rainfall dramatically increases in many states, especially for the Northern Territory, New South Wales, Queensland, Western Australia and, to a lesser extent, for Victoria.
Seasonal Climate Outlooks 2000-2011 (% variance explained)
-10%
-5%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
NT QLD NSW VIC TAS SA WA ALL
State
% v
aria
nce
07.MAX-T-JAS 07.MIN-T-JAS 07.RAIN-JAS
Seasonal Climate Outlooks 2000-2011 (% variance explained)
-10%
-5%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
NT QLD NSW VIC TAS SA WA ALL
State
% v
aria
nce
08.MAX-T-ASO 08.MIN-T-ASO 08.RAIN-ASO
SON/OND
18
Through spring and early summer, the skill at forecasting seasonal rainfall remains high in many states, especially for the Northern Territory, New South Wales, Queensland, Victoria and Western Australia.
Seasonal Climate Outlooks 2000-2011 (% variance explained)
-10%
-5%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
NT QLD NSW VIC TAS SA WA ALL
State
% v
aria
nce
09.MAX-T-SON 09.MIN-T-SON 09.RAIN-SON
Seasonal Climate Outlooks 2000-2011 (% variance explained)
-10%
-5%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
NT QLD NSW VIC TAS SA WA ALL
State
% v
aria
nce
10.MAX-T-OND 10.MIN-T-OND 10.RAIN-OND
NDJ/DJF
19
Into summer, and the skill at forecasting seasonal rainfall begins to decline, but maximum temperature forecasts remain skillful in Queensland , Northern Territory and Western Australia.
Seasonal Climate Outlooks 2000-2011 (% variance explained)
-10%
-5%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
NT QLD NSW VIC TAS SA WA ALL
State
% v
aria
nce
11.MAX-T-NDJ 11.MIN-T-NDJ 11.RAIN-NDJ
Seasonal Climate Outlooks 2000-2011 (% variance explained)
-10%
-5%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
NT QLD NSW VIC TAS SA WA ALL
State
% v
aria
nce
12.MAX-T-DJF 12.MIN-T-DJF 12.RAIN-DJF
Trends
20
Trends in the accuracy of the seasonal forecasts appear to be vary in concert with corresponding long-term variations in the strength of the relationship between predictors and the predictands.
The following graphics illustrate this feature for rainfall outlooks.
Medium term trends
21
Medium term (20+ years) trend in the potential predictive skill(over the preceding 5 years) of the IOD and the SOI (pink, LHS scale),
and the skill actually displayed over the by the official forecasts (blue, RHS scale)
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7M
ay-9
4
May
-95
May
-96
May
-97
May
-98
May
-99
May
-00
May
-01
May
-02
May
-03
May
-04
May
-05
May
-06
May
-07
May
-08
May
-09
May
-10
May
-11
5-year period ended
Aver
age
(NT,
QLD
, NS
W, V
IC, T
AS
, SA
, W
A) m
ultip
le c
orre
latio
n co
effic
ient
be
twee
n su
bseq
uent
rai
nfal
l and
a
com
bina
tion
of p
redi
ctor
s: IO
D, S
OI,
seas
on
-9%
-6%
-3%
0%
3%
6%
9%
12%
15%
18%
21%
Aver
age
(NT,
QLD
, NS
W, V
IC, T
AS
, SA
, W
A) %
var
ianc
e in
rai
nfal
l exp
lain
ed b
y fo
reca
sts
Medium term trends
22
The graphic shows that the skill of the rainfall outlooks during the early 1990s was close to zero, but this period also corresponded to a period when the relationship between seasonal rainfall and its predictors was weak (multiple correlation coefficients between predictors and observed rainfall also being close to zero).
The graphic shows that there was a peak in the skill displayed during the early 2000s (% variance explained ~ 10%), this period corresponding to a period when the relationship between seasonal rainfall and its predictors being strong (multiple correlation coefficients between predictors and observed rainfall ~ +0.3).
The strongest performance was registered during recent years (% variance explained being nearly 20%), when the strength of the relationship also peaked ((multiple correlation coefficients between predictors and observed rainfall ~ +0.5).
Long term trends
23
Long term (100+ years) trend in the potential predictive skill(over the preceding 5 years) of the IOD and the SOI (pink, LHS scale),
and the skill actually displayed over the by the official forecasts (blue, RHS scale)
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7De
c-04
Dec-
09
Dec-
14
Dec-
19
Dec-
24
Dec-
29
Dec-
34
Dec-
39
Dec-
44
Dec-
49
Dec-
54
Dec-
59
Dec-
64
Dec-
69
Dec-
74
Dec-
79
Dec-
84
Dec-
89
Dec-
94
Dec-
99
Dec-
04
Dec-
09
5-year period ended
Aver
age
(NT,
QLD
, NSW
, VIC
, TA
S, S
A,
WA
) mul
tiple
cor
rela
tion
coef
ficie
nt
betw
een
subs
eque
nt r
ainf
all a
nd a
co
mbi
natio
n of
pre
dict
ors:
IOD,
SO
I, se
ason
-9%
-6%
-3%
0%
3%
6%
9%
12%
15%
18%
21%
Aver
age
(NT,
QLD
, NSW
, VIC
, TA
S, S
A,
WA
) % v
aria
nce
in r
ainf
all e
xpla
ined
by
fore
cast
s
Long term trends
24
Looking back over more than 100 years, it may be seen that there have been two periods, in addition to the most recent one, when the relationship between seasonal rainfall and its predictors was strong (multiple correlation coefficients between predictors and observed rainfall being close to +0.5).
It may be suggested that it is no coincidence that the first of these periods (circa1920) precipitated the early work into seasonal forecasting by Quayle and others.
It also may be suggested that the reason interest in this work subsequently declined was the weakness in the relationship between seasonal rainfall and its predictors during the decades following.
The revitalisation of interest by McBride, Nicholls and others during the 1980s follows the peak in the skill displayed by the relationship during the 1970s.
Any Questions?
Thank You