View
31
Download
0
Category
Preview:
DESCRIPTION
The Expanded UW SREF System and Statistical Inference STAT 592 Presentation Eric Grimit. OUTLINE. 1. Description of the Expanded UW SREF System (How is this thing created?) 2. Spread-error Correlation Theory, Results, and Future Work 3. Forecast Verification Issues. - PowerPoint PPT Presentation
Citation preview
The Expanded UW SREF System and Statistical Inference
STAT 592 PresentationEric Grimit
1. Description of the Expanded UW SREF System(How is this thing created?)
2. Spread-error Correlation Theory, Results, and Future Work
3. Forecast Verification Issues
OUTLINE
Core Members of the Expanded UW SREF System
M = 7 + CENT-MM5
Is this enough???
MM5Multiple Analyses /
Forecasts
ICs
LBCs
Generating Additional Initial Conditions
POSSIBILITIES:•Random Perturbations•Breeding Growing Modes (BGM)•Singular Vectors (SV)•Perturbed Obs (PO) / EnKF / EnSRF•Ensembles of Initializations•Linear Combinations*
May be the optimal approach (unproven)
Simplistic approach (no one has tried it yet)
Uses Bayesian melding (under development)
Insufficient for short-range, inferior to PO, and computationally expensive (BGM & SV)}
Selected Important Linear Combinations (SILC) ?
•Founded on the idea of “mirroring” (Tony Eckel)IC* = CENT + PF * (CENT - IC) ; PF = 1.0
•Computationally inexpensive (restricts dimensionality to M=7)•May be extremely cost effective•Can test the method now•Size of the perturbations is controlled by the spread of the core members
Why Linear Combinations?
cmcg*
Illustration of “mirroring”
STEP 1: Calculate best guess for truth (the centroid) by averaging all analyses.
STEP 2: Find error vector in model phase space between one analysis and the centroid by differencing all state variables over all grid points.
STEP 3: Make a new IC by mirroring that error about the centroid.
cmcgC cmcg*
Sea
Lev
el P
ress
ure
(mb)
~1000 km
1006
1004
1002
1000
998
996
994
cent
170°W 165°W 160°W 155°W 150°W 145°W 140°W 135°W
eta
ngps
tcwbgasp
avn
ukmo
cmcg
IC* = CENT + (CENT - IC)
Two groups of “important” LCs: (x) mirrors
Xm* = Xi – Xm ; m = 1, 2, …, M
(+) inflated sub-centroidsXmn* = Xi - (Xm+Xn) ; m,n = 1, 2, …, M ; mn
2M i = 1
M
1+PFM
PF2i = 1
M
PF2 = ( )2*(M-1)(M-2)
•Must restrict selection of LCs to physically/dynamically “important” ones•At the same time, try for equally likely ICs•Sample the “cloud” as completely as possible with a finite number
(ie- fill in the holes)
Root Mean Square Error (RMSE) by Grid Point VerificationR
MS
E o
f M
SL
P (
mb
)
36kmOuter
Domain
cmcg
cmcg*
avn
avn*
eta
eta*
ngps
ngps*
ukmo
ukmo*
tcwb
tcwb*
cent
12h
24h36h48h
12kmInner
Domain
cmcg
cmcg*
avn
avn*
eta
eta*
ngps
ngps*
ukmo
ukmo*
tcwb
tcwb*
cent
Summary of Initial Findings
• Set of 15 ICs for UW SREF are not optimal, but may be good enough to represent important features of analysis error
• The centroid may be the best-bet deterministic model run, in the big picture
• Need further evaluation...• How often does the ensemble fail to capture the truth?• How reliable are the probabilities?• Does the ensemble dispersion represent forecast uncertainty?
1. Evaluate the expanded UW MM5 SREF system and investigate multimodel applications
2. Develop a mesoscale forecast skill prediction system3. Additional Work
– mesoscale verification– probability forecasts– deterministic-style solutions– additional forecast products/tools (visualization)
Future Work
Spread-error Correlation Theory
Houtekamer 1993 (H93) Model:
“This study neglects the effects of model errors. This causes an underestimation of the forecast error. This assumption probably causes a decrease in the correlation between the observed skill and the predicted spread.”
agrees with...
Var[Q | D] = Ek[Var(Q | D,Mk)] + Vark(E[Q | D,Mk])
Raftery BMA variance formula:
“avg within model variance” “avg between model variance”
[ ( )2 1 - exp(-2) 1 - exp(-2)2
Corr(S,|E|) = sqrt ; log S ~ N(0,2) , E ~ N(0,S2) ]
Observed correlations greater than those predicted by the H93 model
RESULTS: 10-m WDIRJan-Jun 2000 (Phase I)
Possible explanations:
• Artifact of the way spread and error are calculated!
• Accounting for some of the model error?
• Luck?
RESULTS: 2-m TEMPJan-Jun 2000 (Phase I)
What’s happening here?
Error saturation?
Differences in ICs not as important for surface temperature
Another Possible Predictor of SkillSpread of a temporal ensemble ~ forecast consistency
Temporal ensemble = lagged forecasts all verifying at the same time
F36 F24 F12F48
CENT-CENT-MM5MM5
CENT-CENT-MM5MM5
CENT-CENT-MM5MM5
CENT-CENT-MM5MM5
CENT-CENT-MM5MM5
CENT-CENT-MM5MM5
CENT-CENT-MM5MM5
CENT-CENT-MM5MM5
CENT-CENT-MM5MM5
CENT-CENT-MM5MM5
00 UTCT - 48 h
12 UTCT - 36 h
00 UTCT - 24 h
12 UTCT - 12 h
00 UTCT
F00* Does not have mesoscale features* “adjusted” CENT-MM5 analysisM = 4
verification
BENEFITS:•Yields mesoscale temporal spread
•Less sensitive to one synoptic-scale model’s time variability
•Best forecast estimate of “truth”
Temporal Short-range Ensemble
with the centroid runs
•Are spread and skill well correlated for other parameters?ie. – wind speed & precipitationuse sqrt or log to transform data to be normally distributed
•Do spread-error correlations improve after bias removal?
•What is “high” and “low” spread?need a spread climatology, i.e.- large data set
•What are the synoptic patterns associated with “high” and “low” spread cases?use NCEP/NCAR reanalysis data and compositing software
•How do the answers change for the expanded UW MM5 ensemble?
•Can a better single predictor of skill be formed from the two individual predictors?IC spread & temporal spread
Future Investigation:Developing a Prediction System for Forecast Skill
Mesoscale Verification Issues
Will verify 2 ways:•At the observation locations (as before)•Using a gridded mesoscale analysis
SIMPLE possibilities for the gridded dataset:
•“adjusted” centroid analysis (run MM5 for < 1 h)Verification has the same scales as the forecastsUseful for creating verification rank histograms
•Bayesian combination of “adjusted” centroid withobservations (e.g.- Fuentes and Raftery 2001)Accounts for scale differences (change of support problem)Can correct for MM5 biases
TRUEVALUES
OBSERVATIONSCENT-MM5“adjusted”
OUTPUT
Bias parameters
Noise
Measurement error
Large-scale structure Small-scale structure (after Fuentes and Raftery 2001)
Limitations of Traditional Bulk Error Scores
•biased toward the mean•can get spurious zero errors by coincidence, not skill also can be blind to position, phase, and/or rotation errors
This affects measurements of both spread & error!
Need to try new methods of verification…
1. consider the gradient of a field, not just the magnitudeaddresses false zero errors / blindness to errors in the first derivative of a fieldstill biased toward the mean
2. pattern recognition softwarewould penalize the mean for absence/smoothness of features
Recommended