Seth Linden and Jamie Wolff NCAR/RAL Evaluation of Selected
Winter 04/05 Performance Results Slide 2 Weather Forecast
Verification Consensus (RWFS) forecast is compared to individual
model components Air-temperature, dewpoint, wind-speed and cloud-
cover forecasts 18 UTC runs for the entire season (1 November 2004
to 15 April 2005) Error (RMSE) calculated for: Colorado Plains: 176
sites Mountains: 119 sites Blizzard of March 2003 Slide 3 Air
temperature RMSE Colorado Plains RWFS Colorado Mountains Slide 4
Dewpoint RMSE Colorado Plains Forward Error Correction Colorado
Mountains Due to 3-hour MOS data Slide 5 Wind Speed RMSE Colorado
Plains Colorado Mountains Slide 6 Colorado Plains Cloud Cover RMSE
Colorado Mountains Slide 7 The ensemble approach utilized by the
RWFS does improve the predictions on average for all verifiable
parameters No single model performs better for all parameters A
blend of weather models will provide better results
Summary/Recommendations Slide 8 Forecast Model Weights Used by the
RWFS System automatically weights forecasts based on skill
Distribution of weight values per lead time for air-temperature,
dewpoint, and wind- speed 18 UTC run on 3 May 2005 Weights looked
at for two sites: Denver International Airport I-70 at Genesse
Which models have the most skill? Slide 9 Air Temperature Model
Weights Denver Int. Airport ETA GFS MOS I-70 at Genesee RUC Slide
10 Dewpoint Model Weights Denver Int. Airport I-70 at Genesee Slide
11 Denver Int. Airport Wind Speed Model Weights MM5 I-70 at Genesee
WRF Slide 12 Insolation Weights No one model consistently
outperforms the others MM5 and WRF forecast hourly instantaneous
values, ETA forecasts 3-hour instantaneous values and GFS forecasts
3-hour averages Clear Conditions For MDSS static weights were
applied: - 50/50 split between MM5 and WRF for the 0-23 hour
forecast - All Eta for the 24-48 hour forecast Slide 13 QPF Weights
ModelGFSEta MM5 2hr MM5 3hr MM5 4hr Total MM5 WRF 2 hr WRF 3hr WRF
4hr Total WRFRUC MAV- MOSTotal % TOTAL MM5+WRF Contribution QPF
Weights (%) 911151210371714124300 100.0080 Due to a lack of quality
precipitation observations static weights were applied Weights
fixed based on expert opinion MM5 and WRF were given 80% of the
total weight Slide 14 Weight distribution reflects that the
corrected (dynamic MOS) NWS models (ETA, GFS, and RUC) had the most
overall skill No one model dominates for all parameters The
limitation of the NWS models is their 3-hr temporal resolution WRF
and MM5 were given the highest static weights for Insolation and
QPF Summary/Recommendations Slide 15 Road Temp Observation Variance
T r variance across E-470 corridor Shading by permanent structures
or passing clouds Make/model/installation/age of temperature
sensors Slide 16 E-470 Road/Bridge Sites Colorado Blvd Platte
Valley (road and bridge) 6 th Ave Pkwy Plaza A Smokey Hill Rd (road
and bridge) Slide 17 SCTBKNOVC LOCAL TIME (19 = noon, 07 =
midnight) 27 Nov 200428 Nov 2004 Slide 18 OVCCLRBKN SCT LOCAL TIME
(19 = noon, 07 = midnight) 29 Nov 200430 Nov 2004 Slide 19
Summary/Recommendations Large variations in observed road and
bridge temperatures Over relatively small area (10s of miles) Makes
prediction and verification of pavement temperatures very
challenging Difficult to establish ground truth Slide 20
Road/Bridge Forecast Verification Road and bridge temperature
forecasts Using recommended treatments from MDSS Error (MAE) and
bias calculated for: For each lead time (0-48hrs) 18 UTC runs
E-470: 6 roads/2 bridge (1 Nov 2004 15 Apr 2005) Mountains: 5 roads
(1 Feb 2004 15 Apr 2005) East bound lane of I-70 at the summit of
Vail Pass Slide 21 Consistent low bias Lead Time (0 = 18 UTC =
noon, 18 = 12 UTC = 6am) Peak insolation Morning hours E-470 road
sites Perfect forecast Slide 22 Lead Time (0 = 18 UTC ~ noon, 18 =
12 UTC ~ 6am) Shadowing? evening morning E-470 bridge sites Slide
23 Lead Time (0 = 18 UTC = noon, 18 = 12 UTC = 6am) evening morning
CDOT mountain road sites Slide 24 Summary/Recommendations Larger T
r differences during times of high solar insolation likely due to
several factors: Errors in measuring pavement skin temp Mountain
shading during low sun angle Limitations in insolation prediction
in models Limitations in pavement heat balance model Simplified
assumptions about pavement characteristics T b analysis compromised
by: Sensors shadowed by bridge rail Bias results suggest tuning may
be beneficial Overall Issue: Actual/Recommended treatments not the
same Slide 25 Case Study Analysis 183 day demonstration 16 winter
weather days 10 light snow 5 moderate snow 1 heavy snow Slide 26
November 27-29, 2004 First significant snow storm of the season 5-8
in the Denver area Large variations in parameter predictions
Forecast vs. observations Denver International Airport Ta, Td,
Wspd, Cloud Cover and Precipitation 12 UTC 28 th examined Captured
the start time of event Slide 27 LOCAL TIME (19 = noon, 06 =
midnight) 28 Nov 2005 8C/14F diff 2C/4F diff Air Temperature Snow
Slide 28 LOCAL TIME (19 = noon, 06 = midnight) 28 Nov 2005 6C/11F
diff Dewpoint Temperature Snow Slide 29 LOCAL TIME (19 = noon, 06 =
midnight) 28 Nov 2005 Snow Wind Speed Slide 30 FEC LOCAL TIME (19 =
noon, 06 = midnight) 28 Nov 2005 Cloud Cover Snow Slide 31 LOCAL
TIME (19 = noon, 06 = midnight) 28 Nov 2005 Quantitative
Precipitation Forecast Snow Slide 32 March 13, 2005 Moderate Snow
Event 4-6 along the E-470 corridor Warm air temps before start of
snow Dropped from 11C (52F) to -2C (29F) in 5 hours Large
variations in parameter predictions Forecast vs. observations
Denver International Airport Ta, Wspd, Cloud Cover and
Precipitation 00 UTC 13 March 2005 run examined Captured both start
and end times Slide 33 LOCAL TIME (18 = noon, 07 = midnight) 13
March 2005 Air Temperature Snow Slide 34 LOCAL TIME (18 = noon, 07
= midnight) 13 March 2005 Wind Speed Snow Slide 35 LOCAL TIME (18 =
noon, 07 = midnight) 13 March 2005 SCT - OVC Cloud Cover Snow Slide
36 actualforecast Start time actualforecast End time LOCAL TIME (18
= noon, 07 = midnight) 13 March 2005 Quantitative Precipitation
Forecast Slide 37 April 10, 2005 00 and 18 UTC 10 April 2005 run
Capture start and end time of the event, respectively QPF only
presented Heavy snow event 10-20 along E-470 15-25 along southern
Denver CDOT routes 20-30 along western Denver CDOT routes Air temps
near freezing (-1C) throughout the event Initial transition from
rain to snow Slide 38 10 April 2005 Act/Fore start time Nearly 3!!
LOCAL TIME (18 = noon, 06 = midnight) Quantitative Precipitation
Forecast Slide 39 10 April 2005 1 diff in total liquid equivalent
precip 0.25 diff LOCAL TIME (18 = noon, 06 = midnight) Quantitative
Precipitation Forecast Slide 40 Summary/Recommendations Large
discrepancies between weather models in predicting state weather
parameters All too dry for Td and cloud cover Low wind speed bias
during windy conditions Overall, no ONE model outperforms =>
Ensemble approach key Supports probabilistic forecast presentation
Atmosphere is unpredictable Best approach to present uncertainty to
end users? Slide 41 Thank You! Questions?
