Embed Size (px)
DESCRIPTION
Evaluating NWP Ensemble Configurations for AT&D Applications. Jared A. Lee 1,2 , Walter C. Kolczynski 1 , Tyler C. McCandless 1,2 , Kerrie J. Long 2 , Sue Ellen Haupt 1,2 , David R. Stauffer 1 , and Aijun Deng 1 - PowerPoint PPT Presentation
Citation preview
UNCLASSIFIED
Evaluating NWP Ensemble Configurations for AT&D Applications
Jared A. Lee1,2,Walter C. Kolczynski1, Tyler C. McCandless1,2, Kerrie J. Long2,Sue Ellen Haupt1,2, David R. Stauffer1, and Aijun Deng1
The Pennsylvania State University 1Department of Meteorology 2Applied Research Lab
University Park, PA
21 January 201016th Conference on Air Pollution90th AMS Annual Meeting - Atlanta, GA12.3
UNCLASSIFIED
UNCLASSIFIED
Why Use Ensembles?
• Single deterministic forecasts could be outliers in the forecast probability density function (PDF)
• NWP ensembles provide a better estimate of the atmospheric variability in a given situation by approximating the PDF of the atmospheric state
• Grimit and Mass (2002) - WAF: Correlation exists between ensemble spread and forecast uncertainty
• NWP ensemble uncertainty information can improve AT&D forecasts [e.g., Lee et al. (2009) - JAMC]Four MREF members on Penn State e-Wall
UNCLASSIFIED
UNCLASSIFIED
Sources of Uncertainty
• Many atmospheric transport & dispersion (AT&D) models are driven by numerical weather prediction (NWP) model output
• Uncertainty estimates for concentration predictions by AT&D models must account for both AT&D and NWP model uncertainty
• Many sources of uncertainty in NWP models• Initial conditions (ICs)• Lateral/lower boundary conditions (LBCs)• Model physics parameterizations• Numerics
Initial Conditions (ICs)
Lateral Boundary Conditions (LBCs)
Model Physics Parameterizations
NWP Model AT&D Model
UNCLASSIFIED
UNCLASSIFIED
Winter Evaluation Period
• WRF-ARW v3.1.1• 36-km domain, no nests• 45 vertical levels• GFS 0.5° ICs/LBCs• 18-member physics (PH)
ensemble• 48-h forecasts starting at 00
UTC daily (12 UTC in the future as well)
• 04–17 Jan 2009
• Different synoptic regimes in both weeks:• 04-10 Jan: Deep, digging trough moving across U.S.• 11-17 Jan: Persistent ridge in west & trough in east
• Important to evaluate performance of ensemble members in a range of regimes
UNCLASSIFIED
UNCLASSIFIED
Ensemble Configuration
Member Microphysics LW Radiation SW Radiation Land Surface Surface Layer Boundary Layer Cumulus
1 WSM 5-class RRTM Dudhia Thermal Diff. MM5 Similarity YSU Kain-Fritsch
2 WSM 5-class RRTM Dudhia Thermal Diff. MM5 Similarity YSU Grell-Devenyi
3 WSM 5-class RRTM Dudhia Noah MM5 Similarity YSU Kain-Fritsch
4 WSM 5-class RRTM Dudhia Noah MM5 Similarity YSU Grell-Devenyi
5 WSM 5-class RRTM Dudhia RUC MM5 Similarity YSU Kain-Fritsch
6 WSM 5-class RRTM Dudhia RUC MM5 Similarity YSU Grell-Devenyi
7 WSM 5-class RRTM Dudhia Thermal Diff. Eta Similarity MYJ Kain-Fritsch
8 WSM 5-class RRTM Dudhia Thermal Diff. Eta Similarity MYJ Grell-Devenyi
9 WSM 5-class RRTM Dudhia Noah Eta Similarity MYJ Kain-Fritsch
10 WSM 5-class RRTM Dudhia Noah Eta Similarity MYJ Grell-Devenyi
11 WSM 5-class RRTM Dudhia RUC Eta Similarity MYJ Kain-Fritsch
12 WSM 5-class RRTM Dudhia RUC Eta Similarity MYJ Grell-Devenyi
13 WSM 5-class RRTM Dudhia Thermal Diff. Pleim-Xu ACM2 Kain-Fritsch
14 WSM 5-class RRTM Dudhia Thermal Diff. Pleim-Xu ACM2 Grell-Devenyi
15 WSM 5-class RRTM Dudhia Noah Pleim-Xu ACM2 Kain-Fritsch
16 WSM 5-class RRTM Dudhia Noah Pleim-Xu ACM2 Grell-Devenyi
17 WSM 5-class RRTM Dudhia RUC Pleim-Xu ACM2 Kain-Fritsch
18 WSM 5-class RRTM Dudhia RUC Pleim-Xu ACM2 Grell-Devenyi
UNCLASSIFIED
UNCLASSIFIED
Why Down-select?
• We want to include other sources of variability in addition to physics options, including IC/LBC and/or multi-model uncertainty
• We know that IC/LBC and PH variability sample different parts of the forecast PDF• Fujita et al. (2007) – MWR:
• IC variability – spread in dynamic variables (u,v)• PH variability – spread in thermodynamic variables (θ,q)
• Potential way to do this: select a small number of physics runs as “control” members around which to perturb the ICs/LBCs using an EnKF (from DART)• e.g., 5 ctrl * 5 pert = 25 members
• We lack computational resources to run large numbers (dozens) of ensemble members• We want to create a long-term stable NWP ensemble dataset for
AT&D research
UNCLASSIFIED
UNCLASSIFIED
Ensemble Down-SelectionPrincipal Component Analysis (PCA)
PCA Weighting
-Principal Component Analysis is a mathematical procedure that transforms a number of possibly correlated variables (ensemble members, in this case) into a smaller number of uncorrelated variables called principal components
-The first principal component accounts for as much variability in the data as possible
-Uses the factors (how much each ensemble member contributed) from the first principal components
-The factors from the first principal component are the ensemble member weights
Credit: Tyler McCandless
UNCLASSIFIED
UNCLASSIFIED
• The selection of ensemble members appears directly related to the parameter being forecast• 2-m Temperature: No member using Thermal Diffusion land surface
scheme (1,2,7,8,13,14) was selected at any forecast lead time• 10-m Wind Speed: Members that were selected varied somewhat by
parameter (u-wind, v-wind) and forecast lead time (24h, 36h, 48h)
• Ensemble members 3,4,10,11,12 were always selected• Varying the cumulus scheme appears to have had almost no
effect during this time period• PCA Weighting evaluates the deterministic predictive ability of
the ensemble, while other methods examine both the deterministic and probabilistic predictive ability of the ensemble
Ensemble Down-SelectionPCA Weighting Results
Credit: Tyler McCandless
UNCLASSIFIED
UNCLASSIFIED
Post-ProcessingBayesian Model Averaging (BMA)
• Bayesian Model Averaging (BMA) main tool for calibration in this study• Assumes a normally distributed conditional probability
around each ensemble member• Estimates the optimal weights and standard deviations
using expectation-maximization
• Also use correlation to identify impact of different parameterizations and possible redundancy
Credit: Walter Kolczynski
UNCLASSIFIED
UNCLASSIFIED
Post-ProcessingBMA Demonstration
Temperature (K)
Credit: Walter Kolczynski
Legend:
BlueConditional probability for individual member
RedCumulative
probability for full ensemble
BlackObserved
temperature
UNCLASSIFIED
UNCLASSIFIED
Post-ProcessingBMA Weights
Thermal DiffusionLand Surface Model
Credit: Walter Kolczynski
UNCLASSIFIED
UNCLASSIFIED
Post-ProcessingBMA Weights
NoahLand Surface Model
Credit: Walter Kolczynski
UNCLASSIFIED
UNCLASSIFIED
Post-ProcessingBMA Weights
RUCLand Surface Model
Credit: Walter Kolczynski
UNCLASSIFIED
UNCLASSIFIED
Post-ProcessingBMA Weights
ACM2 PBL &Pleim-Xu Sfc
Credit: Walter Kolczynski
UNCLASSIFIED
UNCLASSIFIED
Post-ProcessingMember Correlations for 2-m Temp
mem01 mem02 mem03 mem04 mem05 mem06 mem07 mem08 mem09 mem10 mem11 mem12 mem13 mem14 mem15 mem16 mem17 mem18
mem01 1.000 1.000 0.989 0.989 0.984 0.984 0.997 0.997 0.987 0.987 0.983 0.983 0.997 0.997 0.986 0.986 0.981 0.981
mem02 1.000 1.000 0.989 0.989 0.984 0.984 0.997 0.997 0.987 0.987 0.983 0.983 0.997 0.997 0.986 0.986 0.981 0.981
mem03 0.989 0.989 1.000 1.000 0.996 0.995 0.984 0.984 0.994 0.994 0.991 0.991 0.984 0.984 0.994 0.994 0.991 0.991
mem04 0.989 0.989 1.000 1.000 0.995 0.996 0.984 0.984 0.994 0.994 0.991 0.991 0.984 0.984 0.994 0.994 0.991 0.991
mem05 0.984 0.984 0.996 0.995 1.000 1.000 0.980 0.981 0.992 0.992 0.995 0.995 0.980 0.980 0.992 0.992 0.996 0.996
mem06 0.984 0.984 0.995 0.996 1.000 1.000 0.980 0.981 0.991 0.992 0.995 0.995 0.979 0.980 0.992 0.992 0.996 0.996
mem07 0.997 0.997 0.984 0.984 0.980 0.980 1.000 1.000 0.989 0.989 0.985 0.985 0.998 0.998 0.986 0.986 0.981 0.981
mem08 0.997 0.997 0.984 0.984 0.981 0.981 1.000 1.000 0.989 0.989 0.985 0.985 0.998 0.998 0.986 0.986 0.981 0.981
mem09 0.987 0.987 0.994 0.994 0.992 0.991 0.989 0.989 1.000 1.000 0.996 0.996 0.988 0.988 0.998 0.998 0.994 0.994
mem10 0.987 0.987 0.994 0.994 0.992 0.992 0.989 0.989 1.000 1.000 0.996 0.996 0.987 0.988 0.998 0.998 0.994 0.994
mem11 0.983 0.983 0.991 0.991 0.995 0.995 0.985 0.985 0.996 0.996 1.000 1.000 0.982 0.983 0.993 0.993 0.998 0.998
mem12 0.983 0.983 0.991 0.991 0.995 0.995 0.985 0.985 0.996 0.996 1.000 1.000 0.982 0.983 0.993 0.993 0.998 0.998
mem13 0.997 0.997 0.984 0.984 0.980 0.979 0.998 0.998 0.988 0.987 0.982 0.982 1.000 1.000 0.987 0.987 0.981 0.981
mem14 0.997 0.997 0.984 0.984 0.980 0.980 0.998 0.998 0.988 0.988 0.983 0.983 1.000 1.000 0.987 0.987 0.981 0.981
mem15 0.986 0.986 0.994 0.994 0.992 0.992 0.986 0.986 0.998 0.998 0.993 0.993 0.987 0.987 1.000 1.000 0.994 0.994
mem16 0.986 0.986 0.994 0.994 0.992 0.992 0.986 0.986 0.998 0.998 0.993 0.993 0.987 0.987 1.000 1.000 0.994 0.994
mem17 0.981 0.981 0.991 0.991 0.996 0.996 0.981 0.981 0.994 0.994 0.998 0.998 0.981 0.981 0.994 0.994 1.000 1.000
mem18 0.981 0.981 0.991 0.991 0.996 0.996 0.981 0.981 0.994 0.994 0.998 0.998 0.981 0.981 0.994 0.994 1.000 1.000
2-m T correlation Credit: Walter Kolczynski
UNCLASSIFIED
UNCLASSIFIED
Post-ProcessingMember Correlations for 10-m U-wind
mem01 mem02 mem03 mem04 mem05 mem06 mem07 mem08 mem09 mem10 mem11 mem12 mem13 mem14 mem15 mem16 mem17 mem18
mem01 1.000 0.995 0.993 0.989 0.986 0.981 0.976 0.975 0.969 0.969 0.965 0.965 0.976 0.974 0.972 0.970 0.966 0.963
mem02 0.995 1.000 0.989 0.993 0.982 0.986 0.974 0.976 0.966 0.970 0.963 0.966 0.975 0.976 0.971 0.972 0.964 0.964
mem03 0.993 0.989 1.000 0.995 0.989 0.985 0.972 0.971 0.974 0.973 0.969 0.968 0.969 0.967 0.974 0.972 0.967 0.964
mem04 0.989 0.993 0.995 1.000 0.986 0.989 0.970 0.972 0.971 0.974 0.967 0.969 0.968 0.969 0.973 0.974 0.966 0.966
mem05 0.986 0.982 0.989 0.986 1.000 0.995 0.967 0.966 0.969 0.969 0.974 0.974 0.967 0.965 0.973 0.971 0.977 0.974
mem06 0.981 0.986 0.985 0.989 0.995 1.000 0.964 0.967 0.966 0.970 0.971 0.975 0.965 0.966 0.970 0.971 0.974 0.974
mem07 0.976 0.974 0.972 0.970 0.967 0.964 1.000 0.995 0.992 0.989 0.989 0.985 0.976 0.973 0.973 0.969 0.965 0.962
mem08 0.975 0.976 0.971 0.972 0.966 0.967 0.995 1.000 0.987 0.992 0.984 0.989 0.976 0.976 0.972 0.971 0.965 0.964
mem09 0.969 0.966 0.974 0.971 0.969 0.966 0.992 0.987 1.000 0.995 0.995 0.990 0.968 0.965 0.978 0.975 0.970 0.966
mem10 0.969 0.970 0.973 0.974 0.969 0.970 0.989 0.992 0.995 1.000 0.990 0.994 0.968 0.968 0.979 0.977 0.970 0.969
mem11 0.965 0.963 0.969 0.967 0.974 0.971 0.989 0.984 0.995 0.990 1.000 0.995 0.965 0.962 0.974 0.970 0.975 0.972
mem12 0.965 0.966 0.968 0.969 0.974 0.975 0.985 0.989 0.990 0.994 0.995 1.000 0.966 0.965 0.974 0.973 0.976 0.974
mem13 0.976 0.975 0.969 0.968 0.967 0.965 0.976 0.976 0.968 0.968 0.965 0.966 1.000 0.994 0.985 0.983 0.981 0.979
mem14 0.974 0.976 0.967 0.969 0.965 0.966 0.973 0.976 0.965 0.968 0.962 0.965 0.994 1.000 0.983 0.984 0.978 0.981
mem15 0.972 0.971 0.974 0.973 0.973 0.970 0.973 0.972 0.978 0.979 0.974 0.974 0.985 0.983 1.000 0.993 0.989 0.986
mem16 0.970 0.972 0.972 0.974 0.971 0.971 0.969 0.971 0.975 0.977 0.970 0.973 0.983 0.984 0.993 1.000 0.986 0.988
mem17 0.966 0.964 0.967 0.966 0.977 0.974 0.965 0.965 0.970 0.970 0.975 0.976 0.981 0.978 0.989 0.986 1.000 0.992
mem18 0.963 0.964 0.964 0.966 0.974 0.974 0.962 0.964 0.966 0.969 0.972 0.974 0.979 0.981 0.986 0.988 0.992 1.000
10-m U correlation Credit: Walter Kolczynski
UNCLASSIFIED
UNCLASSIFIED
06 Jan 2009 00z – YSU PBLSCIPUFF 36-h Continuous Releases
mem01
mem03
mem05
mem02
mem04
mem06
ThermalDiff.
Noah
RUC
UNCLASSIFIED
UNCLASSIFIED
06 Jan 2009 00z – MYJ PBLSCIPUFF 36-h Continuous Releases
mem07
mem09
mem11
mem08
mem10
mem12
ThermalDiff.
Noah
RUC
UNCLASSIFIED
UNCLASSIFIED
06 Jan 2009 00z – ACM2 PBLSCIPUFF 36-h Continuous Releases
mem13
mem15
mem17
mem14
mem16
mem18
ThermalDiff.
Noah
RUC
UNCLASSIFIED
UNCLASSIFIED
18 Jan 2009 00z – YSU PBLSCIPUFF 36-h Continuous Releases
mem01
mem03
mem05
mem02
mem04
mem06
ThermalDiff.
Noah
RUC
UNCLASSIFIED
UNCLASSIFIED
18 Jan 2009 00z – MYJ PBLSCIPUFF 36-h Continuous Releases
mem07
mem09
mem11
mem08
mem10
mem12
ThermalDiff.
Noah
RUC
UNCLASSIFIED
UNCLASSIFIED
18 Jan 2009 00z – ACM2 PBLSCIPUFF 36-h Continuous Releases
mem13
mem15
mem17
mem14
mem16
mem18
ThermalDiff.
Noah
RUC
UNCLASSIFIED
UNCLASSIFIED
ConclusionsWinter Evaluation Period PH Ensemble
• Changes to the cumulus parameterization have little effect on the ensemble predictions for surface variables
• All methods of investigation show that the Thermal Diffusion Land Surface Model performs far poorer than the others for 2-m temperature
• BMA weights and ensemble member correlations indicate the PBL/Surface Layer scheme as dominant for 10-m winds (of those parameters varied)
UNCLASSIFIED
UNCLASSIFIED
Future Work
• Examine additional combinations of physics parameterizations
• Investigate performance of ensemble for additional evaluation periods and initial forecast times
• Create an ensemble that also perturbs initial conditions and boundary conditions
• Adjust the sensitivity of the PCA Weight Guided Feature Selection to determine the optimal number of ensemble members
• Find BMA weights for vector winds, not just components, and compute CRPS & RMSE for verification
• Find & compare results using a random 10 of 14 days for PCA & BMA methods
UNCLASSIFIED
UNCLASSIFIED
Thanks for listening!
UNCLASSIFIED
UNCLASSIFIED
Supplementary
Slides
UNCLASSIFIED
UNCLASSIFIED
Year-Long Ensemble
• ~20 WRF members, run for 1 year• 2 cycles daily, 48 h starting at 00 & 12 UTC • Use ensemble data to drive SCIPUFF case studies
• Very few regional scale field experiments• Create “truth” using SCIPUFF driven by a higher-resolution
NWP dynamic analysis, as in Kolczynski et al. (2009)
• Compute meteorological statistics for several ABL parameters (e.g., wind direction, temperature)
• Compare performance of this ensemble to an existing ensemble (e.g., NCEP SREF)
27
UNCLASSIFIED
UNCLASSIFIED
Why Is This Important?
• Most current short-range ensembles built for spread in QPF, like NCEP SREF, and not AT&D
• No current agreement on best way to configure an NWP ensemble for AT&D forecasting
• This will provide a long-term, consistent, short-range ensemble dataset for research purposes, particularly for the connection between meteorological and dispersion uncertainty
• Potential to be the basis for an operational NWP ensemble configuration used by DTRA for emergency response after hazardous chem/bio releases
28
UNCLASSIFIED
UNCLASSIFIED
WRF-ARW Physics SchemesMicrophysics & Cumulus schemes
• WSM 5-class microphysics scheme• Vapor, rain, snow, cloud ice, cloud water
• Kain-Fritsch cumulus scheme• Moist updrafts & downdrafts, with entrainment &
detrainment effects and simple microphysics
• Grell-Devenyi cumulus scheme• An ensemble of 144 cumulus schemes are run in every
grid box, the average is fed back to model• Differing entrainment/detrainment parameters,
precipitation efficiencies, dynamic control closures to determine cloud mass flux
UNCLASSIFIED
UNCLASSIFIED
WRF-ARW Physics SchemesLand surface models (LSMs)
• Thermal Diffusion LSM• 5-layer soil temperature model, 31cm depth• Soil moisture constant with land use type and season• No explicit vegetation effects
• Noah LSM• 4-layer soil temperature and moisture model, 2m depth• Includes canopy moisture, fractional snow cover, soil ice• Explicit vegetation effects (incl. evapotranspiration, runoff)
• Rapid Update Cycle (RUC) LSM• 6-layer soil temperature and moisture model, 5.25m depth• Multi-layer snow model• Includes vegetation effects and canopy water• Layer approach to solving moisture & energy budgets
UNCLASSIFIED
UNCLASSIFIED
WRF-ARW Physics SchemesSurface layer schemes
• MM5 Similarity• Stability functions compute heat, moisture & momentum
surface exchange coefficients• No thermal roughness length parameterization
• Eta Similarity• Based on Monin-Obukhov similarity theory• Includes viscous sub-layer parameterizations• Surface fluxes computed iteratively
• Pleim-Xu• Based on similarity theory• Includes viscous sub-layer parameterizations• Similarity functions estimated from analytical
approximations from state variables
UNCLASSIFIED
UNCLASSIFIED
WRF-ARW Physics SchemesAtmospheric boundary layer (ABL) schemes
• Yonsei University (YSU)• Critical bulk Richardson number (0.0) defines ABL top• Entrainment proportional to surface buoyancy flux
• Mellor-Yamada-Janjić (MYJ)• Critical TKE value defines ABL top• Level 2.5 turbulence closure
• Asymmetrical Convective Model v2 (ACM2)• Thermal profile defines ABL top• Combined local and non-local closure in convective
boundary layer (CBL), eddy diffusion in stable boundary layer (SBL)
UNCLASSIFIED
UNCLASSIFIED
Sun 04 Jan 2009NARR, 00 UTC
UNCLASSIFIED
UNCLASSIFIED
Tue 06 Jan 2009NARR, 00 UTC
UNCLASSIFIED
UNCLASSIFIED
Thu 08 Jan 2009NARR, 00 UTC
UNCLASSIFIED
UNCLASSIFIED
Sat 10 Jan 2009NARR, 00 UTC
UNCLASSIFIED
UNCLASSIFIED
Mon 12 Jan 2009NARR, 00 UTC
UNCLASSIFIED
UNCLASSIFIED
Wed 14 Jan 2009NARR, 00 UTC
UNCLASSIFIED
UNCLASSIFIED
Fri 16 Jan 2009NARR, 00 UTC
