Embed Size (px)
Citation preview
Statistical Weather Forecasting
Independent StudyDaria Kluver
From Statistical Methods in the Atmospheric Sciences by Daniel Wilks
Perfect Prog and MOSClassical statistical forecasts for
projections over a few days are not used. Current dynamical NWP models are more accurate.
2 types of classical statistical wx forecasting are used to improve aspects of NWP forecasts. (by post-processing the NWP data)
Both methods used large multiple regression equations.
3 reasons why statistical reinterpretation of dynamical NWP output is useful for practical weather forecasting:
1. NWP models simplify and homogenize sfc conditions. Statistical relationships can be developed btwn NWP output and desired forecast quantities.
2. NWP model forecasts are subject to error. To the extent that these errors are systematic, statistical forecasts based on NWP info can correct forecast biases.
3. NWP models are deterministic. Using NWP info in conjunction with statistical methods allows quantification of the uncertainty associated with different forecast situations.
1st Statistical approach for dealing with forecasts from NWP
Perfect Prog (Klein et al. 1959)◦Takes NWP model forecasts for future
atmosphere assuming them to be perfect◦Perfect prog regression equations are similar to
classical regression equations except they do not incorporate any time lag.
Example: equations specifying tomorrows predictand are developed using tomorrow’s predictor values.
If the NWP forecasts for tomorrow’s predictors really are perfect, the perfect-prog regression equations should provide very good forecasts.
2nd Statistical approachModel Output Statistics (MOS)
◦ Preferred because it can include directly in the regression eqns the influences of specific characteristics of different NWP models at different projections into the future.
◦ To get MOS forecast eqns you need a developmental data set with historical records of predictand, and records of the forecasts by NWP model.
◦ Separate MOS forecast equations must by made for different forecast projections.
Example: predictand is tomorrows 1000-800mb thickness as forecast today by a certain NWP model.
Advantages and Disadvantages of Perfect-Prog and MOS:
Perfect ProgAdvantages:
Large developmental sample (fit using historical climate data)
Equations developed without NWP info, so changes to NWP models don’t require changes in regression equations
Improving NWP models will improve forecasts Same equations can be used with any NWP models
Disadvantages:◦ Potential predictors must be well forecast by the
NWP model
MOSAdvantages:
◦ Model-calculated, but un-observed quantities can be predictors
◦ Systematic errors in the NWP model are accounted for
◦ Different MOS equations required for different projection times
◦ Method of choice when practical
Disadvantages:◦ Requires archived records (several years) of
forecast from NWP model to develop, and models regularly undergo changes.
◦ Different MOS equations required for different NWP models
Operational MOS ForecastsExample: FOUS14MOS equations underlying the FOUS14 forecasts
are seasonally stratified: warm season (Apr- Sep) and cool season (Oct-Mar).◦ A finer stratification is preferable with sufficient
developmental dataForecast equations (except t, td and winds) are
regionalized. Some MOS equations contain predictors
representing local climatological valuesSome equations are developed simultaneously
to enhance consistency (example: a td higher than t doesn’t make sense)
Ensemble Forecasting
First, lets talk about the birth of Chaos…LorenzRoyal McBeeSensitive dependence on initial
conditions
Lorenz’s data
What does Chaos have to do with NWP?
The atmosphere can never be completely observed
A NWP model will always begin calculating forecasts from a state slightly different from the real atmosphere.
These models have the property of sensitive dependence on initial conditions.
Stochastic Dynamical Systems in Phase Space
Stochastic dynamic prediction- physical laws are deterministic, but the equations that describe these laws must operate on initial values not known with certainty ◦can be described by a joint probability
distribution.This process yields, as forecasts,
probability distributions describing uncertainty about the future state of the atmos.
Phase spacePhase space is used to visualize
the initial and forecast probability distributions
Phase space – geometrical representation of the hypothetically possible states of a dynamical system, where each of the coordinate axes defining this geometry pertains to one of the forecast variables of the system.
Pendulum Animation
Lorenz attractor and the butterfly
Phase space of a atmospheric model has many more dimensions. ◦Simple model has 8 dimensions!
Operational NWP models have a million dimensions
More Complications:◦The trajectory is not attracted to a
single point like the pendulum◦Pendulum did not have sensitive
dependence to initial conditions.
Uncertainty about initial state of the atmosphere can be conceived of as a probability distribution in phase space.
The shape of the initial distribution is stretched and distorted at longer forecast projections.◦Also, remember there is no single attractor.
A single point in phase space is a unique weather situation.
The collection of possible points that equal the attractor can be interpreted as the climate of the NWP model.
Ensemble ForecastsThe ensemble forecast procedure begins by
drawing a finite sample from the probability distribution describing the uncertainty of the initial state of the atmos.
Members of the point cloud surrounding the mean estimated atmospheric state are picked randomly
These are the ensemble of initial conditionsThe movement of the initial-state probability
distribution through phase space is approximated by this sample’s trajectories
Each point provides the initial conditions for a separate run of the NWP model.
Ensemble Average and Ensemble DispersionTo obtain a forecast more accurate than 1 model
run with the best estimate of the initial state of the atmosphere, members of the ensemble are averaged.
The atmospheric state corresponding to the center of the ensemble in phase space will approximate the center of the stochastic dynamic probability distribution at the future time.
Doing this with weather models averages out elements of disagreement and emphasizes shared features.
Over long time periods, the average smoothes out and looks like climatology.
We get an idea of the uncertainty◦ More confidence if the dispersion is small◦ Formally calculated by ensemble standard deviation
Graphical Display of Ensemble Forecast InformationCurrent practice includes 3
general types of graphics: ◦displays of raw ensemble output, ◦displays of statistics summarizing
the ensemble distribution, and ◦displays of ensemble relative
frequencies for selected predictands.
Effects of Model Errors 2 types of model errors:
◦ 1. models operate at a lower resolution than reality◦ 2. certain physical processes- predominantly those
operating at scales smaller than the model resolution- are represented incorrectly.
To represent the residuals of fig 6.31, random numbers can be added to the parameterization function. Called “stochastic physics” and used at ECMRF
The parameterization (smooth curve) does not fully capture the range of behaviors for the parameterized process that are actually possible (scatter of points)
Statistical Postprocessing: Ensemble MOSYou can do MOS post processing
on the ensemble mean:◦There is still research on how best to
do this.◦Multiple ways, which involve
probability distributions, which will not be discussed here.
Subjective Probability Forecasts
The nature of subjective forecasts:◦Subjective integration and interpretation of
objective forecast info forecast guidance.◦ Includes deterministic forecast info from
NWP, MOS, current obs, radar, sat, persistence info, climate data, individual previous experiences.
◦Subjective forecasting – the distillation by a human forecaster of disparate and sometimes conflicting info. A subjective forecast- one formulated on the basis
of the judgment of 1 or more individuals. Good one will have some measure of uncertainty.
Assessing Discrete Probabilities
Tricks forecasters can use, like spinning wheels or playing betting games.
Chapter 7: Forecast Verification
Purposes of Forecast Verification◦Forecast verification- the process of assessing
the quality of forecasts.◦Any given verification data set consists of a
collection of forecast/observation pairs whose joint behavior can be characterized in terms of the relative frequencies of the possible combinations of forecast/observation outcomes.
◦This is an empirical joint distribution◦ It is important to do verification to improve
methods, evaluate forecasters, estimate error characteristics.
The Joint Distribution of Forecasts and ObservationsForecast = Observation = The joint distribution of the forecasts
and observations is denoted
This is a discrete bivariate probability distribution function associating a probability with each of the IxJ possible combinations of forecast and observation.
The joint distribution can be factored in two ways, the one used in a forecasting setting is:
Called calibration-refinement factorization
The refinement of a set of forecasts refers to the dispersion of the distribution p(yi)
If yi has occurred, this is the probability of oj happening.Specifies how often each possible weather event occurred on those occasions when the single forecast yi was issued, or how well each forecast is calibrated.
The unconditional distribution, which specifies the relative frequencies of use of each of the forecast values yi sometimes called the refinement of a forecast.
Scalar Attributes of Forecast PerformancePartial list of scalar aspects, or
attributes, of forecast quality
Accuracy Average correspondence between individual forecasts and the events they predict.
Bias The correspondence between the average forecast and the average observed value of the predictand.
Reliability Pertains to the relationship of the forecast to the average observation, for specific values of the forecast.
Resolution The degree to which the forecasts sort the observed events into groups that are different from each other.
Discrimination Converse of resolution, pertains to differences between the conditional averages of the forecasts for different values of the observation.
Sharpness Characterize the unconditional distribution (relative frequencies of use) of the forecasts.
Forecast SkillForecast skill- the relative
accuracy of a set of forecasts, wrt some set of standard control, or reference, forecast (like climatological average, persistence forecasts, random forecasts based on climatological relative frequencies)
Skill score- a percentage improvement over reference forecast.
accuracyAccuracy of reference
Accuracy that would be achieved by a perfect forecast.
Next timeContinue to talk about forecast verification
◦ Looking at some forecast data NWS vs weather.com vs climatology
◦ 2x2 contingency tables◦ Conversion from probabilistic to nonprobabilistic◦ Quantile plots◦ Probability forecasts of discrete predicands◦ Probability forecasts for continuous predictands◦ Accuracy measures, Skill scores, Brier Score,
MSE◦ Multi-category events
