ECMWF DA/SAT Training Course, May 2010 1 The Operational Data
Assimilation System Lars Isaksen, Data Assimilation, ECMWF Overview
of the operational data assimilation cycle Computational issues
Observations used by the ECMWF Assimilation System
Multi-incremental 4D-Var Why is 4D-Var performing better than
3D-Var? Recent improvements of ECMWFs Assimilation System Near
future data assimilation implementations Content of talk Slide 2
ECMWF DA/SAT Training Course, May 2010 2 Data assimilation system
The observations are used to correct errors in the short forecast
from the previous analysis time. Every 12 hours we assimilate 7
9,000,000 observations to correct the 80,000,000 variables that
define the models virtual atmosphere. This is done by a careful
4-dimensional interpolation in space and time of the available
observations; this operation takes as much computer power as the
10-day forecast. Slide 3 ECMWF DA/SAT Training Course, May 2010 3
The operational configuration at ECMWF Configuration: Deterministic
model: T1279L91 (~16km) Outer loop of 4D-Var T1279L91 and inner
loops T159/T255/T255 (~125km/80km/80km) EPS target resolution
T639L62 (to 10 days) and T319L62 thereafter Wave model (25km and 36
directions) Implemented in operations 26 January 2010 Slide 4 ECMWF
DA/SAT Training Course, May 2010 4 Extract data for 12h period
2100-0900UTC 70sec (min. 8x1PEs) Pre-process satellite data. Cloud
clearing. Scatterometer winds. 340sec (min. 16x1PEs) Observation
pre-processing for 0000UTC main cycle Slide 5 ECMWF DA/SAT Training
Course, May 2010 5 Analysis and forecast for 0000UTC main cycle
BUFR to ODB. 200sec 4x(8-16PEs) Fetch background forecast 275sec
2x(1PE) Analysis: trajectory, minimization and update 4320sec
(3072PEs) 10 day forecast. 1440 t-steps 3070sec (3072PEs) (or 15h
fc for cycling: 260sec) Surface analysis. 1010sec 4x(1PEs) 430s
260s 880s 270s 820s 490s 1110s Slide 6 ECMWF DA/SAT Training
Course, May 2010 6 The ECMWF operational schedule Slide 7 ECMWF
DA/SAT Training Course, May 2010 7 Operational schedule Early
delivery suite introduced June 2004 3hFC 6h 4D-Var 21-03Z 00 UTC
analysis (DA) T1279 10 day forecast 51*T639/T399 EPS forecasts
03:40 04:00 04:40 06:05 05:00 Disseminate 06:35 Disseminate 02:00
12h 4D-Var, obs 09-21Z 18 UTC analysis 03:30 3hFC 6h 4D-Var 9-15Z
12 UTC analysis (DA) T1279 10 day forecast 51*T639/T399 EPS forec.
15:40 16:00 16:40 18:05 17:00 Disseminate 14:00 12h 4D-Var, obs
21-09Z 06 UTC analysis 15:30 Slide 8 ECMWF DA/SAT Training Course,
May 2010 8 Data extraction Thinning Some data is not used to avoid
over-sampling and correlated errors Departures and flags are still
calculated for further assessment Blacklisting Data skipped due to
systematic bad performance or due to different considerations (e.g.
data being assessed in passive mode) Departures and flags available
for further assessment Model/4D-Var dependent QC First guess based
rejections VarQC rejections Used data Increments Check out
duplicate reports Ship tracks check Hydrostatic check Analysis
Slide 9 ECMWF DA/SAT Training Course, May 2010 9 Conventional
observations used MSL Pressure, 10m-wind, 2m-Rel.Hum. DRIBU: MSL
Pressure, Wind-10m Wind, Temperature, Spec. Humidity
PILOT/Profilers: Wind Aircraft: Wind, Temperature SYNOP/METAR/SHIP:
Radiosonde balloons (TEMP): Note: We only use a limited number of
the observed variables; especially over land. Slide 10 ECMWF DA/SAT
Training Course, May 2010 10 Satellite data sources used in the
operational ECMWF analysis Geostationary, 4 IR and 5 winds 5
imagers: 3xSSM/I, AMSR-E, TMI 4 ozone 13 Sounders: NOAA AMSU-A/B,
HIRS, AIRS, IASI, MHS 2 Polar, winds: MODIS 3 Scatterometer sea
winds: ERS, ASCAT, QuikSCAT 6 GPS radio occultation Slide 11 ECMWF
DA/SAT Training Course, May 2010 11 Significant increase in number
of observations assimilated Conventional and satellite data
assimilated at ECMWF 1996-2010 Slide 12 ECMWF DA/SAT Training
Course, May 2010 12 The observation operator Observations are not
made at model grid points Satellites often measure radiances, NOT
temperature and humidity We calculate a model radiance estimate of
the observation to enable comparison. This is done with the
observation operator H. H may be a simple interpolation from model
grid to observation location H may possibly perform additional
complex transformations of model variables to radiance space for
satellite data. Model T,u,v,q Observation Satellite Radiance
compare H Model radiance Slide 13 ECMWF DA/SAT Training Course, May
2010 13 Model Radiance The variational method allows model
radiances to be compared directly to observed radiances Enables use
of advanced observation operators Model T and q H compare
Observation Satellite Radiance Slide 14 ECMWF DA/SAT Training
Course, May 2010 14 J b : Ensures that the background model fields
are adjusted meteorologically consistently in the region close to
the observation location Increments due to a single observation of
geopotential height at 1000hPa at 60N with value 10m below the
background. Slide 15 ECMWF DA/SAT Training Course, May 2010 15 J b
: The Balance Operator ensures the height and wind field balance is
retained Increments for a single observation of geopotential height
at 1000hPa. Left panel: wind increment near 300hPa. Right panel:
wind increment about 150m above the surface. wind increments at
300hPa wind increments at 150 metre above surf. Slide 16 ECMWF
DA/SAT Training Course, May 2010 16 Observation minus model
differences are computed at the observation time using the full
forecast model at T1279 (16km) resolution 4D-Var finds the 12-hour
forecast evolution that optimally fits the available observations.
A linearized forecast model is used in the minimization process
based on the adjoint method It does so by adjusting surface
pressure, the upper-air fields of temperature, wind, specific
humidity and ozone The analysis vector consists of 80,000,000
elements at T255 resolution (80km) 4D-Var implementation used at
ECMWF All observations within a 12-hour period (~9,000,000) are
used simultaneously in one global (iterative) estimation problem
09Z 12Z 15Z 18Z 21Z Slide 17 ECMWF DA/SAT Training Course, May 2010
17 4D-Var incremental formulation Courtier, Thpaut and
Hollingsworth (1994) The i-summation is over N=25 -hour long time
slots of the 12-hour assimilation period. In the incremental
formulation the cost function J is expressed in terms of increments
x with respect to the background state, x=x-x b, at initial time.
and are the TL of and, linearized around The innovations d i are
calculated using the non-linear operators, H i and M i. This
ensures the highest possible accuracy for the calculation of the
innovations d i, which are the primary input to the assimilation!
Slide 18 ECMWF DA/SAT Training Course, May 2010 18 Revision of
operational 4D-Var algorithm Implementation: Inner/outer iteration
algorithm Hessian pre-conditioning Conjug. Gradient minimisation
Improved TL approximations Multi-incremental T159/T255/T255 These
developments facilitate: Use of higher density data Higher
resolution inner loop (T255) and outer loop (T1279) Enhanced use of
(relatively costly) TL physics Cloud and rain assimilation More
work has been done to improve the representation of the smallest
scales in the inner loop i.e. the analysis increments. Slide 19
ECMWF DA/SAT Training Course, May 2010 19 Multi-incremental
quadratic 4D-Var at ECMWF T1279L91 T159L91 T255L91 Slide 20 ECMWF
DA/SAT Training Course, May 2010 20 4D-Var with three outer loop:
efficient, accurate and allows non-linearity T95 T159 T255 Analysis
increments at 900hPa for temperature for each of the three
minimizations. Plot shows 2009 setup. From 2010: T95 T159 and T799
T1279. Decreasing amplitude of increments: T95>T159>T255.
Last analysis step adds small corrections where data density is
highest. Note model and observation operators are re-linearized
twice. Add T95 increment to T799 background and re-linearize M, H
Add T159 increment and re-linearize M, H Add T255 increment = final
T799 analysis Slide 21 ECMWF DA/SAT Training Course, May 2010 21
The atmosphere does not evolve in isolation, interactions between
the atmosphere and the underlying land and ocean are also important
in determining the weather. Ocean ice processes, ocean surface
waves, land surface, soil, hydrological and snow processes are all
represented at ECMWF in the most advanced operational Earth-system
model available anywhere. These physical processes have smaller
scales than the model grid (16 km) and are therefore represented by
so-called Parametrization Schemes which represent the effect of the
small-scale processes on the large-scale flow. Physical processes
in the ECMWF model An accurate model representation of the
atmosphere is an important part of the assimilation system Slide 22
ECMWF DA/SAT Training Course, May 2010 22 T1279 (16km) Since
January 2010 2,140,704 grid-points per level Outer loop model
resolution is now T1279L91 Important for accurate comparison
against observations L60 L91 T799 (25km) 2006-2010 843,490
grid-points per level Slide 23 ECMWF DA/SAT Training Course, May
2010 23 OPS (35r2) 980hPa (35r3) Use of improved QC (Huber norm)
961hPa (36r1) High-res system T1279+T159/T255/T255 945hPa Hurricane
Bill, 20 Aug. 2009 Observed MSL pressure~944hPa Resolution and QC
benefits analysis of extreme events Improved hurricane analyses due
to Huber norm and resolution Slide 24 ECMWF DA/SAT Training Course,
May 2010 24 Can we use our parallel computers efficiently?
Scalability of IFS model T1279 L91 Cores 1024 2048 3072 4096 5120
6144 7168 8192 Nodes Slide 25 ECMWF DA/SAT Training Course, May
2010 25 4D-Var is more difficult to parallelize efficiently T1279
model =2304 cores ideal T1279 model T255an T1279obs T159an =1728
cores=1152 cores=576 cores Slide 26 ECMWF DA/SAT Training Course,
May 2010 26 768 1024 1536 512 Parallel performance is important.
Scalability is not perfect for the 4D-Var analysis 4D-Var T799
outer,T95/T255 inner, 91 levels on IBM system Slide 27 ECMWF DA/SAT
Training Course, May 2010 27 Hurricane Lili. Surface scatterometer
winds. An example how 4D-Var propagates information vertically No
SCAT analysis NSCAT analysis First guess MSL pressure Analysis MSL
pressure MSL pressure Analysis increments S.M. Leidner, L. Isaksen
and R.S. Hoffman Impact of NSCAT Winds on Tropical Cyclones in the
ECMWF 4DVAR assimilation system Mon. Wea. Rev. 131,1,3-26 (2003)
Slide 28 ECMWF DA/SAT Training Course, May 2010 28 No SCAT analysis
NSCAT analysis First guess MSL pressure S.M. Leidner, L. Isaksen
and R.S. Hoffman Impact of NSCAT Winds on Tropical Cyclones in the
ECMWF 4DVAR assimilation system Mon. Wea. Rev. 131,1,3-26 (2003)
First guess MSL pressure Analysis MSL pressure Analysis increments
Hurricane Lili. Surface scatterometer winds. An example how 4D-Var
propagates information vertically Slide 29 ECMWF DA/SAT Training
Course, May 2010 29 Surface scatterometer wind information is
propagated vertically and improve the analysis. Due to
flow-dependent structure functions in 4D-Var Slide 30 ECMWF DA/SAT
Training Course, May 2010 30 4D-Var is using more a-synoptic data
than 3D-Var 4D-Var is using more data from frequently reporting
stations. The plots show the use of SYNOP surface pressure
observations. Column height gives the number of observations
available, while the black part displays those actually used in the
assimilation. 4D-Var SYNOP Screening 3D-Var SYNOP Screening 3D-Var
is like 4D-Var without the time dimension. The analysis is
performed at synoptic times only (0000, 0600, 1200 and 1800 UTC).
Mostly only data valid a synoptic time is used. Slide 31 ECMWF
DA/SAT Training Course, May 2010 31 4D-Var versus 3D-Var and
Optimum Interpolation 4D-Var is comparing observations with
background model fields at the correct time 4D-Var can use
observations from frequently reporting stations The dynamics and
physics of the forecast model in an integral part of 4D-Var, so
observations are used in a meteorologically more consistent way
4D-Var combines observations at different times during the 4D-Var
window in a way that reduces analysis error 4D-Var propagates
information horizontally and vertically in a meteorologically more
consistent way 4D-Var more complex: needs linearized perturbation
forecast model and its adjoint to solve the cost function
minimization problem efficiently Slide 32 ECMWF DA/SAT Training
Course, May 2010 32 Recent revisions to the assimilation system Use
of many new satellites and new instruments (will be presented by
Peter Bauer tomorrow) Variational bias correction of satellite
radiances Adaptive bias correction for radiosondes and SYNOP
pressure data Increased resolution from T799/ T95/T159/T255 to
T1279/ T159/T255/T255 More advanced Tangent Linear physics scheme
in 4D-Var Improved handling of supersaturation in the humidity
analysis Huber norm Variational Quality Control Weak-constraint
4D-Var accounting for model error Advanced diagnostic tools to
understand impact of observations on analysis/forecast: Forecast
Sensitivity to Observations Slide 33 ECMWF DA/SAT Training Course,
May 2010 33 First version (SSM/I radiances) 2005; extended to
SSMIS, TMI, AMSR-E in 2007 Direct 4D-Var radiance assimilation from
March 2009 Main difficulties: inaccurate moist physics
parameterizations (location/intensity), formulation of observation
errors, bias correction, linearity. 4D-Var first guess SSM/I Tb
19v-19h [K]SSM/I observational Tb 19v-19h [K] Improved assimilation
of satellite moisture data Assimilation of rain-affected microwave
Slide 34 ECMWF DA/SAT Training Course, May 2010 34 Humidity
analysis improvements New humidity background error model: takes
into account how humidity errors are affected by temperature errors
in cloudy conditions (statistical relationship) extension of the
humidity errors to cater for supersaturated conditions (with
respect to ice) Changes in cloud cover spin-up. Monthly average
zonal mean (Sep 2008) analysis minus a 15h forecast valid at the
same time. Old New Slide 35 ECMWF DA/SAT Training Course, May 2010
35 Improved quality control of observations Huber-norm observation
cost function Gaussian Huber norm (New operational) Gaussian + flat
(operational) L1 metric far from centre, L2 metric close to centre.
Robust method: a few erroneous observations does not ruin analysis
Adds some weight on observations with large departures A set of
observations with consistent large departures will influence the
analysis Most obs-bg innovations follow Huber-norm distribut. Slide
36 ECMWF DA/SAT Training Course, May 2010 36 Q only applied in
stratosphere. Cost function including model error and bias: Model
error Model bias (cycled) b How to account for model error in
4D-Var First step is a weak-constraint 4D-Var that handles biases.
Slide 37 ECMWF DA/SAT Training Course, May 2010 37 Weak-constraint
4D-Var formulation captures part of model error biases in the
stratosphere AMSU-A first guess departures reduced Slide 38 ECMWF
DA/SAT Training Course, May 2010 38 J is a measure of the forecast
error (as defined through e.g. dry energy norm) Example: Impact of
each observing system in ECMWFs 2009 assimilation system Enhance
diagnostics of the assimilation system Forecast Sensitivity to
Observations Slide 39 ECMWF DA/SAT Training Course, May 2010 39
Soil moisture assimilation using Ext. Kalman Filter Will be
implemented in June 2010 Analysis increments (mm) Slide 40 ECMWF
DA/SAT Training Course, May 2010 40 Ensembles of data assimilations
Is being implemented now, first for EPS later for 4D-Var Run an
ensemble of analyses with randomly perturbed observations and SST
fields, and form differences between pairs of analyses (and
short-range forecast) fields. These differences will have the
statistical characteristics of analysis (and short-range forecast)
error. To be used in specification of background errors = errors of
the day. To indicate where good data should be trusted in the
analysis (yellow shading). Also for initialization of the EPS!
Slide 41 ECMWF DA/SAT Training Course, May 2010 41 Estimating
Background Error Statistics from Ensembles of Data Assimilations
(EDA) Run an ensemble of analyses with random observation and SST
perturbations, plus stochastic model error representation. Form
differences between pairs of background fields. These differences
will have the statistical characteristics of background error (but
twice the variance). AnalysisForecast x b + b AnalysisForecast x b
+ b AnalysisForecast x b + b AnalysisForecast x b + b
AnalysisForecast x b + b AnalysisForecast x b + b Background
differences Slide 42 ECMWF DA/SAT Training Course, May 2010 42 To
estimate analysis uncertainty To improve the initial perturbations
in the Ensemble Prediction To calculate static and seasonal
background error statistics To estimate flow-dependent background
error in 4D-Var - errors-of-the-day To improve QC decisions and
improve the use of observations in 4D-Var Why implementing Ensemble
Data Assimilation? Slide 43 ECMWF DA/SAT Training Course, May 2010
43 Ongoing developments in data assimilation at ECMWF EKF for soil
moisture analysis EDA to provide flow-dependent background error
information to 4D-Var Vertical resolution increase planned for
later in 2010 (~140+ levels TBD) Long window 4D-Var: extend to 24
hour window, improve model error term Improved assimilation of
cloud/aerosols/rain observations Flow-dependent data selection
Account for observation error correlations Modularisation of the
IFS and scalability of the assimilation system Slide 44 ECMWF
DA/SAT Training Course, May 2010 44 Summary of todays lecture
Overview of the operational data assimilation cycle Computational
issues Observations used by the ECMWF Assimilation System
Multi-incremental 4D-Var Why is 4D-Var performing better than
3D-Var? Recent improvements of ECMWFs Assimilation System Near
future data assimilation implementations
