2.11 Assimilation methods, including use of data in cloudy regions

Daryl Kleist

1

Data Assimilation and Observing Systems Working GroupBeijing, China

27-28 October 2015

Univ. of Maryland-College Park, Dept. of Atmos. & Oceanic Science

With sincere thanks to everyone that provided material. I have tried to condense as much as possible, but was left with much more than the time allotted. With apologies for leaving anything out…..

Hybrid-4DEnVar(Courtesy UKMO)

• Hybrid-4DVar uses PF model to propagate B implicitly

• 4DEnVar uses ensemble instead of PF, but much more IO required.

– Each iteration ~8 times faster with 200 members.

• Analysis consists of two parts:– A 3DVar-like analysis based on the climatological covariance

Bc

– A 4D analysis consisting of a linear combination of the ensemble perturbations.

• Localisation is currently in space only: same linear combination of ensemble perturbations at all times.

• The same executable can do an ensemble of analyses: En-4DEnVar:

– (For each member, analyse increments relative to its own background trajectory.)

• Additional routines required to deal with perturbed observations, inflation, etc..

ECMWF Testing ofHybrid Gain

Figure 5. Root mean square background forecast observation departures for radiosonde zonal wind (left column) and radiosonde temperature (right column) for the northern hemisphere (top row), tropics (middle row) and southern hemisphere (bottom row). Curves refer to TL399 EnKF (continuous line), TL399 static 4DVar (dashed line), TL399 Hybrid Gain EnDA (dash-dot line) and present statistics computed over the 2012-06-01 to 2012-08-31 period. Triangle symbols indicate statistically significant difference at the 95% confidence level between the Hybrid Gain EnDA departures and the 4DVar departures.

Bonavita, M., M. Hamrud, and L. Isaksen, 2015: EnKF and Hybrid Gain Ensemble Data Assimilation Part II: EnKF and Hybrid Gain Results. Mon. Wea. Rev. doi:10.1175/MWR-D-15-0071.1, in press.

Page 4 – April 21, 2023

Alternative ensemble DA approaches(Mark Buehner, Environment Canada)

• Our EnKF provides high-quality 256-member ensembles for both data assimilation (deterministic and ensemble) and initializing the ensemble forecasts

• Due to computational cost, current EnKF algorithm limits the volume of observations (maybe 10-20% of obs used in EnVar?)

• Only a limited amount of fortran code is shared between the EnKF and EnVar systems – significant work required to increase the amount of shared code

• Theoretically better method for applying localization and ability to use hybrid covariances are advantages of variational approach vs. current approach in EnKF

• These ideas motivate the testing of variational approaches for ensemble data assimilation (MetOffice and MeteoFrance are also pursuing similar approaches)

Page 5 – 23-4-21

Scale-dependent covariance localizationForecast impact – Comparison against ERA-Interim

T+24hZonal mean

Control is better

Scale-Dependent is better

Std Dev difference for U

North Pole

South Pole

NCEP (Planned) 2016 Implementation

6

• Hybrid 4D EnVar to become operational for GFS/GDAS in early 2016 (tentative)– Tests at low resolution helped design configuration – Real time and retrospectives underway, operational package quasi-frozen

• Package Configuration– T1534 deterministic GFS with 80 member T574L46 ensemble with fully

coupled (two-way) EnKF update (87.5% ensemble & 12.5% static), same localization as current operations

– Incremental normal mode initialization (TLNMC) on total increment– Multiplicative inflation and stochastic physics for EnKF perturbations– Full field digital filter– All-sky MW radiance assimilation, aircraft temperature bias correction– Minor model changes

Full Resolution (T1534/T574) Trials

500 hPa AC for the Operational GFS (Black, 3D Hybrid) and Test 4D configuration (Red) for the period covering 02-01-2015 through 10-03-2015 (242 cases!).

Atlantic E-Pac

C-Pac W-Pac

Mean TC Track Errors for the operational GFS (Green, 3D Hybrid) and Test 4D configuration (Red) for the period covering 05-01-2015 through 10-03-2015 ().

4DEnVar(Courtesy UKMO)

Increasing model resolution produces smaller error than increasing

ensemble size in NCEP GFS EnVar

Exp. Name Horiz. localization Vert. localiza1on Weight of static B

T254T670 Operational localization table -0.5 0.125

T254T670Ens320 Operational localization table ‐0.5 0.0

T670T670 75% of the operational localiza4on table

‐0.375 0.125

c/o Lili Lei and Jeff Whitaker

Use of data in cloudy regions

Progression of All-sky Microwave Radiance in NCEP’s GFS System Previous work

Preference given to clear-sky in the data thinning was removed; Cloud signal removal for radiances with thin cloud was turned off in bias correction; Thick cloud filtering was turned off; AMSU-A observation error: symmetric observation error (Geer et al. 2011).

Major upgrade in 2012: provided basic framework for all-sky radiance assimilation study for GFS, NAM and HWRF Introduced individual hydrometeors ql,qi,qr,qs,qg,qh into GSI as state variables; Passed Jacobians w.r.t. hydrometeors into the GSI inner loop; Control variable(s): cloud water (cw) or individual hydrometeors. Observation operator revision and bug fixes

To present: in the hybrid 3D EnKF-VAR data assimilation system Situation-dependent observation error inflation; AMSUA-A observation error re-tuned; All-sky radiance bias correction strategy (Zhu et al. 2014) Additional quality control: cloud effect (Geer et al. 2013) and emissivity sensitivity screening; Normalized cloud water control variable; New static background error variance and correlation lengths for cloud

water; Non-zero Jacobian for locations of clear-sky or small amount clouds; Validation and improvement of CRTM under all-sky (see posters from Emily Liu 8p.01 & Paul van Delst 2p.03) Other changes and bug fixes.

Parallel test in the 4D EnVar system for Q1FY16 implementation Included in the pre-implementation package (see poster from Andrew Collard 6.01) 11

Clear-sky OmF vs. All-sky OmF

CLW More data coverage: Thick clouds that are excluded from clear-sky assimilation are now assimilated under all-sky conditionRainy spots are excluded from both conditions

AMSUA NOAA19 CH1 00Z 20131029

12

Two approaches to the problem:

1- the “complete” physical approach to clouds, where the analysis control variable is augmented to include microphysical variables. Not much progress on this side since I left NCAR,

2- the “dynamical” approach, where the analysis control variable is limited to 3D cloud fraction and the model is reduced to dynamical advection (the physics being switched off). This approach is useful for nowcasting. We are further developing this for the US Air Force. Worth mentioning, the current testing of:

- the use of particle filters to retrieve cloud fraction profiles for every Infrared satellite sounder field-of-view,- the Adaptive Advection Adjustment via Realigned Grid mesH (AAARGH).

JCSDA Perspective

AAARGH: Components of the Adjustment Technique

• Observations: GOES-imager radiances

• Obs operator + TL/AD: simple operator using CRTM (clear-sky) + cloud fractions

• Grid warping: compute & store 2D displacement field

• Following cycle: read previous displacement field and apply to WRF forecast

using dWRF limited to cloud fractions

The following cycle:

WRF

WRF + AAARGH

Difference

RMSE of forecast vs. GOES radiances

WRF + AAARGH

WRF Difference

ECMWF Progress on All-SkyRecent progress in all-sky assimilation: - Consolidation of the all-sky technique as the way to assimilate microwave water vapour and imaging channels: - Activation of GMI and AMSR-2 in operations (August 2015) - Transfer of 4 * MHS to the all-sky framework in operations (May 2015) - All-sky water-vapour channels are in 2nd/3rd place in FSOI measures now (at around 15%) - Better understanding of forecast model cloud issues (which give biases in allsky O-B): - Maritime stratocumulus diurnal cycle problems - Lack of supercooled liquid water in high-latitude cold-air outbreaks (CAO)

Plans for next year: - Improved all-sky framework: - Add coverage over snow-covered land surfaces - Improve all-sky observation error modelling in clear and light-cloud situations - Address forecast model cloud biases: - Changes to the shallow convection scheme to produce supercooled liquid water at lower temperatures. - Move HIRS (and possibly IASI) infrared water vapour channels to the all-sky approach

Further challenges: - Try again to transfer microwave temperature sounding channels to the all-sky approach - Observation error correlations in cloudy situations - Low-frequency rain-focused channels (e.g. 10 GHz) - Simplified 3-D radiative transfer modelling - Further improve simulation of optical properties of frozen particles - Support ICI, the future mm-wave instrument targeting ice clouds 17

Status of tropospheric microwave at ECMWF: Operations at 41r2 configuration for early 2016

Slide 18

Instrument Ocean Sea-ice Land Snowy land

High land

SSMIS-F17 imager

GMI

AMSR2

SSMIS-F17 sounder

SSMIS-F18 sounder

MHS on Metop-A/B and NOAA-18/19

MWHS

MWHS-2

ATMS

AMSU-A on Metop-A/B and NOAA-15/18/19

Imager channels

19-90 GHz

183 GHz WV

50 GHz temperature

=clear-sky =cloud and precipitation

Towards an Advanced Cloudy IR Radiance Assimilation

(T,Q,V)

model physics (M)

(T,Q,V,ciw,clw,cc)

Cloudy(RT)

(Rcal)

Jo=(Robs-Rcal)

Cloudy(RT) *

model physics (M*)

(T,Q,V,ciw,clw,cc)*

(T,Q,V)*

(Rcal)*

Cloudy radiances Rcal are simulated via a chain of forward operators (M,RT).

The fit of the analysis to the observations is computed (Jo)

Jo is minimized by perturbing the analysis variables according to gradients from a chain of adjoint operators (RT*,M*)

ECMWF Summary and PlansAll Sky IR

• The simplified assimilation of overcast HIRS, AIRS and IASI works very well and became operational at ECMWF in 2009.

• We are in the process of developing the advanced cloudy assimilation system (drawing on experience from a similar scheme for rain affected microwave radiances)

• Initially, cloud parameters (clw,ciw,cc) will be diagnostic and driven by the model physics via adjustments in the analysis control vector variables (T,Q and V).

• Later the cloud parameters will be tested as full variables in the analysis control vector (but this will require accurate background error observations

Backup and Reference Slides