21
1 Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid Testbed Arthur P. Mizzi ([email protected]) NCAR/MMM HFIP Ensemble Design Subgroup Meeting October 31, 2011 Teleconference Boulder, CO

1 Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid Testbed Arthur P. Mizzi ([email protected]) NCAR/MMM HFIP Ensemble

  • View
    217

  • Download
    2

Embed Size (px)

Citation preview

Page 1: 1 Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid Testbed Arthur P. Mizzi (mizzi@ucar.edu) NCAR/MMM HFIP Ensemble

1

Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid

TestbedArthur P. Mizzi

([email protected])

NCAR/MMM

HFIP Ensemble Design Subgroup MeetingOctober 31, 2011TeleconferenceBoulder, CO

Page 2: 1 Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid Testbed Arthur P. Mizzi (mizzi@ucar.edu) NCAR/MMM HFIP Ensemble

2

Overview:1. Introduction to the MMM Regional Hybrid

Testbed (MRHT).

2. Results from the study of ETKF inflation factor schemes and data reduction experiments.

3. Preliminary results from the study of the GSI/ETKF, GSI/LETKF, and GSI/DART-EnKF regional hybrids.

Page 3: 1 Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid Testbed Arthur P. Mizzi (mizzi@ucar.edu) NCAR/MMM HFIP Ensemble

3

MMM Regional Hybrid Testbed:1. A community resource to facilitate introduction to

and testing of variational-hybrid cycling strategies.

2. 80 member, low resolution (200km), CONUS domain, initial ensemble for the Hurricane Dean (August 15, 2007 to September 15, 2007) test case.

3. 10 member, higher resolution (45km) ensemble for the same test case.

4. Script to generate initial ensembles.

5. Observations for the test case in prep.bufr, ob.ascii, and obs.seq formats.

Page 4: 1 Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid Testbed Arthur P. Mizzi (mizzi@ucar.edu) NCAR/MMM HFIP Ensemble

4

MMM Regional Hybrid Testbed:6. Cycling script using:

(i) GSI or WRFDA regional hybrids or DART for updating the ensemble mean (other assimilation algorithms can be easily added),

(ii) ETKF, LETKF, or DART-EnKF for updating the ensemble perturbations (other perturbation update strategies - like the Whitaker EnKF - can be easily added),

(iii) Wang et al. (2003), Wang et al. (2007), Bowler et al. (2008), and MMM experimental ensemble spread inflation algorithms, and

(iv) WRF-ARW as the forecast (other models - like HWRF - can be easily added).

Page 5: 1 Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid Testbed Arthur P. Mizzi (mizzi@ucar.edu) NCAR/MMM HFIP Ensemble

5

MMM Regional Hybrid Testbed:

7. Script for calculating hybrid single observation increments.

8. Post-processing scripts to display:(i) Single observation increments,

(ii) Inflation factor, prior, and posterior ensemble spread time series,

(iii) Vertical profile and time series plots of the verification of the analyses and forecasts against observation in observation space, and

(iv) Spread/error verification diagnostics.

Page 6: 1 Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid Testbed Arthur P. Mizzi (mizzi@ucar.edu) NCAR/MMM HFIP Ensemble

6

MMM Regional Hybrid Testbed:

9. Available on web at https://svn-mmm-hybrid-testbed.cgd.ucar.edu with the appropriate password.

Page 7: 1 Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid Testbed Arthur P. Mizzi (mizzi@ucar.edu) NCAR/MMM HFIP Ensemble

7

GSI/ETKF Regional Hybrid Cycling Experiments:

• 20-member ensemble.• 12-hr cycling (Aug. 15 to Sep. 11, 2007)• CONUS low resolution grid (200km)• ETKF with Wang et al. (2003), Wang et al.

(2007), Bowler et al. (2008), and MMM experimental inflation schemes.

• β=0.75, H=750 km, V=20 grid pts.

Page 8: 1 Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid Testbed Arthur P. Mizzi (mizzi@ucar.edu) NCAR/MMM HFIP Ensemble

8

GSI/ETKF Regional Hybrid Cycling Experiments cont.:

• Verification in observation space against radiosonde and surface synoptic observations.

• Statistical significance testing with Student T-test and Wilcoxon Sign Test.

Page 9: 1 Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid Testbed Arthur P. Mizzi (mizzi@ucar.edu) NCAR/MMM HFIP Ensemble

9

Comparison of GSI/ETKF Regional Hybrid with 3DVAR and Ensemble 3DVAR

• GSI/ETKF regional hybrid gave best fit to observations. • GSI/ETKF differences from the other schemes were statistically significant.

Page 10: 1 Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid Testbed Arthur P. Mizzi (mizzi@ucar.edu) NCAR/MMM HFIP Ensemble

10

Comparison ETKF Inflation Schemes

• WG03 and BW08 gave large inflation.• TRNK gave moderate inflation.• WG07 gave small inflation due to ρ-factor.• WG03, WG07, and BW08 had oscillation.

Page 11: 1 Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid Testbed Arthur P. Mizzi (mizzi@ucar.edu) NCAR/MMM HFIP Ensemble

11

Comparison Ensemble Spread from Different Inflation Schemes

• WG03, WG07, and BW08 gave comparable ensemble spread and oscillation.• TRNK gave lower ensemble spread and damped oscillation.

Page 12: 1 Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid Testbed Arthur P. Mizzi (mizzi@ucar.edu) NCAR/MMM HFIP Ensemble

12

Comparison of Hybrid using Different ETKF Inflation Schemes with 3DVAR

• TRNK provided best fit of 12-hr forecasts to the observations for the non-surface variables. • TRNK results were significant different from the other schemes.• Results for the surface variables were mixed.

FCST-RMSE

ANAL-RMSE

Lowest FCST-RMSE

Page 13: 1 Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid Testbed Arthur P. Mizzi (mizzi@ucar.edu) NCAR/MMM HFIP Ensemble

13

Results Summary for the ETKF Inflation Scheme Study:

• Compared to 3DVAR and ensemble-3DVAR, the GSI/ETKF regional hybrid gave statistically significant improvements of the fit to observations for the 12-hr forecasts.

• Compared to WG03, WG07, and BW08, the TRNK inflation scheme gave statistically significant improvements of the fit to observations for the 12-hr forecasts.

• All schemes had oscillations in the inflation factor and ensemble spread (due to changes in number of ETKF observations for one cycle to the next).

Page 14: 1 Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid Testbed Arthur P. Mizzi (mizzi@ucar.edu) NCAR/MMM HFIP Ensemble

14

Summary of Results for ETKF Inflation Scheme Study cont. (Not Presented):

• Holding the number of ETKF observations constant from one cycle to the next eliminated the oscillations.

• The ETKF observation reduction experiments showed that:

Small reductions did not have a significant impact on forecast skill.

Moderate reductions significantly improved the forecast skill.

Large reduction significantly degraded the forecast skill. Those results were due to the contraction and expansion of

spread in the ETKF.

Page 15: 1 Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid Testbed Arthur P. Mizzi (mizzi@ucar.edu) NCAR/MMM HFIP Ensemble

15

GSI ETKF/LETKF/DART-EnKF Regional Hybrid Cycling Experiments:

• 60 member ensemble.• 12-hr cycling (Aug. 15 to Sep. 11, 2007)• CONUS low resolution grid (200km)• ETKF – Wang et al. (2003), Wang et al. (2007)

and MMM TRNK inflation schemes.• LETKF – Loc = 3000 km, Inf = 1.036 Szunyogh et

al. (2005).• EnKF – Prior_Inf = 2,0, Inf_damping = 0.9,

Inf_sd_initial = Inf_sd_lower_bound = 0.6

Page 16: 1 Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid Testbed Arthur P. Mizzi (mizzi@ucar.edu) NCAR/MMM HFIP Ensemble

16

ETKF/LETKF/DART-EnKF Hybrids: Ensemble Spread – (Pre-Results)

• WG03 and WG07 gave large ensemble spread.• TRNK gave second largest ensemble spread.• DART-EnKF gave third largest ensemble spread.• LETKF gave smallest ensemble spread.

Page 17: 1 Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid Testbed Arthur P. Mizzi (mizzi@ucar.edu) NCAR/MMM HFIP Ensemble

17

ETKF/LETKF/DART-EnKF Hybrids: UPR ANAL RMSE – (Pre-Results)

• GSI/TRNK, GSI/WG07, GSI/LETKF, and DETR gave comparable fit of their analyses to the observations.• GSI/DART-EnKF gave slightly degraded fit of its analyses to the observations.

Page 18: 1 Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid Testbed Arthur P. Mizzi (mizzi@ucar.edu) NCAR/MMM HFIP Ensemble

18

ETKF/LETKF/DART-EnKF Hybrids: UPR ANAL BIAS – (Pre-Results)

• All hybrids and DETR gave comparable bias in their analyses.

Page 19: 1 Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid Testbed Arthur P. Mizzi (mizzi@ucar.edu) NCAR/MMM HFIP Ensemble

19

ETKF/LETKF/DART-EnKF Hybrids:UPR 12-hr FCT RMSE – (Pre-Results)

• All hybrids and DETR gave comparable fit of their 12-hr forecasts to the observations.

Page 20: 1 Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid Testbed Arthur P. Mizzi (mizzi@ucar.edu) NCAR/MMM HFIP Ensemble

20

ETKF/LETKF/DART-EnKF Hybrids:UPR 12-hr FCST BIAS – (Pre-Results)

• For u, v, and T at 850 mb, all hybrids and DETR gave comparable bias in their 12-hr forecasts. • For T at 500 mb, GSI/LETKF and GSI/DART-EnKF gave lower bias than the other schemes.• For q, GSI/TRNK gave lower bias than the other schemes.

Page 21: 1 Comparison of GSI-based ETKF, LETKF and DART-EnKF Hybrids from the MMM Regional Hybrid Testbed Arthur P. Mizzi (mizzi@ucar.edu) NCAR/MMM HFIP Ensemble

21

Summary for Comparison of GSI/ETKF, GSI/LETKF, and GSI/DART-EnKF

Regional Hybrids: The different hybrids gave differing amounts of ensemble

spread. The fit of the analyses to observations was comparable for

GSI/TRNK, GSI/WG07, GSI/LETKF, and DETR and was slightly degraded for GSI/DART-EnKF.

All schemes gave comparable analysis bias. All schemes gave comparable fit for their 12-hr forecasts

to observations. Generally, all schemes gave comparable forecast bias,

except that GSI/DART-EnKF and/or GSI/LETKF gave lower bias for T and/or q.