Deutscher Wetterdienst GRIB2 and its implementation in INT2LM and the COSMO-Model Ulrich Schättler Source Code Administrator COSMO-Model and (still) INT2LM

Deutscher Wetterdienst

GRIB2 and its implementationin INT2LM and the COSMO-Model

Ulrich Schättler

Source Code Administrator

COSMO-Model and (still) INT2LM


Contents Structure of GRIB2

Usage of grib_api

Local Use Areas

New: General Vertical Coordinate

21.04.23 WG 6 Meeting / Sibiu 2


GRIB2 ? - WMO - Code for forecast products

GRIdded Binary : Edition 0 (1985/88), Edition 1 (1990ff) General Regularly-distibuted Information in Binary form: Edition 2 (2001)

self explaining, compressed / packed GRIB message contains metadata (explaining the product, grid,…) plus data

Edition 2 is more flexible through a template/table structure,

ammendments by adding templates/tables.

Version 2: 2003 Version 3: 2005 Version 4: 2007 Version 5: 2008 Version 6: 2009 etc.


Changes include

• coding of new elements/parameter

• new product definition templates

• new packing/compressing formats

• new horizontal grid(s), e.g. ICON

• new vertical coordinate type Dörte Liermann


WG 6 Meeting / Sibiu 4

Comparing GRIB1 – GRIB2GRIB 1 vs. 2: Different StructureGRIB 1SECTION 0 Indicator (GRIB)

SECTION 1 Product definitionSECTION 2 Grid Description

SECTION 3 BitmapSECTION 4 Binary DataSECTION 5 End (7777)

GRIB 2SECTION 0 Indicator(GRIB,…)SECTION 1 IdentificationSECTION 2 Local UseSECTION 3 Grid DefinitionSECTION 4 Product DefinitionSECTION 5 Data RepresentationSECTION 6 BitmapSECTION 7 Binary DataSECTION 8 End (7777)

In GRIB 2 several variables are defined with more precision (e.g. angles are in micro-degrees, more vertical coordinates possible)

see: www.cosmo-model.org/content/model/documentation/grib/grib2keys.htm In GRIB 2 the description of the data (parameter, time, statistics,…) is more

complex and is template/table based

21.04.23

Dörte Liermann


Element coding of model fields in GRIB2 GRIB1: Element coding by ee (element) and tab (table)

Example: temperature is ee=11 in tab=2 (WMO)

GRIB2: Element coding by a triplet

Discipline

Category

Parameter

Example: temperature is (0,0,0)

Through setting of shortName, these three parameters are set (and for some products also some other keys; see shortName.def in definitions.edzw)


Dörte Liermann


GRIB_API (Application Programmers Interface)

ECMWF source code for de-/encoding of GRIB1 AND GRIB2 no internal knowledge of GRIB structure needed each element of a grib message has an alphanumeric name (key) that can be

used to access the information linked to it (value) key – value approach: shortName=T

typeOfLevel = hybridLayer

topLevel = 37

bottomLevel = 38 How to find keys: grib_keys –F file.grib lists all keys of a file flexible – local definitions for each centre possible

i.e. local definition tables shortName.def for edzw (GRIB1 and GRIB2), local definition of GME icosahedral grid for GRIB1

But: needs more coordination within COSMO


Dörte Liermann


Product Identifying Keys significanceOfReferenceTime (GRIB2-WMO Tab. 1.2)

productionStatusOfProcessedData (GRIB2-WMO Tab. 1.3)

typeOfProcessedData (GRIB2-WMO Tab. 1.4)

generatingProcessIdentifier (local definition in GRIB1/2)

backgroundGeneratingProcessIdentifier (local definition in GRIB2)

typeOfGeneratingProcess (GRIB2-WMO Tab. 4.3)

localDefinitionNumber (local section = 254, 253, 252; 250 for COSMO)

localNumberOfExperiment (GRIB2)

localInformationNumber (GRIB2) / localElementNumber (GRIB1)

localDecodeDate:s



Product Identifying Keys (II)

Key Values

significanceOfReferenceTime 0 analysis 1 start of forecast 2 verifying time of forecast 3 observation time

productionStatusOf ProcessedData

(possible local use)

0 operational – Routine 1 operational test – Parallelsuite 2 research – Experiments 3 re-analysis products

typeOfProcessedData 0 analysis1 forecast2 analysis and forecast3 control forecast4 perturbed forecast5 control and perturbed forecast



Product Identifying Keys (III)

Key Values

typeOfGeneratingProcess

(local use)

0 analysis 1 initialization2 forecast195 interpolated analysis / forecast

backgroundGeneratingProcessIdentifier

(local use)

0 main run1 pre-assimilation2 assimilation3 test

generatingProcessIdentifier

(local use)

Represents data base identifier



Local Use Section (localDefinitionNumber=254)

localDefinitionNumber Identifier for content (historical: 254)

localHostIdentifier

localCreationDateYear/Month/Day/Hour/Minute/Second

localValidityDateYear/Month/Day/Hour/Minute/Second

localNumberOfExperiment Number of Experiment

localInformationNumber

Identifier for host system/ computer

Also need a local use section for COSMO (localDefinitionNumber = 250)



Implementation of Local Use

SELECT CASE (ncenter)

CASE (78) ! DWD

…

CASE DEFAULT

END SELECT

We soon need to know about your local use of special keys!

We want to document local use on the Web Page!



General Vertical Coordinate

typeOfLevel = 150


Why a new vertical coordinate? As a non-hydrostatic model, COSMO needs a special vertical grid: fixed in space

and time

Also post-processing programs have to be aware of this grid (or the HHL)

But the algorithm to compute it, is rather complex (not just ak + bk * ps)

Therefore a proposition was made to WMO, to introduce a new typeOfLevel=150

To process atmospheric data using that typeOfLevel, another 3D field is necessary: the HHL fields

If a product has typeOfLevel=150, then there are 6 additional meta data in the Product Definition Section, which replace the vertical coordinate parameters

numberOfVGridUsed to identify a special vertical coordinate

nlev number of levels of the HHL file

uuidOfVGrid: unique universal identifier

to ensure correct identification of HHL



Current Situation INT2LM and the COSMO-Model both compute the HHL fields and the

reference atmosphere p0.

The necessary vertical coordinate parameters (for HHL) and the reference atmosphere parameters are given to

INT2LM by Namelist variables

COSMO-Model by GRIB1 meta data: but this always was a non-standard GRIB usage!

The new generalized vertical coordinate does not know meta data for vertical coordinate parameters and for the reference atmosphere.

How to compute HHL and p0 when using GRIB2?

Or what can we do?



What can be done? HHL can be transferred from INT2LM to COSMO or within the assimilation

cycle from COSMO to COSMO by an extra file.

Do we then still need to transfer the vertical coordinate parameters vcoordk and vcflat?

YES! For computations in the nudging and some diagnostices (meanvalues, convective indices). Try to discuss this with the code experts.

Can we also transfer p0? Or the full pressure?

For full pressure the problem is the GRIB packing. But we could use more bits for the packing (pressure deviation is a more homogeneous field with less precision loss in the packing)

Other question: Has anybody ever changed the Namelist variables in INT2LM (p0sl, t0sl, dt0lp, delta_t, h_scal) to compute another p0?



Current implemented Method for HHL INT2LM can compute HHL and write an extra file COSMO_HHL_xxx.g2 (if

lnewVGrid = .TRUE.) or read it from such a file (is done in IEEE 64-bit)

(Note: for COSMO COSMO such a HHL file has also to be read for the coarse grid)

INT2LM computes the atmospheric variables based on this HHL

The COSMO-model has to read the same COSMO_HHL_xxx.g2 file to get the same HHL field as the INT2LM (checking the UUID should ensure using the correct HHL field)

INT2LM and the COSMO-Model define several sets of vertical coordinate parameter defaults (for COSMO-EU, COSMO-DE, COSMO-I7, COSMO-7, etc.). Every set has a special ID

HHL stores the ID of the default set used in the localInformationNumber in HHL



Current implemented Method for p0

INT2LM and the COSMO-Model both compute p0 (as before).

To get the correct reference atmosphere parameters, both programs store several sets of default parameters. Every set has a special ID

pp stores the ID of the used set in the local information number

INT2LM and the COSMO-Model now use the new module vgrid_refatm_defaults.f90



Problems and Drawbacks of this Approach Have to deal with another file in our suites

Have to hard-code all possible COSMO configurations in the source code



What can be done about HHL Can we transfer HHL through the laf-file? Perhaps in higher precision? But

this imposes problems to archiving!

Do we need full precision?

Check , whether usage of vcoordk, vcflat is really necessary or can be avoided



What can be done about p0

Can we transfer full pressure (or p0) through the laf-file? Perhaps in higher precision? Again this imposes problems to archiving!

Which precision is really needed?

Check , whether usage of reference atmosphere parameters can be „fixed“



Migration to GRIB2: Timetable The plans already pronounced by DWD have been revised last week

A prerequisite for GRIB2 at DWD is the stable operation of the new database software SKY (old software is not able to process GRIB2 meta data). And this is not the case at the moment.

New date for operational use of SKY („only the SKY is the limit“): November

If this date can be kept, then we could switch to GRIB2 in January 2014. If not, GRIB2 will not come before May / June 2014. Because in February / March we are migrating to the new CRAY computer system.

Message for COSMO and other partners:

Be prepared to read GME data in GRIB2 with INT2LM



21.04.23 WG 6 Meeting / Sibiu 2210.09.2012 COSMO General Meeting 2012 22

Thank you

very much

for your

attention

Documents

Deutscher Wetterdienst GRIB2 and its implementation in INT2LM and the COSMO-Model Ulrich Schättler Source Code Administrator COSMO-Model and (still) INT2LM