GRIB2 usage at NCEP

GRIB2 usage at NCEP

Dr. V. KRISHNA KUMAR &

BOI VUONGNOAA/NWS/NCEP/Central Operations

NCEP WWB GRIB2 Seminar20 December 2010

Room 209, WWB Camp Springs, MD

Overview• GRIB1 and GRIB2 Sections

• GRIB2 Documentation• Format differences between GRIB1 and GRIB2• GRIB2 in NCEP Operations – Encoding & Decoding

routines, NCEP Utilities (cnvgrib, degrib2, overdateg2, overmodelg2, copygb2……..)

• NCEP Transition to GRIB2 and NWS FTP Servers that provide GRIB2 data

• GRIB2 – Section 7 Data Section • GRIB2 – Packing Options• How to diagnose a Degrib2 output from NCEP GRIB2

tables?• How to write GRIB2 data directly? – A specific example• NCO GRIB2 Templates software

• Parallel GRIB2 in NCEP Post• GRIB2 Options in POST• Parallel generating and writing GRIB2 messages• Parallel GRIB2 results• GRIB2 Encoding routines• GRIB2 Decoding routines• Declaration of Derived type “gribfield”• How to use routine “getgb2”?• How to use routine “putgb2”?• GRIB2 read and write examples• Summary and Conclusions

Overview

GRIB1 Sections

0 – Indicator Section - ‘GRIB’ (octet 1-4 = 32bits), length of message (octet 5-7=24bits), edition number -1 (octet 8=8bits) - 1 octet = 1byte = 8bits - octets are numbered 1, 2, 3, etc., Bit positions within octets are from bit 1 to bit 8. - bit 1 = most significant bit & bit 8 = least significant)

1 – Product Definition Section (PDS) - Length of PDS, parameter table version number, ID of center, generating process ID, grid

identification (Table B), Ref time, parameter and units, level, valid time …

2 – Grid Definition Section (GDS) – optional - Length of sec., sec. no, defn of grid surf and geometry of data values within the surface

3 – Bit-Map Section (BMS) – optional - Length of sec., sec. no, indication of presence or absence of data at each of the grid points, as

applicable example sst

4 – Binary Data Section (BDS) - Length of sec., sec. no, data values

5 – End Section = ‘7777’

GRIB2 Sections

• 0 – Indicator Section - ‘GRIB’, length, version, discipline – (16 octets long)

• Section 1 – Identification - Ref time, orig center, table versions, …(length of the section is expressed as the first four octets (32 bits)

-the section number in the fifth octet and octets beyond 21 are reserved for future use)

• 2 – Local Use Section – optional ==================

• 3 – Grid Definition Section ======= |• 4 – Product Definition Section ==== | |

- parameter, level, valid time | | |

• 5 – Data Representation Section | | | - packing algorithm used, info needed to unpack | | |

• 6 – Bit Map Section | | | • 7 – Data Section ==================

-packed data

• 8 – End Section = ‘7777’

GRIB2 – Documentation• NCEP GRIB2 Documentation (internet - www)http://www.nco.ncep.noaa.gov/pmb/docs/grib2/http://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc.shtm

• NCEP GRIB2 Resources (Detailed Documentation) (intranet – www2) http://www2.nco.ncep.noaa.gov/sib/grib2/

• GRIB2 specification from WMO contains template definitions and table entries: http://www.wmo.ch/web/www/DPS/grib-2.html

• Local Tables: http://www.nco.ncep.noaa.gov/pmb/docs/grib2 • NCEP source files and utilities on the web for decoding and

encoding data in GRIB2 format http://www.nco.ncep.noaa.gov/pmb/codes/GRIB2/

GRIB1 & GRIB2 Major Differences

• One GRIB2 message can contain multiple fields. - Sequences of GRIB sections 2 to 7, sections 3 to 7 or sections 4 to 7 may be repeated within a single GRIB message.

- Once a section is repeated all the following sections must appear in order. - A section remains in effect until redefined. - Could be used for vector fields such as u & v components, or ensemble members - IEEE single precision floating point representation used instead of standard IBM

representation. The representation occupies four octets.

• GDS, PDS, DRS, and DS are all template driven. Octet assignments within a section are based on the Template specified.

• Parameters are defined by three values: Discipline (specified in Section 0) Category (PDS) Parameter number (PDS).

GRIB1 and GRIB2 Major differences contd.

• Table versions, originating center, initial reference time and grid ID removed from GRIB1 PDS. Most are now in Identification Section.

• All longitudes MUST be in degrees East and be between 0 and 360 degrees. No negatives. No degrees West. No change in latitude definitions –900 to 900.

• GDS and BMS no longer optional. They must be included. Only Section 2 (Local Use) is optional.

• Latitudes, Longitudes are in units of micro-degrees (10 –6 degrees). Includes distance between grid points for Lat/Lon grids (GRIB1 was

milli-degrees)• All other grid point increments are in millimeters (GRIB1 was in meters)• Floating point values are now stored in IEEE format (GRIB1 was IBM

360/195 format HDS)

GRIB2 in NCEP operations

Encoding Routines

GRIB2 message can contain gridded fields for many parameters on a number of different grids.

Start a new GRIB2 message with a call to:

(1) call GRIBCREATE – encodes sections 0 and 1 at the beginning of message.

(2) call ADDLOCAL – adds a Local Use Section (Section 2) optional

(3) call ADDGRID – encodes a grid definition into Section 3. Defines the geometry of the data values in the fields that follow it. ADDGRID can be called again to change the grid definition describing subsequent data fields.

(4) call ADDFIELD – adds each data field which adds Sections 4, 5, 6, and 7 to the message.

(5) call GRIBEND – adds the final section 8 to the message which updates the length of the message. GRIBEND is required for each GRIB2 message. Boi will demonstrate these routines with examples in the latter half of this seminar

GRIB2 in NCEP operations contd.Decoding Routinescall GB_INFO – finds out how many Local Use sections and data fields are contained in a given GRIB2

message. Also returns the number of octets of the largest Local Use section in the message. Ensures the dimensionality of the input array of GETLOCAL

call GETLOCAL – returns the requested occurrence of Section 2 from a given GRIB2 message.

call GF_GETFLD – gets all information pertaining to the nth data field in the message – returns all the unpacked values for each Section and Template in a f90 “type gribfield” with options to unpack the Bit-map (if applicable) and the data values or just return the field description information.

call GF_FREE(gfld) – frees up memory held by gfld as many elements in derived type gribfield are dynamically allocated when decoded.

Example: use grib_mod type(gribfield) ::gfld … … call gf_getfld(cgrib,lengrib,1,.true.,.false.,gfld,ierr) print*,’Num of grid points =‘,gfld%ngrdpts print*,’First data value=‘,gfld%fld(1) call gf_free(gfld)

How to extract GRIB2 fields from a GRIB2 file?

GETGB2 – extracts a specified field from a file containing many GRIB2 messages.

Boi will demonstrate these with examples in the latter half of this seminar.

GRIB2 in NCEP operations contd.

Create GRIB2 Messagecall PUTGB2 – if a variable of derived type of “gribfield” already exists, this subroutine can be used to pack it up and write it to a file – Boi will demonstrate this

NCEP GRIB2 Utilities /grb2index grib2file grib2indexfile

- creates a GRIB2 index file /cnvgrib –h (packing options: 1 => 2, 2=>1, 2=>2) -converts all fields in a file between GRIB1 and GRIB2 /degrib2 grib2file -dumps the meta-data for each GRIB2 field in grib2file - basically prints contents of each element in derived type gribfield/wgrib2 grib2file

Grads for GRIB2 sh convert_grib2_to_grads.sh#!/bin/shset -xa#file_list=`ls grib2.t00z.wafs_grbf06.45 gfs.t00z.wafs_grb45f06.grib2`for list in $file_listdo# /usrx/local/grads/bin/g2ctl -verf $list > ${list}.ctl /usrx/local/grads/bin/g2ctl $list > ${list}.ctl /usrx/local/grads/bin/2.0.a3/xgribmap -i ${list}.ctldoneexit

GRIB2 in NCEP Operations$ /nwprod/util/exec/cnvgrib -hUsage: cnvgrib [-h] {-g12|-g21|-g22} [-m] [{-p0|-p2|-p31|-p32|-p40|-p41|-p40000|-p40010}] ingribfile outgribfilecnvgrib: version cnvgrib-1.1.1Must use one of the following options: -g12 converts GRIB1 to GRIB2 -g21 converts GRIB2 to GRIB1 -g22 converts GRIB2 to GRIB2 (used to change packing option)Optional packing options: (for use with -g12 and -g22 only) -p0 simple packing -p2 complex packing -p31 complex pack with 1st order diffs -p32 complex pack with 2nd order diffs -p40 JPEG2000 encoding -p41 PNG encodingOther Optional options: -m Use missing values instead of bitmap

(valid with -p2, -p31 or -p32 options only)

GRIB2 in NCEP Operations/nwprod/util/sorc/overmodelg2.fd

$ more sample.script

# THIS SCRIPT READS A FORECAST FILE (UNIT 11), REPLACES MODEL INDENTIFIER IN

# PRODUCT TEMPLATE 4.x FROM A ID FILE UNIT 51.

# STANDARD INPUT IS A ID (2-DIGIT INTEGER), FOR EXAMPLE: 96 (GFS Model)

set -x

export XLFRTEOPTS="unit_vars=yes" # Allow overriding default names.

export XLFUNIT_11=gfs.t00z.pgrb2f12.2p5deg

export XLFUNIT_51=gfs.t00z.pgrb2f12.2p5deg.new

overmodelg2 << EOF

98

EOF

/nwprod/util/sorc/overdateg2.fd# THIS SCRIPT READS A FORECAST FILE (UNIT 11), MODIFIES REFERENCE TIME YEAR MONTH

# DATE AND HOUR IN SECTION 1 AND IN PRODUCT TEMPLATE 4.x TO REPLACE INTERNAL

# REFERENCE TIME FROM A DATE IN FILE UNIT 51.

# STANDARD INPUT IS A YYYYMMDDHH (10-DIGIT INTEGER), FOR EXAMPLE 2010090112

set -x

export XLFRTEOPTS="unit_vars=yes" # Allow overriding default names.

export XLFUNIT_11=gribfile

export XLFUNIT_31=gribfile_indx

export XLFUNIT_51=gribfile.new

overdateg2 << EOF

2010090112

EOF

GRIB2 in NCEP Operations/nwprod/util/sorc/copygb2.fdNAME

copygb2 - copies GRIB2 files

SYNOPSIS

copygb2 [-g "kgdtn [kgdt]"] [-i "ip [ipopts]"]

[-k "kpdtn kpdt"] [-v "uparms"] [-X]

[-B mapgrib [-b mapindex] [-A "<> mapthreshold"] [-K "mapkpds"]]

[-M "mask"|mergegrib [-m mergeindex]] [-a]

then either:

grib2in index2in grib2out

or:

-x grib2in grib2out

DESCRIPTION

The command copygb2 copies all or part of one GRIB2 file to another

GRIB2 file, interpolating if necessary. Unless otherwise directed

(-x option), the GRIB2 index file is also used to speed the reading.

The fields are interpolated to an output grid if specified (-g option).

The interpolation type defaults to bilinear but may be specified directly

(-i option). The copying may be limited to specific fields (-k option).

It may also be limited to a specified subgrid of the output grid

or to a subrange of the input fields (-B and -b, -A, and -K options).

Fields can be identified as scalars or vectors (-v option), which are

interpolated differently. The invalid data in the output field can be

filled with mask values or merged with a merge field (-M and -m options).

The output GRIB2 message can also be appended to a file (-a option).

If grib2out is specified as '-', then the output GRIB file is written

to standard output.

GRIB2 in NCEP Operations#

# Script to convert a RTMA 2p5 grib2 file to a 5p0 grib2 file

#

# Author: V. Krishna Kumar October 5, 2010 NCO/Systems Integration Branch

#

set -x

#

# clean the output file prior to running this script

#

# I created a single field grib2 file from the original production grib2 file using

# wgrib2 -d 1 in_grb2file -grib out_grb2file

#

rm rtma5p0.t12z.2dvaranl_ndfd.1field.grb2

#

# HGT - Conversion from 2p5 to 5

# Input file : rtma2p5.t12z.2dvaranl_ndfd.1field.grb2

# Output file : rtma5p0.t12z.2dvaranl_ndfd.1field.grb2

#

# 2p5 Grid template 3.30 : 6 0 0 0 0 0 0 2145 1377 20191999 238445999 8 25000000 265000000 2539703 2539703 0 64 25000000 25000000 -90000000 0

# 5p0 Grid template 3.30 : 6 0 0 0 0 0 0 1073 689 20191999 238445999 8 25000000 265000000 5079406 5079406 0 64 25000000 25000000 -90000000 0

export grid_new="30 6 0 0 0 0 0 0 1073 689 20191999 238445999 8 25000000 265000000 5079406 5079406 0 64 25000000 25000000 -90000000 0 0"

export grib_in=rtma2p5.t12z.2dvaranl_ndfd.1field.grb2

#

# Create a grib2 index file of the input grib file

#

/nwprod/util/exec/grb2index $grib_in ${grib_in}_ix

export grib_index=${grib_in}_ix

/nwprod/util/exec/copygb2 -g "${grid_new}" -i0 ${grib_in} ${grib_index} rtma5p0.t12z.2dvaranl_ndfd.1field.grb2

echo "Max Min value from the original 2p5 grid "

/nwprod/util/exec/wgrib2 -grid -max -min rtma2p5.t12z.2dvaranl_ndfd.1field.grb2

echo "Max Min value from the converted 5p0 grid "

/nwprod/util/exec/wgrib2 -grid -max -min rtma5p0.t12z.2dvaranl_ndfd.1field.grb2

echo " Max Min from the current operational 5 degree grid "

/nwprod/util/exec/wgrib2 -grid -max -min rtma.t12z.2dvaranl_ndfd.1field.grb2

exit

GRIB MESSAGE 1 starts at 1

SECTION 0: 0 2 60463 SECTION 1: 7 0 2 1 1 2010 11 9 6 0 0 0 1 Contains 0 Local Sections and 1 data fields.

FIELD 1 SECTION 0: 0 2 SECTION 1: 7 0 2 1 1 2010 11 9 6 0 0 0 1 SECTION 3: 0 65160 0 0 0 GRID TEMPLATE 3. 0 : 6 0 0 0 0 0 0 360 181 0 0 90000000 0 48 -90000000 359000000 1000000 1000000 0 NO Optional List Defining Number of Data Points. PRODUCT TEMPLATE 4. 0 : 3 5 2 0 96 0 0 1 36 100 0 100000 255 0 0 TEXT: HGT 1000 mb valid at 36 hr after 2010110906:00:00 NO Optional Vertical Coordinate List. Num. of Data Points = 65160 NO BIT-MAP DRS TEMPLATE 5. 3 : -979087360 0 1 11 0 1 0 0 0 5124 0 4 1 1 32 5 1 2 Data Values:MIN= -525.79998779 AVE= 70.36888123 MAX= 313.10000610



FIELD 1 SECTION 0: 0 2 SECTION 1: 7 0 2 1 1 2010 11 9 6 0 0 0 1 SECTION 3: 0 65160 0 0 0 GRID TEMPLATE 3. 0 : 6 0 0 0 0 0 0 360 181 0 0 90000000 0 48 -90000000 359000000 1000000 1000000 0 NO Optional List Defining Number of Data Points. PRODUCT TEMPLATE 4. 0 : 3 5 2 0 96 0 0 1 36 100 0 97500 255 0 0 TEXT: HGT 975 mb valid at 36 hr after 2010110906:00:00 NO Optional Vertical Coordinate List. Num. of Data Points = 65160 NO BIT-MAP DRS TEMPLATE 5. 3 : -984977408 0 1 11 0 1 0 0 0 5130 0 4 1 1 32 5 1 2 Data Values:MIN= -323.89999390 AVE= 278.48535156 MAX= 514.00000000



FIELD 1 SECTION 0: 0 2 SECTION 1: 7 0 2 1 1 2010 11 9 6 0 0 0 1 SECTION 3: 0 65160 0 0 0 GRID TEMPLATE 3. 0 : 6 0 0 0 0 0 0 360 181 0 0 90000000 0 48 -90000000 359000000 1000000 1000000 0 NO Optional List Defining Number of Data Points. PRODUCT TEMPLATE 4. 0 : 2 2 2 0 96 0 0 1 36 100 0 100000 255 0 0 TEXT: U GRD 1000 mb valid at 36 hr after 2010110906:00:00 NO Optional Vertical Coordinate List. Num. of Data Points = 65160 NO BIT-MAP DRS TEMPLATE 5. 3 : -1014333440 0 1 9 0 1 0 0 0 4835 0 4 1 1 32 5 1 2 Data Values:MIN= -27.70000076 AVE= .24823742 MAX= 27.20000076

FIELD 2 SECTION 3: 0 65160 0 0 0 GRID TEMPLATE 3. 0 : 6 0 0 0 0 0 0 360 181 0 0 90000000 0 48 -90000000 359000000 1000000 1000000 0 NO Optional List Defining Number of Data Points. PRODUCT TEMPLATE 4. 0 : 2 3 2 0 96 0 0 1 36 100 0 100000 255 0 0 TEXT: V GRD 1000 mb valid at 36 hr after 2010110906:00:00 NO Optional Vertical Coordinate List. Num. of Data Points = 65160 NO BIT-MAP DRS TEMPLATE 5. 3 : -1015873536 0 1 9 0 1 0 0 0 4934 0 4 1 1 16 5 1 2 Data Values:MIN= -24.30000114 AVE= .03596115 MAX= 30.39999962


GRIB2 in NCEP OperationsCurrently about 90-95 % of the GRIB2 products are generated using CNVGRIB

from GRIB1 files. Users are encouraged to generate GRIB2 files directly for all new operational products. Source codes in NCEP operations that generate GRIB2 products directly: /nwprod/sorc

$ ls -d *grb2* *grib2* *rtma*post*fd

cdas_ex_grib2.cd mdl_grb2mdlp.fd mdl_ra2grib2.fd sstoiqd_grib2sst.fd

gfdl_mkgrib2n.fd mdl_grb2tomdlp.fd multiwavegrib2.fd wavesteepgrb2.fd

akrtma_post.fd hirtma_rtmapost.fd rtma2p5_post.fd

gurtma_post.fd prrtma_rtmapost.fd rtma_post.fd

See Manuel Pondeca’s – rtma_post.fd (well written code) cnv_to_grib2.f for GRIB2 creation

/nwprod/util/sorc

$ ls -d *grib2* *grb2*

degrib2.fd grb2index.fd tocgrib2.fd wgrib2.cd

TRANSITION TO GRIB2 ON NCEP AND NWS FTP SERVERS http://www.nco.ncep.noaa.gov/pmb/docs/GRIB1_to_GRIB2.shtml Current GRIB2 data sets available on the TOC ftp servers. Access

ftp tgftp.nws.noaa.gov through anonymous FTP and go to: For NAM:

/SL.us008001/ST.opnt/MT.nam_CY.CC/RD.YYYYMMDD/PT.grid_DF.gr2/ For GFS:

/SL.us008001/ST.opnt/MT.nam_CY.CC/RD.YYYYMMDD/PT.grid_DF.gr2/ where CC is the cycle (00,06,12,18) and YYYYMMDD is the date.

GRIB2 Section 7 – Data Section

• Data coded with the minimum number of bits necessary to provide the accuracy.

• Simple scaling – data multiplied by an appropriate power of 10 before forming the non-negative differences, and then using the binary scaling to select the precision of the transmitted value.

• Data coded in the form of non-negative scaled differences from a reference value of the whole field plus, if applicable, a local reference value. (reference value is normally the minimum value of the data set which is represented).

GRIB2 Section 7 – Data Section contd.

• Complex packing for grid-point values – split the whole set of scaled data values into groups on which local references (local minima) are removed – pre-processing may be needed on the scaled data.

• Complex packing for spectral data.• The original data value Y can be recovered with the

formula Y*10D=R+(X1+X2)*2E

where E=Binary scale factor, D=Decimal scale factor, R=Reference value of the whole field, X1=0, X2=Scaled (encoded) value.

GRIB2 – Packing Methods

• GRIB2_com – Group packing – DR Template 5.2• GRIB2_1od – Group packing –with 1st order differencing Template

5.3• GRIB2_2od – Group packing – with 2nd order differencing

Template 5.3

Image compression algorithms within the GRIB2 standard.

Both treat grid point data as a grey scale image (single component)

• JPEG_2000 (Part-1 standard) (ISO/IEC 15444-1)

http://www.jpeg.org/JPEG2000.html • PNG (Portable Network Graphics image compression format)

http://www.libpng.org/pub/png/.

How to diagnose a degrib2 output from NCEP GRIB2 tables? Example

SECTION 0: 0 2 201721

SECTION 1: 9 170 2 1 1 2007 10 23 23 0 0 0 1

Contains 0 Local Sections and 1 data fields.

FIELD 1

SECTION 0: 0 2

SECTION 1: 9 170 2 1 1 2007 10 23 23 0 0 0 1

SECTION 3: 0 987601 0 0 20

GRID TEMPLATE 3. 20 : 0 0 0 0 0 0 0 1121 881 23097000 240964000 8 60000000 255000000 4762000 4762000 0 64

NO Optional List Defining Number of Data Points.

PRODUCT TEMPLATE 4. 8 : 1 8 2 0 190 0 0 1 0 1 0 0 255 0 0 2007 10 24 0 0 0 1 0 1 2 1 1 255 0

FIELD: A PCP Surface valid 0 hr before 2007102323:00:00 to 2007102400:00:00

NO Optional Vertical Coordinate List.

Num. of Data Points = 987601 with BIT-MAP 0

DRS TEMPLATE 5. 2 : 0 -3 0 7 0 1 0 0 0 12587 0 3 1 1 49 10

Data Values:

MIN= .00000000 AVE= .14565143 MAX= 19.25000000

Total Number of Fields Found = 1

$


SECTION 0: 0 2 1111248

SECTION 1: 9 170 2 1 1 2007 10 23 23 0 0 0 1

Contains 0 Local Sections and 1 data fields.

-----------------------------------------------------------

FIELD 1

SECTION 0: 0 2

0 = GRIB2-Table0.0 Discipline Section 0, Octet7 => Meteorological Products

2 = GRIB2 - Section 0 Indicator Section Octet8 => Edition Number

------------------------------------------------------------

SECTION 1: 9 170 2 1 1 2007 10 23 23 0 0 0 1

9 = ON388-Table0 Grib2 (National/International Orig Ctr) - Section1 , Octets6-7 => US NWS -

Other (WMC)

170 = ON388-TableC Grib2 (National Subcenters) - Section 1, Octets8-9 => Locally defined by originating center

How to diagnose a degrib2 output from NCEP GRIB2 tables? Example 2 = GRIB2-Table1.0 Section1, Octet10 => Future

Operational Version Numbers (2-254)

1 = GRIB2-Table1.1 GRIB Local Tables Version Number Section 1, Octet 11 => Number of local table version used

1 = GRIB2-Table1.2 Significance of reference time Section 1, Octet12 => Start of Forecast

2007 10 23 23 0 0 => year,month,day hour minutes second

0 = GRIB2-Table1.3 Production Status of Data Section1, Octet20 => Operational products

1 = GRIB2-Table1.4 Type of data Section 1, Octet 21 => Forecast Products

----------------------------------------------------------------------

SECTION 3: 0 987601 0 0 20

*********************************

From GRADS Analysis of the file GRIB2400 (GRIB1)

Gridded data that are mapped onto a particular map

projection are called ‘pre-projected’. An example of pre-

projected data is the output from a weather forecast model

that is mapped onto a north polar stereographic grid.

Number of data points = 987601 => dtype grib 255

pdef 1121 881 nps 401.03817080455 1601.18158182199

-105.000000 4.762

PDEF isize jsize NPS ipole jpole lonref gridinc

PDEF isize jsize SPS ipole jpole lonref gridinc

Example: PDEF 1121 881 nps 401.03817080455 1601.18158182199 -105.000000 4.762

Args:

isize The size of the native grid in the x direction

jsize The size of the native grid in the y direction

ipole the i coordinate of the pole referenced from the lower left corner, assumed to

be at (1,1)

jpole the j coordinate of the pole referenced from the lower left corner, assumed to

be at (1,1)

lonref reference longitude

gridinc distance between gripoints in km. Value should be negative for Southern Hemisph

ere data.

Notes: Polar stereographic projections (N and S) are defined as at NCEP. Wind rotation

has also been

added so that vector data will be properly displayed.

***********************

How to diagnose a degrib2 output from NCEP GRIB2 tables? Example 0 = GRIB2-Table3.0 Source of Grid Definition Section3,

Octet 6 => Specified in Table 3.1 (Grid Definition

template number)

987601 = GRIB2-Section3 Grid Definition Section Octet7-10 Number of data points

0 = GRIB2-Section3 Grid Definition Section Octet 11 => Number of octets for optional list of numbers defining number of points (See note 2 below)

0 = GRIB2-Section3 Grid Definition Section Octet 12 => Interpolation of list of numbers defining number of points (See Table 3.11 Interpretation of list of numbers defining number of points) Section 3, Octet 12 => There is no appended list

20 = GRIB2-Section3 Grid Defintion Section Octets13-14 Grid definition template number(=N) (See Table3.1 -GRIB2-Table3.1 Grid Definition Template Number Section3,Octets13-14) => Polar Stereographic Projection (North or South) (See Template 3.20)

GRID TEMPLATE 3. 20 : 0 0 0 0 0 0 0 1121 881 23097000 240964000 8 60000000 255000000 4762000

4762000 0 64

GRIB2 - GRID Template 3.20 Polar stereographic projection has 18 values from octets 15-65

0 = GRID Template 3.20 Octet15 - Shape of the Earth (see Table3.2) => Earth assumed spherical with radius = 6,367,470.0 m

0 = GRID Template 3.20 Octet16 => Scale factor of radius of spherical Earth

0 = GRID Template 3.20 Octet17-20 => Scaled value of radius of spherical Earth

0 = GRID Template 3.20 Octet21 => Scale factor of major axis of oblate spheroid Earth

0 = GRID Template 3.20 Octet22-25 => Scaled value of major axis of oblate spheroid Earth

0 = GRID Template 3.20 Octet26 => Scale factor of minor axis of oblate spheroid Earth

0 = GRID Template 3.20 Octet27-30 => Scaled value of minor axis of oblate spheroid Earth

1121 = GRID Template 3.20 Octet31-34 => Nx - number of points along the x-axis

881 = GRID Template 3.20 Octet35-38 => Ny - number of points along the y-axis

23097000 = GRID Template 3.20 Octet39-42 => La1 -latitude of first grid point in millidegrees (10**(-6))

240964000 = GRID Template 3.20 Octet43-46 => Lo1 - longitude of first grid point in millidegrees (10**(-6))

8 = GRID Template 3.20 Octet 47 => Resolution and component flags (see Flag Table 3.3 and Note 1)

Note1-The resolution flags (bits3-4 Flag table3.3) are not applicable

60000000 = GRID Template 3.20 Octet48-51 => LaD - latitude where Dx and Dy are specified (in millidegrees)

How to diagnose a degrib2 output from NCEP GRIB2 tables? Example 255000000 = GRID Template 3.20 Octet52-55 => LoV –

orientation of the grid (see Note2)

Note2-LoV is the longitude value of the meridian which is

Parallel to the y-axis (or columns of the grid) along

which latitude increases as the y-coordinate increases (the

orientation longitude may or may not appear on a

particular grid)

4762000 = GRID Template 3.20 Octet56-59 => Dx

direction grid length (see Note2)

4762000 = GRID Template 3.20 Octet60-63 => Dy - y-

direction grid length (see Note3)

Note3-Grid length is in units of 10**(-3) m at the latitude

specified by LaD.

0 = GRID Template 3.20 Octet64 => Projection centre

flag (see Flag Table 3.5) (see Note4)

Note4-Bit2 of the projection flag is not applicable to the

polar stereographic projection.

64 = GRID Template 3.20 Octet65 => Scanning mode (see Flag Table 3.4)

----------------------------------------------------------------------

NO Optional List Defining Number of Data Points.

PRODUCT TEMPLATE 4. 8 : 1 8 2 0 190 0 0 1 0 1 0 0 255 0 0 2007 10 24 0 0 0 1 0 1 2 1 1 255 0

29 values from GRIB2 - GRID Template 4.8

Product definition template - average, accumulation and/or extreme values or other statistically-processed

values at a horizontal level or in a horizontal layer in a continuous or non-continuous time interval.

1 = GRIB2 - GRID Template 4.8 Octet10 - Parameter category (see Code Table 4.1) Parameter Category by product discipline section 4, octet 10 => Moisture (see Table 4.2-0-1)

8 = GRIB2 - GRID Template 4.8 Octet11 – Parameter number (see Code Table 4.2) => Total Precipitation

(see Table 4.2-0-1) kg m**(-2) APCP

2 = GRIB2 - GRID Template 4.8 Octet12 - Type of generating process (see Code Table 4.3) => Forecast

0 = GRIB2 - GRID Template 4.8 Octet13 - Background generating process identifier (defined by originating center)

190 = GRIB2 - GRID Template 4.8 Octet14 - Analysis or forecast generating process identified (see Code ON388 Table A) => National Convective Weather Diagnostic generated by NCEP/AWC

How to diagnose a degrib2 output from NCEP GRIB2 tables? Example 0 = GRIB2 - GRID Template 4.8 Octet15-16 - Hours after

reference time data cutoff (see Note 1)

Note1 - Hours greater than 65534 will be coded as 65534

0 = GRIB2 - GRID Template 4.8 Octet17 - Minutes after reference time data cutoff (see Note1)

1 = GRIB2 - GRID Template 4.8 Octet18 - Indicator of unit of time range (see Code Table4.4) => Hour

0 = GRIB2 - GRID Template 4.8 Octet19-22 - Forecast time in units defined by octet 18 (see Note2)

Note2-The reference time in section 1 and the forecast time

together define the beginning of the overall time interval.

1 = GRIB2 - GRID Template 4.8 Octet23 - Type of first fixed surface (see Code Table4.5) => Ground or water surface

0 = GRIB2 - GRID Template 4.8 Octet24 - Scale factor of first fixed surface

0 = GRIB2 - GRID Template 4.8 Octet25-28 - Scaled value of first fixed surface

255 = GRIB2 - GRID Template 4.8 Octet29 - Type of second fixed surface (see Code Table 4.5) => Missing

0 = GRIB2 - GRID Template 4.8 Octet30 - Scale factor of second fixed surface

0 = GRIB2 - GRID Template 4.8 Octet31-34 - Scaled value of second fixed surfaces

2007 10 24 0 0 0 = GRIB2 -GRID Template 4.8 Octet35-36(year) 37(month) 38(day) 39(hour) 40(minute) 41(second)

1 = GRIB2 - GRID Template 4.8 Octet42 - n - number of

time ranges specifications describing the time intervals used

to calculate the statistically-processed field

0 = GRIB2 - GRID Template 4.8 Octet43-46 Total number

of data values missing in statistical process 47-58

specification of the outermost (or only) time range over which

statistical processing is done.

1 = GRIB2 - GRID Template 4.8 Octet47 - Statistical

process used to calculate the processed field from the field

at each time increment during the time range (see Code Table

4.10) => Accumulation

2 = GRIB2 - GRID Template 4.8 Octet48 - Type of time

increment between successive fields used in the statistical

processing (see Code Table 4.11) => Successive times

processed have same start time of forecast, forecast time is

incremented.

1 = GRIB2 - GRID Template 4.8 Octet49 - Indicator of unit

of time for time range over which statistical processing

is done (see Code Table 4.4) => Hour

How to diagnose a degrib2 output from NCEP GRIB2 tables? Example 1 = GRIB2 - GRID Template 4.8 Octet50-53 - Length of

time range over which statistical processing is done, in units

defined by the previous octet => same value as for Octet49

255 = GRIB2 - GRID Template 4.8 Octet54 - Indicator of

unit of time for the increment between the successive fields

used (see Code Table 4.4) => Missing

0 = GRIB2 - GRID Template 4.8 Octet55-58 - Time increment between successive fields, in units defined by the previous octet (see Notes 3 and 4) 59-nn These octets are included only if n>1, where nn=46+12 x n

Note3-An increment of zero means that the statistical processing is the result of continuous (or near continuous) process, not the processing of a number of discrete samples. ......

Note4-The reference and forecast times are successively set to their initial plus or minus the increment, as defined by the type of time increment (one of octets 48,60,72...). ........

----------------------------------------------------------------------

TEXT: A PCP Surface valid 0 hr after 2007102323:00:00 to 2007102400:00:00

NO Optional Vertical Coordinate List.

Num. of Data Points = 987601 with BIT-MAP 0

DRS TEMPLATE 5. 0 : 0 -3 0 8 0

GRIB2-GRID TEMPLATE 5.0 (Grid point data - simple packing) has 5 values

GRIB2 - Section 5 Data Representation Section

987601 = Octet6-9 - Number of data points where one or

more values are specified in Section 7 when a bit map is

present, total number of data points when a bit map is

absent => total number of data points with bit map = 0

DRS TEMPLATE 5. 0 = GRIB2 - Section 5 Octet10-11 –

Data representation template number (See Table 5.0) =>

0 is Grid Point Data - Simple Packing (see Template 5.0)

0 = GRIB2 - Grid Template 5.0 Octet12-15 – reference

value (R) (IEEE 32-bit floating-point value)

-3 = GRIB2 - Grid Template 5.0 Octet16-17 - Binary scale

factor (E)

0 = GRIB2 - Grid Template 5.0 Octet18-19 - Decimal scale

factor (D)

8 = GRIB2 - Grid Template 5.0 Octet20 - Number of bits

used for each packed value for simple packing, or for each

group reference value for complex packing or spatial

differencing

0 = GRIB2 - Grid Template 5.0 Octet21 - Type of original field values (see Code Table 5.1) => Floating Point

-----------------------------------------------------

Data Values: MIN= .00000000 AVE= .14565143

MAX= 19.25000000

total number of fields found = 1

How to write GRIB2 directly? An example to convert APCP data into GRIB2

The degrib2 diagnostic was generated from APCP GRIB2 output and the source code is in:

$ hostname

c2n6.ncep.noaa.gov

$ pwd

/nco/save/sib/wx12kk/degrib2_analysis/sorc

Input binary data file: XMRG2417

run script: sh runGRIB2.sh

output: GRIB2417_grib2

Usage: HEtoGrib2_exec <input XMRG DIR> <XMRG file name> <output GRIB DIR> <output GRIB2 file name>

For full details, see the source code: HEtoGrib2.f90

call write_nesdis_grib2(gnesdis,nx,ny,nflds,&

varname,typeproc,isign, &

rtime(1),rtime(2),rtime(3),rtime(4),rtime(5),rtime(6), &

ityped,g2Path, accumHour, modelId, subCenter)

NCO GRIB2Templates

The template source code is in /nwprod/util/sorc/g2tmpl.fdMakefile grib2_all_tables_module.f params_grib2_tbl_newgdtsec3.f incmod process_4dot2.cgdtsec3.template libg2tmpl.aparams_grib2_tbl_new!!Table History Log:!2010-01-21 V. Krishna Kumar/NCO/SIB!!GRIB2 parameter table for all disciplines and categories!The fields in order are grib2_discipline grib2_category!grib2_parameter_number grib2_table_information (0 for the parameter!from the WMO table and 1 for the parameter from teh NCEP local table)!grib2_NCEP_mnemonic (locally defined at NCEP). WMO parameters have values!ranging from 0 to 191 and NCEP local table parameters have values from 192!to 254 and the parameter value 255 is a missing or undefined.!Comment cards with the !will not be read by the program!!! GRIB2 - TABLE 4.2-0-0 PARAMETERS FOR DISCIPLINE 0 CATEGORY 0! 0 0 0 0 TMP 0 0 1 0 VTMP 0 0 2 0 POT 0 0 3 0 EPOT 0 0 4 0 TMAX 0 0 5 0 TMIN 0 0 6 0 DPT 0 0 7 0 DEPR

NCO Grib2 Templates …

grib2_all_tables_module.f containssubroutine get_g2_subcenters(key,value,ierr)subroutine get_g2_sigreftime(key,value,ierr)subroutine get_g2_prodstatus(key,value,ierr)subroutine get_g2_typeofdata(key,value,ierr)subroutine get_g2_typeofgenproc(key,value,ierr)subroutine get_g2_versionno(key,value,ierr) subroutine get_g2_loctabversno(key,value,ierr)….subroutine get_g2_fixedsurfacetypes

type sigreftime character(len=20) :: sigrefkey integer :: sigrefvalend type sigreftime

! Declare a variable of type discipline

type(sigreftime),dimension(MAXREFTIME) :: table1_2

data table1_2(1) /sigreftime('anal',0)/data table1_2(2) /sigreftime('fcst',1)/data table1_2(3) /sigreftime('vfcst',2)/data table1_2(4) /sigreftime('obstime',3)/data table1_2(5) /sigreftime('missing',255)/

!!!

NCO Grib2 Templates

integer(4) listsec0(2) ! Length of section 0 octets 7 & 8integer(4) listsec1(13) ! Length of section 1 from octets 6-21integer(4) ipdstmpl0(15) ! Length of Section 4 PDS Template 4.0

subroutine g2sec0(idisc,listsec0)

subroutine g2sec1(origin_key,subcen_key,vers_key,lvers_key, sigreftime_key,refyear_val, refmon_val,refday_val,refhour_val, refmin_val,refsec_val,prodstatus_key,typeofdata_key,listsec1)

subroutine g2sec4_temp0(icatg,iparm,typ_gen_proc_key, & gen_proc_or_mod_key,hrs_obs_cutoff,min_obs_cutoff, & unit_of_time_key,fcst_time,lvl_type1,scale_fac1, & scaled_val1,lvl_type2,scale_fac2,scaled_val2, & ipdstmpl0)

Subroutine g2sec4_temp8(icatg,iparm,typ_gen_proc_key,gen_proc_or_mod_key, & hrs_obs_cutoff,min_obs_cutoff,unit_of_time_key, & fcst_time,lvl_type1,scale_fac1,scaled_val1,lvl_type2, & scale_fac2,scaled_val2,year_intvl, & mon_intvl,day_intvl,hour_intvl,min_intvl,sec_intvl, & num_time_range,stat_miss_val,type_of_stat_proc, & type_of_time_inc,stat_unit_time_key, & leng_time_range_stat,stat_unit_time_key_succ, & time_inc_betwn_succ_fld,ipdstmpl8)

subroutine g2sec5_temp0(dec_scale_fac,type_of_field,ifield5)

subroutine g2sec5_temp40(dec_scale_fac,bin_scale_fac,tlnumbits, & type_of_field, type_of_compression,ifield5,len,fld)Makefile

SHELL=/bin/sh

LIB = ./libg2tmpl.a -------------Library Name

INCMOD = ./incmod/g2tmpl

FC = ncepxlf90CC = ncepxlcFFLAGS = -qnosave -O3 -q free=f90 -I$(INCMOD)AR = arARFLAGS = -rv -X64

$(LIB): $(LIB)(process_4dot2.o) $(LIB)(grib2_all_tables_module.o) $(LIB)(gdtsec3.o)

$(LIB)(process_4dot2.o): process_4dot2.c $(CC) -O3 -c $< $(AR) $(ARFLAGS) $@ $*.o rm -f $*.o

NCO Grib2 Templates

NCO Grib2 Templates

gdtsec3.template (Fix file of all NCEP Grids)#

# PREDEFINED GRIB2 GRID DEFINITION TEMPLATE

#

# The format of the file scanned by routine getgridnum is as follows.

# Each line contains one Grid ID entry containing eight fields, each separated

# by a space " ". The fields are:

# 1) - The first character is a space

# 2) - predefined grid ID number

# 3) - Grid Definition Template number

# 4) - Number of entries in the Grid Definition Template

# 5) - Contains information read from the appropriate GRIB Grid

# 6) - A list of values for each entry in the Grid Definition Template.

# 7) - The number of entries in array ideflist i.e., number of rows

# (or columns) for which optional grid points are defined. eg. WAFS grid

# 8) - Optional integer array containing the number of grid points contained

# in each row (or columns).

#

# Comments can be included in the file by specifying the symbol "#" as the first

# character on the line.

#

#

1 10 19 0 1679 0 0 10 6 0 0 0 0 0 0 73 23 -48090000 0 48 22500000 48090000 0 64 0 513669000 5 13669000 0 0

gdtsec3.f (Fortran code that reads the fix grid files)

Parallel GRIB2 In NCEP POST Jun Wang

Changes are made into NCEP POST to add an option to generate GRIB2 output files directly

1. Using XML post control files 2. Apply NCO’s grib2 templates and g2 lib to generate grib2 message3. parallel generate grib2 messages and parallel write out the messages The changes are backward compatible, nothing need to be changed to run NCEP POST for grib1 output Details are on Cirrus:/climate/noscrub/wx20wa/grib2See Jun’s power point on SIB WWW2 GRIB2 Resources page

Courtesy of Jun Wang

Add grib2 option in post• Grib2 option is passed into post from scripts• All post fields are computed and saved in a data array, field

information related to the post control file is save in fld_info• Gribit2 is called to parallel generate grib2 messages and to

parallel write out grib2 message

Courtesy Jun Wang

Parallel generating and writing grib2 message

Each processor holds partial domain of data for all the output data fields in post computation; the data is then redistributed to all processors through mpi_alltoallv so that each processor holds whole domain data, but only with certain number of data fields.

All processors generate grib2 messages simultaneously All processors exchange the information of their total grib2

message length All processors will write out simultaneously through

mpi_write_all

Courtesy Jun Wang

MPI_ALLTOALLV

MPI_ALLTOALLV

gengrb2msg

gengrb2msg

gengrb2msg

MPI IO

MPI IO

MPI IO

Courtesy Jun Wang

Parallel generating and writing grib2 message

PE1 PE1 PE1

PE3

PE2 PE2 PE2

PE3 PE3

Grb2 file

Partial domain data with all the

output fields

Whole domain data with certain number of output

fields

Grib2 messages for certain fields

Contains subroutines to parallel generate grib2 messages using grib2 templates and to parallel write out grib2 file

Public num_pset,nrecout

type(paramset_t),save :: pset,

subroutine grib_info_init(first_grbtbl)

subroutine grib_info_finalize

subroutine gribit2(post_fname)

Private:

subroutine gengrb2msg(idisc,icatg, iparm,nprm,nlvl,ntrange,tinvstat,datafld1,cgrib,lengrib)

subroutine getgds(ldfgrd,len3,ifield3len,igds,ifield3) use gridspec_mod, only: DXVAL,DYVAL,CENLAT,CENLON,LATSTART, LONSTART, & LATLAST, LONLAST,MAPTYPE,STANDLON,latstartv,cenlatv,lonstartv, cenlonv, TRUELAT1, TRUELAT2

Courtesy Jun Wang

Grib2_module.f

Parallel GRIB2 Results from Jun Wang

NMMB Disk space Wall time Total Memory

grib1 950MB 275 740MB

parallel grb2

(JPEG)

296MB 69% savings

220 20% savings 1168MB 57% increase

GFS master files from NEMS GFS (1152*576, 664 fields)

B grid 8KM output from NEMS NMMB (954*835, 1098 fields)

GFS Disk space Wall time Total Memory

grib1 605MB 72 0.85G

parallel grb2 (JPEG)

195MB 68% savings

60 17% savings 1.1G 29% increase

GRIB2 Encoding RoutinesTo write a new GRIB2 message, the following routines are called: gribcreate, addlocal, addgrid, addfield and gribend.

• Gribcreate (gribcreate.f) This routine must be called first to initialize a new grib2 message. It uses to encode and pack section 0 and section 1. call gribcreate (cgrib,lcgrib,listsec0,listsec1,ierr)

Section 0: Indicator Section This section uses to serve as the beginning of GRIB2 message. The length of this section is 16 bytesOctet 1-4 : Identifier ‘GRIB’ 5-6: Reserved 7: Discipline 8: GRIB edition number 2 9-16 Total length of GRIB message Example: Section 0 : 0 2 37048

Section 1: Identification The content of section are: originating center, sub-center, GRIB master table version, local table version, start of forecast, reference time, production status and type of processed data in this GRIB message (i.e., analysis, forecast, ...) Example: Section 1: 7 0 2 1 1 2010 12 17 12 0 0 0 1

GRIB2 Encoding Routines • Addlocal (addlocal.f) optional is used to add a Local Use Section 2 to GRIB2 message call addlocal (cgrib,lcgrib,csec2,lcsec2,ierr)

• Addgrid (addgrid.f) is used to add a Grid Definition Section (grid geometry of the data values in the field) to a GRIB2 message. call addgrid (cgrib, lcgrib, igds, igdstmpl, igdstmplen, ideflist, idefnum, ierr)For example:Section 3: 0 41760 0 0 0Where octet (6 to 12) numbers are:0 Lat-Lon grid41760 number of data points0 regular grid0 regular grid0 grid definition template 3.0 For example: 3.0 is Lat/Lon grid, 3.10 is Mercator grid See grid definition template 3.0 for details in NCEP GRIB2 documentation.Example: GRID TEMPLATE 3.0: 6 0 0 0 0 0 0 288 145 0 0 90000000 0 48 -90000000 385750000 1250000 1250000 0

GRIB2 Encoding Routines• Addfield (addfield.f) This routine uses to add and packs up: Production Definition Section (section 4), Data Representation Section (section 5), Bit-Map section (section 6) and Data Section (section 7) to a GRIB2 message.

call addfield (cgrib, lcgrib, ipdsnum, ipdstmpl, ipdstmplen, coordlist, numcoord, idrsnum, idrstmpl, idrstmplen, fld, ngrdpts, ibmap, bmap, ierr)For example:- Product Definition Section (section 4): 34 4 0 0Where:34 is length of section 44 number of section (4)0 No values present (number of coordinate after template)0 Product template number 0 (analysis or forecast at a point in time) See product template 4.0 for details in NCEP GRIB2 documentation.

For example:PRODUCT TEMPLATE 4.0: 3 5 2 0 96 0 0 1 6 100 0 10000 255 0 0FIELD: HGT 100 mb valid at 6 hr after 2010121700:00:00

NOTE:Parameter HGT defines by Discipline 0 (find in Section 0), category 3 and parameter 5

GRIB2 Encoding Routines

-Data Representation Section (section 5) DRS TEMPLATE 5. 40 : 1156161536 0 1 9 0 0 255Where:1156161536 Reference value of IEE (degrib2 will provide this value) – users will need to give only0 Binary scale factor binary scale or decimal scale and number of bits (precision)1 Decimal scale factor 9 Number of bits required to hold the resulting scaled and referenced data values.0 Floating point (see table 5.1)0 Lossless (see table 5.40)255 Target compression ratio

-Bit-Map Section (section 6)1-4 Length of the section in octets (nn)5 Number of the section 66 Bit-map indicator see table 6.0 i.e., 0 bit-map applied to this product, 1-253 a bit-map predetermined by generating center and 255 No bit-map7-nn Bit-map

-Data Section (section 7)Where:1-4 Length of the section in octet (nn)5 Number of the section 76-nn Data in format described by template 7.x where x is the data representation template number given in octets 10-11 of section 5

GRIB2 Encoding Routines

• Gribend (gribend.f)

This routine finalizes a GRIB2 message after all grids and fields have been added. It adds

the End Section (“7777”) to the end of the GRIB2 message and calculates the length of GRIB2 message

and stores it in the section 0 in (octet 9 – 16)

call gribend (cgrib, lcgrib, lengrib, ierr)

GRIB2 Decoding RoutinesTo Read a GRIB2 message with call routines are following:

• Gb_info (gb_info.f) Is used to search through a GRIB2 message to find out how many local use sections and data fields found in the a given GRIB2 message call gb_info (cgrib, lcgrib, listsec0, listsec1, numfields, numlocal, maxlocal, ierr)

• Gf_getfld (gf_getfld.f) Is used to get all information of grid definition (section 3), product template (section 4), Bit-map (if applicable), and the unpack data values for a given data field. All information returned is store in a derive type variable “gfld” call gf_getfld (cgrib, lcgrib, ifldnum, unpack, expand, gfld, ierr)

gfld is a type of gribfield which is defined in module grib_mod (see grib_mod.f for declaration of derive type gribfield) Users of this routine have to include the line below in your code. Example:

use grib_mod parameter (maxsize=200000) integer :: listsec0(3), listsec1(13) type (gribfield) :: gfld . . . See the example of program degrib2.f that how to call routines gb_info.f and gf_getfld.f to get the information and data values in the GRIB2 message.

Declaration of Derived type gribfield• type gribfield

integer :: version,discipline integer,pointer,dimension(:) :: idsect integer :: idsectlen character(len=1),pointer,dimension(:) :: local integer :: locallen integer :: ifldnum integer :: griddef,ngrdpts integer :: numoct_opt,interp_opt,num_opt integer,pointer,dimension(:) :: list_opt integer :: igdtnum,igdtlen integer,pointer,dimension(:) :: igdtmpl integer :: ipdtnum,ipdtlen integer,pointer,dimension(:) :: ipdtmpl integer :: num_coord real,pointer,dimension(:) :: coord_list integer :: ndpts,idrtnum,idrtlen integer,pointer,dimension(:) :: idrtmpl logical :: unpacked logical :: expanded integer :: ibmap logical*1,pointer,dimension(:) :: bmap real,pointer,dimension(:) :: fld end type gribfield

• If variable “gfld” is declared of type gribfield. For example: Type (gribfield) :: gfld

The variable “gfld” would have the following components: gfld%version = GRIB edition number ( currently 2 )

Declaration of Derived Type Gribfield

gfld%discipline = Message Discipline ( see Code Table 0.0)

gfld%idsect() = Contains the entries in the Identification section 1

gfld%idrtmpl() = Contains the data values for the specified Data Representation Template

gfld%idsectlen = Number of elements in gfld%idsect().

gfld%local() = Pointer to character array containing contents of Local Section 2, if included

gfld%locallen = length of array gfld%local()

… etc …

See routine (gribmod.f) in documentation block for more components of

variable “gfld” is declared of type gribfield

How to use routine getgb2?

• Getgb2 (getgb2.f)

Is used to search for selected fields and unpack from a GRIB2 message

Call getgb2 (lugb,lugi,j,jdisc, jids, jpdtn, jpdt, jgdtn,jgdt, unpack, k, gfld, iret) Where:

J INTEGER NUMBER OF FIELDS TO SKIP ( J=0 search from beginning)

JDISC GRIB2 DISCIPLINE NUMBER OF REQUESTED FIELD ( IF = -1, ACCEPT ANY DISCIPLINE) JIDS INTEGER ARRAY OF VALUES IN THE IDENTIFICATION SECTION (=-9999 FOR WILDCARD) JPDTN INTEGER PRODUCT DEFINITION TEMPLATE NUMBER (N) ( IF = -1, DON'T BOTHER MATCHING PDT - ACCEPT ANY ) JPDT INTEGER ARRAY OF VALUES DEFINING THE PRODUCT

DEFINITION TEMPLATE 4.N OF THE FIELD FOR WHICH TO SEARCH (=-9999 FOR WILDCARD) JGDTN INTEGER GRID DEFINITION TEMPLATE NUMBER (M) ( IF = -1, DON'T BOTHER MATCHING GDT - ACCEPT ANY ) JGDT INTEGER ARRAY OF VALUES DEFINING THE GRID DEFINITION TEMPLATE 3.M OF THE FIELD FOR WHICH TO SEARCH (=-9999 FOR WILDCARD)

How to Use Getgb2? An example of using getgb2 to search for field (Temperature at 100 mb) byproduct definition template.

Initialize all variables to default -9999 or -1 for wildcard

J = 0 Search from the beginning of GRIB2 file DJDSC=-1 Discipline number JIDS=-9999 Identification Section JGDTN=-1 Grid Definition Template Number JGDT=-9999 Grid Definition Template JPDTN=-1 Product Definition Template Number JPDT=-9999 Product Definition Template

Search selected field by product definition template numberand the values of product definition template 4.n of the field

JPDTN = 0

Production template 4.0 for Temperature at 100mb

JPDT(1:15)=(/ 0,0,2,0,96,0,0,1,6,100,0,10000,255,0,0 /)

How to use Getgb2?

How do we get this information for field Temperature at 100mb fromGRIB2 message ?

We can use utility degrib2 to display the content of GRIB2 message degrib2 grib2file



FIELD 1 SECTION 0: 0 2 SECTION 1: 7 0 2 1 1 2010 12 13 0 0 0 0 1 SECTION 3: 0 41760 0 0 0 GRID TEMPLATE 3. 0 : 6 0 0 0 0 0 0 288 145 0 0 90000000 0 48 -90000000 358750000 1250000 1250000

0 NO Optional List Defining Number of Data Points. PRODUCT TEMPLATE 4. 0 : 0 0 2 0 96 0 0 1 6 100 0 10000 255 0 0 FIELD: TMP 100 mb valid 6 hr after 2010121300:00:00 NO Optional Vertical Coordinate List. Num. of Data Points = 41760 NO BIT-MAP DRS TEMPLATE 5. 40 : 1156022272 0 1 9 0 0 255 Data Values: MIN= 185.19999695 AVE= 207.57528687 MAX= 232.00000000

How to use routine putgb2?

• Putgb2 (putgb2.f)

Is used to pack a single field into a GRIB2 and writes out that message to the

file associated with unit number.

Call putgb2 (lugb,gfld,iret)

NOTE:

The information to be packed into the GRIB field is stored in a derived type variable, gfld.

Gfld is of type gribfield, which is defined in module grib_mod, so users of this routine will

need to include the line "USE GRIB_MOD" in their calling routine. Each component

of the gribfield type is described in the INPUT ARGUMENT LIST section below.

See documentation block in routine putgb2.f for more information and an example of

program wavesteepgrib2.f (Courtesy Vera Gerald)

GRIB2 Reads and Writes Examples

• The program readwriteg2.f reads an entire GRIB2 file and extract selected field(s) unpacked from GRIB2 message(s) and write it out to a new GRIB2 file.

• The program readgrib2.f reads an entire GRIB2 file and extract selected field(s) from data card and return packed GRIB2 message(s) to write them out to a new GRIB2 file.

The content of data card is following:

&GRIBIDS DESC=' HGT 100 mb ',PDTN= 0 ,PDT= 3 5 2 0 96 0 0 1 6 100 0 10000 255 0 0 /&GRIBIDS DESC=' TMP 100 mb ',PDTN= 0 ,PDT= 0 0 2 0 96 0 0 1 6 100 0 10000 255 0 0 /&GRIBIDS DESC=' U GRD 100 mb ',EXTRACT=.true.,PDTN= 0 ,PDT= 2 2 2 0 96 0 0 1 6 100 0 10000 255 0 0 /&GRIBIDS DESC=' V GRD 100 mb ',EXTRACT=.true.,PDTN= 0 ,PDT= 2 3 2 0 96 0 0 1 6 100 0 10000 255 0 0 /

• The program degrib2.f reads an entire GRIB2 file and display the content of an entire GRIB2 file.

The examples of these programs are available to download (tar file) from this url:

http://www2.nco.ncep.noaa.gov/sib/grib2/examples/

• Reads and displays the content of GRIB2 files • Reads and extracts selected fields from data card

and write them out to a new file • Reads and searches selected fields (by product definition template)

and write them out to a new file

SUMMARY & CONCLUSIONS• NCEP GRIB2 tools hopefully will provide guidance to all users

to generate GRIB2 messages directly

• WMO has mandated all operational centers to transition to GRIB2 data for all the operational products

• Usage of CNVGRIB type of tools to convert from GRIB1 to GRIB2 should be minimized

• Generation of GRIB1 products take heavy storage and are also computationally expensive

• Jun Wang’s parallel GRIB2 generation of NCEP POST using NCO’s g2tmpl library for NEMS, NMMB and GFS show encouraging results and are very valuable for NCEP's codes to run more efficiently as well as occupy less disk and tape space

• NCO will continually improve the GRIB2 Template source code to incorporate all the Product Definition Templates.

• NCO will improve the existing GRIB2 tools in the future.

THANK YOU

Documents

GRIB2 usage at NCEP