Upload
others
View
14
Download
0
Embed Size (px)
Citation preview
Slide 1
Com Intro 2014 - BUFR decoding ©ecmwf Slide 1
BUFR decoding
Dominique Lucas
User Support
Slide 2
Com Intro 2014 - BUFR decoding ©ecmwf Slide 2
Content
What is BUFR
BUFR tools
BUFR format
BUFR decoding
Practical examples
Slide 3
Com Intro 2014 - BUFR decoding ©ecmwf Slide 3
What is BUFR
Binary representation of meteorological data or Binary Universal Form for data Representation.
Continuous bit stream made of sequence of octets.
Table driven code.
Self descriptive code.
Machine independent.
Compression available for improved transmission speed.
Slide 4
Com Intro 2014 - BUFR decoding ©ecmwf Slide 4
BUFR tools – data validation
http://www.ecmwf.int/products/data/d/check/
Slide 5
Com Intro 2014 - BUFR decoding ©ecmwf Slide 5
BUFR tools – data validation
Slide 6
Com Intro 2014 - BUFR decoding ©ecmwf Slide 6
BUFR data examiner – metview4
$ metview4 –e BUFR ~trx/bufr_decode/bufr_file
Slide 7
Com Intro 2014 - BUFR decoding ©ecmwf Slide 7
BUFR FORMAT
Indicator section
Identification section
Optional section
Data description section
Data section
End section
All sections are padded with “0”s if needed to occupy even number of
octets.
Slide 8
Com Intro 2014 - BUFR decoding ©ecmwf Slide 8
Section 0 - Indication section
4 characters BUFR
Length of message
Edition number.
Slide 9
Com Intro 2014 - BUFR decoding ©ecmwf Slide 9
Section 1 - Identification section
Originating Centre
Data Category (Table A) and sub-category
Version number of tables
Date and time
Slide 10
Com Intro 2014 - BUFR decoding ©ecmwf Slide 10
Section 2 - Optional section
Data used by MARS
Slide 11
Com Intro 2014 - BUFR decoding ©ecmwf Slide 11
Section 3 - Data description section
Number of data subsets
Flag for compression
Data descriptors
Slide 12
Com Intro 2014 - BUFR decoding ©ecmwf Slide 12
Section 3 - Data descriptors
F
type
X
category
Y
entry
2 bits 6 bits 8 bits
F = 0 Element Descriptor – Bufr table B
F = 1 Replication descriptor
X = number of descriptors to repeat
Y = number of times the descriptors are repeated
F = 2 Operator Descriptor – “Internal table C”
F = 3 Sequence Descriptor – Bufr table D
Slide 13
Com Intro 2014 - BUFR decoding ©ecmwf Slide 13
“Table C” - Data Descriptor operators
201yyy - Change data width
202yyy - Change scale
203yyy - Change reference value
222000 - Quality information
This table is internal to the BUFR software. See:
http://www.wmo.int/pages/prog/www/WMOCodes/WMO306_vI2/LatestVERSION/LatestVERSION.html
Slide 14
Com Intro 2014 - BUFR decoding ©ecmwf Slide 14
Section 4 - Data section
Binary data
Section 5 - End section
4 digits 7777
Slide 15
Com Intro 2014 - BUFR decoding ©ecmwf Slide 15
BUFR Tables
Table A - Data category
Table B - Classification of elements
Table C - Code and flag table (*)
Table D - List of common sequences
http://www.wmo.int/pages/prog/www/WMOCodes/WMO306_vI2/LatestVERSION/Latest
VERSION.html
(*) Not to mix with the table C with data operator descriptors mentioned before.
Slide 16
Com Intro 2014 - BUFR decoding ©ecmwf Slide 16
Table A - Data category
Used in the Section 1 (element 9) of the BUFR message
Example:
Code figure Meaning
0 Surface data - land
1 Surface data – sea
2 Vertical soundings (not satellite) …
31 Oceanographic data
Slide 17
Com Intro 2014 - BUFR decoding ©ecmwf Slide 17
Table B - Classification of elements
Element Name Unit Scale Reference #bits
005001 Latitude (high accuracy) Degree 5 -9000000 25
007003 Geopotential m**2/s**2 -1 -400 17
002019 Satellite instruments Code table 0 0 11
008001 Vertical sounding signifi Flag Table 0 0 7
001006 Aircraft flight number CCITTIA5 0 0 64
011012 Wind speed at 10m m/s 1 0 12
(obs. * 10**scale – Reference) is encoded into #bits bits
For coded or flagged values, the element descriptor indicates the number of the table
describing the codes/flags.
0 - Table B entry
05 - Location (horizontal 1) class
01 - Identification
08 - Significance qualifiers
Slide 18
Com Intro 2014 - BUFR decoding ©ecmwf Slide 18
Table C – Code and flag tables
0 20 003 – Present Weather
Code figure Meaning
0 Cloud development not observed or not observable
1 Clouds generally dissolving or becoming less developed
…
10 Mist
11 Patches of shallow fog or ice fog
…
61 Rain, not freezing, continuous; slight at time of obs.
…
171 Snow, slight
172 Snow, moderate
173 Snow, heavy
…
511 Missing
Slide 19
Com Intro 2014 - BUFR decoding ©ecmwf Slide 19
Table D - List of common sequence
Table D can contain sequences of table B entries, Table D entries and Operators. It is not needed but saves a lot of space.
301027 301001 WMO block and station
002011 Radiosonde type
002012 Radiosonde computational method
301011 Date
301012 Time
301022 Lat/Long and station height
Slide 20
Com Intro 2014 - BUFR decoding ©ecmwf Slide 20
Bufr software
PBIO routines
PBOPEN - open bufr file for read/write
PBBUFR - read bufr message
PBWRITE - write bufr message
Slide 21
Com Intro 2014 - BUFR decoding ©ecmwf Slide 21
BUFR decoding - BUFREX
CALL BUFREX ( kbufl, kbuff, ksup, ksec0, ksec1, ksec2, ksec3, ksec4, kelem, cnames, cunits, kvals, values, cvals, kerr )
Input arguments
kbufl - length of bufr message in words
kbuff - array containing bufr message
kelem - expected number of expanded elements
kvals - size of values array
Slide 22
Com Intro 2014 - BUFR decoding ©ecmwf Slide 22
BUFR decoding - BUFREX
Output arguments
ksup - array containing supplementary
information
ksec[0-4] - array containing section [0-4] information
cnames - character array containing element
names
cunits - character array containing element units
values - real array containing element values
cvals - character array containing char. elem.
Values
kerr - return code
Slide 23
Com Intro 2014 - BUFR decoding ©ecmwf Slide 23
BUFR decoding – BUSEL (expanded
descriptors)
CALL BUSEL(KTDLEN,KTDLST,KTDEXL,KTDEXP,KERR)
Output arguments
KTDLEN - An INTEGER variable containing number of data
descriptors in KTDLST array
KTDLST - An INTEGER array containing the list of KTDLEN data
descriptors
KTDEXL - An INTEGER variable containing number of expanded data
descriptors
KTDEXP - An INTEGER array containing the list of KTDEXL data
descriptors
KERR - An INTEGER containing error
Slide 24
Com Intro 2014 - BUFR decoding ©ecmwf Slide 24
BUFR decoding
To access character strings from cvals array, say, for element i.
index = values(i)/1000
string = cvals(index)
values(i) = index * 1000 + length
For example, if values of a character descriptor is 2008, one will have to
look at cvals(2) and the string will be 8 characters long.
To access i-th element in multi subset message from values array
index = i +(nsub-1) * kelem
Slide 25
Com Intro 2014 - BUFR decoding ©ecmwf Slide 25
BUFR Tables
BUFR Edition 4 (and 3) naming convention
vssswwwwwxxxxxyyyzzz
v - Bufr table (B, C, D) sss - Master table number (000) wwwww - Originating subcentre xxxxx - Originating centre yyy - Version number of master table used zzz - Version number of local table used
e.g. B0000000000098013001.TXT C0000000000098013001.TXT D0000000000098013001.TXT
Slide 26
Com Intro 2014 - BUFR decoding ©ecmwf Slide 26
Where to find more about BUFR
WMO manual on Codes, Volume I, International Codes, Part B - Binary Codes, WMO No.306, FM-94-IX Ext BUFR
BUFR User Guide and Reference Manual, ECMWF 2008
http://www.ecmwf.int/publications/technical_notes/
Guides to WMO Table Driven Code Forms
http://www.wmo.int/pages/prog/www/WMOCodes.html
http://www.wmo.int/pages/prog/www/WMOCodes/WMO306_vI2/Latest
VERSION/LatestVERSION.html
Python interface to BUFR available from KNMI
https://code.google.com/p/pybufr-ecmwf/
Slide 27
Com Intro 2014 - BUFR decoding ©ecmwf Slide 27
Practical examples – to be run on ecgate
Copy files across …
$ cd $SCRATCH
$ tar xvf ~trx/bufr_decode/practicals.tar
$ cd bufr_decode
Familiarise yourself with the bufr examinar in metview, e.g.
$ metview4 –e BUFR bufr_file
The script “Retrieve_decode_bufr.cmd” decodes a BUFR Synop
message. The source code file ‘bfdemo.f90’ is also available
together with the Makefile. The file ‘bfdemo.c’ contains C code.
$ make bfdemo
$ ./bfdemo
Slide 28
Com Intro 2014 - BUFR decoding ©ecmwf Slide 28
Practical examples – to be run on ecgate
Can you adapt the above script/code to write part of the observation
data out to a file, e.g.
! 001001 WMO BLOCK NUMBER
! 001002 WMO STATION NUMBER
! 004001 YEAR
! 004002 MONTH
! 004003 DAY
! 004004 HOUR
! 004005 MINUTE
! 005001 LATITUDE (HIGH ACCURACY)
! 006001 LONGITUDE (HIGH ACCURACY)
! 010004 PRESSURE
! 011011 WIND DIRECTION AT 10 M
! 011012 WIND SPEED AT 10 M
An example job is available in Retrieve_extract_bufr.cmd. The
source code file is bfextract.f90, with the Makefile:
$ make bfextract
$ ./bfextract
$ cat fort.2