NOAA’s National Weather Service NDFD File Dissemination Services Allan Darling Office of the CIO, Telecommunication Operations Center National Digital

NOAA’sNOAA’sNational Weather ServiceNational Weather ServiceNDFD File Dissemination ServicesNDFD File Dissemination Services

Allan DarlingAllan Darling

Office of the CIO, Telecommunication Operations CenterOffice of the CIO, Telecommunication Operations Center

National Digital Forecast Database Technical WorkshopNational Digital Forecast Database Technical Workshop

November 2, 2006November 2, 2006

NDFD Technical Workshop 2006NDFD Technical Workshop 2006

NDFD File Dissemination ServicesNDFD File Dissemination Services

Where to find NDFD product files –

Files are available via:

ftp://tgftp.nws.noaa.gov

http://weather.noaa.gov/pub

ftp://ptgftp.nws.noaa.gov

ftp://tgdata.nws.noaa.gov

Access to ptgftp.nws.noaa.gov and tgdata.nws.noaa.gov are limited to SAS customers.

See http://weather.gov/datamgmt/fos/fospage.html for more information about Server Access Services.




On each server, NDFD data is found under the paths:

/SL.us008001/ST.opnl/DF.gr2/DC.ndfd/...or

/SL.us008001/ST.expr/DF.gr2/DC.ndfd/...

The path indicates that the data is on the public NWSTG FTP server, is operational or experimental, is encoded in WMO GRIB Edition 2 format, and is all related to the NDFD data category.

See http://www.nws.noaa.gov/datamgmt/filstnd.html for more information about the directory and file naming standards used on the NWSTG FTP and HTTP Servers



How files are named –Following the data format and category directories, the last directories and the file name specify:

• the geographic area (/AR.aaaaaaaa)• the valid period (/VP.ddd-ddd)• the data subcategory (/ds.sssss)

A list of all the NDFD Sectors (geographic areas) defined by lat/lon and grid points is available at: http://www.nws.noaa.gov/datamgmt/doc/ndfdelem.xls

A list of data subcategories is available at: http://www.weather.gov/ndfd/resources/NDFD_element_status.pdf



How files are constructed –

Files are comprised of:

• A flag field separator block (option 2)

• A super heading

• One or more GRIB products each comprised of– A flag field separator block (option 2)

– A WMO Heading

– A GRIB product



A flag field separator block (option 2) is a file delimiter that provides the length of the product that follows it. The format is:

****nnnnnnnnnn****lf

Where nnnnnnnnnn is the length of the product following the seperator, in bytes and lf is an ASCII line feed. See http://weather.gov/tg/fstandrd.html for more details.



What is a “super heading”?

In order to aggregate forecast periods for one data subcategory and one sector, the “super heading” was developed.

The super heading specifies the data subcategory in T2 and the geographic sector in A1. A2 and ii are fixed as “Z98” or “Z97”:

NDFD Super Heading: YT2A1Z98 KWBN



How product headings are formed –

Generic NDFD Heading: YT2A1A2ii KWBN

T2 specified the data type (data subcategory)

A1 specifies the geographic designator (NDFD sector)

A2 and ii together specify the NDFD reference time (not represented in the directory or file names)

The possible values for T2 , A1 , A2 , and ii and their meanings are specified on the NDFD WMO Heading Reference page at

http://weather.gov/datamgmt/doc/ndfdref.html



How files are constructed –****0004911342****YEUZ98 KWBN 301549****0000266234****YEUB18 KWBN 301549GRIB..................7777****0000018847****YEUB21 KWBN 301549GRIB.....................7777...****0000018051****YEUE00 KWBN 301549GRIB.................7777




Files are available via:

ftp://tgftp.nws.noaa.gov

http://weather.noaa.gov/pub

ftp://ptgftp.nws.noaa.gov

ftp://tgdata.nws.noaa.gov

Access to ptgftp.nws.noaa.gov and tgdata.nws.noaa.gov are limited to SAS customers.

See http://weather.gov/datamgmt/fos/fospage.html for more information about Server Access Services.



Legacy NWSTG file services:

HTTP access via Internet – http://weather.noaa.gov (5 servers)

FTP access via Internet –

ftp://tgftp.nws.noaa.gov (4 servers)

FTP access via dedicated communication paths –

ftp://ptgftp.nws.noaa.gov (2) servers



Significant system upgrades as part of the Replacement NWSTG implementation, including:• Linux servers• Redundant, highly available NFS servers• Network-based load balancing• Gigabit Ethernet internal network• Improved network devices



Replacement NWSTG file services:

HTTP access via Internet – http://weather.noaa.gov (6 servers)

FTP access via Internet – ftp://tgftp.nws.noaa.gov (5 servers)

FTP and HTTP access via dedicated communication paths –

ftp://tgdata.nws.noaa.gov (3 shared servers)

http://tgdata.nws.noaa.gov (3 shared servers)



tgdata.nws.noaa.gov available for testing in the Replacement NWSTG since August

– Supported operationally as of November 8, 2006

– Four week overlap between startup of tgdata.nws.noaa.gov and shutdown of ptgftp.nws.noaa.gov

weather.noaa.gov and tgftp.nws.noaa.gov still in Legacy NWSTG

– Service will move (transparently) to the Replacement NWSTG by December 1



Questions Now?

Questions Later? [email protected]



Additional Information



****0000149136****YGCZ98 KWBN 031906****0000018449****YGCB00 KWBN 031906GRIB..................7777****0000018847****YGCC00 KWBN 031906GRIB.....................7777...****0000018051****YGCI00 KWBN 031906GRIB.................7777

****0000149136****YGCZ98 KWBN 031906

149136 is the length, in bytes, of the super heading and all the following groups of flag field separators, WMO headings, and GRIB products.

Super heading flag field separator -



****0000149136****YGCZ98 KWBN 031906****0000018449****YGCB00 KWBN 031906GRIB..................7777****0000018847****YGCC00 KWBN 031906GRIB.....................7777...****0000018051****YGCI00 KWBN 031906GRIB.................7777

****0000018449****YGCB00 KWBN 031906GRIB.............7777

18449 is the length, in bytes, of the single heading and GRIB product.

“Regular” flag field separator -



Online References

NWSTG Data Management NDFD Reference - http://weather.gov/datamgmt/doc/ndfdindex.html

Directory and File Naming Standards –

http://weather.gov/datamgmt/filstnd.html

Family of Services Main Page –

http://weather.gov/datamgmt/fos/fosindex.html

Gateway File Standards and Content Structures –

http://weather.gov/tg/fstandrd.html



Backup NWSTG file services -

Data access service backup will be split– Public Internet service will be provided at the Primary NWSTG

and at NWS Central Region Headquarters– Non-public service will be provided at the Primary NWSTG and

the Backup NWSTG

Back-end NFS servers will be populated at the active NWSTG

– Primary active: data copied to the backup NWSTG and CR HQ– Backup active: data copied to CR HQ



Access via Dedicated Communications



Access via Internet

Documents

NOAA’s National Weather Service NDFD File Dissemination Services Allan Darling Office of the CIO, Telecommunication Operations Center National Digital