Embed Size (px)
DESCRIPTION
Funded by NOAA's Scientific Data Stewardship (SDS) program, we will investigate how to store NASA ECS data and metadata into HDF5 Archival Information Packages (AIP). To achieve this, the HDF4 to HDF5 conversion tool will be enhanced so that converted ECS data can be read through the NetCDF4 interface. In addition, metadata tools that convert ECS collection and granule level metadata to NOAA's collection level and NARA's METS standard will be developed.
Citation preview
11/8/2007HDF and HDF-EOS Workshop XI, Landover,
MD 1
Storing EOS data using HDF5 Archival Information
Package(AIP)
MuQun YangThe HDF Group
Ruth DuerrNSIDC
Overview
• NOAA Scientific Data Stewardship(SDS) program
• Operational production of high quality Climate Data Records- “Generation of CDR context capable of surviving transformative migration”
• NOAA CLASS may archive EOS2 data as well as future NPOESS HDF5 data
• One year project • Collaboration between THG and NSIDC
11/8/2007HDF and HDF-EOS Workshop XI, Landover,
MD 2
Goals
• An alternative solution to long-term archiving EOS2 data
• Provide openly available tools to convert ECS metadata into FGDC compliant forms
11/8/2007HDF and HDF-EOS Workshop XI, Landover,
MD 3
11/8/2007HDF and HDF-EOS Workshop XI, Landover,
MD 4
HDF5 Archive Information Package
Data file HDF5
METS
Primary Schema Extension Schema
|<mets>|---<dmdSec>----------------<MODS>|---<amdSec>--------------|--<techMD>| |--<rightsMD>| |--<sourceMD>|----<fileGrp>|----<structMap>
http://www.hdfgroup.uiuc.edu/papers/papers/AIP/HDF5_AIP_White_Paper.pdf
HDF5 AIP Components
Metadata file
11/8/2007HDF and HDF-EOS Workshop XI, Landover,
MD 5
Coordinate Systems
Common Data Model Layers
Data Access
Scientific Datatypes
Grid
Point
Radial
Trajectory
Swath
Station Profile
NetCDF4/HDF5-data
NetCDF4 / HDF5 Data
METS
NSIDC/ ECS
HDF4-data
NCDC:CLASS
NMMR
HDF5-AIP
H4toH5
ECS to NMMR
NSIDC/ECS
Metadata
CDM/NetCDF4
ECS to METS
Deliverables
• Three conversion tools1. Enhanced HDF4 to HDF5 conversion tool2. ECS to METS metadata extraction tool 3. ECS system to NMMR conversion tool
• Tool tests• Final report
11/8/2007HDF and HDF-EOS Workshop XI, Landover,
MD 7
11/8/2007HDF and HDF-EOS Workshop XI, Landover,
MD 8
Acknowledgement
This work was supported under NOAA Scientific Stewardship Program grant number NA07OAR4310286. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of NOAA.
