Configuring NVODS Datasets in LAS

Joseph McLeanTMAP/PMEL/NOAA and JISAO/UW

OAR WebShop 2002

Part 1 -Getting a Dataset "into" LAS-

Best Case - COARDS Compliant NetCDF File The metadata ingestion process. The relational database and user interface. Ready to access and process. The LAS Data Configuration XML File operations/institution information data set attributes variable attributes back-end properties lon/lat/depth/time axis definitions

-Data Configuration XML

File-

<?xml version='1.0' ?> <!DOCTYPE spec SYSTEM "spec.dtd" [<!ENTITY StdOperations SYSTEM "operations.xml">]>

<lasdata> <institution name="Pacific Marine Environmental Lab“ url="http://www.pmel.noaa.gov"/> <operations url="http://foo.server.com/las-bin/LASserver.pl">

&StdOperations; </operations><!-- Define datasets/variables -->

<datasets> <coads_climatology_cdf name="COADS Climatology" url="http://ferret.pmel.noaa.gov/cgi-bin/dods/nph-dods/data/PMEL/coads_climatology.ncl" doc="doc/coads_climatology.html"> <variables> <airt name="Air Temperature" units="DEG C"> <link match="/lasdata/grids/coads_climatology_cdf_grid"/> <!-- Define properties --> <properties> <ferret> <palette>light_centered</palette> <init_script>COADS_clim_init</init_script> <ferret> <properties> </airt> </variables> </datasets> <!-- Define grids --> <grids> <coads_climatology_cdf_grid> <link match="/lasdata/axes/coads_climatology_cdf_COADSX"/> <link match="/lasdata/axes/coads_climatology_cdf_COADSY"/> <link match="/lasdata/axes/coads_climatology_cdf_TIME"/> </coads_climatology_cdf_grid> </grids> <!-- Define axes --> <axes> <coads_climatology_cdf_COADSX type="x" units="degrees_east"> <arange start="21" step="2" size="180"/> </coads_climatology_cdf_COADSX> <coads_climatology_cdf_COADSY type="y" units="degrees_north"> <arange start="-89" step="2" size="90"/> </coads_climatology_cdf_COADSY> <coads_climatology_cdf_TIME type="t" units="month"> <arange start="1-1-16" step="1" size="12"/> </coads_climatology_cdf_TIME> </axes> </lasdata>

LAS attributes

Data set attributes

variable attributes

back-end properties

Axis definitions

Part 1 -Getting a Dataset "into" LAS-

Configuring the XML File when needed custom palettes or contour levels variable name conflicts virtual variables incomplete/incorrect metadata Worst Case - No Metadata frequently missing or inadequate supply metadata in XML file by hand need adequate documentation explore the dataset custom initialization scripts

Part 1 -Getting a Dataset "into" LAS-

Back-end Scripts multiple back-ends "NCL","CDAT",GrADS, Matlab,IDL Ferret is free and distributed with LAS. NVODS LAS(via Ferret) uses the NetCDF API to access DODS datasets. Custom User Interfaces high resolution maps data organization

Part 2 -Examples of Dataset Problems- Missing Attributes bad dataflag/missing_value attribute Usually this is straightforward. Dimensional Metadata COARDS Coordinate Variables Fixing this can be time consuming. need documentation. custom scripts Time axes some time units not supported "modulo" attribute Wrong Metadata Freeform file format descriptor Ancillary DAS Files

-DODS .das File with Missing Attributes- Attributes { time { } level { } mean_freq { } NC_GLOBAL { String dtg "2002010700“ String model_name "WAM“ String dset_name "fcst_ops“ String geom_name "adriatic“ Float32 southern_latitude 40.; Float32 northern_latitude 46.; Float32 western_longitude 12.; Float32 eastern_longitude 20.; Float32 delta_x 0.08333332837; Float32 delta_y 0.08333332837; } DODS_EXTRA { String Unlimited_Dimension "time" }

Part 3 -NVODS/DODS Dataset Statistics

How does Metadata affect Performance?

A summary of Tests on a sample of the DODS Dataset List Dataset Ready to Use as accessed - 30% Dataset Usable after Ferret repairs - 35% Unable to use Dataset - 35%

Please Note that not all of the problems with opening and using datasets were related with unavailable use metadata. Some of the reasons for not being able to use a Dataset included password needs, bad links, and incompatible file types. With the use of back-end scripts to enhance and correct use metadata, we were able to successfully access 65% of the DODS datasets tested.

- Conclusion -

How can the Ancillary Information Service Help?

The NVODS LAS is Not Unique Fix the Dataset, Not the Application Allow a User to Determine the Fix