The OPeNDAP/OGC Gateway A NASA ACCESS Project integrated from two proposals:

-- The Development and Deployment of a CEOP Satellite Data Server (Ken McDonald, GSFC)

-- Gateway for Interoperability of Atmosphere, Land, Ocean, and Modeling Science Data (Liping Di, GMU)

Wenli YangCenter for Spatial Information Science and Systems (CSISS)

School of ScienceGeorge Mason University

http://csiss.gmu.edu

OPeNDAP Developer’s Meeting, Feb. 21-23, 2007, Boulder, Colorado

Project TeamPI and Co-Is:

Ken McDonald (PI, NASA/GSFC), Yonsook Enloe (SGT Inc.), Liping Di and Wenli Yang (GMU), Dan Holloway (OPeNDAP), Ben Domenico (Unidata), Glenn Rutledge (NOAA/NCDC).

Other members:Chengfang Hu and Min Min (GMU), Sr. S/W Engr.

(OPeNDAP)

Science advisors and user feedback:Professor Toshio Koike (University of Tokyo), Dr. Mike

Bosilovich (NASA GSFC)

Project Goal

To address the interoperability of two data system infrastructures widely used by different segments of the Earth science research and applications community, namely the Earth science community which uses OPeNDAP and THREDDS protocols and the geospatial community which uses OGC protocols.

Specific Objectives

To allow a user of a DAP client to discover and access data provided through OGC servers. To leverage WCS rectification/reprojection and

interpolation operations with DAP access to satellite products for the CEOP science community. To allow a user of a OGC client to discover data

available through THREDDS servers.

CEOP Satellite Data Server OPeNDAP/WCS Gateway Components

The original design was to develop the gateway components. The gateway can then be installed with the OPeNDAP server to

link the server to a WCS server. With the development of server4, many of the components are already included in the server. Thus, an independent gateway is not needed. The CF-1.0 compliant netCDF format handler is embedded into the

WCS server. Server4 can always expect a valid CF-netCDF from the WCS server.

THREDDS catalog generator will be developed as a THREDDS server at the front end and as a WCS 1.1 client, which sends describeCoverage requests, at the back end. Currently, the catalog generator is implemented by making use of an

XML configuration file (the WCS server’s capabilities XML file) without issuing requests to the WCS server.

CEOP Satellite Data Server OPeNDAP/WCS Gateway Components

OLFS BES

THREDDS

WCS

Local cache

DAP

OPeNDAP Server Implementation Approach

OLFS interacts with local catalog to identify data source as WCS. OLFS instructs BES to set container type WCS; passes name, target, type to BES. BES sets container to WCS, uses name, target, type to interact with remote WCS. BES interns WCS response to local cache. BES uses handler (NetCDF, HDF, <type>) to process cached file to satisfy DAP request. Subsequent DAP requests operate against local cache until cache refresh signaled.

The Test Implementationhttp://test.opendap.org:8080/opendap/data/wcs/Georectified-Grid/MYD11A2.A2004137.h10v05.004.2004147190109_EPSG.MODIS_Grid_8Day_1km_LST.nc.html

http://data.laits.gmu.edu/cgi-bin/ACCESS/wcs300?service=WCS&version=1.0.0&request=getCoverage&coverage=/home/mmin/grid1/MYD11A2.A2004137.h10v05.004.2004147190109_EPSG.hdf:Grid:MODIS_Grid_8Day_1km_LST:LST_Day_1km&crs=EPSG:4326&bbox=-100.8,38,-92.1,39.9&format=netCDF&width=300&height=200

Corresponding WCS call

The Test Implementationhttp://test.opendap.org:8080/opendap/data/wcs/Georectified-Grid/MYD11A2.A2004137.h10v05.004.2004147190109_EPSG.MODIS_Grid_8Day_1km_LST.nc.html

DAS responsehttp://test.opendap.org:8080/opendap/data/wcs/contents.html

DDS response

WCS Coverage

f(x,y,z,t) = {T, RH, P,…} Domain => Range

WCS Domain

WCS Range

DescribeCoverage Request

DescribeCoverage Response

GetCoverage Request Example

GetCoverage Request Example

GetCoverage Response Example<?xml version="1.0" encoding="UTF-8"?><OperationResponse xmlns="http://www.opengis.net/ows/1.1" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/ows/1.1 ../owsInputOutputData.xsd"> <ReferenceGroup>

<Abstract>Coverage created from GetCoverage operation request to a WCS</Abstract>

<Reference xlink:href="coverage/image.tiff" xlink:role="urn:ogc:def:role:WCS:1.1:coverage"/>

<Reference xlink:href="coverage/metadata.xml" xlink:role="urn:ogc:def:role:WCS:1.1:metadata"/>

</ReferenceGroup></OperationResponse>

x

y

u

v

(x1,y1)

(x2,y2)

(u1,v1)

(u2,v2)

(u0,v0)

p2

p1(x0,y0)

P1=(deltU,0)P2=(0,deltV)

Figure 2 Source coverage and transformed boundingBox in source coverage’s baseCRS (x,y). Blue area shows boundingBox being extended to the nearest grid posts.

Figure 3 Output coverage and transformed boundingBox and extended boundingBox in output coverage’s baseCRS (u,v). Blue area shows the minimum subset in source coverage.

1. A client specifies a boundingBox in CRS (a,b) and specifies the output grid’s origin (u0,v0) and offset p1 and p2 in the baseCRS of the output grid (u,v) (this baseCRS is specified in wcs:GridCRS).

2. The boundingBox is transformed to the wcs:GridCRS of the source gird (green area in fig. 2) and the extent is extended to a nearest minimum area (blue area in fig.2) that covers the boundingBox. The grid point values of the are subsetted.

3. The boundingBox is transformed to the wcs:GridCRS (u,v) of the output gird, whose grid point locations are determined by the origin (u0,v0) and offset values p1 and p2 defined in the output part of the getCoverage request. The transformed boundingBox (green area in fig. 3) is extended to a nearest grid points that completely include the transformed boundingBox. This output grid is shown in the grid points constructed by the black lines. Note that this output grid may not necessary cover all areas of the minimum subset in the source grid (the blue area in figure 2) as shown in figure 3, dependent on such factors as baseCRSs and offset values of source and output grid.

4. The minimum subset from the source grid (blue area in figure 2) is transformed to the output grid points. 5. In this method, some values in the minimum subset may not be used (e.g., point Pin in figure 3) while some output grid points may not

be available (e.g., point Pout in figure 3). Such issues can be avoid in approach one discussed in the previous chart. 6. In the output grid, the positions are defined at grid points, not grid cells. If the grid is interpreted as composed of grid cells, the grid cells

look like something as shown by the grid constructed by the dashed dark red lines. The position of the center of each such cell is defined (or, cell position is defined at the cell center).

First grid point

Last grid point

a

b

(a1,b1)

(a2,b2)

Figure 1 boundingBox specified in CRS (a,b).

point Pin

point Pout

The following steps describe one of (many?) approaches, called approach two, of a getCoverage request/response for a 2D grid, assumingthat the boundingBox CRS, source grid baseCRS, and output grid baseCRS are different:

a

b

u

v

(a1,b1)

(a2,b2)

(u0,v0)

p2

p1

P1=(deltU,0)P2=(0,deltV)

Figure 1 boundingBox specified in CRS (a,b).

Figure 2 Output coverage extent (blue) and transformed boundingBox (green) in output coverage’s baseCRS (u,v). The output grid origin is at (u0,v0).

Figure 3 sourceCoverage, its baseCRS (x,y) and origin (x0,y0). The The green and blue areas are correspondent to the boundingBox and output grid.

x

y

(x0,y0)First grid point

Last grid point

1. A client specifies a boundingBox in CRS (a,b) and specifies the output grid’s origin (u0,v0) and offset p1 and p2 in the baseCRS of the output grid (u,v) (this baseCRS is specified in wcs:GridCRS).

2. The boundingBox is transformed to the wcs:GridCRS of the output gird and the extent of the output grid is determined based on the transformed boundingBox and the origin (u0,v0) and offsets p1 and p2, by extending the transformed boundingBox to the closest grid points to completely include the boundingBox. The origin (u0,v0) and offsets p1 and p2 are defined in the output coverage’s basedCRS, which is specified/included in the wcs:GridCRS. The transformed boundingBox is shown in green and the output grid extent is shown in the blue area (grid points constructed by the black lines) in figure 2.

3. Values for the grid points in the output grid are derived by determining their positions in the source grid. In the source grid, the green and blue areas show the areas of the boundingBox and the output grid if they would be transformed to the baseCRS of the source coverage. These areas, however, usually need not to be transformed to the source coverage’s baseCRS. A server may chose to transform the blue area so that only a minimum subset of the source grid needs to be read (note that the extent of the minimum subset in the source grid is also dependent on interpolation method.).

4. In the output grid, the positions are defined at grid points, not grid cells. If the grid is interpreted as composed of grid cells, the grid cells look like something as shown by the grid constructed by the dashed dark red lines. The position of the center of each such cell is defined (or, cell position is defined at the cell center).

The following steps describe one of (many?) approaches, called approach one, of a getCoverage request/response for a 2D grid, assumingthat the boundingBox CRS, source grid baseCRS, and output grid baseCRS are different:

MODIS Data in Swath and Lat/Lon Coordinates

ASTER Data in Swath and Lat/Lon Coordinates

OGC Geoscience Gateway

DataCatalog

SemanticCatalog

THREDDSCatalog

Catalog Ingestor

GeoscienceServers

Tools

WC

S

WF

SCS/W protocol WCS protocol WFS protocol

Geoscience Protocols

DataCatalog

SemanticCatalog

THREDDSCatalog

Catalog Ingestor

GeoscienceServers

Tools

WC

S

WF

SCS/W protocol WCS protocol WFS protocol

Geoscience Protocols

WCS-geoscience Gateway Prototype in THREDDS

OGC CSW

The OGC Catalog Service for Web specifies the interfaces, bindings, and a framework for defining application profiles required to publish and access digital catalogues for geospatial data and services.

OGC Catalog UML Model

Common Queryable Elements

OGC CSW Application Profiles

The ISO19115/19119 profile explains how catalogue services based on the profile are organized and implemented for the discovery and management of geospatial data and service metadata which are compliant with the ISO19115 and 19119 standards.

The ebRIM profile explains how services based on the more general OASIS ebXML Registry Information Model are organized and implemented.

Connecting THREDDS to CSW

The first of the following two approaches are adopted:

Mapping THREDDS metadata to ISO metadata and implementing a CSW server based on the ISO profile. Implementing a CSW server for THREDDS metadata based on ebRIM profile.

ISO19115 Metadata Information

MD_KeywordTypeCode

+ discipline+ place+ stratum+ temporal+ theme

<<CodeList>>

MD_TopicCategoryCode

+ farming+ biota+ boundaries+ climatologyMeterologyAtmosphere+ economy+ elevation+ environment+ geoscientificInformation+ health+ imageryBaseMapsEarthCover+ intelligenceMilitary+ inlandWaters+ location+ oceans+ planningCadastre+ society+ structure+ transportation+ utilitiesCommunications

<<CodeList>>

MD_ProgressCode

+ completed+ historicalArchive+ obsolete+ onGoing+ planned+ required+ underDevelopment

<<CodeList>>

MD_Format(from Distribution information)

MD_Usage

+ specificUsage : CharacterString+ usageDateTime[0..1] : DateTime+ userDeterminedLimitations[0..1] : CharacterString+ userContactInfo [1..*] : CI_ResponsibleParty

MD_MaintenanceInformation(from Maintenance information)

MD_Metadata(from Metadata entity set information)

MD_Keywords

+ keyword[1..*] : CharacterString+ type [0..1] : MD_KeywordTypeCode+ thesaurusName[0..1] : CI_Citation

MD_Constraints(from Constraint information)

MD_Identification

+ citation : CI_Citation+ abstract : CharacterString+ purpose [0..1] : CharacterString+ credit [0..*] : CharacterString+ status [0..*] : MD_ProgressCode+ pointOfContact [0..*] : CI_ResponsibleParty

<<Abstract>>

0..*

+resourceFormat

0..*

0..*

+resourceSpecificUsage

0..*

0..*+resourceMaintenance 0..*

1..*+identificationInfo

1..*

0..*

+descriptiveKeywords

0..*

0..*+resourceConstraints 0..*

MD_BrowseGraphic

+ fileName : CharacterString+ fileDescription[0..1] : CharacterString+ fileType[0..1] : CharacterString 0..*

+graphicOverview

0..*

MD_Resolution

+ equivalentScale : MD_RepresentativeFraction+ distance : Distance

<<Union>>

MD_CharacterSetCode

+ ucs2+ ucs4+ utf8+ utf16+ isoIec8859oneTo15+ jis+ shiftJIS+ eucJP

<<CodeList>>

MD_DataIdentification

+ spatialRepresentationType [0..*] : MD_SpatialRepresentationTypeCode+ spatialResolution [0..*] : MD_Resolution+ language [1..*] : CharacterString+ characterSet [0..1] : MD_CharacterSetCode = "utf8"+ topicCategory [1..*] : MD_TopicCategoryCode+ geographicBox [0..*] : EX_GeographicBoundingBox+ geographicDescription [0..*] : EX_GeographicDescription+ environmentDescription [0..1] : CharacterString+ extent [0..*] : EX_Extent+ supplementalInformation [0..1] : CharacterString

MD_ServiceIdentification=

MD_RepresentativeFraction

/+ denominator : Integer

<<DataType>>

Scale(from Uni ts of Measure)

/Scale

Where MD_RepresentativeFraction.denominator = 1/Scale.measure And Scale.targetUnits = Scale.sourceUnits

characterSet: documented if ISO 10646-1 is not used

{MD_Metadata.hierarchyLevelCode = "dataset" implies count (geographicBox) + count (geographicDescription) >=1}

MD_SpatialRepresentationTypeCode

+ vector+ grid+ textTable+ TIN+ stereoModel+ video

<<CodeList>>

Conditional statements:language: documented if not defined by the encoding standardcharacterSet: documented if ISO 10646-1 not used and not defined by the encoding standardhierarchyLevel: documented if hierarchyLevel not equal to "dataset"?hierarchyLevelName: documented if hierarchyLevel not equal to "dataset"?

MD_SpatialRepresentation(from Spatial representation information)

<<Abstract>>

MD_ApplicationSchemaInformation(from Application schema information)

MD_PortrayalCatalogueReference(from Portrayal catalogue information)

MD_MetadataExtensionInformation(from Metadata extension information)

MD_ContentInformation(from Content information)

MD_ReferenceSystem(from Reference system information)

DQ_DataQuality(from Data quality information)

MD_Distribution(from Distribution information)

MD_MaintenanceInformation(from Maintenance information)MD_Metadata

+ fileIdentifier [0..1] : CharacterString+ language [0..1] : CharacterString+ characterSet [0..1] : MD_CharacterSetCode = "utf8"+ parentIdentifier [0..1] : CharacterString+ hierarchyLevel [0..*] : MD_ScopeCode = "dataset"+ hierarchyLevelName [0..*] : CharacterString+ contact : CI_ResponsibleParty+ dateStamp : Date+ metadataStandardName [0..1] : CharacterString+ metadataStandardVersion [0..1] : CharacterString

0..*+spatialRepresentationInfo 0..*

0..*+applicationSchemaInfo0..*

0..*

+portrayalCatalogueInfo

0..*

0..1

+metadataMaintenance

0..1

0..*

+metadataExtensionInfo

0..*

0..*+contentInfo

0..*

0..*

+referenceSystemInfo

0..*

0..*+dataQualityInfo

0..*

0..1

+distributionInfo

0..1

MD_Constraints(from Constraint information)

0..*+metadataConstraints

0..*

MD_Identification(from Identification information)

<<Abstract>>

0..*+resourceMaintenance

0..*

1..*

+identificationInfo

1..*

0..*

+resourceConstraints

0..*

Identification Information

THREDDS Catalog Information Model

THREDDS/CSW Mapping

The first step is to mapping the semantically equivalent metadata itemsbetween the ISO19115 and the THREDDS information models.

THREDDS Catalog Ingestor

The ingestor is a THREDDS catalog server client at the front end. It obtains information of data sets in the THREDDS catalog maps the information to ISO metadata. At the back end, it writes to the CSW database through JDBC. This will require that the ingestor have write permission to the CSW database. It is also planned, if resources are available, to implement the ingestor as a CSW client at the back end. The client can register the THREDDS metadata to any compliant CSW server through CSW protocol.

THREDDS Dataset Inventory Catalog

THREDDS Catalog Ingestor Tool Design

THREDDS DataServer(TDS)

CSI SSSpati al

DB

THREDDS DatasetI nventory Catal og

Parsi ng&Mappi ngCatal og. xml

DataGranul e

BBOX

Organizati on

…...

GMU CSW Search Interface

http://geobrain.laits.gmu.edu/csw/discovery/

GetRecord Request