IPY Data Management

Archival and Data Access StatusCatalogue Interoperability Status

WIS-relation

Øystein Godøy

The IPY operational snapshot

Still working on cataloguing WMO essential data (and to a certain extent data access)

ECMWF and HIRLAM NWP (METNO) with online access (OpeNDAP, HTTP, WMS)

OSISAF data with online access

THORPEX data sets in the North Atlantic winter 2008 mainly available

METNO surface observations added

Working on drifters, POPs, ITPs, IMBBs, ...

Constraints

Limited funding requires prioritisingThe full funding of METNO IPY-activities is

unlikely to be received, implies a change in time plan/ambitions

Focus on ECMWF access and transformationDone

Focus on Metadata exchangeAdvantage OAI-PMH, easier/cheaper to

implementCIP is ISO23950 related, needed in the future,

but too resource consuming within the current funding scheme

Exchange of metadata

Various technologiesOAI-PMH

IPY, CEOS WGISS

CSW 2.0.2/ISO23950INSPIRE, WIS, GEOSS, CEOS WGISS CIP

OAI-PMHLow cost/easy to implementRecommended for harvesting metadata by

CEOS WGISS Interoperability Handbook 1.1 http://wiki.ieee-earth.org/Documents/CEOS_WGISS_Interoperability_Handboo

k

OAI-PMH

Software developed and testedNorwegian project involving NP, HI, NILU,

METNOInternational Operational Data CoordinationUsing DIF for exchangeGaining experience with controlled vocabulariesSetting up run time rulesRegular (daily) export/import at 3 sites, to be

extendedEvaluated for NDACC-AVDC-EVDC-GEOMON-

GECA-EARLINET interoperability

Catalogue interoperability

DOKIPY do harvest IPY-metadata from GCMDWorking adaptions to local structures requried

DOKIPY will link with NSIDC

DAMOCLES will link with CADIS

IPY in Sweden is using the Open Source system for metadata developed by METNO, will implement OAI-PMH supporthttp://wiki.met.no/metamod/

ArcticNet asked for exchange of metadata

AADC to be asked

Catalogue interoperability continued

Mark Parsons coordinating efforts with othersPANGEA, AADC, ... ???

IPY Discussion Forum have been used to highlight problemsInsufficient ISO Topic CategoriesHarmonisation of controlled vocabularies

Community developed like CF versus single institution controlled like GCMD

How to handle changes, updates, backwards compatibility, etc...

IPY is interdisciplinary...

Catalogue interoperability continued

Main show stoppers so far has not been implementation of the software

But rather handling various controlled vocabularies and mapping between these

The main reason being that IPY is interdisciplinary and data centres need to maintain existing commitments and catalogues

Controlled vocabularies are essential for smooth catalogue interoperability

Remote Sensing data sets

No dialogue with space agencies since last meetingSoftware implementation required first

Experience with exchange mechanisms required to set up a proper system

Transformations of vocabularies neededRAMP added at metadata levelIDN and eoportal added at metadata level

In principle exchange with other OAI-PMH systems should be easier now

Remote Sensing data setscontinued

Adding support for CIP does not seem feasible in the present situation

More likely to be added after IPY after GEOSS, INSPIRE and WIS harmonisation of systems

IPY Operational Data Coordination Technology

http://ipycoord.met.no/

METAMOD 2.x DAR -> WIS DCPC To integrate IPY or scientific needs more seamless with the

WMO community

Archives

OAI-PMH (adding CSW/ISO23950 in time)

Shopping basket in time

THREDDS Data Server HTTP/OpeNDAP

Better support for WMO formats

Although trouble experienced with the BUFR format used

OGC WMS (focusing at high latitudes)

WIS-relation

VGISC project temporarily stranded

Different needs within consortium (GISC/DCPC)The focus for METNO in support of IPY is

DCPC, not GISC functionality

VGISC as a discussion forum still exist and attempts to find a solution

METNO has discussion with EU-funded BALTRAD-project concerning possible SIMDAT-use

WIS-relation

VGISC partners examine possible software solutions to fulfil WIS requirementsGap analyses based upon what the individual

partners have got and the required functionality have been and is being performed by partners

Integration and adaptation of existing components

In most cases a mix of internal and external software

UserRequests

I3

User Role AssignmentProcedure Applicable Data Policy

I2

I1

Observations, Productsand Associated Metadata

Reference Information(WIGOS, IOC, etc)

AuthorizedRequests forInformation

Information SearchRequests

Catalogue BrowseRequests

Subscription

Information

« Ad Hoc »

metadata

Monitoring DataFrom all functions

Control Actions toAll functions

O2

Reports of Performanceagainst Requirements

Delivered Informationvia « Push » Services(time-critical &non-time-critical)

Delivered Informationvia « Pull » Services(time-critical &non-time-critical)

O1

Information Search Results

Information Services

DisseminationMetadata

DAR MetadataCatalogue

POSSIBLE CHOICES

SIMDAT base

Sun OpenSSO baseGeonetwork base

MSSSMART / GEMS

A3

Maintain and ExposeCatalogue of Servicesand Information

ManageRequests

CatalogueAccess

« Ad Hoc »

Routine« Push »

Cache

A6Manage SystemPerformance

Monitoring

Administration

Assign User Role &Authorise Access toInformation A2 & A4

Authentication

Authorisation

User &Data Policies

Portal

Make / FollowRequest

Browse /Search DAR

Admin /Monitoring

Portal

A1

Collect Observations,Create Products and Archive Information

Synchro &Harvesting

Replication

A5Deliver Informationto Users

Legacy Databases

Collection

Retrieval