The use of standard OGC web services in integrating distributed model, satellite and in-situ

datasets

Alastair GemmellJon BlowerKeith HainesEnvironmental Systems Science Centre & Reading e-Science Centre, University of Reading, UK

Martin PriceUK Met Office

Keiran MillardQuillon HarphamHR Wallingford, UK AGU Fall Meeting December AGU Fall Meeting December

20082008

Talk Outline

Context to the work

Potential problems

Useful tools and technologies

Our solution

Summary

Context to the work

Part of ECOOP project: European Coastal Operational Oceanography. 71 partners across most European countries.

Task was to integrate model, in-situ, and satellite data feeds from various project partners into one portal.

Users should be able to compare observed data with the model outputs.

Initial focus was on Ecosystem-relevant data in the North Sea. This has been expanded to other data.

Potential problems

Lack of interoperability

Heterogeneity in data formats

Data dispersed around Europe

Useful tools and technologies

OGC standards (www.opengeospatial.org)

Web Map Service (WMS)

Web Feature Service (WFS)

Climate Science Modelling Language (CSML)

THREDDS / OPeNDAP

Useful tools and technologies – OGC standards

WMS = Web Map Service. Serve geo-referenced images. Ideal for model output (also satellite data).

WFS = Web Feature Service. Serve geo-referenced points, lines, polygons. Ideal for in-situ observations, trajectories etc.

OGC has specifications for these services, allowing data to be served in a consistent manner.

Applications know what data format to expect and how to ask for it.

ncWMS: OGC-compliant WMS for NetCDF developed at Reading e-Science Centre (ncwms.sf.net).

Connects to Godiva2 web client which uses OpenLayers to display data.

Useful tools and technologies – CSML

Climate Science Modelling Language (csml.badc.rl.ac.uk)

A standards-based way of representing data features pertinent to the Climate Sciences.

13 main feature types including profiles, trajectories, swaths, timeseries.

Provides a common view onto datasets, independent of their storage format or physical location. Ideal for integrating diverse data products

We are developing a set of reusable Java libraries that embody the CSML concepts - can then apply these techniques to a number of other projects.

Useful tools and technologies – THREDDS

THematic Realtime Environmental Distributed Data Services (www.unidata.ucar.edu/projects/THREDDS/)

Enables data providers to serve NetCDF and similar data easily online via OPeNDAP protocol. Subsetting of data can be built into request

THREDDS now contains ncWMS bundled in (http://www.unidata.ucar.edu/projects/THREDDS/tech/TDS.html).

Data can be served by THREDDS either via OPeNDAP (e.g. obs data), or WMS (e.g. model images).

Diverse datasets held in different places can read in via THREDDS servers at each institute.

Our solution

ESSCWeb Portal

ESSCncWMS

web app.

CSML

ESSCECOOP Obs

web app.

ESSCPOLCOMS

MRCSmodel

(Physical)

CEFASSmartBuoys

SMHISEPRISE In-situ data

ESSCFerrybox (NetCDF)

HR WallingfordWFS

ASCII

LOCAL

REMOTE

PMLTHREDDS Server

CF-NetCDF

PMLTHREDDS Server

PMLPOLCOMS-

ERSEM MRCS model(Biological)

Comparing / co-plotting datasets can catch errors!

This looks like a suspiciously large and constant difference between obs and model

Summary

Historically data providers in different disciplines used different formats. This created barriers to multidisciplinary work.

Increasingly there are common standards and conventions for formatting and serving data. There is gradual uptake of these by providers.

We have found OGC WMS and WFS effective solutions for working with model and observed data respectively.

CSML was useful as a means of representing the diverse data as common feature types.

These tools and technologies allows diverse data to be brought together for the first time – this can reveal new trends in the data, as well as helping with quality control.

Summary

Thanks for your attention

a.l.gemmell@reading.ac.uk

www.resc.reading.ac.uk