Ocean Information Technology New approaches to ocean data management

COD, Brussells, 25-27 Nov

Ocean Information TechnologyOcean Information TechnologyNew approaches to ocean data New approaches to ocean data

managementmanagement

Neville Smith

BMRC, Australia


The The Motivation … The Motivation … The IssuesIssues

We have only 5-10% of the needed telemetry capacity.

Data Assembly - no broad agreement on how we ensure coherent, integrated data sets.

Lack of agreement on standards, formats and practices.

Poor modes of data transport, both routinely for operational requirements and for science.

Poor or inappropriate uptake of modern information technology.

No proper assessment of the future need. …


Some FoundationsSome Foundations

Some good examples of D&IM at work IODE, JCOMM OBIS (CoML), Argo DM, ODINAfrica, US IOOS Data plans …

but poorly integrated

• “not invented here” syndrome many realizations of the same function/form

Too much in-house technology, too little out-sourced functionality• Too little fusion / cross-sector adoption

No / little engagement with scientific community• Innovation not within

– …


Rationale (The Why)Rationale (The Why)

• Approach IT as you would a Science Program

Effective telecommunications Common standards, practices and protocols

(metadata management) Data and product service matched to the

participants and users Innovative data inquiry, access and delivery Intra-operability and interoperability …


The Components … the The Components … the “How”“How”

1. Telemetry

2. Metadata Management

3. Data assembly, data set integrity, quality control

4. Data circulation and transport

5. Archives and archaeology

6. Applications and user interfaces

7. Capacity enhancement, training

8. Governance, oversight, metrics


1. Telemetry and 1. Telemetry and CommunicationsCommunications

… getting the data “in” (instructions out?)

a) Availability and capabilityBandwidth, 2-way comms, connectivity, …

b) Requirements for climate, open ocean

c) Requirements for coastal regions

d) Polar / remote regionsBandwidth

… enabling


2. Metadata Management2. Metadata Management

IODE, JCOMM (CBS) and others have made considerable progress

Data

Syntactic Metadata

Semantic Metadata• Data about data …

• Its environment

• Its structure

• Its meaning

• Keywords


2. Metadata …2. Metadata … Developing an agreed vocabulary

– Perhaps agreeing on a Markup Language: MarineXML or … (e.g., IODE/ICES Project)

– An essential element for intra-operability, interoperability and exchange

– Non-physical / unconventional: a challenge– Complying with mandated requirements

Complete, consistent, searchable metadata– Keywords, geospatial references– Adopt standardized, shared vocabularies– Developing catalogues– How to do highly effective searches in distributed

systems? How to automate, encourage, …


3. Data assembly, quality 3. Data assembly, quality control, …control, …

Performance level monitoring

More formality, accepted procedures, …– From Level 0 (raw) through Level 3 (analysed)– TOGA, WOCE, JGOFS experience


3. Datum and Data Set 3. Datum and Data Set IntegrityIntegrity

Develop methodology for uniquely identifying original data and variants

• Argo is being used as one test-bed• Original data given and always retains “tag”• Variations referenced against original

Accreditation– Eg, at level 2: a) no QC; b) auto; c) scientific– Institute A, A+, A++ system, esp. for c)

• Recognize value adding • Recognize scientific involvement• Agreed peer-review system• …

?


4. Data Circulation and 4. Data Circulation and ServiceService

TimelinessTimeliness, efficiency Architecture for data/product serving and exchange

• Data packets and sets arrive with standard metadata and “tag”• Systems like IDD are used to “push” data to routine users• The sources may be real-time or high-quality delayed-mode• Each ocean data distribution centre has instruction sets for

distribution; event driven Also request-reply (pull) servers

– US Data Communicationinitiative …


US IOOS ModelUS IOOS Model

Circuits, data conveyers


Together Together New New FunctionalityFunctionality

The XML standard (or an equivalent) should ensure all data are properly described / characterized

Some of the needed metacode can be generated automatically

The “tags” allow identification and removal of duplicates

There is immediate identification of non-conforming data insertion

No need to be a major centre to participate (provider or user)

Some traditional data archive (backup) could be automated


5. Archives, Centres5. Archives, Centres

Reference other sessions Centres might specialise in a type of service

– Routine versus ad hoc / itinerant– Sophisticated versus non-specialist– Strong link to User Interface theme– Catalogues and Portals: Modern Library

Irreplaceable observations, data products of lasting value and associated metadata preserved in perpetuity– Accessible using standards, keywords– Integrated into real-time and delayed mode data transport

mechanism

Data archaeology


6. The User Interface6. The User Interface

Must make data and information:– Available, usable, accessible, …

Scope– Assess “external” user requirements

– “Wizards” to simplify interface

– Assess “internal” user requirements– Robust interfaces; guaranteed responses

Several existing innovative client interfaces– Live Access Server and derivatives– GIS Systems; Web Map Server


Menu for UKFOAM &POLCOMSdata

http://www.nerc-essc.ac.uk/las Daily 1/9o FOAM N Atlantic data

SST for 7 Jun 2002

E.g., E.g., GODAE GODAE LAS LAS Product Product Server: UKServer: UK


7. Capacity, “education” … 7. Capacity, “education” …

Some of the greatest challenges are in the area of community outreach and organizational behavior rather than in technology! [US IOOS DCS SC]

Challenges– Breaking down traditional barriers– Capacity enhancement: even, “fair”– Adoption and acceptance– Uneven access to technology, people

Reference other presentations


ConclusionConclusion

There is broad consensus on the need to act We have an outline of the strategy: why, how Enormous potential to impact science and

applications; not discipline bound Architects and engineers are already busy Critical that we have broad participation Starting now– http://ioc.unesco.org/oit


Thank youThank you

MOI?

Documents

Ocean Information Technology New approaches to ocean data management