Page 1 © Crown copyright 2005 NESC Workshop 6th-8th September 2005 V-GISC – SIMDAT Gil Ross (Met Office UK) NESC Workshop 6th to 8th September 2005

© Crown copyright 2005 NESC Workshop 6th-8th September 2005Page 1

V-GISC – SIMDATGil Ross (Met Office UK)

NESC Workshop 6th to 8th September 2005


Contents

Part 1 World Meteorological Organisation

Data - Communications now WMO Operational Meteorology

Real-Time Daily Data Flows

WMO Information System GISC - Global Information System Centre

data Portal data Mirror

Part 2 vGISC and SIMDAT

SIMDAT EU Project SIMDAT Meteorology


WMO is….

UN Agency (1950)derived from IMO (1873)

Co-ordinate meteorologyweather

observations forecasts

climate changehydrologydisaster mitigation

187 Members


WMO - Data Exchange

Now GTS Global Telecommunications System

Observations ForecastsWarnings

Private NetworkNode to Node networkNOT an Internet

Future WIS - WMO Information System

Replacement for GTSPortal for Operational MeteorologyCatalogue driven based on -

ISO 19115 Metadata WMO Core Profile


GTS Real-Time Data in Met Office UK

Main Message Switch300,000 bulletins received per day5.2 million bulletins sent/day.

Main File Server17,000 files received/day27,000 files sent/day

Other routes – e.g. Satellite data ~ 27GB/day

Ephemeral and Persistent databases Ephemeral completely repopulated every dayPersistent < 1 TB new data per day


Surface Observations 0000Z 3rd September

http://www.metoffice.gov.uk/research/nwp/observations/data_coverage/index.html


Aircraft observations 000Z 3rd September


WIS - Information collection data flow

Global Information System Centre GISC

GISC

GISC

GISC

GISC

NC NC

NC

NC

NC

NC

NC NC

NC

NC

Data Collection or Product Centre

National Centre NC

DCPC

DCPC

DCPC

DCPC

NC NC

Various communication networks

Global Information System Centre GISC GISC

GISC

GISC

GISC

GISC

NC NC

NC

NC

NC

NC

NC NC

NC

NC

Data Collection or Product Centre

National Centre NC

DCPC Data Collection or Product Centre

National Centre NC

DCPC

DCPC

DCPC

DCPC

NC NC



WIS - Information distribution data flow

National disseminationRequest/reply (Internet)

Nation

al u

sers

Nat

ion

al u

sers

Routine dissemination(Internet, private network,satellite, etc.)

GISC

GISC

GISC

GISC

NC

NC

NC

NCNC

NC

NCNC

DCPC

DCPC

DCPC

DCPC


National disseminationRequest/reply (Internet)National disseminationRequest/reply (Internet)

Nation

al u

sers

Nat

ion

al u

sers



GISC

GISC

GISC

GISC

NC

NC

NC

NCNC

NC

NCNC

DCPC

DCPC

DCPC

DCPC



Part 2

SIMDAT / NESC Workshop Sept 05 11 SIMDAT

V-

GISC

Gil Ross

Met Office

V-GISC – SIMDAT


V-

GISC

vGISC

Instead of each National Met Service having a GISC (Global Information System Centre)

a “distributed” or “virtual” GISC

3 NMSs – Meteo France Deutcher Wetterdienst DWD Met Office UK – MetO

and two DCPCs Data Collection and Production Centre ECMWF EUMETSAT


V-

GISC

SIMDAT

4 years project funded by the EU

Contract with EU was signed on 1 September 2004

Data Grids for Process and Product Development using Numerical

Simulation and Knowledge Discovery

SIMDAT focuses on 4 application area:

product design in automotive and aerospace,

process design in pharmacology

service provision in meteorology

Budget of 11 M € of which 10.5% for meteorological activity

project managed by ECMWF


V-

GISC

SIMDAT - Consortium


V-

GISC

SIMDAT

Seven Grid-technology areas have been identified to achieving SIMDAT objectives:

Integrated Grid infrastructure offering basic services to applications Access to data distributed on Grid sites Management of Virtual Organisation Workflows Ontologies Integration of analysis services Knowledge Services

Not all will be addressed in Meteo - SIMDAT


V-

GISC

SIMDAT - Strategy

. Virtual Data Repository. Introduction of grid technologies research

The heart of the issue is data

Phase 1: Connectivity Phase 2: Interoperability Phase 3: Knowledge

. Deployment of Grid infrastructure with particular attention to data transport and management.. Distributed DB access

Workflows for next-generation aggregated knowledge capture, discovery and mining.


V-

GISC

SIMDAT - vGISC

vGISC working groups adopted by SIMDAT (meteorology)

SIMDAT is a technical solution for vGISC type operations

will NOT deliver a vGISC

has specified requirements similar to vGISC

is investigating a vGISC Virtual Organisation

is developing catalogue operations

has developed message infrastructure

SIMDAT 18 month test system due - NOW!


V-

GISC

SIMDAT requirements


V-

GISC

DCI Distributed Architecture


V-

GISC

VGISC Demonstrator: UKMO local db (1)

JEDDS

Internet

Internet


V-

GISC


Local database is a derivative of JEDDS – the ‘Joint Environmental Dynamic Data Server’

Originally developed by Met Office to provide data on demand to UK military

Features:

Live data feed; continuously refreshed

Serving global TAF, METAR & GRIB bulletins

Request-oriented SOAP web-service interface


V-

GISC


Database DMZ

EAL4Firewall

Internal

JEDDS ++

FROST

Application DMZ

Portal &Catalogue

(local)

Router

SIMDATDemonstratorDeployment

July 2005Jeremy Tandy

Push (ftp)

Internet

Pu

ll

User

Pull (http)

Portal &Catalogue(Remote)

Synchronisation


V-

GISC

Catalogue Statusin Test system


V-

GISC

Working decisions (1)

Teleconference, wiki & email used for remote collaboration

focus on HOW the catalogue operates, not the minutiae of metadata standards

use a ‘random’ UID scheme and not try to ‘overload’ identifiers with semantics

use a simple ‘Google-like’ and/or directory-based search interface

3 classes of search – by Keyword, time and/or location

ignore internationalisation for now

Issue: WMO Core (& ISO19115) is TOO flexible – there are too many options for spatial & temporal descriptions … we need best practice ‘application schemas’


V-

GISC

Working decisions (2)

Metadata

create linking schemas using the WMO Core Profile by using additional namespaces (i.e. <v:id>)

metadata documents should be split into a ‘hierarchy’ of XML ‘fragments’ are used within the catalogue

model for merging metadata fragments will uses Java templating rather than an XSLT-based approach … key issue to demonstrate the concept is WHAT the fragments are not the mechanics of HOW they are merged. This position may be revised post-demonstrator

Agreement: nomenclature …• a specific SYNOP (12:00UTC, 16/06/05 at Heathrow) is an instance• the SYNOP ‘product’ is a generalised instance that requires instantiation parameters to

identify a specific synop instance


V-

GISC

What Now?

Metadata demo

http://www.ecmwf.int/products/realtime/d/vgisc/

Re-consider parked issues

extend beyond real-time data

Project tasks

Documents

Page 1 © Crown copyright 2005 NESC Workshop 6th-8th September 2005 V-GISC – SIMDAT Gil Ross (Met Office UK) NESC Workshop 6th to 8th September 2005