ECMWF14-16 June 2006 Slide 1

Access to ECMWF data for Research

Manuel Fuentes

Data and Services Section, ECMWF

ECMWF Forecast Products User Meeting

ECMWF14-16 June 2006 Slide 2

Introduction

ECMWF makes data available for Research & Education freely or at a handling charge only

For Commercial use, data is also available through Data Services (Archived and Real-time), with additional cost (information charge)

This presentation focuses on data available for Research & Education

Data access methods

Data sets

Data formats

ECMWF14-16 June 2006 Slide 3

Data Access Services

Archived or Real-time data:Meteorological Archival Retrieval System (MARS)

Dissemination System

Data Services

Data from special projects:Public Data Server

THREDDS/OpenDAP (experimental)

ECMWF14-16 June 2006 Slide 4

MARS: Contents

Main repository of meteorological data:

Operational runsDeterministic Forecast: Atmospheric and Wave models

Ensemble Prediction System

Monthly Forecast

Seasonal Forecast

Monthly DatasetsMonthly means

Climatology

Data from other NWP Centres

ECMWF14-16 June 2006 Slide 5

MARS: Contents (cont.)

Project DatasetsERA-15, ERA-40

PROVOST

DEMETER

Observations

ECMWF Research experiments

Member States experimentsUsing ECMWF models

Using own models: COSMO/LEPS

ECMWF14-16 June 2006 Slide 6

MARS: Access

Direct access to more than 2 PetaBytes of data using pseudo-meteorological language

Access modes:Batch

Interactive: Metview, WebMARS

Remotely from Member States

Discovery (Data Finder) and Browsing (Catalogue)

Restrictions to access valid data

Users: Member States for Research & EducationAccess granted by NMSs

ECMWF14-16 June 2006 Slide 7

Dissemination system

Real-time products

Subscription basedUsers edit requirement specification:

which data (parameter, step, forecasting system, …)

what post-processing (sub-area, grid, resolution, …)

Product generationAll products are generated in parallel on the supercomputer

Data is pushed from ECMWF to users

Operates under a fixed scheduleEveryone gets same data at the same time

ECMWF14-16 June 2006 Slide 8

Data Services

Provide Data and Software to external organisations under rules approved by Council

Data Services CataloguesCatalogue of Archived data and products

Catalogue of Real-time products

Software catalogue

ECMWF14-16 June 2006 Slide 9

Data Services: Archive Catalogue

Contents: MARS contents

Access: On-line costing systemERA-15, ERA-40

Operational data

ObservationsMarine, Aircraft, Vertical soundings

Cost: Handling charge only for Research & Education

Handling charge and Information charge for commercial use

ECMWF14-16 June 2006 Slide 10

Data Services: Archive Catalogue, delivery

TapeDAT

Exabyte

DLT

LTO2

CD

DVD

Internet transferFTP Push

Downloadable

ECMWF14-16 June 2006 Slide 11

Data Services: Real-Time Catalogue

Contents:Selected parameters from Operational forecasting systems

Web products (as a whole)

Access: Dissemination system

www

Cost: Handling charge (Research & Education)

Additional Information charge (Commercial)

ECMWF14-16 June 2006 Slide 12

Data Services: Software Catalogue

Decoders: Free of chargeGRIB (GRIBEX, grib_api)

BUFR

Interpolation software (EMOSLIB)

Magics

MetviewGraphical User Interface

Plotting

Field manipulation

Macro language

ECMWF14-16 June 2006 Slide 13

Public Data Server

1 Terabyte of data online, 2.5 ° resolution:DEMETER

ERA-15

ERA-40

ENACT

Data is downloadable in GRIB, NetCDF or in graphical form

Field manipulation capabilities: sub-area extraction and change of resolution

ERA-40 contents can be obtained on 2 LTO2 tapes at handling charge

ECMWF14-16 June 2006 Slide 14

Public Data Server (cont.)

Service started in 20037000 users

Average 700 GBytes/month (peaks of 1 TB/month)

For Research and Education only

ECMWF14-16 June 2006 Slide 15

THREDDS/OpenDAP

Data access method under investigation

Contents:ENACT

DEMETER

ENSEMBLES

Link to Climate Explorer at KNMI

For Research and Education only

ECMWF14-16 June 2006 Slide 16

Data Formats vs. Data Access Service

GRIB NetCDF Graphical

MARS

Data Services

Dissemination

Data Server

THREDDS/DoDS

ECMWF14-16 June 2006 Slide 17

Future developments

Thorpex Interactive Grand Global Ensemble, TIGGECreation of a Database containing all available global

ensemble in near real-time

3 Archive Centres (NCAR, CMA, ECMWF), 9 Data Providers (BoM, CMA, JMA, KMA, NCEP, CMC, CPTEC, UKMO, ECMWF)

GRIB Edition 2

Initial exchange of data started (2 models)

First users expected by end 2006

ENSEMBLESSome experiments already available in MARS

Interim Re-Analysis

ECMWF14-16 June 2006 Slide 18

Summary

Data Access ServicesMARS, Data Services, Dissemination

Data Server, THREDDS/OpenDAP

Data setsOperational Archive

Archive Catalogue and Real-Time Catalogue

Special projects: ERA, DEMETER, ENSEMBLES, TIGGE

Data formatsGRIB (Edition 1, 2)

NetCDF

Graphical

ECMWF14-16 June 2006 Slide 19

More information

MARS: www.ecmwf.int/services/archive

Data Services: www.ecmwf.int/products/data

Dissemination: www.ecmwf.int/services/dissemination

Data Server: data.ecmwf.int/data

TIGGE: tigge.ecmwf.int

ENSEMBLES: www.ecmwf.int/research/EU_projects/ENSEMBLES

Thank you !