Upload
nodin
View
49
Download
0
Tags:
Embed Size (px)
DESCRIPTION
JERICO WP5 Data management and distribution. Alessandro Crise Istituto Nazionale di Oceanografia e di Geofisica Sperimentale. JERICO Project Kick-Off meeting Maison de la Recherche Paris, 24-25 May 2011. JERICO WP5 Data management and distribution: main goal. - PowerPoint PPT Presentation
Citation preview
JERICO WP5
Data management and distribution
Alessandro Crise Istituto Nazionale di Oceanografia e di Geofisica Sperimentale
JERICO Project Kick-Off meetingMaison de la RechercheParis, 24-25 May 2011
JERICO WP5Data management and distribution: main goal
The main goal is to design the JERICO Data Flow and Management infrastructure based on the implementation of an end-to-end ‘open and free’ conduit from data to users and upon the guidelines proposed in the GOOS Report 148 (2005).
The implementation will be consistent and complementary with the major European initiatives for the establishment and coordination of infrastructures for the management and distribution of data and products in Europe which can use JERICO data (SeaDataNet, EuroGOOS Regional Centres, MyOcean).
YET ANOTHER DATA MANAGEMENT SYSTEM?: NO!
WP5 Structure
JERICO will need to ensure that the flow of real-time and delayed mode will be reliable, accessible and easy to distribute. The WP5 structure:
• Creating value for measured data: from raw data issued from the sensors to validated data stored and marked. • Facilitating data and meta-data flow to existing data repository infrastructures (SeaDataNet); • Helping in delivering real-time JERICO data to ROOSs and My Ocean
Task 5.1: Quality assurance: create value for measured data
Task responsible: OGS
Partners: HCMR
Duration M1-M42
• This task is aimed at adding value to observations by associating them with characteristic uncertainties.
• By definition, the “expected” accuracy associated with an observation is an attribute of the measurement chain that produces it.
Task 5.1: Quality assurance objectives
The objectives are:
Collecting pre-existing knowledge regarding the key identified sensors and critically revise state-of-art literature on in lab and in situ performances of the most popular sensors (in conjunction with WP4)
defining procedures for ascribing a total uncertainty to measurements that will account for the actual circumstances and conditions under which the measurements were taken.
Testing and verifying uncertainty computational procedures
Task 5.1: Quality assurance methodologies
. MethodologyA novel model of a-priori uncertainty will be proposed as a sum of fixed and time-dependent source of error and uncertainty (precision and accuracy, drifts, experimental uncertainties, the representativity error, etc.) The variables temperature, salinity and fluorescence/chlorophyll will be considered at first. Controlled experiments will be performed in order to document and validate the different uncertainty components composing the final uncertainty estimates. All the relevant assessments carried out in WP4 will be included as basic information for the combined uncertainty estimation. WP4.1 will provide an updated summary of these achievements to WP5.1, no later than the dates set down in the WP4 D4.2 delivery schedule (M14)
Task 5.2: Harmonization of Delayed Mode data management procedures with SEADATANET
Task responsible: IFREMER,
Partners: HCMR, MUMM, OGS
Duration: M1-M42
• This task will define, establish and oversee the data management infrastructure for dealing with delayed-mode data in JERICO.
• The goal is to provide JERICO users with relevant informations for seamless data integration with the EU SEADATANET initiative.
Task 5.2 Methodology
- Survey the existing delayed-mode data handling practices of JERICO partners.
- Formulate a viable proposal for a common JERICO delayed-mode data management platform that can serve to reinforce the SEADATANET effort;
- the guidelines established by DG-Mare/EMODNET and WISE-Marine will be
also taken in account when doing this.- Create common vocabularies for JERICO delayed-mode data
formats and meta-databases, building upon the work that has already been done in SEADATANET.
- Implement delayed-mode data and metadata formats that are compatible with those of SEADATANET (in compliance with the EU INSPIRE Directive);
- the Sensor Web Enablement family of the OGC consortium family of standards (SensorML, Sensor Registry, O&M) will be considered in the implementation.
- Ensure easy sharing and secure archival of delayed-mode data within JERICO by employing common data transport formats and storage criteria making use of the experience gained by SEADATANET.
Task 5.2 Metadata implementation
Discovery metadataTo let all know what have been deployed during the project (where, what, how, who…)
To help to retrieve data related to observatories
To be compliant with European directives (Inspire)
Enabling technology
Use of SeaDataNet EDIOS directory to describe observatories
Typing / Generation of metadata using SeaDataNet procedures and tools (Mikado) complemented with automatic tools developed in Emodnet-Physics to better use the RT Netcdf files provided in RT by the ROOSs
Task 5.2 Data implementation
Delayed mode dataDelayed mode quality checkUniform file format (based on OceanSites NetCDF, compatibility with CF convention)
+ distribution in ASCII flat file format (Ocean Data View)
Full metadata (Common Data Index) generated by Mikado for long term reuse
Data access via web portal (all users)
Task 5.3 Harmonization of Real Time data anagement procedures with MyOcean and EuroGOOS & the institution of
data access services for JERICO TOP activities
Task responsible: IFREMER,
Partners: CNR, NIVA, IMR, HCMR,PUERTO, SMHI
Duration: M1-M42
• This task will define, establish and oversee the data management infrastructure for dealing with real-time data in JERICO.
• The goal is to provide for seamless integration with the EuroGOOS and EU Marine Core Service’s MyOcean component and in order to render JERICO real-time data accessible for use in generating downstream services and products.
Task 5.3 Methodology
• Survey the existing real-time data handling practices of JERICO partners.
• Formulate a viable proposal for a common JERICO real-time data management platform that can serve to reinforce EuroGOOS and MyOcean
• Create common vocabularies for JERICO real-time data formats and metadatabases, based on any existing
• recommendations by MyOcean/EuroGOOS for doing this.
• Implement real-time data and metadata formats based on any existing recommendations by EuroGOOS and MyOcean for doing this.
• Reconcile, wherever possible, the data quality assurance procedures/protocols for real-time data amongst the JERICO partners (link to WP4, WP5.1).
• Ensure easy exchange of real-time data within JERICO by employing common data transport formats and storage criteria, based on any existing recommendations by EuroGOOS/MyOcean for doing this.
WP5.3 Implementation
Real-Time dataReal-Time Quality check based on EuroGOOS/MyOcean procedure that would need to be extended to new parameters whenever possible
Uniform file format (based on OceanSites NetCDF, compatibility with CF convention)
ftp file server (with an index file) per region
Oceanotron (OpenDap protocol) for direct access by remote software
WP5 in JERICO Data flow
WP5 act as mediator in JERICO Data Flow ModelA priori quality assurance (WP4)
Data management procedures (WP5)• Time-dependent data uncertainty estimation• Real-time data management• Delayed mode data management
Value added products and data dissemination (WP6)
EuroGOOS ROOSs engagement (WP2)
Demonstation of data provision (WP7)
SeaDataNet
MyOcean
Emodnet Physical Parameters
EuroGOOS
MyOcean Insitu TAC implementation of EuroGOOS regional approach
MyOcean In-situ TAC role
Limited number of parameters: Physical: T&S, current, sea levelBiogeochemical: Chlorophyll/Fluorescence, Oxygen, Nutrients
Integrate in-situ data in product accessible through global and regional portals
Common formatCommon NRT QCCommon Quality flagsCommon distribution toolsSingle access point
Ensure a minimum level of quality on the data deliveredIn Near real time ( 24h to a week)In delayed mode
Assessment of the products at basin scale
The European Commission, represented by DG MARE, concluded a service contracts aiming to:
Create a prototype components of the European Marine Observation and Data Network (EMODnet).
Provide access to archived and real-time data on physical variables in Europe's seas and oceans
determine how well the data meet the users needs.
EMODNET PHYSICAL PARAMETERS
THE EMODNET PP PILLARS
EMODnet PP
EMODnet PP aims to bridge the communities behind MyOcean, EuroGOOS and SeaDataNet in order to strengthen their position inside the Commission agenda on IS and expand in future the participation to all partners
MyO
cean
Eu
roG
OO
S
Sea
Dat
aNet
JERICO WP5
NEW FEATURES
Add new stations and models
Add profiles
Make use of the results from ECOOP
Add data from FerryBox and RV
Common quality control
Link to EDIOS
SEPRISE data in SDN ODV format
Put data on GTS
E-HYPE
EuroGOOS SEPRISE network
Cooperation between MyOcean in-situ TAC and SeaDataNetMoU signed Beginning 2010
Observing systems
Marine governmental agencies and research institutes (including NODCs)
Other observations(e.g. scientific
cruises…)International organisations
(IOC/IODE, ICES, ICSU World Data System…)
delayed-mode
real-time
real-
time
Metadata(e.g. EDIOS)
Metadata(e.g. EDIOS)
VocabulariesVocabularies
Metadata (EDIOS, CDI, CAMIOON,
SensorML)
Metadata (EDIOS, CDI, CAMIOON,
SensorML)
Yearly regional products(climatologies, datasets)
Yearly regional products(climatologies, datasets)
Re-analysis users
Re-analysis users
UsersUsers
Operational users
Operational users
MyOcean in-situ TAC
SeaDataNet
Direct usersDirect users
Direct usersDirect users
Data integration(QC, « best copy », archival, re-analysis
datasets
Data integration(QC, « best copy », archival, re-analysis
datasets
Technical WGTechnical WG
JERICO role in the European Data Management landscape
International organisations (IOC/IODE, ICES, ICSU World Data System…)
delayed-mode
real-
time
Metadata
(e.g. EDIOS)
Metadata
(e.g. EDIOS)
VocabulariesVocabularies
Metadata (EDIOS, CDI, CAMIOON,
SensorML)
Metadata (EDIOS, CDI, CAMIOON,
SensorML)
Yearly regional products(climatologies, datasets)
Yearly regional products(climatologies, datasets)
Re-analysis users
Re-analysis users
UsersUsers
Operational users
Operational users
MyOcean in-situ TAC
SeaDataNet
Direct usersDirect users
Data integration(QC, « best copy », archival, re-analysis
datasets
Data integration(QC, « best copy », archival, re-analysis
datasets
Technical WGTechnical WG
ObservingNetwork
Information & evidence pipeline
JERICO Coastal
Observatory Network (WP7)
Web Portal
real-time
Assessmentdatabase
‘EM
EC
O’ like
DataXML, CSV
‘EMECO-like’ Web –
enabled Datatools
(WP6)
WP5 Milestones/Deliverables
ACTIONS
To propose a formal agreement with SDN, EuroGOOS/MyO for JERICO data provision, to ensure more visibility to JERICO project
To find an agreement with DG MARE/EMODNET for the availability to JERICO of the developments achieved/to be achieved in EMODNET Physical Parameter lot.
Thanks!