Safe Replication and Data StagingCore Services of the EUDAT CDI

Johannes Reetz, RZG

2nd EUDAT conference

Rome, 29 Oct 2013

2

domain of registered data

individual value

preservationacquisitiongenerationdescription

enrichmentprocessingreductionanalysis

Data Stage-Out

publication

Safe Replication

community valueDat

a Sta

ge-In

society value

IdentifierService

EPIC Handle PID

4

Replication from repositories to data storages in different administrative domains

• (long-term) archiving and preservation

• optimize access for users from different regions

• bring data closer to powerful computers for data analytics

Typical policies triggered by Community Data Managers:• Replicate collection X from my repository to data centres A and B• Store the replica safely for N years• Check the integrity of the replica every M years

5

Transferring data from EUDAT storages to compute facilities

• reliable, efficient, easy-to-use tools to manage data transfers

• ingest data into the EUDAT domain of registered data

Each data centre (administrative domain) has a dedicated Handle namespace (prefix)

replica storages

repositories

6

EPICHandleService

http://cmon.eudat.eu

EPICHandleService

enabled EUDAT sites

replica storages

repositories

7

http://cmon.eudat.eu

EPICHandleService

GridFTP

8

CLARIN (MPI-PL/TLA)production

9

CLARIN (EKUT)production

10

CLARIN (CUNI/LINDAT)production

ENES (DKRZ/CMIP5)

11

production

12

EPOS (INGV)production

13

VPH (UCL)production

14

EPOS (ORPHEUS/ KNMI)work in progress

15

INCF (KTH)work in progress

16

DiXa (EBI)work in progress

Track 1 – Agenda – Session 1 17

14:00: B2SAFE and B2STAGE: Core Services of the EUDAT CDI, Johannes Reetz

14:10: B2SAFE adoption in the EPOS community, Claudio Cacciari

14:25: B2SAFE for Replicating Linguistic Resources, Willem Elbers

14:40: B2SAFE Setup with a Dspace Repository, Pavel Stranak

14:55: VPH and the Biomedical Scientific Case for EUDAT, Peter Coveney

15:10: Community integrated B2STAGE tools, Stefan Zasada/Giuseppe Fiameni

15:35: B2SAFE Practical Policies, Willem Elbers

15:45: B2SAFE Data Policy Manager, Maria Francesca Iozzi

Track 1 – Agenda – Session 1

use cases in production

Track 1 – Agenda – Session 1 18

14:00: B2SAFE and B2STAGE: Core Services of the EUDAT CDI, Johannes Reetz

14:10: B2SAFE adoption in the EPOS community, Claudio Cacciari

14:25: B2SAFE for Replicating Linguistic Resources, Willem Elbers

14:40: B2SAFE Setup with a Dspace Repository, Pavel Stranak

14:55: VPH and the Biomedical Scientific Case for EUDAT, Peter Coveney

15:10: Community integrated B2STAGE tools, Stefan Zasada/Giuseppe Fiameni

15:35: B2SAFE Practical Policies, Willem Elbers

15:45: B2SAFE Data Policy Manager, Maria Francesca Iozzi

use case in production

Track 1 – Agenda – Session 1 19

14:00: B2SAFE and B2STAGE: Core Services of the EUDAT CDI, Johannes Reetz

14:10: B2SAFE adoption in the EPOS community, Claudio Cacciari

14:25: B2SAFE for Replicating Linguistic Resources, Willem Elbers

14:40: B2SAFE Setup with a Dspace Repository, Pavel Stranak

14:55: VPH and the Biomedical Scientific Case for EUDAT, Peter Coveney

15:10: Community integrated B2STAGE tools, Stefan Zasada/Giuseppe Fiameni

15:35: B2SAFE Practical Policies, Willem Elbers

15:45: B2SAFE Data Policy Manager, Maria Francesca Iozzi

Track 1 – Agenda – Session 1

practical policies and policy management

Johannes Reetz RZG / MPG

Claudio Cacciari CINECA

Willem ElbersMax-Planck-Institute for Psycholinguistics

Pavel StranakCharles University

Peter CoveneyUniversity College London

Stefan Zasada University College London

Giuseppe FiameniCINECA

Maria Francesca Iozzi SIGMA/University of Oslo

SAFE REPLICATION AND DATA STAGING - Speakers

20