A FAIR, Domain-aware (Meta-)Data Sharing Platform for … · 2019. 7. 12. · meta-data paramount...

A FAIR, Domain-aware (Meta-)Data Sharing Platform

for (Experimental) Material Science Markus Scheidgen

markus.scheidgen@physik.hu-berlin|fhi.mpg.de

Nomad-FAIRDI Workshop, Berlin July 2019

Agenda

data, meta-data, and domains

FAIR and domain aware (meta-)data sharing

NOMAD’s domain aware search for material science data

data and meta-data

data and meta-data

010101 101100 010011 110111

raw datadata

data and meta-data

processed data

010101 101100 010011 110111

raw datadata

data and meta-data

processed data

010101 101100 010011 110111

raw datadata

data and meta-data

meta-datainstrumentationprocessed data

010101 101100 010011 110111

raw datadata

data and meta-data

meta-datainstrumentation human entryprocessed data

010101 101100 010011 110111

raw datadata

data and meta-data

1 2 3 4

preparation

meta-datainstrumentation human entry electronic lab notebooksprocessed data

010101 101100 010011 110111

raw datadata

data and meta-data

1 2 3 4

preparation

meta-datainstrumentation human entry electronic lab notebooksprocessed data

010101 101100 010011 110111

raw datadata

data and meta-data

1 2 3 4

preparation

meta-datainstrumentation human entry electronic lab notebooksprocessed data

010101 101100 010011 110111

raw datadata

data and meta-data

1 2 3 4

preparation

meta-datainstrumentation human entry electronic lab notebooksprocessed data

010101 101100 010011 110111

raw datadata

data and meta-data

HDF5HDF5HDF5JSONJSONXMLJSON

XMLJSONTXT

1 2 3 4

preparation

meta-datainstrumentation human entry electronic lab notebooksprocessed data

010101 101100 010011 110111

raw datadata

data and meta-data

HDF5HDF5HDF5JSONJSONXMLJSON

XMLJSONTXT

1 2 3 4

preparation

meta-datainstrumentation human entry electronic lab notebooksprocessed data

010101 101100 010011 110111

raw datadata

data and meta-data

HDF5HDF5HDF5JSONJSONXMLJSON

XMLJSONTXT

TXT JSONHDF5

1 2 3 4

preparation

meta-datainstrumentation human entry electronic lab notebooksprocessed data

010101 101100 010011 110111

raw datadata

data and meta-data

HDF5HDF5HDF5JSONJSONXMLJSON

XMLJSONTXT

1 2 3 4

preparation

meta-datainstrumentation human entry electronic lab notebooksprocessed data

010101 101100 010011 110111

raw datadata

data formatsdatabases documentation

schemas

specifications

structure, data types, quantity definitions

data and meta-data

HDF5HDF5HDF5JSONJSONXMLJSON

XMLJSONTXT

1 2 3 4

preparation

meta-datainstrumentation human entry electronic lab notebooksprocessed data

010101 101100 010011 110111

raw datadata

HDF5HDF5HDF5JSONJSONXMLJSON

XMLJSONTXT

data formatsdatabases documentation

schemas

specifications

structure, data types, quantity definitions

data and meta-data

1 2 3 4

preparation

meta-datainstrumentation human entry electronic lab notebooksprocessed data

010101 101100 010011 110111

raw datadata

HDF5HDF5HDF5JSONJSONXMLJSON

XMLJSONTXT

data toolsdata formats

databasesdocumentation

schemas

specifications

structure, data types, quantity definitions

data and meta-data

data meta-data

simulated or real things

(meta-)data structure definitions

data

form

ats

1 2 3 4

preparation

meta-datainstrumentation human entry electronic lab notebooksprocessed data

010101 101100 010011 110111

raw datadata

HDF5HDF5HDF5JSONJSONXMLJSON

XMLJSONTXT

data toolsdata formats

databasesdocumentation

schemas

specifications

structure, data types, quantity definitions

data and meta-data

data meta-data

simulated or real things

(meta-)data structure definitions

data

form

ats

meta-data

(meta-)data structure definitions

domains

computational material science

experimental material science

domains

computational material science

experimental material science

chemistry

domains

computational material science

experimental material science

chemistry

domains

computational material science

experimental material science

chemistry

exciting science

XYZ

domains

computational material science

experimental material science

materials

domains

computational material science

transmission electron microscopy (TEM)

scanning transmission

electron microscopy (STEM)

domains

computational material science

transmission electron microscopy (TEM)

scanning transmission

electron microscopy (STEM)

domains

computational material science

transmission electron microscopy (TEM)

scanning transmission

electron microscopy (STEM)

0

100

200

300

400

crystal

octopus

Quant.Espr.

FHI-a

ims

cp2k

excitin

gVA

SPSIESTA

abinit

NWCh

em ELK

ORC

AGa

ussia

nCP

MD

turbom

ole

NAM

DGP

AWPhonopy

onetep

CASTEP

GULP

qBox

Dmol^3

GAMESS

FLEU

Relastic

BigD

FTDL

_POLY ATK

Molcas

TotalMeta-InfoQuantities CodeSpecificQuantities

domains

material science (material: atoms, symmetry, …)

experimental (sample: temperature,

thickness, …)

computational

STEM TEM

FAIR principles

Findable

Accessible

Iteroperable

Re-usable

FAIR principles

Findable

Accessible

Iteroperable

Re-usable

Filterable

FAIR principles

Findable

Accessible

Iteroperable

Re-usable

Filterable

Repurposable

FAIR principles

Findable

Accessible

Iteroperable

Re-usable

Filterable

Repurposable

ReCyclable

FAIR and Domain Aware

domain agnostic

(met

a-)d

ata

repo

sito

ries

domain-agnostic search

domain-agnostic search

FAIR and Domain Aware

FAIRdomain agnostic

(met

a-)d

ata

repo

sito

ries

FAIR and Domain Aware

FAIR

FAIR

domain agnostic

domain specific

(met

a-)d

ata

form

ats

(met

a-)d

ata

repo

sito

ries

40 more

FAIR and Domain Aware

FAIR

FAIR

domain agnostic

domain specific

(met

a-)d

ata

form

ats

(met

a-)d

ata

repo

sito

ries

FAIRdomain aware

40 more

FAIR and Domain Aware

FAIR

FAIR

domain agnostic

domain specific

(met

a-)d

ata

form

ats

(met

a-)d

ata

repo

sito

ries

FAIRdomain aware

FAIR and Domain Aware

FAIR

FAIR

domain agnostic

domain specific

(met

a-)d

ata

form

ats

(met

a-)d

ata

repo

sito

ries

FAIRdomain aware

FAIR and Domain Aware

FAIR

FAIR

domain agnostic

domain specific

(met

a-)d

ata

form

ats

(met

a-)d

ata

repo

sito

ries

FAIRdomain aware

material science (material: atoms, symmetry, …)

experimental (sample: temperature,

thickness, …)

computational (code,

functional)

STEM TEM

Summarymeta-data paramount for data sharing

homogenous domain meta-data for domain-aware NOMAD search

integrating this meta-data from various material science sub-domains

challenges:

agreeing on a schema system

maintaining suitable hierarchies of shared quantities