Virtual Research Environments supporting tailor-made data management services for marine & maritime sector

BlueBRIDGE receives funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 675680 www.bluebridge-vres.eu

Virtual Research Environments supporting tailor-made data management services

for marine & maritime sector

13 October 2016 - Brest

Pasquale PaganoCNR, [email protected]

Virtual Research Environments for supporting tailor-made data management services 2

Challenges and Opportunities

Data and Services

Hosted by different Organizations

Accessible through different Protocols

Described with different Metadata

Policies

Different approaches for Credits

Different Licenses

Different Terms of Use

Heterogeneity

Support Validation, Curation,

Harmonization

Measure Uncertainty

Trace Provenance

Modern science is increasingly global, multi-disciplinary and networked


Data Analytics

• are multidisciplinary, involve members belonging to diverse organisations • require to access data and services that are spread among many providers

dynamically aggregated to address questions/problems

• cannot rely on pre-organised and costly supporting environments managed by dedicated organizations

build and operate their own supporting environments

wish to effectively inject new approaches in daily tasks

cost and time required to implement this approach largely exceed the available capacities

Not performed by individuals but group of data analysts


Requirements for IT systems

• Support collaborative data analysis and experimentation

• Implement Traceability and Reproducibility-Repeatability-Reusability

• Enable secure and controlled data sharing

• Tackle simplified access to existing data and processes

• Tackle simplified access to existing computing and storage resources

• Ensure low operational and maintenance costs

• Manage heterogeneous data access policies


Virtual Research Environment

An operational environment

• Where set of resources (data, applications, computational, and storage resources)

• are assigned to group of users via interfaces

• for a limited timeframe

• by hiding complexity of hardware setup and software configuration

L. Candela, D. Castelli, P. Pagano (2013) Virtual Research Environments: An Overview and a Research Agenda. Data Science Journal, Vol. 12

Created on demand

Regulated by tailored policies

No cost for the resource providers

Open to host and operate custom software


Application BundlesReady to use technologies

To develop applications interfacing gCube facilities

AppsCubeTo aid modelling and analysing of distribuition data, comparing checklists, and producing maps

BiolCube

To facilitate data publication with appropriate tools including semantic technologies

ConnectCube

To properly access, consume and produce geospatial information

GeosCube

To assist tabular data validation, data enrichment ad efficient analytical tools

StatsCube

To support deployment, operation & mgmt of a data infrastructure

IceCube


VRE Creation

Configuration

ApplicationsMetadata

Data

Simple and effective process to define a new environment


Applications vs Services

Registry

Logi

cal

View

Applications Data

Phys

ical

View

Hardware

Software, Tools, Services

Configuration

Data

VRE Workflow EnablerSPD (BiolCube)ecological and biological data

GeoExplorer (GeosCube)geospatial data

Tabular Data (StatsCube)statistical and reference data

SAI (StatsCube)process importer

(Con

nect

Cube

)


Data Miner (StatsCube)data analytics for interdisciplinary domains


VREs

• Cloud computation• Web interface

available for non experts

• Standard WPS API for easily integration

• 90% processing time reduction

Stock AssessmentEstimates Maximum Sustainable Yield, Biomass, CPUE and catchability from catch statistics, biomass, landings etc.


Performance Evaluation In Aquaculture

Aquafarming assessment tools enacting perform evaluation growth analysis and techno economic investment analysis

Capabilities• Production Planning• Financial Forecast• Skill Building (What-IF)• KPI extraction

• Feed Conversion Rate(FCR)• Growth Per Day(GPD)• Specific Growth Rate (SGR)• Suggested Feeding Rate (SFR)• Mortality Rate (MR)


Biodiversity

Fill knowledge gaps on marine speciesAccount for sampling biasesDefine trends for common species

Plankton regime shift

Herring recovered after the fish ban

LME - MEOW


Fishing Activity

ForecastingTrajectories Analysis


Ecology

Atlantic cod

Coelacanth

Giant squid

AquaMaps

Neural Networks

Neural Networks and MaxEnt

15

Geospatial data processing

Maps comparison

NetCDF

Data extraction Signal processing Periodicity detection

Maps generation

Virtual Research Environments for supporting tailor-made data management services


VREs in operation

Data Infrastructures Computing Infrastructures

Mediator Connector Mediator Connector

Data Curation

Data Preparation

Data Analysis

Data Sharing

Data Publication

Data Provenance

VRE Builder

Security

Monitoring

Marine and MaritimeDigital Humanities

Geothermal

Social Mining


VRE Social NetworkingSocial networking is key to share information in the VRE

It offers a continuously updated list of events / news produced by users and applications

Access VREsDiscuss and

Validate

Share Data, News, Processes


VRE Common WorkspaceA folder-based file system allowing

managing and sharing information objects

Information objects can be

• files, dataset, workflows, experiments, etc.

• organized into folders

Users can

• Share with selected users

• disseminate via persistent public URLs


VRE Software Integration

Download the (python, R, Java, …) script and the user’s data

Execute script

Collect output

Destroy local copies of I/O and script

Save Output on the User’s Workspace, with provenance info

Scientist’s provided script

User’s data

Infrastructure


VRE Collaborative Experiments

WS

Shared online folders

Inputs

Outputs

Results

Computational system

In the e-Infrastructure

Through third party software


VRE Enabling New Workflow

Script provider

Updates the script on his private Workspace

The service downloadsthe script on-the-fly

A user executes an experiment on his/her data

The output, the input and the parameters can be shared with another user

This user can execute the experiment againand share the computation with other users

1

2

3

4

5

6

7

89

10


ConclusionsVRE are defined by users and created on demand• New software can be integrated and used as-a-Service• Invoked via standard interfaces

VRE ensures • Provenance management• Access via an easy-to-use storage system• Collaboration and sharing

VRE enables • Complex workflows• Repeatability, Reproducibility and Reusability


Visit us at www.bluebridge-vres.euTry it at i-marine.d4science.org

http://www.bluebridge-vres.eu/

https://i-marine.d4science.org/



Technology

Virtual Research Environments supporting tailor-made data management services for marine & maritime sector