The TARDIS Framework

The TARDIS Framework

A Federated Repository Solution For Raw Diffraction Datasets

Steve Androulakis, Monash University, Melbourne Australia

International Digital Curation Conference, 2010

Started in late 2007

Public Access to Raw Protein Crystallography Data

Fedora Commons Based

Federated Storage

Protein Crystallography Specific Code

The TARDIS Framework

Federated Data Storage

TARDIS.edu.au

Repository

Repository

Repository Repository

Repository

TARDIS’ Internal Structure

Thanks to Alistair Grant

Generic Parameter Support

Thanks to Alistair Grant

Why Fedora wasnʼt suitable for large datasets..

HTTP Protocol Multiple / Large File

Downloading Setup and maintenance

Existing repository solutions at institutions..

Willingness / funds for libraries to store large datasets

Data - From Production to Publication

Publication

Production

ProcessingManaged by an

instrument/facility’s system, such as that

at the Australian Synchrotron

???Data exists on researchers’

personal computers and is

usually NOT managed in any

standard, persistent way

Solutions exist for deposition of data into several sources. TARDIS currently solves for raw Px data

The Need for Data Management at Time Of

Processing

Processing

From Capture to Publication

Also deployed at:University of Mebourne

La Trobe University

TARDIS: Internals

Lightweight Architecture

6 Datasets, 5140 Datafiles ~90GB

Data ‘staging’ mechanism.

Upcoming Features..

MyTARDIS

We’re now the ‘official solution’ for instruments at AS + ANSTO!

..so more instruments + facilities

“TARDIS/MyTARDIS system for high-end instrumentation data capture (e.g. X-ray diffraction images) has been chosen by both the Australian Synchrotron and ANSTO (Australian Nuclear Science and Technology Organisation) as the platform for metadata capture. Congratulations to the developers of TARDIS/MyTARDIS: Monash University biochemist, Associate Professor Ashley Buckle and software engineer Mr Steve Androulakis. (Support received from the Monash e-Research Centre, the Victorian eResearch Strategic Initiative, the Australian Synchrotron and the Australian National Data Service).”

Australian Synchrotron / ANSTO

Test instances already receiving SAXS/WAXS and IR data…

New Authorisation Engine Enables Various Access Rules For:• Groups of users• People in specific locations (ip based)• Custom rules (such as embargo periods)

Powerful Search Interface• Generic search across all parameters by various rules• Discipline specific ‘canned’ searches

Easy Parameter Schema Definition(in conjunction with Monash Metadata Store project)

• Users to define new parameter sets• Coming in a later iteration..

Advanced Parameter Editing(in conjunction with Monash Metadata Store project)

Australian Synchrotron / ANSTO

Molecular Biology Auto-Processing

Thanks Grischa Meyer

Upcoming Projects…

RMIT – Microscopy• 9 Microscopes• New ‘Filter’ Module Functionality• RMIT eResearch Office – Heinz Schmidt / Ian Thomas

The Ian Wark Research Institute – Particle Physics• Instrument Linkage• University of SA - Ryan Green

Peter Mac Institute – Next Gen Sequencing• Instrument Linkage• Courtesy of VeRSI – Lyle Winton

APS Chicago?!

Upcoming Projects

TARDIS’ Modular Framework

MyTARDISMyTARDIS

SynchrotronSynchrotron

TARDIS.edu.auTARDIS.edu.au

MicroscopyMicroscopy

VBL DownloadVBL Authentication

Visual StyleParameter Schemas

Parameter ‘Filters’Visual Style

Parameter Schemas

Federated Storage SupportVisual Style and Templates

Parameter Schemas

http://code.google.com/p/mytardis/ (BSD)

This project is supported by the Australian National Data Service (ANDS)

ANDS is supported by the Australian Government through the National Collaborative Research Infrastructure Strategy Program and the Education

Investment Fund (EIF) Super Science Initiative

Special thanks to..

Find us at:tardis.edu.au