Chronopolis: Preserving Our Digital Heritage

Ardys Kozbial, UCSD LibrariesiPRES, October 6, 2009

Service Providers

• Development of digital preservation services– UCSD

• San Diego Supercomputer Center (SDSC)• UCSD Libraries

– National Center for Atmospheric Research (NCAR)

– University of Maryland Institute for Advanced Computer Studies (UMIACS)

Chronopolis

• Digital preservation data grid framework• Designed to leverage the storage capabilities

at SDSC, NCAR and UMIACS• Emphasizes heterogeneous and redundant

data storage systems• 150 TB of storage capacity (50 TB x 3 nodes)• 3 geographically distributed copies of data• Audit reporting

Data Providers

• Inter-university Consortium for Political and Social Research (ICPSR)– social science research data

• California Digital Library– web-at-risk collections

• North Carolina State University Libraries– geospatial data resources

• Scripps Institution of Oceanography– data from research cruises

TRACING DATA FLOW

ET-1 ServiceLevel Agreement

Data

DP

Data

Node 1

ET-1 ServiceLevel Agreement

Data

DP

ET-2 AcquisitionTransfer

Data

Manifest

Node 1

ET-1 ServiceLevel Agreement

Data

DP

ET-3 AcquisitionValidation

ET-2 AcquisitionTransfer

MCAT

Data

Manifest

Node 1

ET-1 ServiceLevel Agreement

Data

DP

ET-4 AcquisitionRegistration to SRB

ET-3 AcquisitionValidation

ET-2 AcquisitionTransfer

MCAT

Data

Manifest

Node 1

ET-1 ServiceLevel Agreement

Data

DP

ET-4 AcquisitionRegistration to SRB

ET-3 AcquisitionValidation

ET-2 AcquisitionTransfer

ACEACE

ET-5 AcquisitionRegistration into ACE

MCAT

Data

Manifest

Node 1

ET-1 ServiceLevel Agreement

Data

DP

ET-4 AcquisitionRegistration to SRB

ET-3 AcquisitionValidation

ET-2 AcquisitionTransfer

ACEACE

ET-5 AcquisitionRegistration into ACE

Node 3

Node 2

Replication Monitor

Replication Monitor

ET-6 Inter-NodeInventoryCheck

MCAT

Data

Manifest

Node 1

ET-1 ServiceLevel Agreement

Data

DP

ET-4 AcquisitionRegistration to SRB

ET-3 AcquisitionValidation

ET-2 AcquisitionTransfer

ACEACE

ET-5 AcquisitionRegistration into ACE

Node 3

Node 2

Replication Monitor

Replication Monitor

ET-7 AcquisitionReplication

ET-6 Inter-NodeInventoryCheck

MCAT

Data

Manifest

Node 1

ET-1 ServiceLevel Agreement

Data

DP

ET-4 AcquisitionRegistration to SRB

ET-3 AcquisitionValidation

ET-2 AcquisitionTransfer

ACEACE

ET-5 AcquisitionRegistration into ACE

ET-8 File Integrity Check

Node 3

Node 2

Replication Monitor

Replication Monitor

ET-7 AcquisitionReplication

ET-6 Inter-NodeInventoryCheck

MCAT

Data

Manifest

Node 1

ET-1 ServiceLevel Agreement

Data

DP

ET-4 AcquisitionRegistration to SRB

ET-3 AcquisitionValidation

ET-2 AcquisitionTransfer

ACEACE

ET-5 AcquisitionRegistration into ACE

ET-8 File Integrity Check

Node 3

Node 2

Replication Monitor

Replication Monitor

ET-7 AcquisitionReplication

ET-6 Inter-NodeInventoryCheck

SYSTEM STATUS DISPLAY

Future directions

• Update auditing procedures• Updated portal• Automation of collection ingest• New collections and storage nodes• Fully-fledged business model• TRAC certification

http://chronopolis.sdsc.edu

MCAT

Data

Manifest

Node 1

ET-1 ServiceLevel Agreement

Data

DP

ET-4 AcquisitionRegistration to SRB

ET-3 AcquisitionValidation

ET-2 AcquisitionTransfer

ACEACE

ET-5 AcquisitionRegistration into ACE

ET-8 File Integrity Check

Node 3

Node 2

Replication Monitor

Replication Monitor

ET-7 AcquisitionReplication

ET-6 Inter-NodeInventoryCheck