DigCCurr 2007: What digital curators do and what they need to know The CASPAR view on: What digital curators do and what they need to know : Research Perspectives

DigCCurr 2007: What digital curators do and what they need to know

The CASPAR view on:What digital curators do and what

they need to know :Research Perspectives

David Giaretta

CASPAR Project Director


What digital curators do:

• Struggle

with:• Funders

– Reluctant to provide long-term commitments

• Information providers– Unwilling to provide what is needed

• Users– demanding ever more sophistication

Ways to ensure info is understandable

Cost control, Cost estimates

Ways to capture required info


The CASPARConsortium

The CASPARConsortium

http://www.casparpreserves.eu


What do digital curators need to know


Curation: do preservation and publication/access - but do not confuse them

• Needs of access:– Responsive– Sophisticated search techniques– Users often familiar with the material

• Needs of Preservation:– Ensure the information trapped in the bits is authentic

and understandable• To the Designated Community

Transient tools and technologies with changing demands and implementations

Not transient

Curation also implies making fit for purpose – adding to info


There are disincentives for preservation: COST

Mon

ey

Time

Budget available

If cost of preserving old information increases…

Need to show that costs are contained


Preservation can be sold as benefiting Publication/Access :

Use of Unfamiliar Data• Global Cyber-Infrastructures allow users to

find and try to use data from many sources– Some sources will be familiar – Most available sources will be unfamiliar

• How can one be sure that the unfamiliar data is used correctly

• Need understanding– Garbage in – garbage out

• Need to be able to deal with unfamiliar data whether it is contemporary or old (preserved)


Digital Preservation…

• Easy to do…

• …as long as you can provide money forever

• Easy to test claims about tools…

• …as long as you live a long time


Know what is being preserved: the great Data / Document divide

• Need to preserve information & knowledge – not just “the bits”

– Documents, videos are rendered – simple?– Data – must be processed – in new ways - harder

Publication of data as well as documents

What is the cost of publication and preservation?


Information is the important thing

• What information?– Documents……– Data…….

• Original bits?• Look and feel?• Behaviour?• Performance?• Explicit/ Implicit/ Tacit

Information:

Any type of knowledge that can be exchanged. In an exchange, it is represented by data.

Long Term is long enough to be concerned with the impacts of changing technologies, including support for new media and data formats, or with a changing user community. Long Term may extend indefinitely.

Ensure that the information to be preserved is Independently Understandable to (and usable by) the Designated Community.


Things change/disappear

• Software

• Hardware

• Environment– E.g. Network links to related information

• People– What is “common knowledge”

How can we ensure that the information trapped in the “bits” remains understandable despite all these changes?

How can a digital curator even be aware of these changes?


Your time is short…

• Neither you nor your institution (or preservation project) will last forever

• The chain of preservation is only as strong its weakest link

• Need to be prepared to hand over

How can whole collections be handed over?

How can the information in the archive managers’ heads be handed over?


No repository is an island

• You/your organisation/project cannot do everything– Things change– You will not be around forever

• Must somehow tap into other resourcesHow can we find these resources?

How can we share the resources?

Where do the resources come from?


Wisdom of the world

• “given enough eyeballs, all bugs are shallow”

• However it may be a statistical process

Is there one right description of something?

How can we decide between alternatives?


We cannot foretell the future

• Need to manage knowledge to keep archives alive through time – Preservation is a process, not a one-time event– Preservation is expensive – costs need to be

shared– Open Archival Information Systems Reference

Model (ISO 14721) provides a general conceptual framework (http://public.ccsds.org/publications/archive/650x0b1.pdf)

• At least monitor the Designated Community

How can this be done over time?


OAIS – not just the Functional Model diagram

The Information Model is key

InformationObject

RepresentationInformation

1+

interpretedusing1+Data

Object

interpretedusing

PhysicalObject

DigitalObject

BitSequence

1+

Recursion ends at KNOWLEDGEBASE of the DESIGNATED COMMUNITY

(this knowledge will change over time and region)


Representation Information• The Data Object is “interpreted using” the

Representation Information (RepInfo) • The Reference Model is designed to ensure

that an OAIS is not set the impossible task of having to provide all possible RepInfo immediately

• Hence:– Take account of the Designated Community and its

associated Knowledge Base

• The amount of RepInfo is not fixed– Additional RepInfo will be needed over time

How do we define a Designated Community?

How?By whom?


CASPAR information flow architecture

•Rep

•Info

VirtualisationHow do we capture the Representation Information?


Authenticity

• Evidence

• Evidence

• Evidence


Support infrastructure

• Registries of Representation Information

• Representation Information Gap Manager

• Orchestration Manager• Toolkits

– Representation Information– Preservation Description Information


Some shared infrastructure


CASPAR aims• Produce tools and techniques to support digital

preservation and make it easier to share the cost– must be relatively easy to use– must have a low “buy-in” in terms of effort required for

adoption– must avoid requiring wholesale change of everyone else’s

systems– must be decentralised and reproducible so that it can live

on after the formal end of the CASPAR project– must be “preservable”– must be open: open source, open standards

• Cannot do everything– Working closely with the UK Digital Curation Centre


Can you tell who is selling preservation snake oil?

• Write out everything as XML?

• Write things onto holographic storage?

• Etch text onto titanium sheets?

• Just migrate to the newest format?

• CASPAR?

• ….

How to decide?


Validation

• Demonstrate theoretical basis• “Accelerated lifetime” tests

– Changes in hardware– Changes in environment– Changes in Designated Community

• Demonstrate increased trustworthiness – Measured using Certification process

(as/when available)


Links

• CASPAR: http://www.casparpreserves.eu • DCC: http://www.dcc.ac.uk

• OAIS (ISO 14721) http://public.ccsds.org/publications/archive/650x0b1.pdf

• Audit and Certification ISO standard development: http://wiki.digitalrepositoryauditandcertification.org