Preservation, New Media, Oral Cultures How to Build a Digital Library Ian H. Witten and David Bainbridge

Preservation, New Preservation, New Media, Oral Media, Oral

CulturesCultures

How to Build a Digital LibraryHow to Build a Digital LibraryIan H. Witten and David BainbridgeIan H. Witten and David Bainbridge

Mission of a LibraryMission of a Library

The mission of a library is twofold:The mission of a library is twofold: To collect, organize, and provide access To collect, organize, and provide access

to informationto information To pass it down to succeeding To pass it down to succeeding

generations as a record of culturegenerations as a record of culture

The Librarian’s DutyThe Librarian’s Duty

The librarian has twin duties:The librarian has twin duties: AccessAccess

to the world’s literature for today’s to the world’s literature for today’s readersreaders

PreservationPreservationfor future generationsfor future generations

The Problem of The Problem of PreservationPreservation

PaperPaper Acid-based paper decomposes after only Acid-based paper decomposes after only

a few decadesa few decades FilmFilm

Film containing nitrate decays quicklyFilm containing nitrate decays quickly Analog audioAnalog audio

Wax cylinders or magnetic tapes must Wax cylinders or magnetic tapes must be preserved by transferring into digital be preserved by transferring into digital formatsformats

The Problem of The Problem of PreservationPreservation

Technological progress comes at the Technological progress comes at the expense of preservationexpense of preservation

A process of regular copying can be A process of regular copying can be established to preserve digital established to preserve digital material without lossmaterial without loss

““No one understands how to archive No one understands how to archive digital documents”digital documents”

Preservation TechnologyPreservation Technology

Enormous amounts of digital Enormous amounts of digital information are already lost foreverinformation are already lost forever

Information technologies become Information technologies become obsolete very quicklyobsolete very quickly

Document and media formats Document and media formats continue to proliferatecontinue to proliferate

Technology standards will not solve Technology standards will not solve fundamental issues in the fundamental issues in the preservation of digital informationpreservation of digital information

Availability of MaterialAvailability of Material Libraries will shortly see a demographic bulge Libraries will shortly see a demographic bulge

of electronic material as the baby boom of electronic material as the baby boom generation of authors and academics generation of authors and academics contribute material gathered during their contribute material gathered during their careerscareers

Much material will never make it into library Much material will never make it into library collections for preservation because of collections for preservation because of increasingly restrictive intellectual property increasingly restrictive intellectual property and licensing regimesand licensing regimes

Archiving and preservation functions in a Archiving and preservation functions in a digital environment will increasingly become digital environment will increasingly become privatized as information continues to be privatized as information continues to be commodifiedcommodified

Traditional Library Traditional Library FunctionsFunctions

Financial resources available to Financial resources available to libraries and archives continue to libraries and archives continue to decreasedecrease

Libraries and archives will be Libraries and archives will be required to continue their existing required to continue their existing archival and preservation practices archival and preservation practices as the current paper publishing as the current paper publishing boom continuesboom continues

Preservation StrategiesPreservation Strategies

Digital documents are vulnerable to Digital documents are vulnerable to loss because the media on which loss because the media on which they are stored decays and becomes they are stored decays and becomes obsoleteobsolete

They become inaccessible when the They become inaccessible when the software or hardware becomes software or hardware becomes obsoleteobsolete


Digital formats have advantages Digital formats have advantages over analog formatsover analog formats

Digital formats seem to promote Digital formats seem to promote preservationpreservation

The advantages make digital The advantages make digital preservation even harderpreservation even harder


Ease of creation causes information Ease of creation causes information glutglut

Easy of copying makes “copies” Easy of copying makes “copies” seem dispensableseem dispensable

Improvements in hardware and Improvements in hardware and software promote obsolescencesoftware promote obsolescence


““May all your problems be technical May all your problems be technical ones”ones” Computer people recognize that the Computer people recognize that the

technical problems can be solvedtechnical problems can be solved It’s the human part that causes problemsIt’s the human part that causes problems

Administrative and political processes Administrative and political processes take time and cause frustrationtake time and cause frustration

Technical problems have solutions Technical problems have solutions which yield to honest intellectual workwhich yield to honest intellectual work


Paper and MuseumsPaper and Museums Involves printing the material on paper or Involves printing the material on paper or

microfilm and storing in museumsmicrofilm and storing in museums Not considered a long-term preservation Not considered a long-term preservation

strategystrategy Emulation and MigrationEmulation and Migration

Involves preserving the physical stream Involves preserving the physical stream of bits and/or the logical means by which of bits and/or the logical means by which the bits are interpreted as a documentthe bits are interpreted as a document


Preserving the physical bit streamPreserving the physical bit stream Regular copying to new mediaRegular copying to new media Error detection to determine if Error detection to determine if

degradation is occurringdegradation is occurring Error correcting codes to ensure new Error correcting codes to ensure new

generations are faithful copies of the generations are faithful copies of the originaloriginal


EmulationEmulation Keeping the documents in exactly the Keeping the documents in exactly the

same formsame form Emulate the functionality of the Emulate the functionality of the

original, obsolete system on future, original, obsolete system on future, unknown systemsunknown systems


MigrationMigration Translating the document from the old Translating the document from the old

format to a format accepted by new format to a format accepted by new softwaresoftware Designed for near-obsolete softwareDesigned for near-obsolete software

Involves copying the physical bit stream Involves copying the physical bit stream to new mediato new media

Involves translation to a new logical Involves translation to a new logical formatformat


Emulation or Migration?Emulation or Migration? Migration may be cheaperMigration may be cheaper

No special emulation software needs to be No special emulation software needs to be writtenwritten

Conversion software is usually availableConversion software is usually available Conversion is a kind of translationConversion is a kind of translation

May lose features of the dataMay lose features of the data


An important feature of any format used An important feature of any format used for preserving documents it that it is for preserving documents it that it is open - the details are made publicly open - the details are made publicly availableavailable

It must be open in principle as well as It must be open in principle as well as practicepractice Documented well enough for others to Documented well enough for others to

understand and build their own understand and build their own interpretersinterpreters

Examples: PostScript and PDFExamples: PostScript and PDF

New MediaNew Media

Text remains the principal means for Text remains the principal means for searching and browsing collections, searching and browsing collections, even when they contain documents even when they contain documents in other mediain other media

MusicMusic

Music information retrievalMusic information retrieval Motifs in music are analogous to key Motifs in music are analogous to key

phrases in textphrases in text OMROMR

Optical music recognitionOptical music recognition Music analog of OCRMusic analog of OCR

ImagesImages

ThumbnailsThumbnails Visual material can be rapidly browsed Visual material can be rapidly browsed

using thumbnailsusing thumbnails Captures the readers attentionCaptures the readers attention Gives a feeling for what the collection is Gives a feeling for what the collection is

aboutabout Difficult to automatically search images Difficult to automatically search images

rather than manually browse themrather than manually browse them Collection Understanding …Collection Understanding …

VideosVideos

VideoVideo a sequence of pictures?a sequence of pictures?

Cut detectionCut detection Locating techniques where the scene Locating techniques where the scene

changeschanges MoviesMovies

Browsed and manipulated using thumbnailsBrowsed and manipulated using thumbnails Each thumbnail represents a typical image Each thumbnail represents a typical image

or the initial image in a sceneor the initial image in a scene

RealiaRealia

Real artifactsReal artifacts Computer graphics allow three-Computer graphics allow three-

dimensional objects to be captured dimensional objects to be captured in the form of a data setin the form of a data set

Artifacts are indexed and located on Artifacts are indexed and located on the basis of metadatathe basis of metadata

Other Document TypesOther Document Types

Teaching materialTeaching material Multimedia elementsMultimedia elements

Research materialResearch material Laboratory notebooksLaboratory notebooks

Scientific and engineering dataScientific and engineering data Results of experiments, simulations, Results of experiments, simulations,

and surveysand surveys Information is expressed in many formsInformation is expressed in many forms

Digital Libraries for Oral Digital Libraries for Oral CulturesCultures

Libraries are about literatureLibraries are about literature Literature:Literature:

The writings of a society, in prose or The writings of a society, in prose or verseverse

Broadly speaking, literature includes all Broadly speaking, literature includes all types of fiction and nonfiction writing types of fiction and nonfiction writing intended for publicationintended for publication


It should be possible to create digital It should be possible to create digital library collections intended for library collections intended for people in oral culturespeople in oral cultures

Useful for people who may be Useful for people who may be illiterate or semi-literateilliterate or semi-literate

Useful for people who cannot speak Useful for people who cannot speak or read the language of the digital or read the language of the digital librarylibrary


Iconic FormIconic Form Serious practical information can be Serious practical information can be

conveyed in a purely iconic formconveyed in a purely iconic form ExamplesExamples

How to splint a broken forearmHow to splint a broken forearm User manual for underground transport User manual for underground transport

systemsystem Historical precedent of Beggar’s BiblesHistorical precedent of Beggar’s Bibles


Libraries for the illiterateLibraries for the illiterate We are all illiterate with respect to We are all illiterate with respect to

some other languages and culturessome other languages and cultures Media types:Media types:

Static imagesStatic images Motion, sound, video, interaction, 3D Motion, sound, video, interaction, 3D

objects, simulations, virtual realityobjects, simulations, virtual reality

Documents

Preservation, New Media, Oral Cultures How to Build a Digital Library Ian H. Witten and David Bainbridge