Chapter Nine Visions Future, past, and present How to Build a Digital Library Ian H. Witten and David Bainbridge

Chapter NineChapter Nine

VisionsVisionsFuture, past, and presentFuture, past, and present

How to Build a Digital LibraryHow to Build a Digital LibraryIan H. Witten and David BainbridgeIan H. Witten and David Bainbridge

Visions: Future, Past, and Visions: Future, Past, and PresentPresent

Digital libraries have practical Digital libraries have practical advantages over physical onesadvantages over physical ones

Digital libraries offer the promise of Digital libraries offer the promise of far greater universalityfar greater universality

Mission of a LibraryMission of a Library

The mission of a library is twofold:The mission of a library is twofold: To collect, organize, and provide access To collect, organize, and provide access

to informationto information To pass it down to succeeding To pass it down to succeeding

generations as a record of culturegenerations as a record of culture

The Librarian’s DutyThe Librarian’s Duty

The librarian has twin duties:The librarian has twin duties: AccessAccess

to the world’s literature for today’s to the world’s literature for today’s readersreaders

PreservationPreservationfor future generationsfor future generations

Challenges for Digital Challenges for Digital LibrariesLibraries

Today’s collections are mostly textToday’s collections are mostly text The real challenge is to create The real challenge is to create

collections of digital documents in collections of digital documents in diverse media typesdiverse media types

Examples:Examples: Music libraries that can be searched by Music libraries that can be searched by

humminghumming

Libraries of the Libraries of the futurefuture

Libraries of the FutureLibraries of the Future

Digital libraries Digital libraries Have the potential to be far more Have the potential to be far more

flexible than conventional onesflexible than conventional ones Will be largeWill be large Will not be staticWill not be static

Today’s VisionsToday’s Visions

Impersonal and utilitarianImpersonal and utilitarian Example: Figure 9.2Example: Figure 9.2

Real people in real environmentReal people in real environment Example: Figure 9.3Example: Figure 9.3 Kataayi cooperative in UgandaKataayi cooperative in Uganda Low techLow tech

Today’s VisionsToday’s Visions

Libraries are about connecting Libraries are about connecting people with the information they people with the information they needneed

Tomorrow’s VisionsTomorrow’s Visions

Sci-fi imageSci-fi image emphasis of preservation over accessemphasis of preservation over access

Personalized spacePersonalized space A kitchen for knowledge preparationA kitchen for knowledge preparation

WorkshopWorkshop Comfortable, personalized, dynamic, up Comfortable, personalized, dynamic, up

to dateto date Your Visions?Your Visions?

LibrarianshipLibrarianship

Librarianship:Librarianship: Selection, organization, and Selection, organization, and

maintenancemaintenance Wisdom and value judgmentsWisdom and value judgments

What information to includeWhat information to include How to organize the informationHow to organize the information

Working Inside the Digital Working Inside the Digital LibraryLibrary

Digital LibraryDigital Library A library without walls but with A library without walls but with

boundariesboundaries

Working inside the digital library:Working inside the digital library: An environment that surrounds in an An environment that surrounds in an

intellectual senseintellectual sense More or less immersiveMore or less immersive Reacts and respondsReacts and responds

Preserving the Preserving the pastpast

The Problem of The Problem of PreservationPreservation

Technological progress comes at the Technological progress comes at the expense of preservationexpense of preservation


PaperPaper Acid-based paper decomposes after only Acid-based paper decomposes after only

a few decadesa few decades FilmFilm

Film containing nitrate decays quicklyFilm containing nitrate decays quickly Analog audioAnalog audio

Wax cylinders or magnetic tapes must Wax cylinders or magnetic tapes must be preserved by transferring onto be preserved by transferring onto digital formatsdigital formats


A process of regular copying can be A process of regular copying can be established to preserve digital established to preserve digital material without lossmaterial without loss

The Digital Dark AgesThe Digital Dark Ages

““No one understands how to archive No one understands how to archive digital documents”digital documents”

Preservation TechnologyPreservation Technology

Enormous amounts of digital Enormous amounts of digital information are already lost foreverinformation are already lost forever

Information technologies become Information technologies become obsolete very quicklyobsolete very quickly

Document and media formats Document and media formats continue to proliferatecontinue to proliferate

Technology standards will not solve Technology standards will not solve fundamental issues in the fundamental issues in the preservation of digital informationpreservation of digital information

Availability of MaterialAvailability of Material Libraries will shortly see a demographic bulge Libraries will shortly see a demographic bulge

of electronic material as the baby boom of electronic material as the baby boom generation of authors and academics generation of authors and academics contribute material gathered during their contribute material gathered during their careerscareers

Much material will never make it into library Much material will never make it into library collections for preservation because of collections for preservation because of increasingly restrictive intellectual property increasingly restrictive intellectual property and licensing regimesand licensing regimes

Archiving and preservation functions in a Archiving and preservation functions in a digital environment will increasingly become digital environment will increasingly become privatized as information continues to be privatized as information continues to be commodifiedcommodified

Traditional Library Traditional Library FunctionsFunctions

Financial resources available to Financial resources available to libraries and archives continue to libraries and archives continue to decreasedecrease

Libraries and archives will be Libraries and archives will be required to continue their existing required to continue their existing archival and preservation practices archival and preservation practices as the current paper publishing as the current paper publishing boom continuesboom continues

Preservation StrategiesPreservation Strategies

Digital documents are vulnerable to Digital documents are vulnerable to loss because the media on which loss because the media on which they are stored decays and becomes they are stored decays and becomes obsoleteobsolete

They become inaccessible when the They become inaccessible when the software or hardware becomes software or hardware becomes obsoleteobsolete


Digital formats have advantages Digital formats have advantages over analog formatsover analog formats

Digital formats seem to promote Digital formats seem to promote preservationpreservation

The advantages make digital The advantages make digital preservation even harderpreservation even harder


Ease of creation causes information Ease of creation causes information glutglut

Easy of copying makes “copies” Easy of copying makes “copies” seem dispensableseem dispensable

Improvements in hardware and Improvements in hardware and software promote obsolescencesoftware promote obsolescence


““May all your problems be technical May all your problems be technical ones”ones” Computer people recognize that the Computer people recognize that the

technical problems can be solvedtechnical problems can be solved It’s the human part that causes problemsIt’s the human part that causes problems

Administrative and political processes Administrative and political processes take time and cause frustrationtake time and cause frustration

Technical problems have solutions Technical problems have solutions which yield to honest intellectual workwhich yield to honest intellectual work


Preservation is not a technical Preservation is not a technical problemproblem


Four Preservation StrategiesFour Preservation Strategies PaperPaper MuseumsMuseums EmulationEmulation MigrationMigration


Paper and MuseumsPaper and Museums Involves printing the material on paper or Involves printing the material on paper or

microfilm and storing in museumsmicrofilm and storing in museums Not considered a long-term preservation Not considered a long-term preservation

strategystrategy Emulation and MigrationEmulation and Migration

Involves preserving the physical stream Involves preserving the physical stream of bits and/or the logical means by which of bits and/or the logical means by which the bits are interpreted as a documentthe bits are interpreted as a document


EmulationEmulation Keeping the documents in exactly the Keeping the documents in exactly the

same formsame form Emulate the functionality of the Emulate the functionality of the

original, obsolete system on future, original, obsolete system on future, unknown systemsunknown systems


Preserving the physical bit streamPreserving the physical bit stream Regular copying to new mediaRegular copying to new media Error detection to determine if Error detection to determine if

degradation is occurringdegradation is occurring Error correcting codes to ensure new Error correcting codes to ensure new

generations are faithful copies of the generations are faithful copies of the originaloriginal


Preserving the logical interpretationPreserving the logical interpretation Emulate old interpreters on new Emulate old interpreters on new

hardwarehardware Backward compatibilityBackward compatibility


An important feature of any format used An important feature of any format used for preserving documents it that it is for preserving documents it that it is open: the details are made publicly open: the details are made publicly availableavailable

It must be open in principle as well as It must be open in principle as well as practicepractice Documented well enough for others to Documented well enough for others to

understand and build their own understand and build their own interpretersinterpreters

Examples: PostScript and PDFExamples: PostScript and PDF


MigrationMigration Translating the document from the old Translating the document from the old

format to a format accepted by new format to a format accepted by new softwaresoftware Designed for near-obsolete softwareDesigned for near-obsolete software

Involves copying the physical bit stream Involves copying the physical bit stream to new mediato new media

Involves translation to a new logical Involves translation to a new logical formatformat


Emulation or Migration?Emulation or Migration? Migration may be cheaperMigration may be cheaper

No special emulation software needs No special emulation software needs writtenwritten

Conversion software is usually availableConversion software is usually available Conversion is a kind of translationConversion is a kind of translation

May lose features of the dataMay lose features of the data

Generalized Generalized documents: A documents: A

challenge for the challenge for the presentpresent

Generalized Documents: A Generalized Documents: A Challenge for the PresentChallenge for the Present

Text remains the principal means for Text remains the principal means for searching and browsing collections, searching and browsing collections, even when they contain documents even when they contain documents in other mediain other media

Multimedia documents can be Multimedia documents can be displayeddisplayed Linked to text documentsLinked to text documents Text may contain only captionsText may contain only captions Text is browsed and searchedText is browsed and searched

Digital Libraries of Digital Libraries of MusicMusic

Music information retrievalMusic information retrieval Motifs in music are analogous to key Motifs in music are analogous to key

phrases in textphrases in text OMROMR

Optical music recognitionOptical music recognition Music analog of OCRMusic analog of OCR

Other MediaOther Media

ImagesImages VideosVideos ObjectsObjects Other Document TypesOther Document Types

ImagesImages

ThumbnailsThumbnails Visual material can be rapidly browsed Visual material can be rapidly browsed

using thumbnailsusing thumbnails Captures the readers attentionCaptures the readers attention Gives a feeling for what the collection is Gives a feeling for what the collection is

aboutabout Difficult to automatically search images Difficult to automatically search images

rather than manually browse themrather than manually browse them

VideosVideos

VideoVideo a sequence of pictures?a sequence of pictures?

Cut detectionCut detection Locating techniques where the scene Locating techniques where the scene

changeschanges MoviesMovies

Browsed and manipulated using thumbnailsBrowsed and manipulated using thumbnails Each thumbnail represents a typical image Each thumbnail represents a typical image

or the initial image in a sceneor the initial image in a scene

ObjectsObjects

RealiaRealia Real artifactsReal artifacts Computer graphics allow three-dimensional Computer graphics allow three-dimensional

objects to be captured in the form of a data objects to be captured in the form of a data setset

ArtifactsArtifacts In libraries and museums, artifacts are In libraries and museums, artifacts are

indexed and located on the basis of indexed and located on the basis of metadatametadata

BooksBooks Can be modeled as physical objectsCan be modeled as physical objects

Other Document TypesOther Document Types

Teaching materialTeaching material Multimedia elementsMultimedia elements

Research materialResearch material Laboratory notebooksLaboratory notebooks

Scientific and engineering dataScientific and engineering data Results of experiments, simulations, Results of experiments, simulations,

and surveysand surveys Information is expressed in many formsInformation is expressed in many forms

Generalized Documents in Generalized Documents in GreenstoneGreenstone

Digital LibraryDigital Library Focused collection of digital objects, Focused collection of digital objects,

including text, video, and audioincluding text, video, and audio The ChallengeThe Challenge

Integrate objects of all kinds of media Integrate objects of all kinds of media into digital libraries in such a way that into digital libraries in such a way that each becomes a first-class citizeneach becomes a first-class citizen


Greenstone does not incorporate Greenstone does not incorporate searching and browsing techniques searching and browsing techniques for non-textual mediafor non-textual media


Solutions to current Greenstone limitations:Solutions to current Greenstone limitations: New modules can be addedNew modules can be added New search engine can be deployed by New search engine can be deployed by

replacing or augmenting the MG system that replacing or augmenting the MG system that does text searchingdoes text searching

Browsing horizontal and vertical lists can be Browsing horizontal and vertical lists can be handled by adding a new classifierhandled by adding a new classifier

New browsers can be added through Perl codeNew browsers can be added through Perl code New media types can be imported by adding New media types can be imported by adding

new plug-insnew plug-ins

Digital Libraries for Oral Digital Libraries for Oral CulturesCultures

Libraries are about literatureLibraries are about literature Literature:Literature:

The writings of a society, in prose or The writings of a society, in prose or verseverse

Broadly speaking, literature includes all Broadly speaking, literature includes all types of fiction and nonfiction writing types of fiction and nonfiction writing intended for publicationintended for publication


It should be possible to create digital It should be possible to create digital library collections intended for library collections intended for people in oral culturespeople in oral cultures

Useful for people who may be Useful for people who may be illiterate or semi-literateilliterate or semi-literate

Useful for people who cannot speak Useful for people who cannot speak or read the language of the digital or read the language of the digital librarylibrary


Iconic FormIconic Form Serious practical information can be Serious practical information can be

conveyed in a purely iconic formconveyed in a purely iconic form ExamplesExamples

How to splint a broken forearmHow to splint a broken forearm User manual for underground transport User manual for underground transport

systemsystem Historical precedent of Beggar’s BiblesHistorical precedent of Beggar’s Bibles


Libraries for the illiterateLibraries for the illiterate We are all illiterate with respect to We are all illiterate with respect to

some other languages and culturessome other languages and cultures Media types:Media types:

Static imagesStatic images Motion, sound, video, interaction, 3D Motion, sound, video, interaction, 3D

objects, simulations, virtual realityobjects, simulations, virtual reality