What is a Successful Digital Library?

ECDL 2006, Alicante, September 18, 2006

Rao Shen, Naga Srinivas Vemuri, Weiguo Fan, and Edward A. Fox

fox@vt.edu http://fox.cs.vt.edu

Acknowledgements (Selected)

• Sponsors: NSF grant ITR-0325579, ASOR, CWRU, ETANA, Vanderbilt U., Virginia Tech

• Faculty/Staff: Lillian Cassel, Debra Dudley, Manuel Perez, …

• VT (Former) Students: Aaron Krowne, Ming Luo, Fernando Das Neves, Ricardo Torres, Hussein Suleman, …

Acknowledgements (Selected)

• Karen Borstad, MPP• Giorgio Buccellati, UCLA• Douglas Clark, Walla Walla College• Joanne Eustis, CWRU• Nick Fischio, CWRU• Israel Finkelstein, Tel-Aviv University • Paul Gherman, Vanderbilt U.• Andrew Graham, U. Toronto• Tim Harrison, U. Toronto• Larry Herr, Canadian University College

• Christopher Holland, LRP• Paul Jacobs, Mississippi State U.• Douglas Knight, Vanderbilt U.• Stan LaBianca, Andrews U.• David McCreery, Willamette U.• Eric Meyers, Duke U.• Adam Porter, Illinois College• Jack Sasson, Vanderbilt U.• Tom Schaub, Indiana U. of Penn. • Randall Younker, Andrews U.

ETANA-DL Website

ETANA-DL’s Member Collections

Outline

Prior work DL success model

From end user perspective Case study Conclusion

has an exampleIS success model

Prior Work on Measuring DL Success

DeLone et al.

Seddon

has an exampleInformation seekingbehavior model

Ellis

Kuhlthau

has an exampleDL quality model Gonçalves

has an exampleUsability of DLsinspection of NCSTRL

evaluation of ACM, IEEE-CS,NCSTRL, and NDLTD

evaluation of ADL

evaluation of ADEPT

has an indicator

system usage

has an exampleTechnology

acceptance model Venkateshpredict Intention to re/use

Behavioral Attitude & Intension to Use — Venkatesh Model of IT Adoption

intention to use system

performance expectancy

system usage

Unified Theory of Acceptance and Use of Technology

effort expectancy

social influence

facilitating conditions

• Performance expectancy: – perceived usefulness, extrinsic motivation, job-fit, relative advantage,

and outcome expectations

• Effort expectancy– the degree of ease associated with the use of system

• Social influence– Subjective norms, social factors, and image

• Facilitating conditions– the degree to which an individual believes that an organizational and

technical infrastructure exist to support the system

Venkatesh Model of IT Adoption

DeLone and McLean Model of IS Success

System Quality

Information Quality

UserSatisfaction

IndividualImpact

OrganizationImpact

Use

Seddon Model of IS Success

System Quality

Information Quality

UserSatisfactions

Net Benefits to:

IndividualsOrganizations

Society

PerceivedUsefulness

Outline

Prior work DL success model

From end user perspective Case study Conclusion

DL Success Model

5S and minimal DLSynthesize

IS success and adoption models (see above) Information life cycle model (Borgman et al.) 5S-based DL quality model (Gonçalves et al.) Information-seeking behavior models (Ellis’ and

Kuhlthau’s)

From end user perspective

Informal 5S & DL Definitions

DLs are complex systems that

• help satisfy info needs of users (societies)• provide info services (scenarios)• organize info in usable ways (structures)• present info in usable ways (spaces)• communicate info with users (streams)

5S and DL formal definitions and compositions (April 2004 TOIS)

5S

structures (d.10)streams (d.9) spaces (d.18) scenarios (d.21) societies (d. 24)

structural metadataspecification(d.25)

descriptive metadataspecification(d.26)

repository(d. 33)

collection (d. 31)

(d.34)indexingservice

structured stream (d.29)

digitalobject (d.30)

metadata catalog (d.32)

browsingservice

(d.37)

searchingservice (d.35)

digital library(minimal) (d. 38)

services (d.22)

sequence (d. 3)

graph (d. 6)function (d. 2)

measurable(d.12), measure(d.13), probability (d.14), vector (d.15), topological (d.16) spaces

event (d.10)state (d. 18)

hypertext(d.36)

sequence (d. 3)

transmission(d.23)

relation (d. 1) language (d.5)

grammar (d. 7)

tuple (d. 4)*

Borgman et al.:Workshop Report onSocial Aspects ofDigital Libraries: http://www-lis.gseis.ucla.edu/DL/

InformationLifeCycle

Information Life Cycle

AuthoringModifying

OrganizingIndexing

StoringRetrieving

DistributingNetworking

Retention/ Mining

AccessingFiltering

UsingCreating

creatio

n

distributionsee

kingutilization

search

ing,

browsing,

recommending

storing, archiving,

networkingautho

ring,

modify

ing,

descr

ibing

orga

nizing

, inde

xing

pres

erva

bilit

y,

sim

ilarit

y,

timel

ines

s,accuracy, completeness,

conformance accessibility, preservability

pertinence,significance,

timelinessrel

evan

ce

Active Sem

i-active

Inactive

accessibility,accessing,

filteringGonçalves et al.

system quality (SQ)performance expectancy (PE)

society, scenario, spaceservice

accessibility reliability

ease of use joy of use

composabilityefficiency

effectivenessextensibilityreusabilityreliability

accessibility accuracy

completeness consistence conformance

pertinence preservability

relevance significance

similaritytimeliness

DL quality dimension

information quality (IQ)

stream, structuredigital object

metadatacollectioncatalog

repository

adequacyrelevancereliability

scopetimeliness

understandability

social influence (SI)

DL success construct

society

5S and DL concept

DL visibility

DL success manifest variable

system quality (SQ)performance expectancy (PE)

society, scenario, spaceservice

accessibility reliability

ease of use joy of use

composabilityefficiency

effectivenessextensibilityreusabilityreliability

accessibility accuracy

completeness consistence conformance

pertinence preservability

relevance significance

similaritytimeliness

DL quality dimension

information quality (IQ)

stream, structuredigital object

metadatacollectioncatalog

repository

adequacyrelevancereliability

scopetimeliness

understandability

social influence (SI)

DL success construct

society

5S and DL concept

DL visibility

DL success manifest variable

creatio

n

distributionsee

kingutilization

E1:starting

E2: chaining

E3: browsing

E4: differen

tiating

E5: monitoring

E6: extractin

g

storing, archiving,

networking

K1:

in

itiat

ion

K2: sel

ectio

n

K3: exploration

K4: formulation

K5: collection

K6:

presentation

autho

ring,

modify

ing,

descr

ibing

orga

nizing

, inde

xing

pres

erva

bilit

y,

sim

ilarit

y,

timel

ines

s,accuracy, completeness,

conformance accessibility, preservability

DL Success Constructs

Active Sem

i-active

Inactive

E: Ellis’ modelK: Kuhlthau’s model

satisfactionperformanceexpectancy

(PE)

behavioralIntention to

(re)use

relevance adequacy timeliness

reliability understandability scope

accessibilityease of use

joy of use reliability

userinterface

social influence (SI)

information quality(IQ)

system quality(SQ)

DL Success Model

information quality(IQ)

DL Concepts Regarding Information

collection metadata catalog

digital object metadata specification

repository

consist of

associated with

associated with

consist of consist of

accessibilitypreservability

pertinencerelevancesimilarity

significancetimeliness

digital object

accuracycompletenessconformance

metadata specification

impact factorcompletenesscollection

consistencycompletenesscatalog

consistencycompletenessrepository

understandabilityrelevancetimelinessreliabilityadequacy

digital objectmetadata specification

adequacy

catalogcollection

scope

repository

Information Quality (IQ)Digital Library IQ

satisfactionperformanceexpectancy

(PE)

behavioralIntention to

(re)use

relevance adequacy timeliness

reliability understandability scope

accessibilityease of use

joy of use reliability

userinterface

social influence (SI)

information quality(IQ)

system quality(SQ)

DL Success Model

system quality(SQ)

screen designnavigation

user interface

composabilityreusability

extensibilityefficiency

effectivenessreliability

serviceaccessibility

reliabilityease of usejoy of use

System Quality(SQ)

performanceexpectancy

Digital Library SQ

Outline

Prior work DL success model

From end user perspective Case study Conclusion

Case Study

Part of requirements analysis for ETANA-DL Email interviews with 5 prestigious

archaeologists Face to face workplace interviews with 11

archaeologists Associate the 4 constructs of DL success

model with the activities occurring in the seeking and utilization phases

DL successConstruct

seeking phase utilization phasestarting (E1/K1)

selection exploration

(E2-E6)/(K2-K3)

formulation (K4)

collection (K5)

presentation (K6)

social influence DL visibility

information quality adequacy, scope accuracy

system quality ease of use joy of use (interface)

accessibility accessibility accessibility

performance expectancy

usefulness(interface)

DL Success Constructs Associated with Seeking and Utilization Phases

DL Success Constructs Associated with Seeking Phase

E1: Starting’ activity in Ellis’ model (K1: ‘initiation’ stage in Kuhlthau’s model) Social Influence (SI) — DL visibility

Publicize existence of a DLProvide a DL alert service

DL Success Constructs Associated with Seeking Phase

E2-E6: ‘chaining’, ‘browsing’, ‘differentiating’, ‘monitoring’, and ‘extracting’ in Ellis’ model (K2-K3: ‘selection’ and ‘exploration’ stages in Kuhlthau’s model) Information Quality (IQ) System Quality (SQ) Performance Expectancy (PE)

DL Success Constructs Associated with Seeking Phase

E2-E6: ‘chaining’, ‘browsing’, ‘differentiating’, ‘monitoring’, and ‘extracting’ in Ellis’ model (K2-K3: ‘selection’ and ‘exploration’ stages in Kuhlthau’s model) Information Quality (IQ)

Adequacy (degree of sufficiency and completeness) of DL collections and metadata catalogs

Scope of DL repository

DL Success Constructs Associated with Seeking Phases

E2-E6: ‘chaining’, ‘browsing’, ‘differentiating’, ‘monitoring’, and ‘extracting’ in Ellis’ model (K2-K3: ‘selection’ and ‘exploration’ stages in Kuhlthau’s model) System Quality (SQ)

Ease of use Joy of use

DL Success Constructs Associated with Seeking Phases

E2-E6: ‘chaining’, ‘browsing’, ‘differentiating’, ‘monitoring’, and ‘extracting’ in Ellis’ model (K2-K3: ‘selection’ and ‘exploration’ stages in Kuhlthau’s model) Performance Expectancy (PE)

Usefulness

DL Success Constructs Associated with Seeking Phases

E2-E6: ‘chaining’, ‘browsing’, ‘differentiating’, ‘monitoring’, and ‘extracting’ in Ellis’ model (K2-K3: ‘selection’ and ‘exploration’ stages in Kuhlthau’s model) System Quality & Performance Expectancy

DL interface: screen design & navigation

DL Success Constructs Associated with Utilization Phase

K4-K6: ‘formulation’, ‘collection’, and ‘presentation’ stage in Kuhlthau’s model Information Quality

information accuracy information accessibility

Outline

Prior work DL success model

From end user perspective Case study Conclusion

Conclusion

Lay the foundation for defining success of DLs from the view of DL end users

Assume a multi-theoretical perspectiveSynthesize many related research areas in

terms of theory and empirical workExplicate and illustrate our approach by a

case study with ETANA and usabilityConnect with other work on DL quality: led

by Emory funded by IMLS, DELOS …

Questions?Comments?

See http://fox.cs.vt.edu/talks/2006/ 20060918ECDLsuccess.ppt