What is a Successful Digital Library?

ECDL 2006, Alicante, September 18, 2006

Rao Shen, Naga Srinivas Vemuri, Weiguo Fan, and Edward A. Fox

fox@vt.edu http://fox.cs.vt.edu

Acknowledgements (Selected)

• Sponsors: NSF grant ITR-0325579, ASOR, CWRU, ETANA, Vanderbilt U., Virginia Tech

• Faculty/Staff: Lillian Cassel, Debra Dudley, Manuel Perez, …

• VT (Former) Students: Aaron Krowne, Ming Luo, Fernando Das Neves, Ricardo Torres, Hussein Suleman, …

Acknowledgements (Selected)

• Karen Borstad, MPP

• Giorgio Buccellati, UCLA

• Douglas Clark, Walla Walla College

• Joanne Eustis, CWRU

• Nick Fischio, CWRU

• Israel Finkelstein, Tel-Aviv University

• Paul Gherman, Vanderbilt U.

• Andrew Graham, U. Toronto

• Tim Harrison, U. Toronto

• Larry Herr, Canadian University College

• Christopher Holland, LRP

• Paul Jacobs, Mississippi State U.

• Douglas Knight, Vanderbilt U.

• Stan LaBianca, Andrews U.

• David McCreery, Willamette U.

• Eric Meyers, Duke U.

• Adam Porter, Illinois College

• Jack Sasson, Vanderbilt U.

• Tom Schaub, Indiana U. of Penn.

• Randall Younker, Andrews U.

ETANA-DL Website

ETANA-DL’s Member Collections

Outline

Prior work DL success model

From end user perspective

Case study Conclusion

has an exampleIS success model

Prior Work on Measuring DL Success

DeLone et al.

Seddon

has an exampleInformation seekingbehavior model

Ellis

Kuhlthau

has an exampleDL quality model Gonçalves

has an exampleUsability of DLsinspection of NCSTRL

evaluation of ACM, IEEE-CS,NCSTRL, and NDLTD

evaluation of ADL

evaluation of ADEPT

has an indicator

system usage

has an exampleTechnology

acceptance model Venkateshpredict Intention to re/use

Behavioral Attitude & Intension to Use — Venkatesh Model of IT Adoption

intention to use system

performance expectancy

system usage

Unified Theory of Acceptance and Use of Technology

effort expectancy

social influence

facilitating conditions

• Performance expectancy: – perceived usefulness, extrinsic motivation, job-fit, relative advantage,

and outcome expectations

• Effort expectancy– the degree of ease associated with the use of system

• Social influence– Subjective norms, social factors, and image

• Facilitating conditions– the degree to which an individual believes that an organizational and

technical infrastructure exist to support the system

Venkatesh Model of IT Adoption

DeLone and McLean Model of IS Success

System Quality

Information Quality

UserSatisfaction

IndividualImpact

OrganizationImpact

Use

Seddon Model of IS Success

System Quality

Information Quality

UserSatisfactions

Net Benefits to:

IndividualsOrganizations

Society

PerceivedUsefulness

Outline

Prior work DL success model

From end user perspective

Case study Conclusion

DL Success Model

5S and minimal DLSynthesize

IS success and adoption models (see above) Information life cycle model (Borgman et al.) 5S-based DL quality model (Gonçalves et al.) Information-seeking behavior models (Ellis’ and

Kuhlthau’s)

From end user perspective

Informal 5S & DL Definitions

DLs are complex systems that

• help satisfy info needs of users (societies)

• provide info services (scenarios)

• organize info in usable ways (structures)

• present info in usable ways (spaces)

• communicate info with users (streams)

5S and DL formal definitions and compositions (April 2004 TOIS)

5S

structures (d.10)streams (d.9) spaces (d.18) scenarios (d.21) societies (d. 24)

structural metadataspecification(d.25)

descriptive metadataspecification(d.26)

repository(d. 33)

collection (d. 31)

(d.34)indexingservice

structured stream (d.29)

digitalobject (d.30)

metadata catalog (d.32)

browsingservice

(d.37)

searchingservice (d.35)

digital library(minimal) (d. 38)

services (d.22)

sequence (d. 3)

graph (d. 6)function (d. 2)

measurable(d.12), measure(d.13), probability (d.14), vector (d.15), topological (d.16) spaces

event (d.10)state (d. 18)

hypertext(d.36)

sequence (d. 3)

transmission(d.23)

relation (d. 1) language (d.5)

grammar (d. 7)

tuple (d. 4)*

Borgman et al.:Workshop Report onSocial Aspects ofDigital Libraries: http://www-lis.gseis.ucla.edu/DL/

InformationLifeCycle

Information Life Cycle

AuthoringModifying

OrganizingIndexing

StoringRetrieving

DistributingNetworking

Retention/ Mining

AccessingFiltering

UsingCreating

crea

tion

distributionsee

kingutilization

search

ing,

browsing,

recommending

storing, archiving,

networkingau

thor

ing,

mod

ifyin

g,

desc

ribin

g org

anizi

ng, i

ndex

ing

pres

erva

bilit

y,

sim

ilari

ty,

tim

elin

ess,

accuracy, completeness,

conformance accessibility, preservability

pertinence,significance,

timelinessre

levan

ce

Activ

e Semi-active

Inactive

accessibility,accessing,

filteringGonçalves et al.

system quality (SQ)

performance expectancy (PE)

society, scenario,

space

service

accessibility

reliability

ease of use

joy of use

composabilityefficiency

effectiveness

extensibility

reusability

reliability

accessibility accuracy

completeness consistence conformance

pertinence preservability

relevance significance

similarity

timeliness

DL quality

dimension

information quality (IQ)

stream, structure

digital object

metadata

collection

catalog

repository

adequacy

relevance

reliability

scope

timeliness

understandability

social influence (SI)

DL success construct

society

5S and

DL concept

DL visibility

DL success

manifest variable

system quality (SQ)

performance expectancy (PE)

society, scenario,

space

service

accessibility

reliability

ease of use

joy of use

composabilityefficiency

effectiveness

extensibility

reusability

reliability

accessibility accuracy

completeness consistence conformance

pertinence preservability

relevance significance

similarity

timeliness

DL quality

dimension

information quality (IQ)

stream, structure

digital object

metadata

collection

catalog

repository

adequacy

relevance

reliability

scope

timeliness

understandability

social influence (SI)

DL success construct

society

5S and

DL concept

DL visibility

DL success

manifest variable

crea

tion

distributionsee

kingutilization

E1:starting

E2: chaining

E3: browsin

g

E4: diffe

rentia

ting

E5: monito

ring

E6: extra

cting

storing, archiving,

networking

K1:

in

itia

tion

K2:

se

lectio

n

K3:

explorationK4:

formulation

K5: collection

K6:

presentation

auth

orin

g, m

odify

ing,

desc

ribin

g org

anizi

ng, i

ndex

ing

pres

erva

bilit

y,

sim

ilari

ty,

tim

elin

ess,

accuracy, completeness,

conformance accessibility, preservability

DL Success Constructs

Activ

e Semi-active

Inactive

E: Ellis’ modelK: Kuhlthau’s model

satisfaction

performanceexpectancy

(PE)

behavioralIntention to

(re)use

relevance adequacy timeliness

reliability understandability scope

accessibilityease of use

joy of use reliability

userinterface

social influence (SI)

information quality(IQ)

system quality(SQ)

DL Success Model

information quality(IQ)

DL Concepts Regarding Information

collection metadata catalog

digital object metadata specification

repository

consist of

associated with

associated with

consist of consist of

accessibilitypreservability

pertinencerelevancesimilarity

significancetimeliness

digital object

accuracycompletenessconformance

metadata specification

impact factorcompletenesscollection

consistencycompletenesscatalog

consistencycompletenessrepository

understandabilityrelevancetimelinessreliabilityadequacy

digital objectmetadata specification

adequacy

catalogcollection

scope

repository

Information Quality (IQ)

Digital Library IQ

satisfaction

performanceexpectancy

(PE)

behavioralIntention to

(re)use

relevance adequacy timeliness

reliability understandability scope

accessibilityease of use

joy of use reliability

userinterface

social influence (SI)

information quality(IQ)

system quality(SQ)

DL Success Model

system quality(SQ)

screen designnavigation

user interface

composabilityreusability

extensibilityefficiency

effectivenessreliability

serviceaccessibility

reliabilityease of usejoy of use

System Quality(SQ)

performanceexpectancy

Digital Library SQ

Outline

Prior work DL success model

From end user perspective

Case study Conclusion

Case Study

Part of requirements analysis for ETANA-DL Email interviews with 5 prestigious

archaeologists Face to face workplace interviews with 11

archaeologists

Associate the 4 constructs of DL success model with the activities occurring in the seeking and utilization phases

DL success

Construct

seeking phase utilization phase

starting (E1/K1)

selection exploration

(E2-E6)/(K2-K3)

formulation (K4)

collection (K5)

presentation (K6)

social influence DL visibility

information quality adequacy, scope accuracy

system quality ease of use

joy of use (interface)

accessibility accessibility accessibility

performance

expectancy

usefulness

(interface)

DL Success Constructs Associated with Seeking and Utilization Phases

DL Success Constructs Associated with Seeking Phase

E1: Starting’ activity in Ellis’ model (K1: ‘initiation’ stage in Kuhlthau’s model) Social Influence (SI) — DL visibility

Publicize existence of a DLProvide a DL alert service

DL Success Constructs Associated with Seeking Phase

E2-E6: ‘chaining’, ‘browsing’, ‘differentiating’, ‘monitoring’, and ‘extracting’ in Ellis’ model (K2-K3: ‘selection’ and ‘exploration’ stages in Kuhlthau’s model) Information Quality (IQ) System Quality (SQ) Performance Expectancy (PE)

DL Success Constructs Associated with Seeking Phase

E2-E6: ‘chaining’, ‘browsing’, ‘differentiating’, ‘monitoring’, and ‘extracting’ in Ellis’ model (K2-K3: ‘selection’ and ‘exploration’ stages in Kuhlthau’s model) Information Quality (IQ)

Adequacy (degree of sufficiency and completeness) of DL collections and metadata catalogs

Scope of DL repository

DL Success Constructs Associated with Seeking Phases

E2-E6: ‘chaining’, ‘browsing’, ‘differentiating’, ‘monitoring’, and ‘extracting’ in Ellis’ model (K2-K3: ‘selection’ and ‘exploration’ stages in Kuhlthau’s model) System Quality (SQ)

Ease of use Joy of use

DL Success Constructs Associated with Seeking Phases

E2-E6: ‘chaining’, ‘browsing’, ‘differentiating’, ‘monitoring’, and ‘extracting’ in Ellis’ model (K2-K3: ‘selection’ and ‘exploration’ stages in Kuhlthau’s model) Performance Expectancy (PE)

Usefulness

DL Success Constructs Associated with Seeking Phases

E2-E6: ‘chaining’, ‘browsing’, ‘differentiating’, ‘monitoring’, and ‘extracting’ in Ellis’ model (K2-K3: ‘selection’ and ‘exploration’ stages in Kuhlthau’s model) System Quality & Performance Expectancy

DL interface: screen design & navigation

DL Success Constructs Associated with Utilization Phase

K4-K6: ‘formulation’, ‘collection’, and ‘presentation’ stage in Kuhlthau’s model Information Quality

information accuracy information accessibility

Outline

Prior work DL success model

From end user perspective

Case study Conclusion

Conclusion

Lay the foundation for defining success of DLs from the view of DL end users

Assume a multi-theoretical perspectiveSynthesize many related research areas in

terms of theory and empirical workExplicate and illustrate our approach by a

case study with ETANA and usabilityConnect with other work on DL quality: led

by Emory funded by IMLS, DELOS …

Questions?Comments?

See http://fox.cs.vt.edu/talks/2006/ 20060918ECDLsuccess.ppt