Listening to Students: Innovative Responses

Betsy WilsonUniversity of Washington

July 10, 20087th International JISC/CNI Conference

The Role of Library Assessment

The library community needs to invest more in data collection and analysis and to take its examples from commercial leaders that have a much more detailed and insightful understanding of their customer base and preferences. In particular, there is a need for ongoing longitudinal data and intelligence functions to provide a vital early radar warning of oncoming change.

No library we are aware of has a department devoted to the evaluation of the user, how can that be?

(Information Behaviour of the Researcher of the Future, CIBER, January 2008)

North American University Libraries with Notable Assessment Programs

• Alberta “Ongoing student surveys”

• Cornell “In-house research capability”

• Pennsylvania “Data Farm”

• Rochester “Ethnographic approaches”

• Virginia “Balanced Scorecard”

• Washington “User needs”

Building a Community of Practice:Library Assessment Conference

Co-sponsored by ARL, Virginia and Washington– 2006 Charlottesville,

Virginia (220 registrants)– 2008 Seattle, Washington

(August 4-7, 380 registrants)

– 2010 Washington D.C. area

• Focused on practitioners – Papers, panels, posters,

informal contact, workshops

ARL Sponsored Assessment

• Tools– ARL Statistics– LibQUAL+®– MINES for Libraries

• Building a Community of Practice– Library Assessment

Conferences– Service Quality Evaluation

Academy– Library Assessment blog– Workshops

• Individual Library Consultation– Dynamic Duo: Jim Self (Virginia)

and Steve Hiller (Washington– Making Library Assessment

Work (24 libraries)– Effective, Sustainable, Practical

Library Assessment (6 libraries)

UW Libraries Assessment Priorities

• Understanding how faculty and students work

• Information seeking behavior and use

• Patterns of library use• Value of library• User needs• Library contribution to

customer success• User satisfaction with

services, collections, overall

University of Washington

• Established in 1861

• Campuses in Seattle, Tacoma, and Bothell

• Public research university

Students and Learning

• Diverse students• 30,000

undergraduates• 9,300 graduate

students• 1,700 professional

students

Research and Discovery

• Collaborative and interdisciplinary

• $1 billion US in grants and contracts for research

• Engineering, technology, forestry, aerospace, marine sciences, health, biosciences, computer sciences

Seattle and Innovation

• Microsoft• Amazon• RealNetworks• Serials Solutions• Starbucks• Boeing• Biotechnology firms• Gates Foundation

University of Washington Libraries

• Large, public research library

• Over 7,000,000 volumes• 50,000 serial titles• Digital collections• Excellent service• Expert staff• Innovation• Global collections• Assessment and planning

since 1992

Award Winning Library

Culture of Assessment• …is an environment in

which decisions are based on facts, research and analysis, and where services are planned and delivered in ways which maximize positive outcomes and impacts for library clients. A culture of assessment is an integral part of the process of change and the creation of the 21st century research.

All assessment is local

Creating a culture of assessment

Methodological Diversity

• Qualitative and quantitative methods

• Surveys (since 1992)• Focus groups• Observation and

interviews• Usability• LibQual• Data mining• And more…

We learned that…

• Remote use preferred• Self-reliance and

unmediated• Online equals

productivity• Desktop delivery is

top priority• Undergraduates value

place

Biosciences at the UW

• Internationally ranked graduate and research programs

• Largest segment of University research community

• Receive 80% of externally funded research monies

Key Questions

• Who are the bioscience students and researchers?

• What do they do?

• Where are they physically, administratively, and intellectually?

• How do they find and use information?

• What are the barriers?

• How can the Libraries help?

• How will we know if we have made any difference?

Bioscience Findings

• Print is dead, really dead• e-journal provider• Faculty don’t come to the

physical library• Students come to the

physical library• Databases underused• Personal information

management• Buy from Amazon

More Findings

• Transaction cost from discovery to delivery

• Fragmented systems and processes

• Multi-disciplinary collaborators

• Everywhere, in scattered locations

• Independent and self-sufficient researchers

Biosciences Recommendations

• Integrate search/discovery tools into users workflow• Expand/improve information/service delivery options• Make physical libraries more inviting and easier to use

– Consolidate libraries, collections and service points– Reduce print holdings– Focus on services

• Use an integrated approach to collection allocations• Get librarians to work outside library space• Lead/partner in scholarly communications and E-science • Provide more targeted communication and marketing

Biosciences Actions 2007-08

• Appointed a Director, Cyberinfrastructure Initiatives, Biosciences, and eScience

• Strategic priorities:– Improve discovery to delivery

(WorldCat Local, etc.)– Reshape physical facilities as

discovery and learning centers – More rapid delivery services– Enhance the Libraries support

for UW’s scientific research infrastructure (DataNet, etc.)

– Do market research

Can We Generalize?Did themes raised in the interviews/focus groups

reflect the bioscience population? The campus community?

The 2007 Triennial Survey as a corroborating source:• Mode of access• Resource type importance• Sources consulted for research• Primary reasons for using Libraries web sites• Libraries contribution to work and academic

success

• Useful library services (new and/or expanded)

Reasons for In-Person Library Visits 2001

Faculty and Undergrads Visiting Weekly or More Often

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Use Collections Use space orservices

Only Collections Only space orservices

Faculty

Undergrads

Library as Place Change In Frequency of In-Person Visits 1998-2007 (weekly+)

Faculty

Faculty

Grad

Grad

Undergrad

Undergrad

20%

30%

40%

50%

60%

70%

80%

1998 2001 2004 2007

Undergrad Activities DuringLibrary Visits

(% using at least weekly in 2004/2007)

0%

10%

20%

30%

40%

50%

60%

70%

Getassistance

Look formaterial

Use libcomputers

IndividualWork

Group Work

2004 2007

Off-Campus Remote Use 1998-2007(Percentage using library services/collections at least 2x week)

Faculty

Undergrad

Grad

0%

10%

20%

30%

40%

50%

60%

70%

1998 2001 2004 2007

0%

10%

20%

30%

40%

50%

60%

70%

Sources Consulted for Information on Research Topics (Scale of 1 “Not at All” to 5 “Usually”)

Faculty

Grad

Undergrad

2.5 2.75 3 3.25 3.5 3.75 4 4.25 4.5

BibliographicDatabases

Library Catalog

Open Internet RefSource

Open Internet Search

Sources Consulted for Information on Research Topics (Scale of 1 “never” to 5

“usually”)

3

3.25

3.5

3.75

4

4.25

4.5

Health Sci BioSci Phy Sci-Engin Hum-Soc Sci

Bib Databases Open Internet Library Catalog

I wish the interface between scholar and UW libraries was better. I want to search with Scholar, but use UW's credentials to access full-length articles. Right now, there are a lot of intermediary pages I need to visit. Bioengineering grad student

Libraries Contribution to… (Scale of 1 “Minor” to 5 “Major”)

3 3.25 3.5 3.75 4 4.25 4.5 4.75

Academic success

Efficient use of time

Finding info in new orrelated areas

Keeping current inyour field

Being a moreproductive researcher

Grad Faculty

Takeaways after 16 Years

• Undergraduates– Library as Place– Work, meet, learn, live

• Graduate/professional students– Access to information and

services– How can the library save

me time?

• Faculty– Collections (physical and

virtual)

Innovative Responses

• Extend hours in Undergraduate Library (24/7)• Create more diversified student learning spaces• Privilege electronic journals • Enhance usability of discovery tools and

website• Provide standardized service training for all staff• Stop activities that do not add value • Consolidate and merge branch libraries• Change/reallocate collections budget and

staffing• Improve librarian liaison program to academic

areas

WorldCat Local

• Discovery to delivery challenge• One box, one stop shopping

– Formats integrated– UW holdings, regional consortium, WorldCat, 50 million articles– User doesn’t need to know how we do things internally

• Components– Content– Shared platform– Interoperability with local services

• Circulation, ILL, OpenURL

– Common marketing

Results to Date

• 20% increase in use of local collections

• 60% increase in borrowing from regional consortia

• 114% increase in world-wide interlibrary loan

• 200-275,000 searches per month

• WCL is our #2 ranked OpenURL origin and growing

Innovation

UWILL

• University of Washington Information Literacy Learning

lib.washington.edu/uwill/

UWired

• Established in 1994• Multiple partners• Teaching, learning,

and technology• Information smart

global citizens

Successes

• Collaboratories• Center for Teaching• Learning Commons• Technology-

enhanced spaces• Course tools• Workshops• MyUW• ONTECHNews

The Challenge: Maintain Relevancy and Centrality

Faculty 4.25

4.44

4.33 4.33

Faculty 4.56

Undergrad 3.97 3.99

4.22

4.32 UW Seattle UG 4.36

4.34

4.26

4.11Grad 4.18

Grad 4.36

3.8

3.9

4

4.1

4.2

4.3

4.4

4.5

4.6

1995 1998 2001 2004 2007

3.8

3.9

4

4.1

4.2

4.3

4.4

4.5

4.6

Overall Satisfaction by Group 1995-2007

Working without a Net

Suggestion Box in Hell

Listening and Responding

For more information:

http://www.lib.washington.edu/assessment/

Thank You