The Value of Library-Provided Content:

Assessing Usage and Demonstrating Impact

Megan Oakleaf, MLS, PhDNISO Virtual Conference

April 2015moakleaf@syr.edu | meganoakleaf.info

What is value in the context of library-provided content?

Conceptions of Value• Satisfaction | surveys, self-reports• Service Quality | surveys, self-reports• Inputs/Outputs | money, time, numbers, etc. in and

out, comparisons to external standards/lists• Commodity | monetary value• Competing Alternatives | monetary value in

comparison with other options• Use | downloads, citation analysis, impact

factors/SNIP/etc., altmetrics• Return-on-Investment | financial ratios, micro (cost

per use) and macro (overall library investment)• Impact | what users do with content, achievement of

institutional or other goals

WARNINGI am about to oversimplify.

Satisfaction?

Typical:23% of faculty are satisfied with

library-provided content in their area of research.

So what?

Service Quality?

Typical:65% of our users think our collections

are acceptable.

So what?

Inputs/Outputs?

Typical:We increased the collection by X.

So what?

Usage?

Typical:A lot of people downloaded

a lot of things.

So what?

Typical:A lot of people downloaded

a lot of things.We’re not sure who.

We’re not sure what things.We’re not sure what they did with them.

If anything.

Seriously?

Commodity?

Typical:The value of our collection = $$.

So what?

Competing Alternatives?

Typical:If you had to

get the same or similar materials elsewhere

it would cost you = $$, time.

But I don’t do that.

ROI $ ?

Typical:We do more with less.

Then can I give you less again?

Typical:We do more with less.

Well, we do about the same with less.

Ok…?

Typical:We do more with less.

Well, we do about the same with less.Uh, actually we do a lot less with less.

I didn’t notice….

Impact?

Impact:the value of library-provided content

in meeting the needs of users

Impact:not how good is library content,

but rather how much good

do users do with that content

Tell me more!

What do users dowith the content you provide?

What do users dowith the content you provide?

create new content, solve problems, make decisions,

take actions, and so on…

What do users do with the content you provide?

How much do you know about that?Do they do the things your

institution/organization values?

thinking organizationally vs

thinking strategically

Context for Value

Context for Value…

institutional or organizationalmission

goals strategic priorities

focus areas

Tell me more!

What does your institution/organization

care about?

Possible Impacts What does library content

contribute?

A Few Examples

Efficiency Focus Areas:

Time savingsIndustrial productivity, Shortened product cycle

Reduced parts costs, Labor savingsImproved qualityIncreased sales

Quicker response to threatsMoney saved over alternative sources

Risk of inappropriate information decreased

What does library content contribute?

Corporate Focus Areas:

Proceed to the next step in a projectDecide upon a course of actionImprove organizational imageImprove client relationshipsExploit new opportunities

Avoid time lost, bad decisions, loss of funds

What does library content contribute?

Medical Focus Areas:

Handle patient care differently (diagnosis, choice of tests, choice of drugs,

length of hospital stay, advice given)

Avoid hospital admissions, patient mortality, surgeries

What does library content contribute?

Public Library Focus Areas:Student learning, Early education, Literacy

development, Problem solvingFamily cohesion, Social inclusion, Social capital,

Community buildingCommunity capacity building, Employment, Small business support, Local economic developmentEconomic value, Decreased family spending (on

recreation, learning materials, etc.)Democracy support, Voter education

Safety, Community resilience (think Ferguson)

What does library content contribute?

What does library content contribute?

What does library content contribute?

Diagraming the Problem

Is there any intersection

here?

Librarians provide content.

The institution/ organization/etc. has needs, goals,

missions, outcomes it wants to

achieve.

But it’s possible to know…

A couple examples…

Read All About It!

Soria, K., Fransen, J. & Nackerud, S. (2013). Library use and undergraduate student outcomes: new evidence for students’ retention and academic success. portal: Libraries and the Academy. 13(2), 147-164.

Stone, G. & Ramsden, B. (2013). Library impact data project: Looking for the link between library usage and student attainment. College and Research Libraries. 74(6), 546-559.

Cox, B. & Jantti, M. (2012). Discovering the Impact of Library Use and Student Performance. EDUCAUSE Review 47(4).

Want even more information? See the 2014 Library Assessment Conference proceedings and Assessment in Action results too!

Carol Tenopir& The Value of Academic Reading

• Faculty read 22 articles, 7 books, and 10 other publications (on average) per month, and it takes them approximately 40 hours.

• Two thirds of the articles come from library collections.

• Impact: improves research results, changes research focus, inspires new thinking, improves teaching.

Tenopir, C., & Volentine, R. (2012). Scholarly reading habits: demonstrating library value. LINK, 8-9.

Adapted from: Oakleaf, Megan. “Are They Learning? Are We? Learning and the Academic Library.” Library Quarterly. 81(1). 2011.

A Map to Value

What do we need to assess (demonstrate) the value of content in

academic libraries?

We need to know:

The mission, goals, strategic priorities, or focus areas

of the parent institution.

We need to have:

Defined outcomes that describe the impact of library-provided content

on users, written in terms relevant to

the parent institution.

[Users] will be able to do [thing that the institution values].

• Students will be able to evaluate information found in library-provided content effectively.

• Faculty will be able to teach effectively using library-provided content .

• Faculty will be able to complete competitive grant proposals using library-provided content.

We need to get:

Better in-house use data.Better vendor use data.

Better user data.

“Use” data.

What content use is there?

Useful for correlating library content usage with

measures of institutional success.

But what we have right now is inadequate…

“Use” data that doesn’t exist.

What use should there be based on the needs/requirements of

the parent institution, even if that use doesn’t currently exist?

Individual level data.

We can aggregate individual data, but we can’t disaggregate group data.

We can clean personally identifiable information from data,

but we can’t add it back in.

The Question of Correlation

Surrogates for library-provided content:

• Downloads

• Citations

• Library instruction

• Reference transactions

• ILL

• “Traditional” impact factors & newer altmetrics

• Other?

Surrogates for outcomes:

• Better grades

• Increased retention

• Better teaching evaluations

• More publications

• More positive T&P decisions

• More grants or awards

• Other?

Correlations

http://meganoakleaf.info/valresearchquestions.pdf

What if we could demonstrate the contribution of library content...

• To students completing academic work successfully and completing their programs/degrees?

• To actually learning how to be good, lifelong, sophisticated users of information?

• To quick and lucrative employment of students?• To recruiting great new faculty?• To improving teaching?• To placing articles in the right publications?• To the awarding of grants?• To tenuring and promoting successful faculty members?• To the profile, prestige, reputation of the institution in the

global and local community?

More importantly…

Adapted from: Oakleaf, Megan. “Are They Learning? Are We? Learning and the Academic Library.” Library Quarterly. 81(1). 2011.

Identify situation of complexity

Decide to find

solution

Seek information

Decide to act

Take action

REFLECTIVE PRACTICE

Formulate question

Find Evidence

Appraise Evidence

Apply Evidence

Evaluate impact

EBP

Reflect on past experiences &

interpret environment

Engage in inquiry to

reduce doubt

Use reasoning to create new

rules for action

Build knowledge through

experimentation

Incorporate discoveries from action

PRAGMATISM

What if we could use what we learned about the contributions of library content…

• To teach users to be better, lifelong, sophisticated users of information?

• To help students do their academic work even more successfully and achieve their goals in greater numbers or more quickly?

• To assist students to gain faster and more lucrative employment?

• To recruit and promote more and better new faculty?

• To help faculty produce more articles in higher-impactpublications and get more and bigger grants?

• To the grow the profile, prestige, reputation of the institution in the global and local community?

Which conception of value are your assessments of library-provided

content predicated on?

SatisfactionService QualityInputs/Outputs

UsageCommodity

ImpactOr something else?

The Value of Library-Provided Content:

Assessing Usage and Demonstrating Impact

Megan Oakleaf, MLS, PhDNISO Virtual Conference

April 2015moakleaf@syr.edu | meganoakleaf.info