FALL 2010ELIZABETH LEONARD

Using the ACRL Value of Academic Libraries Research

Review @ Your Library

Anne Ciliberti, Elizabeth Leonard, Luis Rodriguez, & Mark Thompson

Sponsored by the VALE-NJ Assessment Committee

Introductions

Anne Ciliberti, Director, William Paterson University Library

Elizabeth Leonard, Director, Online Campus Library, Berkeley College

Mark Thompson, Director, Middlesex County College Library

Luis Rodriguez, Director, Kean University Library

Agenda

Overview, ACRL “Value of Academic Libraries: A Comprehensive Research Review and Report” report (Sept. 2010)

How to operationalize in our own libraries?

Overview

www.acrl.ala.org/value

Why?

Request for Proposal (RFP)

Assumption: “the relevance of academic libraries is under question.” Google Current Economic Climate

Need to demonstrate their value

Are we still seen as the “heart of the university”?

“I began my professional career in the late 1960s, at the tail end of what must have been a ‘golden era,’ when we rarely, if ever, said ‘no’ to any request for new content or services. That era is long behind us. So, too, are the days when arguing that the library deserves more financial support because it is a public good and because the costs of maintaining its collections and services are rising.” (Kaufman 227)

“It used to be that the way you put together a library budget was to look at like institutions and then argue for a little more. Now my provost is saying to me: ‘If I give you x dollars, what is the return on investment to the University?’” [T. Scott Plutchak, Librarian, University of Alabama at Birmingham, as quoted in Luther 3]

If we are seen no longer as the “heart of the university,” what is it that we need to do to show our value to the institution?

RFP, continued

Result must identify measurable surrogates that demonstrate the library’s impact on institutional goals

Include best practices

Explain how the report findings should inform data collection decisions of academic libraries going forward

Other Measurement of Value Projects

Association of Research Libraries and the Institute of Museum and Library Services are partnering on a study to enrich, expand, test, and implement methodologies measuring the return on investment (ROI) in academic libraries.

(“Lib-Value Project” n.p.)

Report Overview/ToC

Parts of Presentation

Elizabeth Leonard: In which ways can libraries measure value?

Mark Thompson: What are the next steps advocated in the report?

Anne Ciliberti: What can we do next?

Value of Academic Libraries, defined

“…this report focuses on library value within the context of overarching institutions. It does not attempt to address methods for assessing library value within a library context. Therefore, this report does not emphasize measures of internal library processes, such as inputs and outputs. Nor does it focus on satisfaction or and service quality approaches. These measures are of great utility to librarians who seek to manage library services and resources, but they may not resonate with institutional decision makers as well as outcomes-based, mission-focused approaches.” (Oakleaf 25)

Value Summation

not input or outputs

not how it defines its own value

how it adds to institutional goals

Value of Academic Libraries

Value= institutional, not library, goals.  The value of an academic library is communicated by how it supports institutional goals Increasing retention/graduation rates Increasing student achievement Increasing faculty research output

Different Types of Value Measurements

1. Use2. Return on Investment (ROI)3. Production of a commodity4. Library impact5. Competing alternatives

#1, Use

Traditional measures Circulation statistics Door clicks

Useful for internal managementNot compelling to external stakeholders/

institutional decision makersMust be tied to institutional outcomes to have

real impact

#2, ROI

Defined:

Perceived costs: price, time, effort

Difficult to use in academic settings

(Oakleaf 20)

Luther, 2008

(Luther 8)

Grant income obtained using library materials/library budget= ROI on grant money on library

#3, Production of a Commodity

Challenges: finding “market price”

Purchase/exchange value What would people be willing

to pay for the service? Ability to pay Use own money or someone

else’s? Students tend to undervalue

immaterial goods (e.g. library services and information)

Fail to account for externalities

(Kelly, McNicoll, and McLellan 13)

value= quantity of commodity produced x price per unit of commodity

#4, Library Impact

Deals more with externalitiesDirect measurement is challenging

Observe use and what is being produced as a result Librarians tend to avoid

Measure surrogates of impact Student success Retention

Seen as in keeping with institutional mission and purpose

#5, Competing Alternatives

Choose library over other options (e.g. Google) Good v. Good enough Develop relationships with users to ensure they

continue to choose libraries rather than alternatives

Which are most important?

Value is entirely dependent on stakeholders’ concept of value

Stakeholders tend to choose financial value (ROI) and Impact Value (#2, #4) The “business of higher education” – money is the bottom

line

Develop studies using both ROI and Impact Value ROI: How do libraries…

…manage resources …bring money into the institution

Impact: What does the library enable users to do?

Challenges and Opportunities

Shift from value of products (e.g., collections) to services (e.g., librarian expertise and teaching)

Value on Investment 1) What did the user get out of the library service? 2) What did the user accomplish as a result? (Saracevic

and Kantor 540)

Value Thoughts

Are all values equal?

Should we treat them as such?

OK, then Next Steps

•If you accept these premises; and agree with the core findings of research;

•Then take steps to assess your library’s value within your institutional perspectives….

•See report, pp. 102-140.

(Oakleaf 103)

Next Steps According to the Report

1. Get Started2. Define Outcomes3. Use Assessment Management Systems4. Gather New Data5. Use Existing Data6. Engage in Higher Education Assessment

Efforts 7. Create Library Assessment Plans 8. Mobilize Library Administration9. Engage in Professional Development10. Leverage Association Connections

Gather Data

Gather data for either: ROI assessment OR Outcomes-Focus: research, learning, & service

contributions.

“Librarians can develop systems that will allow data collection on individual user library behavior.” “identify students who have had substantial library exposure

and compare to those who have not.” (Oakleaf 97)

Three Levels of Effort

Level 1 = Uncover your campus-wide strategic goals [see overall institutional goals, p.29], such as recruit and retain students; support teaching and learning....

Level 2= Review the potential surrogates for each goal and select.

Example: see Figure 4, p.54Level 3 = Select possible areas of

correlation (library measures) with each of these surrogates.  Example: see p.105.

Example

Level 1: Goal:Student Learning

Level 2:Possible surrogates

Must be selected foryour institutionEXAMPLE:1. Learning assessments2. Faculty judgments

(Oakleaf 41)

Surrogate #1: Learning Assessments

Level 3: Brainstormand then choosespecific linkages,correlations, orrelationships thatcan be measured.

Surrogate #2:Faculty Judgments

The top set of correlations factor toward the Outcomes values calculations;and the bottom set factor toward the ROI calculations.

Success with the Process?

What are the possible implications of this process? It assumes that we need to work more directly with

faculty. To engage them in how library resources and services

have impacted their teaching. It assumes that any library services or activities must

directly impact student learning efforts.  Course activities, such as students asking questions on

WebCT; as they request library materials; or as they conduct library research and use library resources.

A Call to Action?

All assessment is local

What does this Report say to us

…as individuals?

…as the VALE collective?

Is there interest in doing something more?

Option One

Share ideas, plans, assessment results?

Informal quarterly think-tank forums?

Option Two

Explore some of the VALUE assumptions among

NJ CAOs? Conduct (replicate?) a study of

how our administrators view/define/describe

“library value”?

• Deborah Grimes, 1993

• Beverly Lynch, 2007

• Leigh Estabrook, 2007

• Barbara Fister, 2010

Option Three

Conduct a VALE-wide study of academic

library impact?

• Dickenson’s 2005 Colorado Academic

Library Impact Study: How Academic

Libraries Help Faculty Teach and Students

Learn?

Option Four

Design a new study that combines CAO views, with those of faculty and students, such as “Teaching, Learning and Research in New Jersey: The Role and Value of Academic Libraries as Expressed by Chief Academic Officers, Faculty and Students”?

Discussion

Bibliography comprehensive Report written as toolkit

What do you think of the report’s central findings? Institutional leaders define value Communicate the value of academic libraries by how they

support institutional goals Relate library activities to report surrogates

Should all institutional priorities be equal? (retention v. student learning)

What happens when externals change (senior leadership)?

Are the steps to determining impact realistic?

Conclusion

What has worked for you?

What’s next for you?

What’s next for us?

Thank you!

www.acrl.ala.org/value

To get your own copy of the report, visit: