Selena Killick Library Quality Officer Leadership Seminar on Strategy, Assessment and Service Development. University of Lund 19th September 2012.
Strategic Metrics
Eras of Library Metrics
• Holdings
• Physical usage
• Customer satisfaction
• Electronic usage
• Value & Impact
Few libraries exist in a vacuum,
accountable only to themselves. There is
always a larger context for assessing
library quality, that is, what and how well
does the library contribute to achieving the
overall goals of the parent constituencies?
(S. Pritchard 1996)
Value & Impact Metrics
• Based on what matters the most to your key
stakeholders
• Identify where the Library impacts upon the goals of
the parent organisation
• Demonstrate how the library is contributing to the
mission of your institution
Strategic Metrics
• Start by defining your outcomes
• What do you want to demonstrate?
• What does your institution care about?
• SMART outcomes:
• Specific
• Measureable
• Attainable
• Relevant
• Time-bound
• If in doubt, KISS.
Good Strategic Metrics
• Linked to the University/Library mission & goals
• Provide useful information
• Reasonably accurate within the data limitations
• Ethical
• Cost effective
Mixed-Methods Approach
Quantitative
• Library Data
• Institutional Data
• National/International
Data
• Activity analysis
• Return on Investment
• Cost benefit
Qualitative
• Customer Surveys
• Focus groups
• Observational Studies
• Interviews
Quantitative Data
• Be prepared to take a fresh look at your data collection
• Look beyond the Library
• Talk to other data providers in your institution
• Combine data sets
• Local (Library & student data), national, international
• Check to see what is available before you start to count things
Case Study: Cornell University Library
Library Value Calculation
• Annual Cost to the institution $56,678,222
If CUL did not exist:
• Sourcing information and answering enquiries would
cost the institution $90,648,785
Source: Cornell University Library Research & Assessment Unit
Case Study: University of West Florida
UWF University Libraries calculated its Return on Investment (ROI) for eight of its services:
• students studying in the library;
• borrowing books, e-books, DVDs and laptops;
• students or faculty members asking reference questions or meeting with reference librarians for individual research consultations;
• conducting library instruction sessions; and,
• students or faculty members using subscription databases when off-campus.
Calculations based on the number of occurrences (service outputs) multiplied by an informal and conservative market value of the occurrence to calculate a summed value for the services. The summed value was then divided by sum of the University Libraries' personnel and operating expenditures.
• It was calculated that for every dollar expended by the University Libraries, at least $5.89 was returned for the services identified.
Source: University of West Florida
Case Study: University of Huddersfield
• Hypothesis: do the students who use the library the
most get the highest grades?
• Library Usage Data
• Book loans
• eResources usage
• Visits to the Library
• Student Attainment Data
• Final grade received when graduating
13
Non/Low Use Project Source: Stone, G. (2011)
Measuring Library Impact Source: Stone, G. (2011)
Analysis of the results consistently revealed a
correlation between e-resource use, book
borrowing and student attainment
This appears to be the case across all
disciplines
Thoughts
• Does correlation equal causation?
• Do our stakeholders like this impact data?
Qualitative Assessment
• Numbers are only ever half of the story
• Customer feedback can provide more insight that
data alone
• Customer quotes can have great impact on
stakeholders
When I started my distance learning
MSc it was the first time I had accessed
a digital library with such resources; you
have played a major positive role in my
effective learning.
The Library now has access
to 15,586 full text electronic
journals and 9,354 eBooks
Data Management
• Data Management is an important part of the
planning cycle
• Assessment management systems
• Qualitative coding software
• MS Excel / MS Access
Considerations
• Bring both the Qualitative and Quantitative together
• Collaborate with local experts
• Not all news will be good news
• Be realistic on what is achievable
How many staff-hours were spent
counting statistics this year?
Common Challenges
• Where do I start?!
• Finding the time
• Getting support from all Library staff
• Requesting data from other units
• Finding other data sets
• Difficulty in analysing and interpreting data
• Fear of change
• What if I fail?
Source: Megan Oakleaf, 2011,
Source: Megan Oakleaf, 2011,
Source: Megan Oakleaf, 2011,
Reporting Back
• Know your audience
• What do they care about?
• What is their level of understanding?
• Will they feel threatened by the data?
• Tell a story with the data
• Begin with the outcomes in mind
• Be mindful of the data limitations
Case Study: McMaster University
http://library.mcmaster.ca/library-scorecard
References
• Institutional Return on Investment. University of West Florida.
http://libguides.uwf.edu/content.php?pid=188487&sid=2184200
• Library Scorecard: Strategic Objectives, Measures & Initiatives 2011/2012. McMaster
University Library, revised April 23, 2012, http://library.mcmaster.ca/library-scorecard
• Library Value Calculations. Cornell University Library Research & Assessment Unit.
http://research.library.cornell.edu/value
• Oakleaf, M. and Matthews, J. (2011). Assessing the Impact of the Academic Library.
Presented at the 9th Northumbria International Conference on Performance
Measurement in Libraries and Information Services. York, England. 22nd – 26th August.
• Stone, G. (2011). The Library Impact Data Project: hit, miss or maybe. Presented at the
9th Northumbria International Conference on Performance Measurement in Libraries
and Information Services. York, England. 22nd – 26th August.
Recommended
