Embed Size (px)
Citation preview
Preparing to Work Effectively withLibQUAL+® Survey Results
CARL ABRCLibQUAL+® & Beyond
Applying Your Survey Results & Other Performance Measures in Library PracticeOttawa, ON
October 24-25, 2007
Raynna BowlbyMartha Kyrillidou
Association of Research Libraries
old.libqual.org
old.libqual.org
Desired OutcomesDesired OutcomesAs a result of our work together today, you will be prepared to…As a result of our work together today, you will be prepared to…
• Perform some simple analyses of the LibQUAL+® survey results data
• Today we’ll be working mainly with the 22 core LibQUAL+® questions and Comments
• Present the results to different stakeholders
• Utilize the data to target areas for improvement
old.libqual.org
Understanding the Data & ResultsUnderstanding the Data & Results
old.libqual.org
Representativeness - Standard Disciplines: Representativeness - Standard Disciplines: An Academic LibraryAn Academic Library
Respondents
Population
old.libqual.org
Representativeness - Customized Disciplines: Representativeness - Customized Disciplines: An Academic LibraryAn Academic Library
Respondents
Population
old.libqual.org
Determining RepresentativenessDetermining Representativeness
• Conclusions and recommendations are grounded in an understanding about your survey respondents and the population you surveyed
Determining Representativeness Worksheet:
LibQUAL+® Survey Population Statistics
old.libqual.org
Dimension SummaryDimension Summary
Key Term: Zone of Tolerance
old.libqual.org
Peer ComparisonsPeer Comparisons
LibQUAL+ 2006Faculty Ratings of Journal Collections
ARL Libraries
5.00
6.00
7.00
8.00
9.00
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37
Red Square = Perceived Service Performance
Top of Blue Bar = Desired Level of Service
Bottom of Blue Bar = Minimum Level of Service
UVA
Source: Jim Self, University of Virginia, Presented at Performance Measurement in Academic Libraries Workshop, EBLIP4, Durham, North Carolina, May 11, 2007
old.libqual.org
Peer ComparisonsPeer ComparisonsLibQUAL+ 2006
Overall Quality of the Service Provided by the Library38 ARL Libraries
7.87
7.637.63
6.51
6.61
5.87
7.487.52
7.87
5.5
Undergraduates
Graduates
Faculty
UVA UVA
UVA
Source: Jim Self, University of Virginia, Presented at Performance Measurement in Academic Libraries Workshop, EBLIP4, Durham, North Carolina, May 11, 2007
old.libqual.org
Peer ComparisonsPeer Comparisons
Adequacy GapThe difference between the minimum and perceived score.
-0.40
-0.20
0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
UT Austin
ARL
Peers
Source: Fred Heath, LibQUAL+™ Results Meeting, ALA Annual Conference, Seattle, WA, January 22, 2007
old.libqual.org
NormsNorms
• Identify your score
• Compare it to a relevant norms table– Year– Subgroup– Dimension
• Norms are stable and are not calculated on an annual basis anymore (see article)
old.libqual.org
AnalyticsAnalytics
• Institution Explorer– User group and Discipline analysis
• Representativeness graphs• Radar chart• Library Use chart• Thermometer chart• Cumulative percentile distribution
• Longitudinal
old.libqual.org
SPSSSPSS
• What is SPSS?– Statistical Package for the Social Sciences
• How do I know that my data are trustworthy?– Reliability and Validity Analysis
• Are there differences between different groups?– Compare mean scores for subgroups
old.libqual.org
Quantitative Analysis: Excel DataQuantitative Analysis: Excel Data
• Excel data files available on the LibQUAL+® Web sitehttp://old.libqual.org/Manage/Results/index.cfm
• Use customized radar chart template to create custom analyseshttp://old.libqual.org/Manage/Resources/
SampleRadarChart/index.cfm
old.libqual.org
Qualitative Analysis: User CommentsQualitative Analysis: User Comments
• About one-half of users include comments on their surveys
• User Comments available on the LibQUAL+® Web site– Download comments in Excel or text file
• Skim the comments
• Conduct ATLAS.ti analysis
old.libqual.org
Identifying What is ActionableIdentifying What is Actionable
• Conclusions and recommendations are grounded in survey data
Identifying What is Actionable Worksheets:
Priorities, Weaknesses, Strengths
old.libqual.org
Developing the OrganizationDeveloping the Organization
old.libqual.org
Presenting Results to StakeholdersPresenting Results to Stakeholders
• Identify all of the stakeholders or constituents who want and need to know about the survey results
• Consider the “stake” of each of the above; what specific aspect of LibQUAL+® will be of most interest / concern
• Determine how to communicate with each identified stakeholder
old.libqual.org
Brown University LibraryBrown University LibraryStakeholder Accountability MatrixStakeholder Accountability Matrix
http://www.arl.org/arldocs/stats/statsevents/laconf/2006/BowbyOMahony.ppt
Stakeholder "Stake"Relationship &
Communication with the Library
Indicators
ProvostEnsure PAE progress & success;
ensure effective Library management & utilization of
resources, and alignment with PAE
Inform indirectly through OIR; followed up by direct dialogue with
University LibrarianAll key indicators at broad level
University Resources Committee
Determine allocation/reallocation of financial resources
Inform directly via annual written budget request and oral presentation
Interested in indicators at more detailed level, particularly those
related to financial aspects of the Library and measures related to
specific PAE goals such as collections and space/facilities
Office of Institutional
Research
Ensure that indicators measure PAE progress
Inform directly; work collaboratively to identify, develop, and refine
appropriate measures; OIR assists in analyses & presentation of PAE
indicators
All key indicators at broad level
Corporation Committee on
Academic Affairs
Determine academic priorities & programs, with recommendations to
the full Corporation; direct the President, Provost, and others to
enact
Inform indirectly via Provost; CCAA receives regular reports from the
University administration containing assessment information on agreed-upon measures of academic quality
All key indicators at broad level
Library Advisory Council
Advise the President, annual report; chair’s influence on CCAA
Inform directly, 2-3 times per year; opportunity to build understanding
ARL size data, especially expenditures; LibQUAL+
satisfaction, especially students; space/facilities
old.libqual.org
Assumption College Assumption College LibQUAL+LibQUAL+®® Marketing Plan: Marketing Plan:old.libqual.org/documents/admin/ThistleLibQUAL+IntroSeattle.ppt
LibQUAL Marketing Theme: Can We Hear You Now?
Dates [2/6-2/26 (3/3)] Notes Owner Done
Library Web site 1/13/06 informational page on the survey linked to from library front page Carole X
January library newsletter 1/23/06 article on the survey - who, what, where, and WHY - WIIFM Dawn X
Public relations office 1/23/06 Heidi - send her survey link for her newsletter Carole X
Academic Council 1/24/06 make the announcement at meeting Carole X
Administrator's meeting 1/24/06 make the announcement at meeting Laurie X
SGA 1/29/06 make the announcement at meeting Dawn X
Bulletin board 1/30/06 Dawn X
Communication with CE & Grad 1/30/06 Dawn X
Faculty Senate 1/30/06 make the announcement at meeting Janice X
Letter 1/30/06 faculty, admin, staff - see LibQUAL mktg. booklet (students receive email) Dawn X
Poster display 1/30/06 large and small posters - pic of Larry holding the iPOD (?) Carole X
Purchase Prizes 1/30/06 iPod, gift certificates (Nina-to get Amazon one)Dawn & Laurie x
Mobiles 2/1/06 mobiles with Larry Dawn X
Tent signs 2/1/06 library Liz X
Connection 2/3/06 survey announcement John X
Public access computers 2/3/06 wallpaper screen - posters
Laurie/ Carole/ Dawn X
Email 2/6/06 take the survey (ask Dawn to send to CE) Carole
Postcards 2/6/06 Can we Hear You Now? - pic of iPod w/ info on back Carole X
Student postcards 2/6/06 iPod pic and message on back Carole X
Announce at BI sessions 2/6/06, ongoing librarian responsible for making announcements All x
PowerPoint at BI sessions 2/6/06, ongoing Carole will create and presentation would run in a loop Carole x
Provoc article 2/8/06 article written by staff writier Dawn X
old.libqual.org
Communicate with your CustomersCommunicate with your Customers(faculty, students, other)(faculty, students, other)
• Particularly those whom you asked to participate in the survey
• Announce incentive award winners• Inform users of highlights of survey results• More importantly, what the library intends to do
““You asked for it…”You asked for it…”
old.libqual.org
The Library Summit:The Library Summit:Clemson University & U. of Texas at AustinClemson University & U. of Texas at Austin
http://old.libqual.org/documents/admin/LibQUAL_Summit.pdf
old.libqual.org
Why Hold a Library Summit?Why Hold a Library Summit?http://www.lib.utexas.edu/vprovost/assessment/libsummit/pdf/info_pkt.pdf
old.libqual.org
Spreading Knowledge about Spreading Knowledge about LibQUAL+LibQUAL+®® throughout the Library Staffthroughout the Library Staff
• Post Report on library web• Make staff aware of LQ tutorial:
– http://old.libqual.org/Information/Tools/index.cfm
• All-staff presentation• Disseminate Comments to depts, units, groups,
branches, librarians, etc.• Work w/department leaders, managers
– Identifying Library Departments with Contribution or Impact on LibQUAL+® Questions
• Other?
old.libqual.org
Identify 2 or 3 LibQUAL+® questions for
which you/your department has primary
responsibility
Do these “cluster” in a specific dimension?
Being User-Centered:Being User-Centered:Guiding Library Staff in Understanding & Using Guiding Library Staff in Understanding & Using LibQUAL+LibQUAL+®® Results Results
old.libqual.org
“We know what’s best”
“They are wrong”
“We don’t have the resources to…”
““...only customers judge ...only customers judge quality; all other quality; all other judgments are judgments are essentially irrelevant.”essentially irrelevant.”
Zeithaml, Parasuraman, Berry.(1999). Delivering Quality Service. NY: The Free Press
Overcoming Resistance & Barriers:Overcoming Resistance & Barriers:Changing Attitudes & BehaviorsChanging Attitudes & Behaviors
old.libqual.org
Using Using LibQUAL+LibQUAL+®® Effectively Effectively
old.libqual.org
Targeting Incremental ImprovementsTargeting Incremental Improvements
• From all of the data, determine what can and should be addressed
• Prioritize some action items– Align with mission, vision and goals of parent
organization– Address users’ top priorities, by user group– Improve areas of strong user dissatisfaction– Build on strengths, if they are truly user needs
and priorities– Identify work that can be de-emphasized and
resources that can be reallocated
old.libqual.org
S.M.A.R.T. GoalsS.M.A.R.T. Goals
• S Specific– the desired outcome or result is clearly defined
• M Measurable– accomplishment can be charted and/or observed
• A Attainable– achievable, goal is challenging but realistic
• R Relevant– results-oriented, in line with institutional goals and library vision
• T Timely– deadlines are set for accomplishment
old.libqual.org
SMART Goals -- ExamplesSMART Goals -- Examples
• 75% of materials acquired from other libraries are received by users within 10 days of request by 12/07
• 50% of new books are on shelf within 5 days after library receipt by 10/07
• 60% courses utilizing WebCT include links to library research materials by 3/08
• Lower the unit cost of each service desk transaction by 10% from FY’08 to FY’08
Versus general goals:Improve ILL turn-around timeMake new materials accessible to users more quicklyDecrease staffing budget
old.libqual.org
old.libqual.org
LibQUAL+LibQUAL+®® Resources Resources
• LibQUAL+® Web site:http://old.libqual.org
• Publications:http://old.libqual.org/publications
• Events and Training:http://old.libqual.org/events
• Gap Theory/Radar Graph Introduction:http://old.libqual.org/Information/Tools/libqualpresentation.cfm
• LibQUAL+® Procedures Manual:http://old.libqual.org/Publications/index.cfm
old.libqual.org
LibQUAL+LibQUAL+®® Team Team
• MaShana DavisTechnical Communications [email protected]
• Yolanda GlassAdministrative [email protected]
• Kristina JusthCustomer Relations [email protected]
• Martha KyrillidouDirector, Statistics and Service Quality [email protected]
• Khyati NayakSenior Applications [email protected]
• Gary RoebuckTechnical Operations [email protected]
old.libqual.org
In development ….
Customized Consulting Services:– Longitudinal data analysis, across multi years– Tailored peer comparisons, libraries you choose– Assistance w/ content analysis for Comments– Facilitation of LibQUAL+® staff retreat or summit– Guidance in preparing communication plan– Customized training in assessment skills– Launch an assessment group or team– Assistance in identifying key customer issues for future
action
