Informing Reference Services @ Columbia University Libraries

Using Today’s Numbers to Plan Tomorrow’s ServicesRUSA Program, ALA Annual Conference

4:00PM, June 26,2011Jennifer Rutner, Assessment & Planning LibrarianColumbia University Libraries

Reference Assessment @ Columbia

Digital Centers User Needs AssessmentService EvaluationVirtual Reference Assessment

Assessment Program Mission Statement

Serve library users and staff through the gathering, analysis, and application of high-quality, actionable information to guide library decision making.

Project Teams and Decision Making

Appoint project team Conduct assessment Make

recommendations Implementation

Defining Information Needs

Identify existing data

Brainstorm questions

Information needs “IWTK”

Prioritize information

needs

Determine audience

Assign methodology

Develop protocol

Library vision

User needs

Assessing IM Reference

IM at Columbia Libraries

•CUL launches Chat Reference

2001•Eval

uation of Virtual Reference transactions

2006

•Switch to Meebo

•Departmental library accounts

2007

•User Assessment

2009

•Switch to LibH3lp

•Expand staffing and hours, consolidate services

2010

Motivation

• Understand how IM services are being used.• Evaluate quality of service provided via IM

reference services.• Identify user-needs for reference support.

Clients• Reference Coordinating Committee• IM Reference Coordinator

Methodology

Survey

• Service evaluation survey sent to IM users

Focus groups

• Students who had never used IM reference

• Libraries staff

Data Analysis

• Compilation of IM statistics from 2007-2009

Findings: User Survey• 46% were graduate students• 41% were between 23-29 years old• 61% were 1st time library IMers• 79% discovered IM through the website• 51% were at home when they IM’d

Findings: User Survey

• 74% were very satisfied with the information they received when using the IM service

• 69% strongly agreed that the resources suggested by the librarian were useful

• 85% strongly agreed that using IM to contact a librarian was very easy

• 80% agreed that the hours were satisfactory

Findings: Focus Groups

• Hours: “It’s not 24 hours? That sucks.” • Communication: “It is a little bit awkward, but it’s good

that librarians are taking this to the instant messaging level. It’s just better service.”

• Marketing: “Very visible… if you’re trying to eventually expand the hours and make it more visible for students…”

• Staffing: “If they could answer the question, whoever can answer the question.”

• Convenience: “I live off campus so, my Columbia experience is minimal. So, this IM thing would probably be helpful for someone like me, who doesn’t chill in the library.”

Findings: Transaction Analysis

• 60% were policy/procedural, research or holdings questions

• 8% were e-resource problems• 90% come through the widget• 57% required no follow-up• 85% are <10 minutes

1702 transactions logged from 2007-2008.

Recommendations

An ideal IM service for our users would be a 24/7 service; staffed by knowledgeable, friendly people; easy to access from the Libraries’ web pages that they use most; where they can get quick efficient service.

Impact: Service• Library-wide participation• MLS interns• 15 additional staffStaffing• 36 additional hours/week

Hours• Original: Live Assistance• Interim: Meebo• Today: Library H3lpPlatform

Impact: Usage

Year # of IM Transaction2007 4682008 3832009 5492010 22932011 (through May) 1722

Digital Centers: User Needs Assessment

Locations

Digital Social Science Center @ Lehman Social Science Library

Digital Humanities Center @ Butler History & Humanities Library

Digital Science Center @ Science & Engineering Library

The Burke Library @ Union Theological SeminaryAvery Architectural & Fine Arts Library

MotivationHow is the changing information environment

impacting library use, and student needs for technology and research support?

Clients• Departmental libraries• Libraries IT Office• Library Leadership

Methodology

Social Sciences2007

• Online questionnaire

• Student focus groups (2)

• Faculty interviews

• Observation studies

Humanities2009

• Online questionnaire

• In-library paper questionnaire

• Focus groups, TBD

Sciences2010

• Online questionnaire

• In-library paper questionnaire

• In-library flip charts

Results

Social Sciences

• Surveyn = 125

Humanities

• Surveyn = 940

Sciences

• Surveyn = 611

Impact: Digital Social Science Center

• From on-call to desk hours

Mode• Increased visibility• Centralized service pointLocation• Promote consultation services• “Info Expo” eventOutreach• Group study• Presentation practice roomStudy space• Presentation practice room• Quantitative support, GIS, bibliographic softwareTechnology

Ongoing!Impact: Digital Humanities Center

• Dedicated 2 reference staff to the DHC

Staffing• Transitioning reference support from

the reading room to the DHCLocation• SCANNERS!

Technology• Extended hours in the DHC• Late night vs. overnightHours

Impact: Digital Science Center

• Increased reference visibilityLocation

• Emerging Technology Librarian hiredStaffing

• Group study rooms and boothsStudy space

• Quantitative software• Visualization softwareTechnology• More workshops

Instruction

Digital Centers: Evaluation

DSSC Evaluation

MotivationUnderstand awareness of and

satisfaction with new services at the DSSC.

Methodology• Online survey• In-library paper survey

Response• 416 respondents

– 47% Graduate Students– 80% from the social sciences

Findings: Technology

• 66% “never heard of” data services, Bloomberg stations

• 47% “heard of/never used” GIS

• Overall satisfaction with technology services

Findings: Research Support

• 34% “never heard of” the reference desk• 42% “never heard of” emailing a librarian• 50% “never heard of” of research consultations• 50% “never heard of” library workshops• 66% “never heard of” IMing a librarian

Findings: Space

• 75% use the library for individual study• 68% use the library for group study

• 67% are satisfied with individual study• 67% are satisfied with group study

• 61% “never heard of” practice presentation room• 73% “never heard of” conference calling room

Impact• Continue renovations• Improve maintenanceFacilities• Continue to expand group study

Study space• Promote IM• Promote GIS/Data servicesOutreach• Offer more workshops• New orientation approachInstruction

Thank you.

jenrutner@columbia.edu

Special thanks to: Amanda Bielskas, Rob Cartolano, Kathleen Dreyer, Mary Giunta, Jean LaPonce, Bob Scott, Jane Winland