Depending On Our UsersCollecting User Feedback to Assess and

Improve Research Consultations

ACRL Virtual ConferenceMarch 31, 2011

Carrie ForbesErin Meyer

University of Denver, Penrose Library

Overview

• Introduction• Research Center Model• Effective Assessment Program• Assessment Instrument• Instructional Assessment• Results• Implementation Strategies

Poll Question 1

Do you currently offer a reference or researchconsultation service?

– Yes– No

Research Center Model

• Changing Nature of Reference• Increased Complexity of Questions• Based on Writing Center Model• Visible, Dedicated Space

Poll Question 2

Where do you offer reference or researchconsultations?

– At the reference desk– In librarian offices– Outside of the library– In a dedicated space in the library– Other

Effective Assessment Program

• Immediate Feedback• Continuous and Comprehensive• Buy-in• Feedback Revision• Well-designed Instrument• Supports Learning Outcomes

Assessment Instrument

• Demographics• Service Awareness• Satisfaction• Learning Outcomes Questions:

– What was the most important thing that you learned in the consultation?

– What questions do you still have about library research?

Instructional Assessment

• Instruction Program Statistics• Minute Paper Assessments• Learning Outcomes Questions:

– What was the most important thing that you learned in the library workshop?

– If you were to attend a follow-up workshop, what topics or resources would you want us to cover?

• Results

Results

• Who receives research consultation assistance? • Satisfaction Rate• When do students seek help?• How do they hear about the Research Center?• What do students say they learned?• Are there any correlations between

– major and what they say they learned?– when they sought help and what they say they

learned?

International Business - 35

International Studies - 22

General Business - 21

Hotel Restaurant & Tourism Management - 20

Communication - 18

Psychology - 18

Top 6 Undergraduate Majors

International Business - 35

International Studies - 22

General Business - 21

Hotel Restaurant & Tourism Management - 20

Communication - 18

Psychology - 18

Social Work - 96

International Studies - 91

Library and Information Science - 49

Hotel Restaurant & Tourism Manage-

ment - 20

Clinical Psychology - 12

Top 5 Graduate Majors

Social Work - 96

International Studies - 91

Library and Information Science - 49

Business - 25

Clinical Psychology - 12

No or N/A - 16 (1.7%)

Yes - 922 (98.3%)

Satisfaction Rate

No or N/A - 16 (1.7%)

Yes - 922 (98.3%)

Would you recommend the Research Center to a friend or classmate?

5 or more days before - 778 (66%)

3-5 days before - 228 (19%)

1-2 days be-fore

- 150 (13%)

Same day – 21 (2%)

When did students seek help?

5 or more days before - 778 (66%)

3-5 days before - 228 (19%)

1-2 days before - 150 (13%)

Same day - 21 (2%)

How far before your paper or project was due did you visit the Research Center?

Professor - 620

Librarian class visit - 213

New student orientation - 139

Same day – 21 (2%)

Visibility - 78

Friend/Classmate - 50 Online - 28

How did you hear about the Research Center?

Professor - 620

Librarian class visit - 213

New student orientation - 139

Brochure - 6

Visibility - 78

Friend/Classmate - 50

Online - 28

Correlations between major and learning outcomes?

Social Work and International Studies graduatestudents had similar numbers of consultations.

Students in International Studies indicated agreater interest in learning about specific databases while students in Social Work wantedto learn about search refinement.

Correlations between when they sought help and what they learned?

Students coming in 5 or more days before weremore likely to want help with search refinement.

Students coming in 1-2 days before theassignment was due indicated that they learnedabout accessing the materials. Overall numbers for students coming in 1-2 days ahead wererelatively low, however.

Poll Question 3

How do you assess your reference services?– Counting reference questions (on-going)– Counting reference questions (sampling)– User surveys or feedback (on-going)– User surveys or feedback (sampling)

Implementation Strategies

• Point-of-use Survey• Integrate Demographic Questions, Service

Feedback, and Learning Outcomes• Online Survey Software• Qualitative and Quantitative Questions

Future Assessment Ideas

• Modify Learning Outcomes Questions• Assessment of Non-Student Users• Faculty Feedback• Assessment of Student Work• Longitudinal Data

Contact Information

Carrie Forbes, Instruction Coordinator andReference Librariancarrie.forbes@du.edu

Erin Meyer, Student Outreach Librarian andResearch Center Coordinatorerin.meyer@du.edu