Assessment Cycle or Circular File: Do Academic Librarians Use Information Literacy Assessment Data? Megan Oakleaf & Lisa Hinchliffe Library Assessment

Assessment Cycle or Circular File:

Do Academic Librarians Use Information Literacy

Assessment Data?

Megan Oakleaf & Lisa HinchliffeLibrary Assessment Conference, August 2008

Oakleaf & Hinchliffe, LAC 2008

Methods - IL Assessment Data

• Surveys• Tests• Performance assessments

– Ex: evaluation of student bibliographies, research papers, or portfolios

• Classroom Assessment Techniques (CAT)– Ex: muddiest points or minute paper

• Etc.• Etc.


From the literature: (Bresciani 2009)

• Too little time/resources

• Lack of knowledge or skills

• Lack of process coordination

• Lack of conceptual framework for assessment

• Lack of collaboration with faculty

• Lack of trust

• Difficulties managing expectations

Barriers to Assessment


Research Questions

• Do academic librarians use information literacy assessment data? – If so, how do they use the data? – If not, why not?

• What can be done to encourage the use of information literacy assessment data in academic libraries?


Survey & Participant Details

• 12-question survey using SurveyMonkey

• Distributed to Information Literacy Instruction Listserv (ILI-L) and the ACRL Immersion Alumni listserv

• 437 participants – 98% academic librarians– 96% conduct IL instruction– 35% supervise people who conduct IL instruction– 31% have been a librarian 1-5 years– 21% have been a librarian 6-10 years– 24% have been a librarian 20+ years


Do academic librarians assess information literacy skills?

76% (332)

assess IL skills

24% (105)

do not

Barriers to Assessing IL Skills


Too Little Time

26% I spend my time on other work responsibilities.

21% I’m unsure how to re-allocate my responsibilities to make time for assessment.


Too Few Resources

29% I don’t have the necessary staff.

19% Other library services and priorities monopolize available resources.


Lack of Knowledge & Skills

21% I have limited understanding of or expertise in assessment.

17% I’m concerned that I am not competent in conducting assessment.

15% I don’t know how to select & implement assessment tools.

14% I don’t know how to analyze assessment data.


Lack of Process Coordination

29% There is no centralized support/commitment for assessment.

22% There is no assessment coordinator to assist me.

18% Assessment is not valued in my library.

15% There is no assessment committee to assist me.


Lack of Conceptual Framework

18% It’s hard to integrate with college-wide assessment efforts.

17% It’s hard to grasp librarians’ role in assessing student learning.


Fear, Anxiety, Lack of Trust

17% I’m concerned that I am not competent in conducting assessment.


Lack of Collaboration with Faculty

21% Faculty involvement is limited.

19% Faculty consider the library an auxiliary service.

11% Faculty do not collaborate with librarians or vice versa.


Difficulty Managing Expectations

17% Many assessment tools don’t adequately measure information literacy.


To Assess IL Skills, Librarians Need:

• More time

• Help re-allocating their time

• More staff

• Greater understanding of IL assessment

• Centralized support and/or a coordinator of assessment

• Increased faculty involvement


Do academic librarians who assess IL skills

have results from these assessments?

86% (276)

have results

14% (46)

do not

Barriers to Producing Assessment Results


Too Little Time

31% I spend my time on other work responsibilities besides producing assessment results.

24% I don’t have time for the added responsibility of producing assessment results.

21% I’m unsure how to re-allocate my responsibilities to make time for producing assessment results.


Too Few Resources

24% I don’t have the necessary staff to produce assessment results.

10% I won’t be rewarded for producing assessment results.



31% I have limited understanding of or expertise in producing assessment results.

14% I don’t know how to select & implement tools for producing assessment results.



24% There is no centralized support/commitment for producing assessment results.

10% There is no assessment coordinator to assist me in producing assessment results.

10% There is no assessment committee to assist me in producing assessment results.

10% Assessment results are not valued on my campus.


Lack of Conceptual Framework for Assessment

17% It’s hard to integrate assessment results with college-wide assessment efforts.



10% I’m concerned that I am not competent in producing assessment results.



24% Faculty do not collaborate with librarians or vice versa to produce assessment results.

17% Faculty involvement in producing assessment results is limited.



31% Many assessment results don’t adequately measure information literacy.

17% Many assessment results don’t provide an adequate level of detail.


To Produce Results, Librarians Need:• More time

• Help re-allocating their time or responsibilities

• More staff

• Greater understanding of how to produce assessment results

• Centralized support for producing assessment results

• Increased faculty collaboration in producing assessment results

• Assessment results that adequately measure IL skills


Do academic librarians who have IL assessment results

use them?

83% (228)

use assessment results

17% (48)

do not

Barriers to Using Assessment Results


Too Little Time

45% I spend my time on other work responsibilities besides using assessment results.

25% I don’t have time for the added responsibility of using assessment results.

18% I’m unsure how to re-allocate my responsibilities to make time for using assessment results.


Too Few Resources15% I don’t have the necessary budget to use assessment

results.

15% I don’t have the necessary professional development opportunities to use the assessment results.

15% I don’t have the necessary staff to use assessment results.

15% I won’t be rewarded for using assessment results.

15% Other library services and priorities monopolize available resources for using assessment results.



40% I have limited understanding of or expertise in using assessment results.

33% I don’t know how to select & implement tools for using assessment results.



28% There is no centralized support/commitment for using assessment results.

15% The use of assessment results is not valued on my campus.

13% The use of assessment results is not valued in my library.

13% There is no assessment committee to assist me in using assessment results.

10% There is no assessment coordinator to assist me in using assessment results.


Lack of Conceptual Framework

35% It’s hard to grasp expectations for using assessment results.

33% It’s hard to integrate the use of assessment results with college-wide assessment efforts.

28% It’s hard to grasp librarians’ role in using assessment results.

13% It’s hard to grasp librarians’ role in using assessment results for accreditation purposes.



25% I’m concerned that I am not competent in using assessment results.



30% Faculty do not collaborate with librarians or vice versa to use assessment results.

28% Faculty involvement in using assessment results is limited.



48% The assessment results can’t be used because they don’t adequately measure information literacy.

40% The assessment results can’t be used because they don’t provide an adequate level of detail.


To Use Results, Librarians Need:• More time• Greater understanding of

how to use assessment results

• Training in how to use assessment results

• Training in how to select & implement tools for using assessment results

• Centralized support for using assessment results

• Clearer expectations for using assessment results

• Greater understanding of librarians’ role in using assessment results

• Increased faculty collaboration/involvement in using assessment results

• Methods for integrating assessment results with college-wide efforts

• Assessments results that adequately measure IL skills

• Assessment results that provide an adequate level of detail about IL skills


What do academic librarians use IL assessment results for?


Ways Academic Librarians Use IL Assessment Results

96% To improve instruction

73% To increase student learning

58% To inform future assessment efforts

52% To provide data for accountability measures

50% To collaborate with faculty

36% To collaborate with other librarians

20% To argue for increased resources


Results Summary (n=437)

Yes No

Assess? 332 105

Analyze Results?

276 46

Use Results? 228 48

42

Closing the Loop…


So, what can be done to encourage academic librarians to assess IL, produce assessment

results, & use the results to achieve the purposes of

assessment?


What can be done to encourage collection & use of assessment data?

From the literature: (Bresciani 2009)

• Educate• Clarify• Collaborate• Coordinate

• Celebrate• Be flexible• Keep it simple


EducateTrain librarians about

– IL assessment in general, – tools for assessing IL skills

adequately (and in detail), – tools for producing data, and – tools for facilitating the use of

assessment data

Inform library administration about – the time and – resources required to assess IL

skills, as well as produce and use assessment results


Clarify

Clarify the role of librarians in assessing information literacy as well as producing & using assessment data


Collaborate

Collaborate with faculty, centralized campus assessment support processes and personnel.

Participate in college-wide efforts to act on assessment results.


Coordinate

Coordinate assessment efforts within the library by creating structures to support IL assessment.


Be Flexible

Re-allocate job responsibilities of those tasked with assessment duties.

Questions?

[email protected]

[email protected]

mailto:[email protected]

mailto:[email protected]

Documents

Assessment Cycle or Circular File: Do Academic Librarians Use Information Literacy Assessment Data? Megan Oakleaf & Lisa Hinchliffe Library Assessment