Project SAILS: Facing the Challenges of Information

Literacy Assessment

Julie GedeonCarolyn Radcliff

Rick WigginsKent State University

EDUCAUSE 2004 ConferenceDenver Colorado

2

What is information literacy?

• Ability to locate, access, use, and evaluate information efficiently and effectively.

• Guiding document: “Information Competency Standards for Higher Education” – Association of College & Research Libraries

(http://www.ala.org/ala/acrl/acrlstandards/informationliteracycompetency.htm)

3

Our questions

Does information literacy make a difference to student success?

Does the library contribute to information literacy?

How do we know if a student is information literate?

4

The Idea of SAILS

Perceived need – No tool available Project goal – Make a tool:

Program evaluation Valid Reliable Enables cross-institutional comparison Easy to administer for wide delivery Acceptable to university administrators

5

Project parameters

TestSystems design approachMeasurement model – Item

Response TheoryTests cohorts of students (not

individuals)A name

Standardized Assessment of Information Literacy Skills

6

The project structure

Kent State teamOhio Board of Regents collaborative

grant with Bowling Green State University (part for SAILS)

IMLS National Leadership GrantAssociation of Research Libraries

partnershipwww.projectsails.org

7

Technical components

Environment Item builderSurvey builderSurvey generatorReport generationChallenges

8

Environment

Linux (Red Hat)ApacheMySQLPHP

9

Survey process

Create survey questions (items)Create survey for this phaseAdd schools for this phaseSchools create web front-endCollect data

10Item Builder

11

Item maintenance

12

Survey Builder

13

Survey items

14

Random selection of items

15

School information

16SAILS front-end

17

Redirection to SAILS web site

Parameters passed: Unique student identifier School code Authorization code

18

Link test

19

Demographic data

20Survey questions

21

Report process

Send schools unique identifiersUpload demographicsScan & upload paper surveysGenerate entire dataset fileOffline IRT analysisUpload IRT resultsGenerate reports

22

Sample report text

23

Sample report graph

24

Technical challenges

Creation of the front-endCustomizations for schoolsAutomating the data analysisSupporting different languages

25

Data analysis

Item Response Theory Measures ability levels Looks at patterns of responses

For test-takersFor items (questions)

Based on standards and skill setsShow areas of strength and areas of

weakness

26

Status

Instrument 126 items developed, tested, and in use Web-based and paper-based administration

Grant Project - IMLS Phase I complete - 6 institutions Phase II complete - 34 institutions Phase III began June 2004 - 77 institutions

27

Next steps for SAILS

Analyze data and other inputAdministrative challenges

Self reported demographic data Testing environment Report generation

Does the instrument measure what we want it to?

Are institutions getting what they need?

28

Summary

Vision: Standardized, cross-institutional

instrument that measures what we think it does

To answer the questions: Do students gain information literacy

skills? Does information literacy make a

difference to student success?

29

For more informationwww.projectsails.orgsails@kent.edu

Julie Gedeon, jgedeon@kent.eduCarolyn Radcliff, radcliff@kent.eduRick Wiggins, rwiggins@lms.kent.edu

Mary Thompson, project coordinatormthomps1@kent.edu; 330-672-1658