Culture of Assessment

NCLB

Outcomes-based assessment

Accountability

Pressure to Assess

At UNO--MyMapp

Programmatic assessment

ACRL training

Model for real-life library assessment

Advantage of developing model before committing resources

Feedback from ACRL instructors

http://www.ala.org/ala/acrl/acrlproftools/assessingstudent.cfm

Ask yourself a few questions Need to Determine:

ScopeMeasurement instrumentPopulationTimeframeDelivery MethodSustainability

What do you want to measure? Determine type of instrument

Pre and Posttest to measure ENGL116 Composition II students’ information literacy skills

Pool of questionsVariety of questions—multiple answer, drag

and drop and fill in the blank

Who can help?

Institutional assistance Faculty with expertise in assessment English faculty to assist in question

development and review Graduate students to assist with data

analysis Other librarians with assessment

experience

Consultants

Use consultants to:

Define your scopeSuggest testing instrument modelsTest your testHelp navigate IRB processReview results and draw conclusions

Delivery

Online vs. Paper Course management software

Blackboard○ Campus IT support○ Familiar○ Convenient○ Accessible 24-7

Time

Planning for assessment—our timeline Timeline for launching Timeline for measuring

Longitudinal?One time measure?

What can you sustain?

Money

Do you need it? What it did for us

IncentiveStipend for consultantsStudent employee wagesData analysis

Staff

How to create buy-in with:Library staffLibrariansEnglish professorsAdjunctsCampus stakeholders

Highlight benefits to all stakeholders

Curriculum

Build on what you have in place Keep it simple to start Be consistent with learning objectives Reinforce key concepts Share decision-making with English or

other faculty stakeholders

Promotion

English Department faculty meetings Library Instruction sessions Library Newsletter—website Continuous need for promotion every

semester until it becomes routine

Benefits of assessment in library Working with faculty consultants

campus-wide opened our eyes to the ways in which different units measure learning

Engaging in larger assessment conversation developing on campus—future committee work opportunities

Theory

Tests

Consistent

Not Consistent

Modify Hypothesis

MeasurementsPre and post questionnairesStudent surveysBlackboard organizationControlled group of librarians, English

instructors, and class sections

QuestionsCollaboration with English departmentPrioritized outcomes with ACRL IL StandardsUsed challenging, performance-based questions

Background and Online Assessment ToolsWhy did we select Blackboard?

Saving time and money with little trainingBlackboard – The Foundation

Sample Size15 sections of English 1160/1164184 students completed both questionnaires

Pre-questionnaire mean = 8.23 out of 15 Post-questionnaire mean = 9.89 out of 15 Statistically significant (p < .001)

Findings from Initial Semester

What did you find most useful in the library instruction?A compleat waste of my time and tuition.Learned what some stuff didAll of the engines that we could use to find things,

and how to use them.The library instructor only showed us a couple,

but after being shown how to use a few, I went through and figured out how to use others without being intimidated by all of the information.

Everything..I woul not have found my sources if it wasnt for the library instruction

Changes implemented after initial semesterAdvice offered from faculty consultants, library

instructors, and English faculty:

147 completed the pre and post questionnaires8 scores rejected as “outliers”139 scores used

Pre-questionnaire mean score = 16.10 Post-questionnaire mean score = 15.61 Results were NOT statistically significant

Results from Second Semester

Second Semester – Survey Questions

Second Semester – Survey Questions

Second Semester – Survey Questions

Correlation between Score Difference and Number of Days Between Tests– Second Semester

Correlations

1 -.081

.344

139 139

-.081 1

.344

139 139

Pearson Correlation

Sig. (2-tailed)

N

Pearson Correlation

Sig. (2-tailed)

N

difference

Number of Days

differenceNumberof Days

Factors affecting success

Rewards Focused and clarified the purposes of the

information literacy sessions Prioritized and used the most relevant

standards and outcomes that fit our program and institution

Continued to build relationships with the English department faculty

Future assessment ideas

Verbal demonstrations Keyboard analysis Surveys—in depth Longitudinal study of information literacy

skills Citation analysis—MyMapp e-portfolio

Practical Tips

Thank You

Questions? Discussion