Embed Size (px)
Citation preview
Making a difference?
Measuring the impact of an
information literacy programme
Ann Craig
Structure of presentation
• Context and rationale• Literature • Methodology:
– Pre- and post-tests and interviews• Results:
– Increased skills and confidence? • Outcomes • Recommendations
Context
• University of Worcester
• Student profile:
– Approx 7500 students
– 60% mature students
– Non-traditional entrants
Context
• Information literacy programme– Embedded across three years of pre-registration
Nursing course– 3 key sessions in first semester
• 2 whole group sessions & 1 set of small group sessions
• Small team of subject librarians: 36 hours of contact time with students during first semester– Two intakes each year (September and February)
• Evaluation methods were mainly subjective
Research questions• How effective is the programme in
developing students’ skills?
• Does the programme increase students’ confidence?
• Are the factors of skill and confidence linked?
• How do demographic characteristics relate to skill and confidence?
Aims and objectives• Aim: to investigate whether the information
literacy programme for pre-registration nursing students is effective in increasing skills and confidence– To test students’ skill levels before and after two key
sessions– To explore factors affecting confidence– To identify any relationships between skills,
confidence and demographic characteristics
• Aim: to inform the development of information literacy at the University of Worcester
Literature: key themes• Assessment of information literacy
– Range of methods employed
• Skills: tests– Fox et al (1996), Shorten et al (2001),
Brettle (2003)
• Confidence– Kurbanoglu (2003) and Ren (2000)
• Demographic factors
Literature: key themes
• Evaluations of programmes for pre-registration nursing students: – Range of methods used
• Little evidence that programmes for nursing students are effective
• Lack of published instruments
Methodology• Skills of cohort tested using pre- and post-
intervention questionnaire (quiz): – To measure change – 14 multiple choice questions: only one correct
answer– Same questions used for both tests, in same
order– Likert scale question(s) about confidence– Information on age and gender
Methodology• Confidence was explored using a
semi-structured interview• Theoretical sampling for divergent cases• Questions covered:
– Educational level, previous experience of skills programmes
– Confidence before, during and after programme– Increase or decrease in confidence during
programme– Factors influencing confidence at all stages
Research Study• 70 students completed pre-test • Change in attendance policy: only 32 students
completed post-test• 29 students completed both
– Mean age: 32– 83% female & 17% male
• Impact on test results• Impact on selection of students for interview:
5 interviews completed
Results: Skills• Change in group is statistically significant:
p<0.05 (Wilcoxon Signed Ranks Test)• Mean number correct (out of 14)
– Pre-test 9.10 Whole group 8.44– Post-test 10.59
• Mean change +1.48• 21 students scored more in post-test• 5 students showed no change• 3 students scored less in post-test
Results: Skills
Pre- and post-tests: correct answers
02468
10
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Number correct
Cas
es Pre-test
Post-test
Results: Skills
• Most correct answers: use of library catalogue, refining searches and use of search logic– Little significant positive change can be demonstrated
when students scored highly at the pre-test
• Fewest correct answers: use of specific techniques e.g. identifying keywords
• Selection of “Internet” as answer for information sources questions
Results: Confidence
• 5 interviewees: all female and 31 or over• Representative of range of levels of
confidence• Confidence often dipped during
programme• All interviewees were “Fairly Confident” or
“Very Confident” by end of programme• Comparison with self-assessment in tests
Results: Confidence• Factors affecting confidence:
+ Programme itself: particularly small group sessions
+ Previous successful use of IT: “Mastery experiences”
+ Experience, and confidence in, using libraries
- Lack of experience with IT
- Problems with passwords!
Results: Confidence
0
5
10
15
20
Not atall
Notvery
Fairly Very
Level
Fre
qu
en
cy
Pre-test
Post-test 1
Post-test 2
Results: Confidence
• “I think I was a bit naïve actually...I thought you’d be attending your lectures…Nowhere was I thinking it’s access wwwdotslash. It completely put me on edge.”
• “I was quite proud of myself for what I’ve managed to achieve.”
Relationship between skills and confidence• Evidence of a link at the pre-test
• Some over assessment of confidence by the post-test: some confidence ratings were not justified
• Interview sample: assessed confidence in line with skill level
Demographic characteristics
• Sample for pre- and post-tests was too small to draw valid conclusions
• Whole group pre-test indicated that students aged 18-30 were marginally more confident than other students
• No difference in scores between age groups• No evidence of differences due to gender: only 5
male students
Outcomes
• Tests provided evidence that sessions should be made mandatory
• Tests highlighted areas of difficulty for whole group: inform planning
• More practical exercises in whole group sessions and brief tours in the induction
• Highlighted need to stress relevance of sessions – working with colleagues
• Tests have been reused
Recommendations
• Longitudinal study though whole course
• Conduct study with comparison group
• Extend use of tests: other disciplines
• New Nursing curriculum: challenges– Less face to face contact– Creation of interactive materials via WebCT– Student support
Summary: key results• Programme is effective in increasing students’
skills and confidence• Confidence levels rose by the end of the
programme• Some evidence of a link between skills and
confidence• No firm evidence of the impact of demographic
characteristics on skills and confidence• Future challenge: providing same learning
opportunities online
References• Brettle, A. (2003). “Information skills training: a systematic review of
the literature.” Health Information and Libraries Journal. 20 (Suppl 1.), 3-9.
• Fox, L., Richter, J.M. and White, N.E. (1996). “A multidimensional evaluation of a nursing information-literacy program.” Bulletin of the Medical Library Association. 84(2), 182-190.
• Kurbanoglu, S. (2003). “Self-efficacy: a concept closely linked to information literacy and lifelong learning.” Journal of Documentation. 59(6), 635-646.
• Ren, W.H. (2000) “Library instruction and college student self-efficacy in electronic information searching.” Journal of Academic Librarianship. 26(5), 323-328.
• Shorten, A., Wallace, M.C. and Crookes, P.A. (2001). “Developing information literacy: a key to evidence-based nursing.” International Nursing Review. 48, 86-92.
