University Library

Making the connections: assessment, training, progression and performance

Dr A. Harrison: Institute of Psychological Sciences

A. Newton: Leeds University Library

Lilac 2007

Why this project?

BSc Psychology (Hons)

A progressive and integrated degree programme that encourages students to become more autonomous thinkers and practitioners of Psychology

Culminating in an independent, empirical project reporting on a substantial piece of research

Do we need Information Literate students?

Quality Assurance Agency

Generic Skills

Retrieve and organise information effectively:- books, journals, computer and internet sources

Handle primary source material critically

Become more independent and pragmatic as learners

The story so far

Lilac 2005

Second year students are more information literate than first year students

First year students have a strong preference for using the internet to source information

Second year students use the Library Catalogue as their primary literature searching tool

Both groups use the Library Catalogue ineffectively

Current study

Aims:

• Assess the reliability of the data produced by the information literacy questionnaire

• Provide detailed information about the progression of information literacy skills acquisition by students

• Assess the relevance of information literacy skills to academic performance

A reliable tool?

Comparing results

No significant differences between the performance of the two first year groups (2005-06/2006-07): t = 0.909, df = 382 p=0.364

No significant differences between the performance of the two second year groups (2005-06/2006-07): t =1.58, df = 383, p>0.115

Year of entry 2005 2006

First year StudentsMean % correct +/- (SED)

48.94 (9.89) 47.96 (11.67)

Second Year StudentsMean % correct +/- (SED)

61.69 (12.80) 59.63 (13.22)

Tracking progression

Second year students do progress in Information Literacy

0

10

20

30

40

50

60

70

80

90

100

Within Group Comparison (N=112) Betw een Group Comparison(N=805)

Mea

n %

Co

rrec

t

First Year

Second Year

Progression

Areas of significant progression

Search strategies

Locating and accessing information

Organising and applying information

Areas of less progression

Recognising an information need

Identifying ways to fill an information need

Comparing and evaluating information

IL progression

Second year students are:

Better at keyword searches

Better users of Boolean logic

Search Strategy

0

10

20

30

40

50

60

70

80

90

100

First Year Second Year

Students

Mea

n %

of

Stu

den

ts A

nsw

erin

g

Co

rrec

tly

IL progression

Second year students are far more competent at referencing

Neither group have sufficient understanding of copyright

Organise and Apply

0

10

20

30

40

50

60

70

80

90

100

First Year Second Year

Students

Mea

n %

of

Stu

den

ts A

nsw

erin

g

Co

rrec

tly

IL progression

Both groups fail to use search facilities effectively

More second year students understand the usefulness of a bibliography in a printed book

Locate And Access

0

10

20

30

40

50

60

70

80

90

100

First Year Second Year

Students

Mea

n %

of

Stu

den

ts A

nsw

erin

g

Co

rrec

tly

Less progression

Good performance by both groups: Very good at acknowledging gaps in their information

Appropriate strategies for dealing with information overload

Room for improvement:

Lack skepticism

First year students use of online encyclopedias

Recognise Need

0

10

20

30

40

50

60

70

80

90

100

First Year Second Year

Students

Mea

n %

of

Stu

den

ts A

nsw

erin

g

Co

rrec

tly

Less progression

Good performance in both groups:

Understand purpose of journal article abstracts

Over ½ correctly identified differences between websites and peer reviewed journal articles

Understand peer review process

Compare and Evaluate

0

10

20

30

40

50

60

70

80

90

100

First Year Second Year

Students

Mea

n %

of

Stu

den

ts A

nsw

erin

g

Co

rrec

tly

Less progression

Poor performance by both groups:

First year students source most information from internet search engines

Second year students are more discerning

Second year students look to the internet for recent academic information

Review articles: both groups unaware of appropriate use

Identify Ways to Fill Need

0

10

20

30

40

50

60

70

80

90

100

First Year Second Year

Students

Academic performance

First year StudentsIn general, Information Literacy ability of students entering Higher Education is strongly related to their academic performance in year one semester one at University

Mean Academic Performance:

(r =0.349, n =130, p<0.001) HIGHLY SIGNIFICANT

Multiple choice examination performance:

(r = 0.326, n=130, P<0.001) HIGHLY SIGNIFICANT

Practical report writing performance:

(r = 0.208, n = 130, p<0.05) SIGNIFICANT

Essay writing performance:

(r = 0.152, n = 130, P=0.084) NOT SIGNIFICANT

Academic performance

Second year studentsAfter some initial Information Literacy training in Higher Education the strong relationship between IL ability and academic performance is enhanced

Mean Academic Performance:

(r =0.355, n =118, p<0.001) HIGHLY SIGNIFICANT

Multiple choice examination performance:

(r = 0.290, n=118, P=0.001) HIGHLY SIGNIFICANT

Practical report writing performance:

(r = 0.269, n = 118, p=0.03) SIGNIFICANT

Essay writing performance:

(r = 0.181, n = 118, P<0.05) SIGNIFICANT

Summary

Reliability

Data produced by the questionnaire is reliable across different cohorts of students and the LILAC 2005 data suggesting a progression of IL ability from year 1 to year 2 has been replicated

Progression

The Questionnaire now provides detailed information about student abilities including areas of substantial or no progression

Academic performance

Information Literacy is strongly related to academic performance

Newton-Harrison 7 pillars of evidence based IL training

Pre-training

Re

co

gn

ise

info

n

eed

of

you

r s

tud

en

ts

Iden

tify

way

s t

o f

ill

the

ne

eds

of

yo

ur

stu

de

nts

Co

ns

tru

ct a

tra

inin

g p

rog

ram

me

De

live

ry o

f tr

ain

ing

Co

mp

are

& e

va

lua

tes

tud

en

t ab

iliti

es

an

d p

rog

res

sio

n

An

aly

se

stu

den

t p

erf

orm

an

ce d

ata

Sy

nth

es

ise

da

ta &

cre

ate

e

vid

en

ce

-bas

ed

tra

inin

g

Post-training Pre-training

Future plans

3rd year student data to be obtained

Re-examine IL in the Psychology curriculum

Further investigation into academic performance:

A-level subjects

Dissertation results (3rd year students)

Final degree results