I-ACTInstitute for Applied Creative ThinkingStaffordshire University

Dr Geoff WaltonSenior Researcher

Dr Mark HepworthSenior Lecturer

Department of Information ScienceLoughborough University

Fostering higher order cognition to enhance undergraduates’ information literacy

PLAN

Introduction - Geoff’s motivation for doing PhD Research project

Intention to build a picture to illustrate what students did during an information literacy learning interventions in an attempt to reveal what their thoughts and feelings were for each stage

Implications for models of information behaviour

Implications for models of information literacy New assessment tool for measuring levels of

information discernment Concluding remarks – A theory and formulae for

further research?

INTRODUCTION

Geoff’s original motivation to prove the ‘correctness’ of information literacy as a model and ‘expose’ VLEs as the ‘emperor’s new clothes’

Having completed PhD his view has changed somewhat!

Geoff’s own personal PhD journey ‘proof’’ and ‘correctness’ are problematic IL models don’t seem to be sensitive to the

cognitive and affective processes that impinge on becoming information literate

VLEs, or more generally e-learning, offer very promising pedagogical opportunities

RESEARCH PROJECT – RESEARCH QUESTION AND HYPOTHESIS

How do the psychological states associated with information behaviour and thinking help explain the learning processes in an information literacy blended learning and teaching intervention?

Students who participate in online social network learning (OSNL) will demonstrate:(1) a greater degree of knowledge about e-resources(2) a greater ability to evaluate information

than those who do not receive this intervention

RESEARCH PROJECT - METHODOLOGY

ParticipantsSport & Exercise first year undergraduates in first year core module Effective Learning, Information and Communication Skills (ELICS)Three tutor groups who received the following delivery:A. Face-to-face workshop plus interactive web

tools plus online social network learning (n=12)B. Face-to-face workshop plus interactive web

tools (n=11)C. Face-to-face workshop only (n=12)

RESEARCH PROJECT – DATA GATHERING INSTRUMENTS

Focus-group interviews (Group A)

Questionnaire (Group A)Post-diagnostic test (All groups)Assessment - portfolio (All groups)Annotated bibliography Reflective practice statementsReflective essay

RESEARCH PROJECT – CODING FRAMEWORK FOR QUALITATIVE DATA

Codes based on:Bloom’s taxonomy Hepworth’s information

behaviour model Moseley et al

RESEARCH PROJECT – FACE-TO-FACE WORKSHOP

Overall context – Roles and normsstudents studying at university and

bring some prior knowledge with themTask – Problem-based scenario part of portfolio assignmentExamples in the one hour ‘hands-on’

activity-based workshop mirrored the problem-based scenario for the assessment (see p1 of handout)

RESEARCH PROJECT – FACE-TO-FACE WORKSHOP

Behaviour Interaction with source

Source character “looking at the library catalogue”, “search the library”

Source behaviour“you can look for a certain subject [on the catalogue]”

“you [the tutor] showed us what to do”

Comprehension Application

metacognition

Sometimes you don’t

know the book exists,

so you put in a litt

le

bit of w

hat your are

looking for a

nd you

get the m

atches

The keyword we did the searches for

I have realised that these sources contain information that can help me to develop while studying at university

Affective state Style state

You can get involved in the sessions yourselves more hands-on, it was active

Being able to find out the books was really interesting

FACE-TO-FACE WORKSHOPKnowledge

Factual knowledge“it [the library catalogue] allowed me to see what books were available and where I could find them”

Process knowledge “you don’t necessarily need to know what book you are looking for [on the catalogue], you can look for a certain subject”

New behaviour “I [now] use the online library to search for e-

books and books”

ONLINE SOCIAL NETWORK LEARNING

Task - to find out how to evaluate web-pages by discussing the issue online Used a questioning approach for example,

So, how would you decide what makes a good quality web page?

Structure of process (see p2-3) Behaviour

Interaction with sources peers, tutors, Berkeley website on evaluating

information, the Internet Detective, and the tutor-summary for example, “we had to feedback on each others. I remember I was commenting on his [posting], he was commenting on mine”

See examples of a postings on p3-4

Questioning Analysis

metacognitionAffective state

“check the author or who it

was created by” , “evaluate

the web page by looking at

other pages with the same

topic and compare”, “look at

the last time when the web

page was updated”

Style state

[The tutor summary] gave the whole group a bit of recognition [...] you read through what other people thought of URLs and took advice from other people not just the lecturer’s, it worked really well, it is a good way of reflecting what you’ve done

I think it w

as quite

interesting gets you

involved as well and

that side of it was

quite fun - as

opposed to being

lectured to”

Allowed me to

see what other

people thought o

f the web

site, the way th

ey had

evaluated it, not ju

st myself.

[...] Y

ou got to see what you

was (sic) m

issing out or

something you hadn’t looked

at, so th

ey could bring up th

e

points saying may be look at

at this.” somebody commenting on your evaluation could highlight things you’d overlooked [...] you always think your own work is perfect, sometimes it’s a bit of an eye opener when somebody says you should have done this, gets you thinking about it

ONLINE SOCIAL NETWORK LEARNING New knowledge

Both in terms of postings to the discussion and the tutor summary and final summary handout – synthesised output from discussion (see p3-6)

Transition to a feeling of less uncertainty “It makes you aware, a little bit more aware of what web

sites are more useful to you than others and there are quite a lot of web sites on line and you don’t want to be writing stuff in your assignments that’s not true. [Before] I didn’t know what the things at the end like .ac. org meant […].”

(Changed) Behaviour “I have used [the evaluation criteria] actually, since we did

it for essays and stuff, since we did these things in Effective learning it actually alerted me to what to look for when looking for a good web site and what to steer clear of.”

COMMERCIAL BREAK All and more in the

book! Practical examples of

learning and teaching interventions underpinned by theory

Face-to-face learning Online learning Based on empirical

research Also, Journal of

Documentation article in 2011

ASSESSMENT

Task Answer the problem-based scenario see p1

Behaviour Found six resources, two books, two journal

articles and two web pages, evaluated them and then wrote about it

Written reflective statement

Application Evaluation

Analysis

I know about scope, audience,

timeliness, scholarly vs. popular,

authority documentation and

objectivity. I have learnt how to

judge how good a book or a

journal is against this criterion.

Evidence via assessed work transcripts see examples on p7k

SynthesisI would evaluate a web

page by; looking at the

ULR address

It [URL] informs

you that it is from

a popular and

reliable source,

the BBC

ASSESSMENT (REFLECTIVE STATEMENT)

New knowledge “I have acquired new knowledge on the Library

Catalogue, Swetswise, e-journals and the Web. I now know how to look for E-journals and E-books on Swetswise and E-brary, something I did not know how to do before.”

Task completion See assessed work transcript on p7

Changed behaviour “I will incorporate this new knowledge and skills

in the future by using these skills when completing a new task e.g. I can use the online library to search for e-books and books.“

ASSESSMENT (REFLECTIVE STATEMENT)

Richness of written language used to express reflections on evaluating information very different between groups

Group A Group B Group C Relevant Relevance Recent Judge Criterion Scope Audience Timeliness Scholarly Popular Authority Documentation Objectivity Published date Reliable (3) Evaluate Evaluating Critically evaluate Deciding Valid (2) Reputable

Reliable Trustworthy (2) Topic related Up-to-date Unreliable Relevant Evaluate

Suitable Breadth Intended audience Published Trustworthy Relevant Evaluating

OTHER EVIDENCE (QUANTITATIVE DATA) Assessment (annotated bibliography)

One-way ANOVA statistical test Variety of evaluation criteria

Experimental group used greater variety of evaluation criteria than either groups B or C, but not statistically significant

However, large effect size (using Eta squared test) found – therefore, if this part of the study was carried out again with nineteen subjects per condition it would have produced statistically significant results

Frequency of evaluation criteria Again, similar result to variety measure However, large effect size - to gain a statistically

significant result in a future study would require between twenty and twenty five subjects per condition.

OTHER EVIDENCE (QUANTITATIVE DATA)

Post-delivery diagnostic test – 14 multi-choice questions on the library catalogue, e-journals, referencing and evaluating web pages (analysed using one way ANOVA statistical test) Test scores between Group A (experimental group) and

Group C (control group) are significantly different at p < .025, t= 2.66 , Degrees of Freedom (df)= 22 (within groups).

IMPLICATIONS FOR MODELS OF INFORMATION BEHAVIOUR a number of cognitive states (knowledge and application)

were evident at the face-to-face stage; the higher cognitive states of analysis, synthesis and

evaluation appeared to be most evident for those students who experienced the OSML process;

Three stages in behaviour transition – existing, new and changed

Affective and style states also appeared to be critical factor and the importance of task in maintaining a positive affective and style state leading to new knowledge;

Task completion via the assessed work indicated: changed behaviour and implied that assessment is an

essential component in facilitating deep learning and enabling students to become information literate

the complex inter-relation of these factors in a learning and teaching intervention.

HEPWORTH’S (2004) IB MODEL

Knowledge state

Cognitive state

Style state

Affective state

Source behaviour

Source Character

1 2

5

4

3

Sociological data

Psychological data

Behavioural data

Source data

Roles

Norms

Tasks

Psyche

Behaviour

Sources

Iterative interaction

Key: 1= associated with 2= associated with 3= interaction with4= has impact on 5= may resolve situation and help complete task(s)  Hepworth (2004, p705)

Key: 1= associated with 2= associated with 3= interaction with4= has impact on 5= may resolve situation and help complete task(s)  Hepworth (2004, p705)

IMPLICATIONS FOR MODELS OF INFORMATION BEHAVIOUR (NEW MODEL)

Walton, G & Hepworth, M. (2011, p470)

IMPLICATIONS FOR MODELS OF INFORMATION LITERACY

They are too rigid and over simplified The step-by-step approach indicated by some

models doesn’t necessarily reflect how the person becomes information literate

They don’t seem to reflect (or harness) the social nature of learning

SCONUL Seven Pillars’ (1999) did not recognise that potentially any student can synthesise information and produce new knowledge, this changed in redesigned model (2011)

Don’t take into account the affective dimension ‘Grand narratives’ which aren’t sensitive to

context

CONCLUDING REMARKS – A THEORY (SEE P9)

Becoming information literate appears to be about an individual completing a task in a given context. This context leads to the interaction with sources (e.g., databases, e-journals, books, e-books, peer and tutors etc) and in so doing brings about the interplay of an individual’s behavioural, cognitive, metacognitive and affective states. It is this interplay which determines the level of new knowledge learnt (or produced or both) and the degree of changed behaviour (i.e., level of information literacy) exhibited.

THREE SPHERES OF INFORMATION LITERACY

Spheres can occur in no particular order

Find/ access/ locate

Evaluate/ discern

Use/ communicate/ produce

Each sphere triggers its own set of behavioural, cognitive, metacognitive and affective states

Becoming information literate takes place in a wider social context determined by roles, norms and tasks

MATHEMATICAL FORMULAE TO FOCUS FURTHER RESEARCH?

HEREKey As = Affective stateB = BehaviourBc = Changed behaviourCana = Cognitive state, analysisCapp = Cognitive state, applicationCcomp = Cognitive state, comprehensionCeval = Cognitive state, evaluationCq = Cognitive questioning stateCsyn = Cognitive state, synthesisKx = Knowledge stateSb = Source behaviourSc = Source characterΣK = Final knowledge stateMcog = Metacognitive stateSs = Style state

REFERENCES Bloom, B. S., Engelhart, D., Furst, E. J., Krathwohl, D. A. and Hill, W.

H. (1956). Taxonomy of educational objectives: the classification of educational goals: handbook 1: cognitive domain. New York: David McKay Company Inc.

Hepworth, M. (2004). A framework for understanding user requirements for an information service: defining the needs of informal carers. Journal of the American Society of Information Science and Technology, 55 (8), pp695-708.

Hepworth, M. & Walton, G. (2009). Teaching information literacy for inquiry-based learning. Oxford: Chandos.

Moseley, D. Baumfield, V., Higgins, S., Lin, M., Newton, D., Robson, S., Elliot, J. and Gregson, M. (2004). Thinking skills frameworks for post-16 learners: an evaluation. a research report for the Learning & Skills Research Centre. Trowbridge: Cromwell Press.

Walton, G. & Hepworth, M. (2011). A longitudinal study of changes in learners’ cognitive states during and following an information literacy teaching intervention.  Journal of Documentation, 67(3), 449-479.

SELECTED PUBLICATIONS Walton, G. (2010). From online discourse to online social

networking, the e-learning Holy Grail?. In Parkes, D. and Walton, G. (eds.). Web 2.0 and libraries: impacts, technologies and trends. Oxford: Chandos, pp33-65.

Walton, G., Barker, J, Hepworth, M. and Stephens, D. (2007a). Using online collaborative learning to enhance information literacy delivery in a Level 1 module: an evaluation, Journal of Information Literacy, 1 (1), pp13-30. [Online] http://jil.lboro.ac.uk/ojs/index.php/JIL/article/view/RA-V1-I1-2007-2/3 (accessed 17 June 2011).

Walton, G., and Barker, J., Hepworth, M. and Stephens, D. (2007b). Facilitating information literacy teaching and learning in a level 1 sport and exercise module by means of collaborative online and reflective learning. In Andretta, S. (Ed.) Change and Challenge: Information Literacy for the 21st Century. Adelaide: Auslib Press, pp169-202.

I-ACTInstitute for Applied Creative ThinkingStaffordshire University

Dr Geoff WaltonSenior Researcher

Dr Mark HepworthSenior Lecturer

Department of Information ScienceLoughborough University