Fit for purpose: diversifying assessment to meet

the needs of a 21st century university education & 21st

century studentsProf. Sue BloxhamS.bloxham@cumbria.ac.uk

Workshop outcomesDevelop participants skills and knowledge to:-enhance the alignment between what you intend students to learn from your courses and the assessment methods you use to test that learning

-diversify assessment methods to: • engage students• involve students in their own assessment and learning• Promote assessment for learning and employability• Provide formative feedback

reasons to diversify assessment methods

• Meeting the different purposes of assessment

• Valid assessment – alignment with learning outcomes/ aims

• Tasks that assess a 21st Century curriculum• Student involvement in assessment• Inclusivity• Student perceptions and motivation• Practicability / marking load

Certification

to identify and discriminate between different levels of achievement, and between students;

providing a license to practice in the case of professional programmes;

enabling selection of students for further study and employment.

This is assessment of learning.

Quality assurance

to provide evidence for relevant stakeholders (for example, external examiners, QAA, professional bodies);

to enable them to judge the appropriateness of standards on the programme.

This is assessment of learning.

Student Learning

to promote effective learning;

formative and diagnostic;

steering students’ approach to studying;

giving the tutor useful information to inform changes in teaching strategies.

This is assessment for learning.

Lifelong learning: sustainable assessment

This is assessment as learning.

to achieve an understanding of standards;

to learn how to make judgments;

to be able to use criteria;

to be able to tell when you really understand something.

The unbalanced purposes of assessment

Certification & QAAssessment for and as learning

Assessment for certification & QA involves:

Formal feedback;2nd marking;Moderation;External examiners;Assessment boards.

Slow, costly & high stakes

Assessment for and as learning can:

Be immediate;Regular;In-class;On-line;Involve students in their

own assessment ;Uses peer assessment.

Quick, cheap & low stakes

Constructive Alignment

Teaching/learning activities

Designed to achieve intended learning outcomes

May be:teacher-controlledpeer-controlledself-controlledas best suits context

Learning outcomes ------------Expressed using verbs that the students have to enact

Verbs chosen to reflect level of achievement(for example lower level learning signalled by verbs such as describe, identify; higher level signalled by verbs such as evaluate, reflect, analyse)

Assessment of learning

Assessment tasks

designed to evaluate how well outcomes are demonstrated

From Biggs,J & Tang, C (2011)

Valid assessment – (QAA)

• the range and types of assessments used measure appropriately students‘ achievement of the knowledge, skills and understanding identified as intended learning outcomes. It is important that each assessment enables students to demonstrate the extent to which they meet the intended learning outcomes in respect of both the subject and any generic skills.*

• Is your assessment FIT FOR PURPOSE?

* QAA Quality Code (2012)

Assessing today?

Develop participants skills and knowledge to:-enhance the alignment between what you intend students to learn from their courses and the assessment methods you use to test that learning

Intended Learning outcomesHow will we know if these intentions have been met?What tool or task could we use?

A 21st Century education?: The USEM account of employability

Skills including key skills

Personal qualities, including self-theories and efficacy beliefs Subject

under-standing

Meta-cognition

Employability, citizenship, life, etc

Yorke & Knight 2004

S

U

E

M

21st Century higher education• Skills for the ‘knowledge economy’*

– Critical thinking and problem-solving– Collaboration across networks and leading by

influence– Agility and adaptability– Initiative and entrepreneurialism– Effective oral and written communication– Accessing and analyzing information– Curiosity and imagination

Do these attributes figure in Programme/ unit outcomes and do assessment tasks foster and test these broader skills and capacities?

* ‘Must have’ skills for the future to tackle the ‘Global achievement gap’ (Wagner, 2008)

Characteristics of learning-oriented assessment

• Formative• Demands higher order learning • Learning and assessment are integrated• Students are involved in assessment• It promotes thinking about the learning process; • Assessment expectations should be made clear;• Involves active engagement of students, developing

independent learning;• Tasks should be authentic and involve choice ; • Tasks align with important learning outcomes• Assessment should be used to evaluate teaching.

Exams

• They come at the end of a course

• They are not integrated into the learning,

• The criteria are oblique, • They rarely result in useful feedback,

• When students do badly we tend to blame the students rather than use the results to diagnose problems with our teaching.

Essays

• Often no formative element• Questions may ask students to ‘evaluate’ or

‘critically assess’ a topic but if students can pass adequately by regurgitating others’ evaluation or criticism (from lectures or reading), they may avoid higher order learning.

• Students not involved in assessment or thinking about learning

• Rarely authentic• Criteria often oblique

have the potential to meet many characteristics but:

Task

Likely to promote learning & employability

Less likely to promote learning and employability

Well aligned with the learning outcomes (or aims) of the course

Not very well aligned with the learning outcomes of the course

In small groups, discuss the assessment methods in the module outlines that you brought along and try to allocate them to a box on this grid.

Teacher education: embedding formative assessment in a portfolio

• First year students, assessment 4000 word portfolio;• Professional Development Activity (PDA) after each taught session;• the PDAs were used in various ways:

– peer reviewing; – collating – applying research to a case or problem– sharing of work.

• Summative assignment 1500 words• eight PDAs as appendices referenced in the text. • Favourable student evaluation• Higher marks, better engagment

Georgia Prescott, Cumbria

Psychology Redesign

• 560 students in groups of 6-7;• 3 week cycle culminating in 700-800 word essay

e.g.’Assess the strengths and weaknesses of Freud’s and Eysenck’s theories of personality. Are the theories incompatible?

• Guidance provided for tackling the question and working in a group;

• Best definitions & essays posted on VLE as feedback;• Students used familiar language to discuss academic

concepts – Dialogue and explanation.

20

Nicol 2009

Peer assessment in lab reports

• Formative• Students develop understanding of quality in analysing and

reporting science• Higher order learning, focus on the science rather than a

description of the process• Students involved in assessment, developing skills of

evaluation

Field-based enquiry: childhood studies

1. Introduced to lecturer’s research

2. Exploring signs of childhood

3. Photos posted on line – annotated by self and others

4. 2 visits to museum, gallery, library to collect other images

5. Round table conference facilitated by more experienced students

6. Submit summative research report

Sambell, (2010).

Some other assessment methods

• Writing tasks: newspaper articles, press releases, executive summaries, information sheets.

• Video about a specific topic – Youtube?• Research Grant applications• Lay commentary on specialist material, e.g. journal article• Poster – presenting information clearly & concisely • Presentation – oral communication• Problems and case study analysis• Reflective Journals, Diaries & learning logs• Wikipedia entry

Opportunities and threats

Task:On your tables, have a discussion about the ideas and examples so far.

Make two lists:Ideas I can take away

--------

Barriers to changing my assessment

-------

Engaging students in formative assessment – key requirements

• It clearly feeds into summative assessment tasks;

• The students must submit it in some way (bring to class, post on line, hand it in) and action is taken if they don’t;

• Students receive useful feedback on it;

• It is not contaminated by summative purposes.

Selling peer assessmentEvidence shows students find their peers a useful and more approachable source of help with assignments but we need to stress the main value in peer assessment is standing in the shoes of the assessor – not being assessed – because:•learning about standards – absolutely crucial to making progress and understanding feedback•Seeing other ways of going about the task – develops strategies for taking action•Key employability skill – being able to judge own performance and assessing and giving feedback to others•More opportunity for dialogue•Chance for more formative feedbackPeer assessment needs to become a regular feature of programmes so that it is

taken seriously and taken for granted as part of learning at this level.

Staff work loadA heavy staff workload in assessment is not necessarily helpful to student learning. Improving assessment needs to be accompanied by a shift in effort from tutors to students:•more student activity and engagement (doing tasks, reviewing progress, self-regulation) and less staff activity (marking)•Formative assessment has the potential to achieve both of these and is most likely to increase achievement.

Tutor friendly formative feedback

• Tutor posts good examples or model answers on VLE;• Students peer assess tasks using assessment criteria;• Tutors give feedback on posters, presentations in class;• Oral/ on-line feedback to group after marking a sample• Tasks done on-line (e.g EMQs, MCQs), auto marked and

give immediate feedback;• Tutors put main effort into marking drafts (agreed with

examiner), just checking for change and putting mark on final piece.

2929

Tutor leads discussion of exemplars previously marked and annotated with feedback

Students write and submit individual assignm-ents

Tutor assesses assignm-ents and prepares feedback

Tutor hands back assignments and leads discussion on feedback

Out

of

clas

s ac

tivity

In-c

lass

act

ivity

Submission pointModule timeline

Example of module-level approaches:the use of exemplars annotated with feedback toencourage dialogue about assessment criteria

From Oxford Brookes FDTL project on feedback

3030

3. In-class discussion of generic feedback

4. Students rewrite and submit assignments with reflective commentary on how they have incorporated the generic feedback

5. Tutor assesses assignm-ents

6. Tutor hands back assignments with minimal formative feedback

Out

of

clas

s ac

tivity

In-c

lass

act

ivity

Submission pointModule timeline

Example of module-level approaches:Generic (non-personalised) feedback on draftsplus reflective commentary

1. Students draft & submit individual assignm-ents

2. Tutor marks sample of assignments and prepares generic feedback

From Oxford Brookes FDTL project on feedback

Changing student learning through assessment design

• Higher order skills may be encouraged by asking students to act on something which requires them to apply, analyse, and evaluate. – E.g. Analysis of a current news item, a piece of

scientific research, a lesson, an ethical dilemma. • With such tasks, they can’t regurgitate – they have to use

higher order skills because the answers are not in the public domain (plagiarism?)

• The level of guidance offered, and the level of complexity demanded, should change as they move up the levels.

• Such assignments are more likely to address a range of learning outcomes rather than one or two specific topic areas.

19/04/23

Caution

• Multiple methods have been associated with negative learning outcomes for students

• You may make malpractice easier • Extra workload for students

Benefits to students of moving to in-class, on-line, ongoing assessment and feedback

• Immediate feedback• More feedback• Assessment & teaching/learning are

integrated• Students involved in assessment – gaining

better understanding of standards and own performance

• Potential for greater student engagement throughout modules

• More independent study• integration with work

experience• Raise expectations regarding

study workload

Quicker, cheaper and low stakes

TaskWorking in groups•Briefly discuss the module outlines that you have brought along and select one to work on as a group.•Design a new assessment plan for the module.

•Particularly think about assessment tasks which might encompass several of the outcomes and how formative work can be embedded

in the process.

Be prepared to present your assessment plan to the group

What are the unanswered questions?

Conclusion

Diversifying assessment can help:-Balance the different purposes of assessment;-Make assessment more valid for a 21st century curriculum;-Encourage and reward student engagement;-Ensure that students’ study efforts are directed towards meaningful learning;-Use tutor time most effectively for learning and limit the time absorbed by marking and QA to essential summative items;

ReferencesBloxham, S (2014) Assessing assessment (case study by Georgia Prescott), in H. Fry et al (Eds) A

handbook for teaching and learning in higher education (4th Edn) London: Routledge.Nicol, D (2009) Assessment for learner self-regulation: enhancing achievement in the first year

using learning technologies Assessment & Evaluation in Higher Education, Vol. 34, No. 3, June 2009, 335–352

Sambell, K (2010) Enquiry-based learning and formative assessment environments: student perspectives, Practitioner Research in Higher Education, Vol 4, No 1 (2010), p52-61

Wagner, T. (2008) The Global Achievement Gap: Why Even Our Best Schools Don’t Teach The New Survival Skills Our Children Need—And What We Can We Do About It Basic Books.

Yorke, M. & Knight, P. (2004) Embedding employability into the curriculum. Higher Education Academy

General texts on assessment design• Bloxham, S & Boyd, P (2007) Developing assessment in Higher Education: a practical guide,

Maidenhead, Open University Press (course reader).. • Bryan, C. & Clegg, K (2006) Innovative assessment in Higher Education. London: Routledge.• Merry, S., Price, M., Carless, D & Tara, M. (2013) Reconceptualising Feedback in Higher

Education. London: Routledge.• Price, M., Rust, C., O’Donovan, B & Handley, K (2012) Asssessment Literacy: The foundation for

improving student learning. Oxford: ASKE.• Sambell, K., McDowell, L & Montgomery, C (2013) Assessment for Learning in Higher

Education, London: Routledge