View
214
Download
0
Category
Tags:
Preview:
Citation preview
Making assessment more efficient and effective
Ray Land, Discovery Point, Dundee, January 13th 2004
‘Streamlining’ assessment
Ashley Havinden – Design for Chrysler Motors Ltd, 1929 (detail)
• Principles and purposes of assessment (effective for what?)
• Effects of massification
• Control strategies
• Independence strategies
• Issues of risk and support –examples of adaptive initiatives
Ashley Havinden 1903-1973 Design for Chrysler Motors Ltd, 1929 (detail) Scottish National Gallery of Modern Art Archive
© The artist’s estate with thanks to DaimlerChrysler UK Limited
Construction of identities through
assessment
Paul Ramsden – ‘a serious and often tragic enterprise’
Sally Brown – ‘a nightmare’
David Boud - ‘more bad practice and ignorance of significant issues in the area of
assessment than in any other aspect of higher education’
Principles of Assessment
• Beneficial• Actively foster
learning• Fair• Diverse and varied• Valid• Reliable
• Transparent• Representative• Effective• Practicable/ cost
effective• Right of redress• Secure
Need to bear in mind that principles of assessment often contradict.
What is efficient may not be effective. What is effective may not be efficient.
What is assessment?
• Sampling evidence of performance
• Making inferences
• Estimating worth
• Representing that worth (symbolically)
I keep six honest serving-men(They taught me all I knew)Their names are What and Why and Whenand How and Where and Who
(Rudyard Kipling – Just-So Stories)
• What kinds of knowledge, skills, abilities or values are we seeking evidence of?
• Why are we seeking to gain evidence of achievement in the first place? For what purpose will this evidence be used?
• When should assessment take place – at the end of the course? at periodic intervals throughout it? continuously? at the start of the course?
• How should we gather the evidence? What is the most appropriate strategy to use for our particular course?
• Where should assessment take place – in the examination hall? during a group presentation? In the workplace? On a computer?
• Who should undertake the assessment – the teacher? Anonymous examiners? A workplace/clinical supervisor? The student? The student’s peers?
What’s assessment for?
• Diagnostic testing
• Motivating/regulating students (and staff)
• Developing student confidence
• Evaluation of teaching
• Certification /licensing
• Providing feedback to students
• Selection for subsequent courses
• Information for employment
• Predicting future performance
• Monitoring progress• Public accountability• Institutional marketing• Social stratification
Effects of massification • Over assessment• Lack of advice on improvement• Lack of knowledge of progress• Slow return of coursework• Same outcomes repeatedly assessed
(redundancy)• Credit / workload not calibrated • ‘Convergence’ of assessments leading
to stress points • Lack of variety/ repetition of method
across modules• Consumerism / commodification • ‘Performativity’
Over assessment an issue for both staff and students
Issues arising from widening participation
• Academic literacies debate / acculturation
• Traditional dominance of particular modes (eg essay)
• Pacing of learning / ‘Slow’ learning
• Assessment in the first term
• Sensitivity in feedback
IndependenceControl
(Gibbs and Jenkins, 1997)
Choice of strategies
‘control’ strategies
traditional methods retained but carried out with increasingly greater (industrial) efficiency cf ‘Performativity’ (Lyotard)
‘independence’ strategies
shift of responsibility for (usually formative) assessment to students themselves, on the grounds that, apart from economies of effort, this is in itself an educational benefit that is likely to enhance student performance
Change issues - Risk and support
• High Risk High Support
• High Risk Low Support
• Low Risk High Support
• Low Risk Low Support
control strategies
• Set fewer assignments• Set shorter assignments• Set fewer exam questions• MCQs• Use standardised feedback forms• CAA – eg computerised objective tests, word-
processed statement banks, optical mark readers• Set limits on assignment length and materials
costs; penalise those who go beyond the limit
control strategies (cont’d)
• Conduct more in-class assessments• On written assignments mark in pencil to make
alterations without time or trouble• Sort scripts into piles according to approximate standard
– helps maintain consistent standard• Scrap grades and percentages and use pass/fail instead.
Scrap degree classifications.• Refuse to accept work that is difficult to read or late
without reason. Stick to this policy• Use placement and clinical assessors – standardise
criteria and provide training
‘independence’ strategies
• ‘front-ending’ assessment• devise group tasks and assess the group – group
reach consensus on scores for individual members• Self assessment - require students to mark their
own work with comments, or submit a completed standardised feedback form with their work
• Peer assessment – ask students to use a marking scheme to mark each other’s work
What staff want are low risk, low support initiatives
1. Introductory Chemistry
‘Course puts multiple choice questions on the course’s website each week for the students to do in their own time. There are no marks for these tests, and no record is kept of who has taken them, but the students do the questions because they know from the beginning of the course that the end of module exam will include a section of MCQs and that half of these questions will have been selected from those questions used in the weekly tests’ (Rust 2003)
2. Anatomy
Students undertake regular MCQ tests but instead of simply marking their chosen response on the test sheet they also have to indicate a ‘confidence measure’ (CF), in order to minimise the element of guesswork common to many MCQs. If they indicate a high CF and get the correct answer they gain a high mark. If they indicate a high CF and get it wrong, they will be heavily penalised.
3. The Geography course• convert all examination questions from 30/40
minute to hour-long essays. • thus fewer questions, but not possible to get away
with only recall. • immediate response that quality of answers
improved and students did more reading. • number of essays also reduced throughout all
modules, with each essay requiring more work on the part of the student, and contributing more to the final grade.
• Class examinations removed from most modules and the number of exemptions given to first year students increased.
• Pass/fail examinations being used to a larger extent, and more planned.
• Project work also increased and now accounts for up to 45% of the module grade.
• prior discussion between staff and students as to what makes a good project, followed up with input in tutorials
• several options now incorporate oral assessment and are peer assessed, with half of the grade being awarded by the students.
• In feedback, some of the discussions revolve around why peer and tutor marks differ.
• Department encouraging the submission of all work in a word-processed format, though concerns have been voiced about the increasingly poor writing styles on exam scripts.
• The dissertation deadline has been changed to later in the degree programme, and the greater experience gained from options and the like has resulted in higher quality dissertations, which have been winning national prizes. (ASSHE Project 1997)
4. An Education Course
Original Course Staff Time Staff Time New Course
Lectures
(18 weeks x 2hrs)
Seminars
(18 wks x 9 groups of 8)
36 hrs
162hrs
36 hrs
36hrs
Workshops
(18 weeks x 2 hrs)
Autonomous seminars
(18 wks x 2 tutors touring 6 groups of 12)
Total teaching hours 198hrs 72hrs
Essays
(72 x 2 essays @ 20mins)
Examination
(72 x 2 questions @ 10 mins)
48hrs
24 hrs
6hrs
18hrs
36 hrs
MCQ tests
(6 x 1hrs administration)
Seminar presentations
(peer assessed,
1 hr/week collation)
Portfolio (72 @ 30 mins)
Total assessment hrs 72 hrs 60 hrs
5. Using ‘Discourse’ in History
A classroom communication system (CCS)
Discourse – what the student would see
What was one effect of World War II on America? Explain.
Women worked in the factories
Discourse – Tutor’s list of responses
This is a list with the student’s name beside it. It is possible for responses to be anonymous.
The responses can be readily kept as a record.
Responses can be kept for the tutor’s view or can be shared with all in the group.
What was one effect of World War II on America? Explain.
Wu, Ken
Instead of looking at domestic affairs, America realised that it needed to be on top of world happenings
Effective uses of CCS systems
• diagnostic/ formative potential• quick-fire tests of understanding• possibilities for peer judgement• promotion of discussion • useful research tool to create an instant
teaching resource• evaluation of content / delivery • makes lectures less boring!
Differences between ‘Discourse’ and previous generations
• used between people that are linked
• bi-directional
• web-based / doesn’t need special equipment
• questions can be asked ‘on the fly’
• open-ended questions can be asked
6. The Patchwork Text
A patchwork is not just a ‘collection’ but a ‘pattern’: in the end it does have a unity, albeit made up of separate components. The unity of the Patchwork Text has two dimensions. To begin with it is defined by academic staff, as they carefully derive a sequence of tasks from the course material. And finally it is, as it were, re-defined by individual students, who review (and perhaps edit) their separate pieces of work in order to write their final section as an interpretation of what this course materials ‘means’, to them, now…(Winter 2003)
References
Gibbs, G. & Jenkins, A. (1992 )Teaching Large Classes in Higher EducationRoutledgeFalmer, London. Lyotard, J. L. (1986) The Postmodern Condition: A Report on Knowledge, Manchester University Press, Manchester.
Rust, C. (2002) ‘The impact of assessment on stuydent learning: how can the research literature practically help to inform the development of departmental assesswmtn strategies and learner-centred assessment practices?’ Active Learning Vol3, 2 145-158 July
Winter, R. (2003) ‘Contextualizing the Patchwork Text: Addressing Problems of Coursework Assessment in Higher Education’, Innovations In Education and Teaching International, Vol 40 No 112-122
ray.land@coventry.ac.uk
Recommended