Download pdf - Self- and Peer-Assessment - UCL - London's Global … · using self- and peer-assessment in their learning and teaching activities, but who may feel they have little understanding

Teaching BioscienceEnhancing Learning SeriesEdited by Stephen Maw, Jackie Wilson & Heather Sears

Self- and Peer-AssessmentGuidance on Practice in the Biosciences

TheHigher

EducationAcademy

Centre forBioscience

Paul Orsmond

Teaching Bioscience Enhancing Learning is a series ofguides intended to be an accessible introduction togood learning and teaching practice within the contextof competing research and institutional pressures. Theaim of each publication is to provide a persuasiveoverview of the pedagogic reasons for adopting aparticular practice and support these reasons withsufficient practical guidance and information to turnideas into reality. The guides are structured around acommon format; Chapter 1 provides a generalintroduction to the topic, Chapter 2 advice on how toimplement the topic and Chapter 3 more in-depthinformation on the topic and the opportunity toinvestigate it further. In addition, each guide contains acollection of bioscience case studies highlighting howothers have introduced the topic into their teachingpractice. It is intended that the guide will be useful toacademics in their first year of lecturing, particularlythose who are studying for Postgraduate Certificates inLearning & Teaching in Higher Education, as well as tothose with many years of teaching experience.

First published in Great Britain in 2004by Centre for Bioscience, The Higher Education Academy, Leeds LS2 9JT.

ISBN 0 9548751 0 9

Copyright of this Guide resides with The Higher Education Academy of which the Centre for Bioscience is part. Thematerial in this Guide is subject to copyright protection and may be reproduced free of charge in any format or mediawithout requiring specific permission, subject to it being reproduced accurately and not being used in a derogatorymanner or in a misleading context. Where the material is being published or issued to others, the sources and copyrightstatus should be acknowledged. This permission does not extend to any material in the Guide that is identified as beingthe copyright of a third party. Copyright over this material sits with the original author.

Whilst every effort has been made to ensure that the information provided is accurate, it does not constitute legal orother professional advice. The Higher Education Academy does not accept any legal responsibility for any errors,omissions or misleading statements (caused by negligence or otherwise). The Higher Education Academy does notassume any legal liability or responsibility for the accuracy or completeness of any information contained in this Guide.

INTRODUCTION 4

CHAPTER 1 INVOLVING STUDENTS IN ASSESSMENT 6

CHAPTER 2 GETTING STARTED WITH SELF- AND PEER-ASSESSMENT 12

CHAPTER 3 GOING DEEPER 21

CLOSING THOUGHTS 26

BIOSCIENCE CASE STUDIES 27

CASE STUDY 1THE EFFECT OF MARKING CRITERIA AND EXEMPLARSON STUDENTS’ LEARNING DURING PEER- AND SELF-ASSESSMENTOF SCIENTIFIC POSTERS 28

CASE STUDY 2ON-LINE CALIBRATED PEER-ASSESSMENT —STUDENT LEARNING BY MARKING ASSIGNMENTS 31

CASE STUDY 3PEER-ASSESSMENT OF SCIENTIFIC POSTERS —THE LEAGUE FIXTURE APPROACH 33

CASE STUDY 4PEER-ASSESSMENT OF GROUP WORK IN A LARGE CLASS — 35DEVELOPMENT OF A STAFF AND STUDENT FRIENDLY SYSTEM

CASE STUDY 5PEER-ASSESSMENT OF PRACTICAL WRITE-UPSUSING AN EXPLICIT MARKING SCHEDULE 37

CASE STUDY 6WRITING AND REVIEWING AN ARTICLE FOR A SCIENTIFIC MAGAZINE — 41A PEER/SELF-ASSESSMENT EXERCISE

CASE STUDY 7PEER-ASSESSED PROBLEM-BASED CASE STUDIES 43

REFERENCES 45

TEACHING BIOSCIENCE ENHANCING LEARNING

3

CONTENTS

This guide is for those bioscientists teaching in higher education who are interested inusing self- and peer-assessment in their learning and teaching activities, but who mayfeel they have little understanding of how to go about doing so. Some of the reasons forchoosing self- and peer-assessment as opposed to other assessment methods areoutlined in Table 1 on page 5.

This book is written to a specific format, made up of three chapters:

Chapter 1 is an introductory chapter that discusses some concerns about the mostcommon assessment practices within higher education today. Through this under-standing, we can proceed towards illustrating how self- and peer-assessment can be apositive instrument for change. In order to be used effectively, it is helpful to understandthe theoretical learning framework which underpins these student-centred assessmentpractices. Chapter 1 concludes with a discussion on effective learning through self- andpeer-assessment. This chapter is very much about providing a rationale for the need forchange and provides a way in which meaningful change can be brought about.

Chapter 2 considers in detail how you can get started with self- and peer-assessment.Specific issues related to the effective design, implementation and evaluation of self- andpeer-assessment, such as the central role played by students engaging with markingcriteria are discussed in detail. Here we take a deeper look at those requirements, fromdeveloping marking criteria to the development of a community of practice, providingpractical suggestions in undertaking self- and peer-assessment, whilst illustrating somerequirements for good practice.

Chapter 3 reflects the world in which students are being prepared. This is a world whichrequires the achievement of complex learning outcomes in order to meet the demands ofemployment and engagement in lifelong learning. Such learning outcomes are part-icularly well served by the application of self- and peer-assessment. The Chapter thenmoves from learning environments that use self- and peer-assessment, such as problem-based and reflective learning, to consider the role of formative and summative assessmentand finally to considering how students and tutors perceive each assessment source.

The structure of the book is therefore to look at the rationale for using self- and peer-assessment, to outline effective application and finally to see how effective a learning toolboth self- and peer-assessment can be.

In order to illuminate the role of self- and peer-assessment in practice, this book alsocontains seven bioscience case studies. Expanded versions of these and other casestudies, marking criteria, and video streams of peer-assessment in action, are availablefrom the website supporting this guide (http://www.bioscience.heacademy.ac.uk/TeachingGuides/). In addition, the chapters draw on a number of specific examples frompublished research of practice, each chosen because they illustrate a particular aspect ofthe assessment process well. Whilst some of these examples are from bioscience, anumber are not, but in these cases the examples can be readily transferred into abiosciences setting.

SELF- AND PEER-ASSESSMENT PAUL ORSMOND

4

INTRODUCTION

http://www.bioscience.heacademy.ac.uk/TeachingGuides/


5

TABLE 1. A COMPARISON BETWEEN SELF- AND PEER-ASSESSMENTAND OTHER ASSESSMENTS

Self- and Peer-Assessment Other Assessments

Student-centred. Students often excluded.

Clear transparent criteria.Norm referenced assessment. Or ifcriteria used, these may be given tostudents without discussion.

Student empowered. There is a strongsense of personal ownership.

Students isolated from the assessmentand therefore from the learningprocess.

Likely to encourage a deep approach tolearning.

Likely to foster a surface approachto learning.

Allows students to actively constructtheir learning.

Does not provide the incentives toconstruct own learning.

Encourages discussion betweenstudents and tutors. Little discussion, sometimes none.

Formative feedback.Feedback misunderstandings due tolapse of time or loss of ongoing comm-unication between student and tutor.

Opportunity to revise or review weakareas of learning.

Results final, with little point in goingback over boxes ‘ticked’.

More trial and less error in studentlearning.

Results received too late in the methodto revisit or be useful in learningprocess. Little trial anda lot of error in learning.

Prepares students for the lifelongongoing journey of learning.

Often end-point destination onlylearning.

For peer-assessment often severalassessors.

One assessor and a moderator or atmost two assessors.

Provides good opportunities forformative assessment. Little formative assessment.

Likely to increases student’sconfidence.

Limited or negative effect onconfidence.

Increased performance/learning qualityof the learning output. –

Often authentic learning tasks. Rarely authentic learning tasks.


6

1I INVOLVING STUDENTS IN ASSESSMENT

The reason why it is desirable and infinitely sensible to have students involved and central to theassessment process is well illustrated by Boud and Falchikov (1989), ‘teachers have limited access tothe knowledge of their students and in many ways students have greater insights into their ownachievements’. The fact that this is not normally recognised in higher education is a serious concern,as reflected by Boud (1995) ‘there is probably more bad practice and ignorance of significant issues inthe area of assessment than in any other aspect of higher education. Assessment acts as a mechanismto control students that is far more pervasive and insidious than most staff would be prepared toacknowledge’. This is unfortunate, as assessment is a foundation of student achievement andtherefore regarded as a measure of institutional success. Why, if assessment is so important toundergraduate learning experiences, should bad practice exist?

There may be a number of reasons for a lack of student involvement. Increasingly in highereducation, there is greater cross-disciplinary teaching taking place. Courses like forensic scienceinvolve tutors from different departments such as biology, chemistry and law, and each departmentmay have their ‘own’ understanding of assessment within their own structure of assessment policies.Institutions may also be presenting tutors with too many assessment options without looking into orunderstanding them fully. In a recent publication, Knight (2001) presented fifty assessment techniques.For some, this diversity may be heaven sent, but for others it could be thoroughly overwhelming; andwithout guidance, many excellent tutors can be left not knowing where to begin.

Because of this mix of practice, assessment processes in higher education generate a mixtureof concerns, such as:

Criteria concerns

• ‘Norm’ referenced marking; grading students according to how they compare against eachother as a class. Norm referencing may still be the ‘naturally’ preferred model of assessmentby most markers, Rust et al. (2003).

• Criteria referenced marking, where grading is expressed according to each student’sperformance, may have criteria and individual weightings that are often unclear and notconstructed with the involvement of students.

Assessment deficiencies

• Learners ill-informed about what they need to know in order to understand or do. Interestingly,Gabb (1981) reported that the only piece of assessment information given to a cohort of studentspreparing to undertake final year projects was the name of the assessor. In response to thislimited information, students deduced and developed their own sets of assessment rules, bywhich they tried to work out how best to pass the assessment.

• The development of a truly hidden and non-transparent curriculum, described by Sambell andMcDowell (1998) as ‘the shadowy, ill-defined and amorphous nature of that which is implicit andembedded in educational experiences in contrast with formal statements about curricula andthe surface features of educational interaction.’

A CONSIDERATION OF CURRENT ASSESSMENT PRACTICE IN HIGHER EDUCATION

Tutor folklore

• Community discussions between academics ina field developed through years of experience,concerning assessing and teaching (Edwardsand Knight, 1995).

Feedback concerns

• Feedback can be given too late to be of benefit.

• Feedback can be diminished in usefulnessbecause students do not understand it orperceive its importance (Chanock, 2000).

Traditionally, so-called ‘summative’ assess-ment, (for example, end of module examinations), hasbeen used to determine how much ‘learning’ hastaken place. Used here, summative refers to an end-point mark, which influences student progression andmay contribute towards their degree classification.Failing an assessment may mean students do notprogress, yet passing does not always indicatemeaningful learning, as demonstrated by thesestudent interview quotes from Brown et al. (1998);‘you shallow learn for an exam but you don’t know thestuff. It’s poor learning which you quickly forget’, and‘you think just let me remember this for the next hourand a half. Then you don’t care’. These studentsappear to see learning as an end product of assess-ment and view the learning quantitatively, whichmeans that to be a good learner is to know more. Thestudent learning which higher education needs toencourage is qualitative learning, where new materialcan be interpreted and incorporated, so that under-standing is progressively changed through an on-going, updating process (Biggs, 1996).

Underpinning many existing assessment pro-cesses is the issue of ownership and hence power.When referring to the goal of education, Rogers (2003)made the distinction between authoritarian ordemocratic philosophies. Heron (1992) distinguishedauthority in education as being either benign, lumin-ous and truly educative, or punitive, indoctrinatingand intimidating. It is the latter which formed thebasis for his authoritarian model (so called because ofthe unilateral control of assessment by staff). ForHeron (1988), power lay with those who makedecisions about other people. Students are consider-ed rationally competent to grasp a major discipline,but perversely are not considered competent toengage with the educational decision-making, where-by this grasp may be fully achieved. If, as Heronbelieved, the objective of the process of education isthe emergence of a self-determining person, i.e.someone who can set their own learning objectives

(outcomes), devise a rational programme to attainthem, set criteria of excellence by which work isassessed and assess their own work, then the‘unilateral control and assessment of students bystaff means that the process of education is at oddswith the objective of that process’ (Heron, 1988).

A NEED FOR CHANGE IN ASSESSMENT PRACTICE

Sixteen years ago, Heron (1988) thought the time wasripe for an educational change from the authoritarianmodel to one which is student inclusive. Almost tenyears after Heron’s call for change, came thepublication of Higher Education in the Learning Society.The Dearing Report (1997), as it became known,perhaps noting that little, or no change had occurred,attempted to prime teaching staff in universities tomake a professional commitment to teaching. Dearingaddressed as a priority the improvement of thestudent learning environments, recommended thatlearning and teaching strategies should now focus onthe promotion of student learning and stressed that aradical change to teaching was needed.

The impact of Dearing on assessment may begauged by the comments of Brown and Glasner (2003),who noted that the range of ways in which studentsare assessed is unfortunately extremely limited witharound 80 per cent of assessment being in the form ofexams, essays or reports of some kind. This may onlypartly reflect what is assessed, consisting of a verylimited range of student skills, knowledge and ability.Students appear to do the same old types of activitiesagain and again. It may have been as a result of thesesame old activities that Boud (2000) was led to assertthat ‘assessment practices in higher educationinstitutions tend not to equip students well for theprocess of effective learning in a learning society’.Boud’s comments shed further light on the impact ofthe Dearing report, as the use of the ‘learning society’formed part of its title. Dearing’s view of the learningsociety reflected a ‘vision’ of a society (individuals, thestate, employers and providers of education andtraining) committed to learning throughout life, moreas a process or journey of discovery, rather than aticked box outcome. Boud (2000) discussed a morecomplex view of the learning society in which ‘thosewho are skilled and flexible learners will flourish,others will languish’. Thus a need for change withinassessment is evident to encourage progressivelearning, as skilled and flexible learners are unlikelyproducts of Heron’s authoritarian model ofassessment. The way forward is to look for a model ofstudent assessment which is inclusive, involvingstudents and tutors working collaboratively. Self- andpeer-assessments provide just that model.


7


8 CHAPTER 1 INVOLVING STUDENTS IN ASSESSMENT

SELF- AND PEER-ASSESSMENT:A WAY TO IMPLEMENT CHANGE

Writing in the early 1950s, Rogers (2003) outlined thegoals of democratic education, in assisting studentsto become individuals. He included such attributes asbeing ‘a critical learner, able to evaluate thecontributions made by others and being able to self-initiate actions and be responsible for those actions’.Furthermore, he went on to say that, ‘we cannot teachanother person directly; we can only facilitate theirlearning. A person learns significantly only thosethings which they perceive as being involved in themaintenance of, or enhancement of, the structure ofself’. These are sentiments which underly self- andpeer-assessment philosophy. The defining character-istic of self-assessment is the ‘involvement ofstudents in identifying standards and/or criteria toapply to their work and making judgements about theextent to which they have met these criteria andstandards’ (Boud, 1986). Peer-assessment has beendefined (Topping et al., 2000) as ‘an arrangement forpeers to consider the level, value, worth, quality orsuccessfulness of the products or outcomes oflearning of others of similar status’. From thesedefinitions it becomes apparent that self- and peer-assessments are not methods of assessment butsources of assessment that may be used within aframework of different methods (Brown et al., 1997).

At the heart of both of these assessmentprocesses is the student. Brew (1995) commenting onthe conceptual shift in higher education from a focuson teaching, to a perspective in which studentlearning is central, illustrates the importance of thisstudent centredness, ‘the essence of the learningperspective is that it considers all decisions aboutteaching and assessment in the light of the impact orpotential impact on student learning’. Both self- andpeer-assessment appear to have an emphasis ondeveloping student autonomy, which, while not aneasy concept to define, does have ‘some of theattributes required by anyone if they are to be effectivelearners’. After all, being dependent on others(teachers) and not being able to plan and manageyour own journey, or process of lifelong learning willnot be effective preparation for learning and the worldof employment (Boud, 1988).

The model shown in Figure 1 from Higgs (1988)is of autonomous learning. It shows how the fourprincipal elements of learner, teacher, task andenvironment interact together. How successful theinteraction is, depends mainly on the extent to whichthe elements are consistent with each other and uponcertain specific assumptions, such as, that self-directed learning needs to be active and not passive.More importantly, the outcomes of learning are

dependent on the assessment process. In the self-determining student who will be self-assessing, theassessment will be included in the process oflearning, as well as work done on the content of thelearning. Therefore assessing how learning takesplace and considering how evidence is provided ofwhat has been learnt is fundamentally moreimportant than assessing what has been learnt ormemorized. The shift to self-determination and self-assessment starts to make the process moreimportant than content (Heron, 1988). Some mayconsider that Heron demotes content too much,believing, with some justification, that a balanceneeds to be established between the process oflearning and the content of learning. However, thestress on the process and the content in self- andpeer-assessment highlights the need for effectivecommunication between students and tutorsconcerning the use of appropriate tasks and activities.This is well illustrated, for example, in the need fortutor and student to discuss and agree assessmentcriteria, which results in students having a greaterdegree of ownership of each assessment they areundertaking (Falchikov and Boud, 1989). While it isevident that greater ownership may also be related toa shift in power, a note of caution needs to beexpressed. Tan (2004) argues that while self-assessment provides students ‘with more autonomyto judge their own work, more is known about thestudents in terms of how they view themselves’. Thishas implications for how power is manifested withinthe assessment process. Therefore, it is vital forstudent empowerment to understand the ways inwhich power is exercised.

Figure 1. Autonomous learning

Environment

TeacherTask

The learner andautonomous

learning

Source: Higgs (1988)


9

With this increased ownership, it may bepossible to reconsider Figure 1 to explicitly includeassessment. In Figure 2, the environment provides theoverall background in which the learning occurs. Theenvironment is formed from a number of things suchas, the learning and teaching beliefs of departments orfaculties, as well as, the human and physical resourceissues. The environment influences in varying ways thestudent, the facilitator, the assessment practice andthe assessment task(s). These four separate com-ponents all overlap at a given focus, becoming one. It ishere that students’ learning is shaped; this is the Zoneof Formative Learning. Assessment is, therefore,inclusive in the learning process.

Hinett (1995) in a study which comparedassessment practice at a British University with thatcarried out at the Alverno College Milwaukee USAreported how effectively this close integration canwork at an institutional level. A major difference in theapproach to assessment was in the use of self- andpeer-assessment. At Alverno; each student was act-ively encouraged to self- and peer-assess. Attitudesranged from ‘it’s painful, but it works and I learnmore’ to ‘I like self-assessment because I can reflectback and know I should study more in this area’.

At the British University little value was given toself- and peer-assessment, which meant studentslacked confidence and faith in their own judgements.On self-assessment, some typical commentsstudents made were that ‘no-one takes it seriously’and ‘it is just a hassle’. Furthermore, students learntin a prescriptive environment, being told ‘you will dothis’. They generally validated their work in terms of

grades and admitted to getting into the mentality of‘what am I going to get out of this in terms of credit’.When asked ‘How do you know what is expected ofyou?’, the majority suggested that they didn’t reallyknow as ‘they never actually say what they are lookingfor’. Students also often talked of ‘guessing’.

Students at Alverno used feedback construct-ively, to help them to plan their work and tounderstand how they were developing as learners.Explicit criteria and learning integrated with theassessment process allowed students, through self-assessment, to take control of their own learning.

Before considering self- and peer-assessmentin a little more detail, it would help to be familiar withsome aspects of the learning process.

THEORETICAL FRAMEWORKS TO LEARNING

Falchikov (2001) observed that ‘too many educationalinitiatives appear to be devoid of theoretical under-pinning, seeming to be driven by expediency,economics or political agendas’. Perhaps educationalinitiatives will go on being at least influenced by suchas factors. However, learning through self- and peer-assessment can only be understood within a theo-retical learning framework. There are two influentialtheories of learning that we need consider.

Piaget and learningPiaget believed that each child passed through aseries of stages of thinking which were qualitativelydifferent from each other (Sutherland, 1992). A childactively constructs their knowledge of the world aroundthem as a result of various encounters with theenvironment, and also, by, communicating with otherchildren, as discussion can challenge existingschemes or concepts leading to a re-think of anoriginal point of view. In this way the child learns by aseries of adjustments to their environment, which isachieved through using two alternative mechanismswithin the process — assimilation and accomm-odation — which are balanced through equilibration.In this way, new material being assimilated by thelearner can be modified against previous concepts,which are stored in the memory as learningprogresses. These individual pieces of information arethemselves up-dated by the mechanism ofaccommodation and transformed into new materialand a more complete understanding. Piaget’s views oflearning are of particular importance to those of us ineducation in a number of rather significant ways. Forexample, they underpin the learning cycle proposedby Kolb, which has prominence in higher education asa model to aid understanding the learning process.Kolb’s learning cycle has frequently been reinterpreted

Figure 2. Requirements for generatingthe Zone of Formative Learning

Environment

Facilitator

Task

Zone ofFormativeLearning

Assessment

Student


10 CHAPTER 1 INVOLVING STUDENTS IN ASSESSMENT

and is often presented in a very simplified form. InFigure 3, the unabridged learning cycle (Kolb, 1984),with its strong reference to Piaget’s work isillustrated. Looking at the cycle, we can see itrepresents a very personal cycle of learning, self-contained from outside social and professionalinfluences. The learner is very reliant on their ownperceptions of their learning experience.

Vygotsky and learningLike Piaget, Vygotsky (1978) believed that childrenconstructed their own learning. Vygotsky was awarethat children, often unable to perform tasks or solveproblems alone often succeeded when an adult helpedthem. Piaget took a dim view of success obtained inthis way, claiming that it involved the teaching andlearning of procedures and not the development of fullyintegrated learning and understanding. For Piaget,genuine intellectual competence was a manifestationof a child’s largely unassisted activities (Wood, 1988),whereas Vygotsky saw intervention as important. ‘Thedifference between twelve and eight, or between nineand eight is what we call the Zone of ProximalDevelopment (ZPD). It is the distance between theactual developmental level as determined byindependent problem solving, and the level of potentialdevelopment as determined through problem solvingunder adult guidance or in collaboration with a morecapable peer’ Vygotsky (1978).

For Vygotsky, ‘learning awakens a variety ofinternal developmental processes that are able to

operate only when the child is interacting with otherpeople in his environment and in cooperation with hispeers. Once these processes are fully internalisedthey become part of the child’s independentdevelopment achievement’. Therefore, while bothPiaget and Vygotsky placed a very strong emphasis onactivity as the basis for learning, Vygotskyemphasised communication and social interaction,where teachers (either adults or more experiencedpeers) retain varying degrees of influence over eachchild’s learning activities. Wood et al., (1976) saw theintervention of a tutor as involving a kind of‘scaffolding’ process that enables a child or novice tosolve a problem or achieve a goal which would bebeyond his or her unassisted efforts.

The theoretical underpinnings of the work ofPiaget and Vygotsky are recognisably used in highereducation today with regards to self- and peer-assess-ment. Peer-assessment is grounded in philosophiesof active learning, and may be seen as being amanifestation of social construction, because itinvolves the joint construction of knowledge throughdiscourse (Falchikov and Goldfinch, 2000). Falchikov(2001) draws our attention to both the work of Piagetand Vygotsky with respect to peer tutoring, andemphasises the role of self- and peer-assessment inpeer tutoring. One of the arguments used by Falchikov(2003) to illustrate that self- and peer-assessmentare for educational and not just training purposes isthat Piagetian theory stresses the importance ofpractical concrete experiences for cognitive develop-ment. The role of experience, with social and culturalinfluences in learning, is very relevant to self-assessment (Brew 1995). MacDonald (2004)discussed the practical implications of implementingonline pedagogies and stressed the communicativepotential of e-learning employing a social construct-ivist approach. It has already been seen how Piagetianthinking is compatible with Kolb’s learning cycle; butKolb (1984) also appears to draw on a Vygotsky socialconstructivism. This, a less discussed aspect of theKolb learning cycle, is of immense importance inrelation to self- and peer-assessment.

APPROACHES TO LEARNING

Deep and surface approachesA number of advocates of self-assessment relateapproaches to learning as so-called ‘deep’ and‘surface’. Marton and Saljo (1976) explored theprocesses and strategies of learning used by studentsas well as the outcomes of that learning, in terms ofwhat is understood and remembered. They found twodifferent levels of processing which they called deep-level and surface-level processing. ‘In the case of

ABSTRACT CONCEPTUALISATION

ACTI

VEEX

PER

IMEN

TATI

ON

REFLECTIVEOB

SERVATION

Figure 3. Structural dimensions underlying theprocess of experiential learning and the resultingbasic knowledge forms

Transformationvia EXTENSION

Accommodativeknowledge

Divergentknowledge

Assimilativeknowledge

Convergentknowledge

CONCRETE EXPERIENCE

Transformationvia INTENTION

Grasping viaAPPREHENSION

Grasping viaCOMPREHENSION

Source: Kolb (1984)

surface-level processing, each student directed hisattention towards learning text itself (the sign), i.e. hehad a reproductive conception of learning whichmeant that he was more or less forced to keep to arote-learning strategy. In the case of deep-levelprocessing, the student was directed towards theintentional context of the learning material (what wassignified), i.e. he was directed towards compre-hending what the author wanted to say about a certainscientific problem or principle’ (Marton and Saljo,1976). Students are not necessarily deep or surfacelearners, but they can take a deep or surfaceapproach to learning depending on the circum-stances. For example, someone might normally adopta deep approach to a subject, but under pressure ofan impending examination they might switch to asurface approach (Brew, 1995).

SUMMARY

Students are central in both self- and peer-assessment. As such, both sources of assessmentcan be used to enthuse, enable and empowerstudents within a variety of assessment methods. Theevidence for the type and approaches to learningencouraged by self- and peer-assessments istheoretically strong, with an emphasis on studentsconstructing knowledge within a formative learningenvironment. With self- and peer-assessment,learning is more fully integrated with assessment,and not just a by-product of assessment. Ideally,students should grow in the use of self- and peer-assessment throughout their university experience,because the ultimate goals, successful andmeaningful learning, are essential in preparation forthe learning society.


11


12

2I GETTING STARTEDWITH SELF- AND PEER-ASSESSMENT

AASSESSMENT CRITERIA

Definition and rationaleThe role of the criteria is fundamental to self- and peer-assessment because criteria provide anobjective structure for those who generate and implement them. Marking criteria are, in effect, theseat of ownership. Sadler (1989) defines a criterion as ‘a distinguishing property or characteristic ofany thing, by which its quality can be judged or estimated, or by which a decision or classification maybe made’. Without criteria, academics rely covertly on an expert’s notion of quality. There is aconnoisseurship; judges ‘know’ what the standards are and how to use them (Sadler, 1989). There is areliance on tacit assessment knowledge, knowledge regarding assessment that is in the head of thetutor and not necessarily made explicit to students (or other tutors). This approach to marking is notacceptable for the reasons already considered in Chapter 1, primarily, the exclusion of students fromthe assessment process. Furthermore, Ecclestone (2001) illustrates other considerations for usingcriteria such as enhanced reliability, as criteria make assessment more amenable to moderation andstandardisation.

Self- and peer- assessment: introducing the criteriaStudents are often unfamiliar with marking criteria. Hence, they need to be clearly introduced to themat the beginning of their course of study. Boud (1986) considered primarily self-assessment, but madesuggestions which are also applicable to peer-assessment. In order to try to resolve the issue ofcriteria, students and staff should attempt to clarify the concepts of assessment criteria. Wherepossible, tutors should not impose, but listen to the student’s perception of the criteria. Jointdiscussion may help avoid any mismatch in interpretation of the criteria or an individual criterion(Orsmond et al., 1996, 1997 and 2000). Further discussion regarding interpretation of the criteria canbe found in Case Study 1 (Merry and Orsmond) at the back of this guide. Rust et al., (2003) alsoinvestigated student understanding of assessment criteria and the assessment process. They toofound a mismatch of interpretation with difficult criteria, such as analysis and evaluation.

Discussions may initially take the form of asking:

‘What would be the factors which characterised a good assignment on this course?’

Once this process has been completed, the elements of satisfactory criteria should be considered. Thisentails such information as:

• the area to be assessed;

• the aims of the assessment;

• the standards to be reached.

Generating assessment criteriaBoud (1986) highlighted the importance of students reaching their own decisions about the criteria forassessing themselves but stressed the facilitative role the teacher plays in this process. Two techniquesfor facilitating understanding of criteria are considered:

1. Structured written schedules for developingindividual criteria

2. Structured group activities for reaching consen-sus on common criteria

1. Structured written schedulesThese provide a list of instructions guiding studentsthrough a sequence of steps involving:

• identifying the criteria, which they considerappropriate to apply to their work;

• clarifying these criteria;

• assessing the priority, or emphasis, which theywish to give to each criterion.

Working with students who have perhapsstarted a task or who are more experienced in self-and peer-assessment might require different types ofquestions. Brown et al., (1997) gave some suggestions,such as:

• what do I think about what I have been doing?

• how could I improve my approach?

Once satisfactory criteria have been generatedin this way, students use them as a yardstick by whichthey are able to judge their own performance. Thismight involve:

• awarding themselves a mark with respect toeach criterion; and then

• making a statement justifying that mark.

2. Structured group activitiesBoud (1986) suggested structured group activities ifcommon criteria for a class are required. This willinvolve the group (groups) identifying, discussing andagreeing upon a common set of criteria. Initiallystudents are briefed that they will be expected toproduce a number of criteria. There then follow anumber of sessions where criteria are clarified anddiscussed.

Freeman and Lewis (1998) also providedsuggestions for group construction of criteria. Thesesuggestions particularly lend themselves to engagingstudents with self- or peer-assessment by askingthem to consider some of their own existing course-work, perhaps with the accompanying feedback:

• ask students to review their returned assign-ment in pairs or as individuals;

• ask them to make brief notes, concerningwhere they gained or lost marks;

• from these notes draw up a list of the criteriastudents thought the tutor seemed to be using.These criteria can be discussed further forgreater clarity.

By viewing student learning as a journey, theongoing or re-evaluating of the initial joint discussionregarding criteria is necessary to support the learningthat the students are undertaking. This can be seen asa first stage in the self- and peer-assessment process.

The above suggestions are helpful, but somecare needs to be taken. For example, students whoundertake a poster assignment in a level onemicrobiology session may also be undertaking a posterassignment in level one physiology. The students maythink that the end product of the assessment is‘making a poster’. However, the purpose of theassessed assignment may be the demonstration ofspecific learning outcomes through a poster, not themaking of the poster per se. So students need to beclear that the physiology assignment requiresdemonstration of certain processing outcomes, whichmay, or may not, be very different from those outcomesdemonstrated through the creation of theirmicrobiology posters.

The more experienced students become, themore their approach to criteria construction changes.Sivan, (2000) illustrated this well when discussinghow students with previous experience of peer-assessment approached criteria construction.Students were seen to take ‘a further step andinitiated the allocation of different weighting to eachcriterion and thus were taking even moreresponsibility for their own learning’. This acceptanceof responsibility further develops a student’s sense of‘ownership’ of the assessment process and furtherstrengthens a student’s claim of being an auto-nomous learner.

The practise and understanding of self- andpeer-assessment develops through use as a courseor as education progresses, leading to a deeperunderstanding of the assessment and criteriarequirements as the learning journey progresses.

Self- and peer-assessment: criteria implementationDifferent types of criteria can be used to generatedifferent forms of judgements. Miller (2003) consider-ed the implementation of criteria within a self- andpeer-assessment context. Miller, looking at oralpresentation over two consecutive years, wanted tochange assessment marking from looking at a fewglobal components of performance such as clarity


13

and interaction, (used in 2000) to multiple, verydiscrete components of performance such as ‘thepresentation included a plan for community and/orwork re-integration’ (used in 2001). The reason forthis shift was to tackle the tendency for markers toassign scores in a very narrow range, concentrating atthe high end of the scoring scale. The resultsindicated that:

• the initial assessment sheet with questionselicited greater feedback from the markers;

• the revised assessment sheet elicited fewercomments and a larger percentage of negativefeedback.

Miller explained this result as a consequenceof markers being more critically analytical of thepresentation when using the revised sheet. However,it could be that, with a set of discrete statements,there was less the students were unclear about, orperhaps the students did not fully understand thestatements, and so were unable to structure ques-tions as well. Furthermore, the use of statementsmay prove limiting or even detrimental to the learningprocess. This could occur in a number of ways. Firstly,this form of listed ‘closed’ statements in anassessment sheet may be less than inspiring tostudents, as the statements could, even if verycarefully worded, be based upon the tutor’s singular,and perhaps biased, view of what may, or may not,have occurred. Secondly, rather than help studentsfocus on their own personal learning experiences,statements may, instead, severely limit free andhonest individual expression from the student, whichwould then be detrimental to both students and tutorsthroughout the course, thereby unwittingly limitingthe students’ learning potential.

Ultimately, Miller (2003) made an importantstatement, the highly specific assessment instru-ment, as opposed to the more global instrument,‘produces better quantitative differentiation of levelsof performance at the expense of losing qualitativefeedback’. It is very important to be aware beforeimplementing specific types of criteria, what exactlyyou are hoping to achieve with them. Using Miller’sexample above, are you looking for the ‘quantitative’or ‘qualitative’? Knowing this not only lends credit-ability to the assessment process, but provides usefulinformation, particularly for evaluation purposes.

There are some situations where tutors do notalways have total control over how a module runs.Tutors are increasingly finding themselves ‘teamteaching’ where the criteria are defined, perhaps byone person, but implemented by the whole team.Tutors, particularly in the case of new tutors, may find

themselves taking over a module where the criteriaare already defined. Often in such situations tutorscannot involve students in construction of the criteria,but that does not necessarily mean that studentsneed be excluded from working with those criteria.There are things that can be done. Here is an example:

• Take the marking criteria and consider theterms used to define the aspects of the assign-ment to be assessed. One example would be iflooking into experimental design then part ofthe criteria may be to consider how ‘robust’and ‘rigorous’ the design is.

• Make a list of the criteria terms used.

• Ask the students to write down their definitionsof each term.

• Collect in the definitions, read them and notethe correct ones for each term and the otherdefinitions used. This second part is important,because incorrect definitions may give you aninsight into student thinking and highlight anycommon misconceptions.

• Feedback to the students the range of defini-tion they have used, indicating the ones whichare correct.

This is not ideal, but at least students do haveinvolvement in the criteria, and furthermore the groundhas now been prepared for further discussions,perhaps in subsequent tutorials. Tutorial work isuseful for discussing assessment issues. Adams andKing (1995) using six tutorial groups engaged studentswith self-assessment in a class of 120. Race (1998)outlines a process list for peer-assessment which canbe used for groups of up to 100 and takes less than anhour to implement.

Self- and peer-assessment: using marking criteria tohelp in making judgements Making judgements is ultimately what assessment isall about. Having explicit and unambiguous criteriahelps this process, but it is still a challenge forstudents to take their own work and make judge-ments about it. Peers can be useful in helpingstudents develop their ability to judge. ‘While peersmay be unwilling to make formal assessment of theirpeers they may be more positive when students haveto give specific feedback of a descriptive nature forthe benefit of their peers and no grading has takenplace’ (Boud, 1986). When using criteria, rememberthey are reference points in the process of judgement,aids not replacements (Knight and Yorke, 2003).


14 CHAPTER 2 GETTING STARTED WITH SELF- AND PEER-ASSESSMENT

Making judgements does not, then, simply involvemarking work. Peer review is a helpful way to approachpeer-assessment (Pond et al., 1995). When askingstudents to make judgements about their own work orthat of their peers, it is important to consider timepressures. For guidance, if considering poster work,students can provide useful written feedback (with orwithout a mark), on approximately three or four pieces ofwork in an hour (depending on their complexity).

Elwood and Klenowski (2002) offered studentsstructured support which considered criteria develop-ment and help in making judgements about theirwork, which incorporates a modelling process whichconsidered six separate topics such as, demonstrationof understanding of criteria by grading an assignment.

Computer-based programmes are increasinglybeing used for assessment purposes. The Many Usingand Creative Hypermedia system (MUCH) is a multi-user hypermedia tool that supports collaborativelearning and has proved to be suitable for peer-assessment (Rushton et al., 1993). Another computersystem called ‘Peers’ has been successfully used toundertake peer-assessment allowing staff andstudents to determine criteria and weightings foreach criterion (Ngu et al., 1995). The case study (CaseStudy 2) provided by Kuri stresses two strengths ofpeer-assessment with respect to criteria, empower-ment and peer use of feedback. The use of onlinelearning often allows detailed and personalisedfeedback to be exchanged quickly thus enhancing itspotential effectiveness.

Time does need to be allocated to help studentsin making judgements using criteria as these commentsfrom a study by Brown et al., (1998) illustrate:

‘Grading is really hard, to know whether to givethem a 2 or a 4, I’ve know idea how you draw the line,I just know if someone is good or not’.

‘I sat there with these numbers and in the endit became a bit random. Perhaps the tutor finds iteasier to break it all down into a section, that’s up toher. But I just get a general feeling that’s all.’

However, the more students undertake exer-cises involving generating and applying criteria, themore comfortable they become. A quote from a studyinterview sums this up well:

‘When I did it the first time, I need longer tothink what grade I should give to this group. Besides,I worried a lot whether I gave a fair mark to others.However, I can do it quite fast this time ... More you do,better you can do it’ (Sivan 2000).

Using criteria and making judgements in a

meaningful way does not just happen after oneattempt, students need practice to develop the abilitiesrequired. Because of this, it is important that, teachingtutors, try their best to get self- and peer-assessmentpractices implemented at an early stage within astudent’s university career, and provide a progressionof self- and peer-activities throughout a student’suniversity course of studies. Adams and King (1995)give an indication how this could be done. Sluijsman,(2002) presents a framework with guidance on how torealise integrated peer-assessment activities.

However, a note of caution: just because youhave worked through a criteria construction processwith students, this is no guarantee that all yourstudents will necessarily understand it. Many will, butsome may not. The more experienced students becomein working with criteria, the fewer the problems ofmisunderstanding. However, as with any assessmentpractice some students may misunderstand.

Lack of understanding may be only one causeof disagreements. Hughes and Large (1993) discussvariability in the marking of oral presentation inpharmacology students despite working with agreedmarking criteria. They identify issues separate fromthe criteria such as, how the ‘voice’ of the speakermay be heard by those near the front of the lecturetheatre and inaudible at the back; or how overheadtransparencies may be readable at close quarters butunreadable from a distance.

ASSESSMENT: VALIDITY AND RELIABILITY

Only a valid and reliable assessment processesshould be used to determine what learning hasoccurred. Reliability of assessment is defined by Fryet al., (1999) as ‘the assessment process wouldgenerate the same results if repeated on anotheroccasion with the same group, or if repeated withanother group of similar students’. Validity is definedas ‘adequacy and appropriateness of the task/test inrelation to the outcomes/objectives of the teachingbeing assessed’, i.e. it measures what it is supposedto measure. These general definitions have beendeveloped; for example, Gielen et al., (2003) considerthe validity of assessment scoring, and whetherscores are valid. In this respect the criterion offairness plays an important role.

Self- assessment: validity and reliabilityThree studies which consider validity and reliabilityhave implications for implementing and evaluating self-assessment. Boud and Falchikov (1989) reviewed 48studies of student self-ratings compared to the ratingsof students by teachers. Some of the outcomes were:


15

• comparing results from a number of studiesshowed that there is no consistent tendency toover or underestimate performance;

• when asked to rate themselves on a markingscale, able and mature students are able to doso. Able students when new to a subject areaware of, and concentrate on, their owndeficiencies, and thus underrate their work.

Falchikov and Boud (1989) undertook a meta-analysis of 57 quantitative self-assessment studiesused in higher education. Some of the outcomes were:

• the role of seniority or duration of enrolment ofthe marker was found to be less important thanexpertise in a given subject;

• explicit criteria led to greater accuracy of ratingas did criteria that students felt they ownedwhen compared to criteria that were provided;

• better-designed study was associated with acloser correspondence between student andtutor compared to poorly designed studies.

Boud (1989) raised the question, ‘if there is a highcorrelation between marks generated by students andthose generated by staff, why bother with involvingstudents if their contribution makes no difference tothe final grade?’ He provided two suggestions:

• self-assessment provides practice in theinterpretation of the often arbitrary require-ments which most public work needs to satisfy.

• expediency: if students can take a greater role inassessment there is the potential for the savingof staff time on the often tedious task of marking.

Regardless of the correlation between marks,considering marks themselves as an importantindicator may be missing the point. Topping (2003)comments ‘that the high correlation between measuresis in any event redundant, and the processes here are atleast as important as the actual judgements’. Engage-ment in self-assessment is a good way to improveperformance and nudge students forward in theirZones of Proximal Development. The case study byRushton (Case Study 3) at the back of this guidestresses the learning that takes place as a result of self-or peer-assessment.

Peer-assessment: validity and reliabilityFalchikov and Goldfinch (2000) carried out a meta-analysis comparing peer-assessed and teacher marks.

This study can be seen as a companion piece to thepaper by Falchikov and Boud (1989). Some of theoutcomes of this peer-assessment study were similarto the outcome for the self-assessment study:

• high-quality studies were associated withbetter peer-faculty agreement than studies oflower quality;

• student familiarity with the ownership of criteriatended to enhance peer-assessment validity.

However, there were differences compared tothe self-assessment study:

• Unlike self-assessment where the level of thecourse appeared to be a salient variable, peer-assessment does not seem to be less valid inlower-level courses. A possible explanation isthat participants in peer-assessment studiesare, in general, well prepared.

• There was no clear subject area difference.

Finally, some areas were measured in peer-assessments that were not considered in self-assess-ment. Peer marking of several individual dimensionsappeared less valid than peer-assessment thatrequired global judgement, based on well-definedand well-understood criteria.

A number of concerns, which have implicationsfor the implementation and evaluation of self- andpeer-assessment, have been expressed concerningbias in peer-assessment. Magin (1993), accepting thecriticism, where a peer mark is based on an individualpeer rating, described a study where multiple ratingsare used. The effect of a lenient or severe mark isdiluted by the marks from the other students. Aninteresting approach to dealing with lenient or harshmarkers is reported in the case study by Cogdell et al.,(Case Study 4 at the back of this guide). In anotherstudy, Magin (2001) studied the relational ‘reciprocity’effect of peer-assessment of group work. Only 1 percent of the variance in peer scores was due to bias. Inconsidering gender bias, Falchikov and Magin (1997)used a system that considered student ratings fromsame and opposite sex, found marginal differencesfavouring females in peer-assessment.

A brief but useful and in-depth review of validityand reliability in self- and peer-assessment exists(Topping, 2003). Within bioscience, Stefani (1994)found student assessment to be as reliable as that oflecturers, and also reflected on the power/ownershipdebate, advocating the early introduction in thestudents’ careers of self- and peer-assessment forsummative and formative assessment.



SELF- AND PEER-ASSESSMENT:HOW IS IT CARRIED OUT AND EVALUATED?

Developing and implementing the marking criteria aremajor parts of carrying out self- and peer-assess-ment. However, there are a number of stages thatneed planning when carrying out and evaluating self-and peer-assessment. These have been documented(Falchikov 2003). Figure 4 is taken from this work.While this diagram is in some ways self explanatoryand a number of issues have been covered already, alittle commentary may clarify and refine.

Preparation

• Remember the principles of good experimentaldesign. In order to evaluate the procedure sub-sequently, the dependant variables need to beidentified, such as the agreement between peersand tutor, or a measure of the benefits to learningexperienced by the participants (Falchikov 2003).

• Students need to be well briefed in advance ofthe assessment practice and this may meanincluding details in module/award handbooks,which are often written months in advance ofteaching.

• Information should be given both in writing andverbally. Try to ensure that students see andare familiar with the use of all documents, such

as evaluation forms, and have the opportunityto question, clarify and check all material.

• Most importantly, clearly articulate the ration-ale for using self- and/or peer-assessment.This may make planning the assessment moremeaningful. If students have motivation, theyare more likely to engage in the assessmenttask. For example, if the reason is primarily toengage students in the assessment processper se, then you will plan differently than if thereason was to make students aware of how touse feedback through peer- or self-formativeassessment. Often this is a case of emphasisand the focus of the task.

• Particularly with students new to the process,discuss issues related to fairness and bias(Sivan, 2000).

• Have an effectively detailed approach as to howself- or peer-assessment is to be ‘policed’ and‘controlled’. Assessment criteria can help withthe marking, but some students may under- orover-mark. You need to talk to the studentsabout the consequences of this type of marking(Adams and King, 1995). Race (1998) has somesuggestions on how this may be done whichinclude, moderation of peer marking, moni-toring student achievement and providingmark-free rehearsal opportunities. Moderation


17

Figure 4. Stages in carrying out and evaluating self- and peer-assessment

Clear instructions given(written) relating to all stagesof the process, including, for

example, mechanisms fordisagreement and whether,

and the extent to which,marks count

Criteria identified by students(and discussed/agreed

with teacher?)

Rationale suppliedto students

Study designed carefully

Preparation

Checklist/criteria used bystudents to mark their

own/peers’ work/performance

(where appropriate)

Feedback given by studentsto peers

Judgements justified bystudents publicly

Disagreements resolved usingagreed mechanisms

Implementation

Feedback collected(formally/informally) usingstandardised instruments

where appropriate

Feedback analysed

Problems identified

Modifications madewhere necessary

Follow-up andevaluation

Exercise repeated withsame cohort

Replication

Checklist prepared withcriteria listed and ranked

Source: Falchikov (2003)

of marks is a difficult issue as it closely relatesto the issues of power discussed in Chapter 1.

ImplementationPerhaps here it may be appropriate to consider when touse self-assessment, when to use peer-assessment,and when to use both. There are no clear rules. However,there are some things you may wish to consider:

• Self-assessment is not undertaken in isolationfrom others. ‘The defining feature of self-assessment is that the individual learnerultimately makes a judgement about what hasbeen learnt, not that others have no input to it’Boud (1995). So, self-assessment should beseen as a continual process, used by students aspart of their natural learning. It may initially lenditself to some exercise rather than others, forexample PDP. The initials PDP are sometimesassumed to mean Personal Development Plan,but the last P indicates planning. Therefore PDPis a process carried out over time. In PDPstudents are often asked to reflect and writeabout those reflections. This is ideally suited toself-assessment or self-evaluation processes.(The distinction between assessment andevaluation is discussed in Chapter 3).

• Sometimes you may wish to use student-centredassessment practices for semi-pragmaticreasons. The case study by Hughes (Case Study5) illustrates this well. Here peer-assessment isused to reduce marking time, but also provides arange of benefits for students.

• Race (1998) lists a range of activities wherepeer-assessment can be used, such as instudent presentation, interviews and practicalwork. Peer-assessment also lends itself togroup work and there are a number ofexamples of this (Cheng and Warren, 1999;Freeman, 1995; Goldfinch, 1994; Li, 2001 andLekj and Wyvill, 2002).

• Assessment practices should relate to the learn-ing outcomes to be achieved. Therefore, checkwhat the outcomes say as this may give guidanceas to which student-centred practice to use.

• Boud (1986) gave guidance on giving andreceiving feedback. It is a key area and studentswill need some preparation in matters such asresolving disagreements. To faciliate studentsin their learning, it is best to introduce guidanceearly, rather than just before the assessedproduct is produced, to allow time for students

to assimilate the process.

• Giving students a greater degree of help withmarking, and also focusing on the provision offeedback, is important when implementingpeer-assessment. The Peer Feedback Marking(PFM) designed to develop peer-assessmentwas rated by students as conferring morebenefits than the more usual lecture markedmethods. It also enhanced reflection and thedelivery of diplomatic criticism (Falchikov,(1995). The case study by Reed (Case Study 6)gives an indication of how to introduce self- andpeer-assessment. Race (1998) also containssome suggestions for getting the most out ofpeer-assessment such as, allow time for theassessment exercise and the need to keepeveryone well informed.

Follow-up and evaluationTry not to give feedback on the process to the group orindividual, but do discuss feedback with students —and receive it, from them. Exchange views with yourstudents as to how the process went. Listen to whatvalue it had for them. Remember, this is an inclusiveprocess. Try not to leave them with a void. Attempt toidentify any concerns or problems, which may meanalteration to how self- and peer-assessment may beused next time. Use a detailed and diverse form ofevaluation methodology, such as questionnaires andgroup interviews.

A number of the case studies included in thisbook have thoughtful guidance on preparing andimplementing self- and peer-assessment; Hughes’scontribution (Case Study 5) is particularly helpful. Theimportance of careful consideration of preparation,implementation and evaluation cannot be overemphasised. The consequences of not doing this maylead to ineptly introduced and delivered practices thatproduce results directly opposite of what is desired.

SELF- AND PEER-ASSESSMENT:A COMMUNITY OF PRACTICE

Encouraging students to engage in self-and peer-assessmentA barrier to new methods of assessment are anindividual’s prior experience of being assessed. Henceit is necessary to consider how to encourage studentsto be involved. Be aware of the problems that studentsmay have. Cheng and Warren, (1997) highlightedsome student concerns. Students may:

• be aware of their own shortcomings in thesubject area;



• have doubts about their own objectivity; and

• feel that the process is unfair.

Often tutors are regarded as specialists in theirfield, which could mean that students feel moreinadequate as they are novices to the subject andtherefore in awe of staff. This should be avoided asmuch as possible and the role of tutors should bemore as guides. This is one reason why a differentnon-specialist tutor should facilitate in some of theimplementation.

(Falchikov, 2003) highlighted other concernssuch as.

• Social effects, such as friendships or hostility.

• It is the ‘job’ of the teacher to mark work.

It may, therefore, be necessary to ‘sell’ to the studentsthe idea that their involvement in assessment is agood thing. There are a number of considerations toremember.

• Ensure students feel ‘safe’ about the process.Sullivan and Hall (1997) considered this animportant issue.

• In peer-assessment, it may be necessary tomake the process as anonymous as possible(Merry and Orsmond, Case Study 1).

• In order to engage students, teachers need toclearly identify why they want students to beinvolved in the assessment process. Studentsneed to see the value (what’s in it for them) ofengaging in a particular form of self- or peer-assessment. The case study by Hughes (CaseStudy 5) has some useful suggestions as to whystudents need to be involved.

• Show students (and other staff) the researchliterature as the evidence that it works. Also itmay help to introduce students to theories oflearning, perhaps discuss with them the Zoneof Proximal Development (see Chapter 1) andillustrate how self- and peer-assessment areinterventions to enhance their learning.

• Use exemplars as a practice run for thestudents, so that they can gain objective con-fidence. Exemplars allow students to under-stand and use the concepts and criteria withthe guidance of their tutor and additional inputof peers at the beginning of a project or course.These exemplars may normally be work gen-

erated from previous cohorts of students whoundertook a similar assessment. A number ofcase studies in this guide advocate the use ofexemplars, Brennan et al. (Case Study 7) is onesuch example.

Encouraging teachers to engage in offering and usingself- and peer-assessmentIt is important to encourage colleagues to be involvedin alternative forms of assessment. There are anumber of reasons for this.

• Within modular and distance-learning frame-works assessment communities are becomingincreasingly fragmented.

• Increasingly, assessment involves more thanjust one person, or even one subject area.

• This greater involvement will challenge tacitnotions of standards shared in a familiaracademic community (Ecclestone, 2001).

• The one-off experience is not good. There is aneed for practice if skills in assessing their ownwork or that of peers are to be developed andintegrated into students’ normal learningpatterns.

• Students should have peer-assessment andexemplar work available to support theirongoing self- and peer-assessment practice,enabling and empowering them to achievehigher standards of learning, and thereforehigher success in their studies.

Falchikov (2003) gave some suggestions as to howcolleagues’ suspicions and hostility can be overcome.

• Help allay fears of colleagues by informingthem about existing research that advises onbest practice.

• Consider using assessment for formativepurposes.

• Help ease the change of role required, bystressing the importance of the teacher insetting up, implementing and running a self- orpeer-assessment initiative, and in helpingstudents acquire the necessary expertise.


19

SUMMARY

Achieving empowerment for students in assessmentprocesses demands their involvement with theassessment marking criteria. Although desirable, it isnot always possible to have students involved incriteria construction. However, the onus is on the tutorto ensure that students have a good workingunderstanding of the criteria. The design of themarking criteria often involves discussion of learningoutcomes. Therefore it is one of the cornerstones topreparing students for assessment and a valuable toolin successful implementation of self- and peer-assessment. How we ‘get started’ and ‘keep going’with self- and peer-assessment involves a lot of effort,reflection and planning on behalf of tutors. This is wellillustrated in the Cogdell et al., case study (Case Study4). To have a truly effective impact on student learningrequires departments and faculties to take on boardboth the culture and underlying philosophies of self-and peer-assessment; students need to perceive thisform of assessment as a natural process in theirlearning and be actively involved in its implementationand its importance in lifelong learning.




21

3I GOING DEEPER

SSELF- AND PEER-ASSESSMENT: A ROLE IN A SUPERCOMPLEX WORLD

Introducing supercomplexityChapter 1 introduced the learning society, one where skilled and flexible learners are required.Learning was seen as a continual process throughout life. Barnett (2000) begins to give some shape tothe type of world higher education may have to prepare students for. He described the world we live inas complex one, where we are assailed by more facts, data, evidence and arguments than we can easilyhandle. However, the world that current students may one day be entering is not a complex world, butrather a supercomplex world. One where everyone is continually being conceptually challenged, andthrough these challenges able to discover the way that they understandthemselves, the world, and how secure they feel about acting in the world.Supercomplexity is already recognisable in the world of work throughterms such as ‘flexibility’, ‘adaptability’ and ‘self-reliance’. The implicationof this terminology is that individuals have to take responsibility for‘continually reconstituting’ themselves. In other words, tutors will have tothink more about the society we are becoming, their role in the work place,and how they can make any necessary changes. To prepare graduates tomeet the challenge and to prosper in a supercomplex world requires acurriculum which fully embraces the domains of being, knowing and action.

Powerful learning environmentsOn first encountering the literature regarding complex worlds the reader is immediately aware of therole which self- and peer-assessment can play. This is well illustrated in a study by Schelfhout et al.(2004) who described a powerful learning environment which is aimed at fostering entrepreneurialskills and incorporated elements of self-, peer- and teacher-assessment. The design principles behindpowerful learning environments such as supporting constructive learning processes, resonate as thecore principles of self- and peer-assessment. One prime reason for using self- and peer-assessmentrelates to the use of feedback, ‘Within the learning process it is important to give students feedback ina way that challenges their perception on how to behave within groups (learning to cooperate, organiseetc.). A combination of self-, peer- and coach-assessments, followed by group discussions can beused’. Here we are reminded of Vygotsky’s theories of learning discussed in Chapter 1. When tutorsand students are involved in assessment, it is often referred to as co-assessment or collaborativeassessment (Dochy et al., 1999).

Problem-based learning (PBL) potentially allows for the creation of a powerful learningenvironment, developing abilities and assessing them, as it does cognitive, metacognitive (heightenedawareness of one’s own learning through developed cognitive processes) and social competencies.Self- and/or peer-assessment are good sources of assessment for PBL, as marking criteria often needto be developed and implemented in the judging of the learning process or product. However, PBLincorporating self- and peer-assessment is not without its difficulties, as reported by Segers andDochy (2001). They found rather mixed results. Students involved in the study were new to both PBLand self- and peer-assessment. As a result there were a number of issues raised in the study thatwould be helpful to those considering similar approaches:

• there were concerns about the commentsof peers;

• the criteria were felt difficult to interpret;

• the self- and peer-assessment process wasnot sufficiently introduced; and

• students tended not to be able to reflect ontheir own functioning.

The authors saw these problems in a positivefashion, identifying room for teachers to improve theireducational practice and to look again at thealignment of assessment with the main goals of theprogramme, with specific attention to certain issueslike critical reflection.

Jackson and Ward (2004) outlined a way ofrepresenting complex learning suitable for meetingthe demands of a supercomplex world. Theydescribed how higher education curricula can reflectthe disciplinary world of knowledge and the world ofprofessional and work-based learning. They proposedfive different curriculum-assessment environments,one of which, the ‘explicit curriculum’, allowsstudents to recognise and record their own learningand achievement through Personal DevelopmentPlanning (PDP). PDP may have a number of differentfocuses, such as encouraging students to takeresponsibility for their own learning, and encouragingstudents to understand more fully the process oflearning as discussed earlier in this book. Self-assessment is seen as a universal assessmentconcept within these processes.

SELF- AND PEER-ASSESSMENT:ENGAGEMENT IN REFLECTION

While tutors can choose to become more effectivereflective teachers as opposed to good teachers (Kuitet al., 2001) it is likely that many tutors will have tofacilitate reflective practice in students as part ofstudents’ Personal Development Planning. Reflectionis a very important part of development planning. Asconsidered by Moon (2001) reflection is ‘a means ofworking on what we know already, thoughts, ideas,feelings, we may add new information and then wedraw out of it something that accords with thepurpose for which we reflected’. Adding newinformation may take place as a solitary process, or itmay involve other people. The latter can lead to thedevelopment of a learning conversation wherediscussion may focus on learning experiences inwhich the learner reflects on some event or activity(Candy et al., 1985).

The learning experiences, which might feedreflections, are given consideration by Schön (1983).Schön moved the reader from thinking about theconcept of knowing-in-action to reflecting-in-action.Knowing-in-action relates to how people in daily lifeintuitively perform the actions of everyday life. Whensomeone reflects on the situations in which they areperforming, and on the know-how implicit in theirperformances, they are, in some cases, reflecting-in-action. Schön accepted that reflecting-in-action maynot always be possible, but that these argumentsadmit the possibility of reflecting-on-action, that is,looking back on an experience where reflecting-in-action was not possible. Pereira (1999) and Cowan(2002) give good examples of both reflection-in-actionand reflection-on-action.

Cowan (2002) identifies a third type of reflectionwhich he called reflection-for-action. Cowan explain-ed that this type of reflection may occur at the start ofa reflective process, where aspirations are beingdefined or problems are being identified with the hopeof finding a resolution. This type of reflection isanticipatory and establishes priorities to supportsubsequent learning.

Proposing a model for carrying out reflectivepractice, Cowan (2002) brought together the inter-pretation of reflection as given by Schön (whichevervariant), which Cowan considered to be open-endedactivity, with reflection (as interpreted by Kolb in hislearning cycle), which Cowan considered to be closed,as it is part of a sequence and as such, may act as abridge to cross between sequences. Cowan’s model,therefore, is one which incorporates reflection- for-,in-, and on-action.

Further refinement of reflective practice hasallowed Cowan to develop models for ‘analytical’ and‘evaluative’ reflection.

In analytical reflection, Cowan (2002) tookanalysis to be a cognitive process in which it is usefulto look for patterns and generalities. These then, helplearning concerning a particular experience. In evalua-tive reflection, Cowan addressed questions such as‘How well can I do it?’ or ‘Should I do it better?’

Self-assessment and peer review within areflective framework, is well illustrated by Cowan(2002). The example involved the generation oflearning contracts, which required students on aweekly basis to summarise their individual personallearning objectives for that week and to produce anoutline of the methods they proposed to use in orderto achieve those objectives. Each student had to askanother student to comment on their personalobjectives. Students were then required to considerthese comments carefully but did not necessarilyhave to agree with them. At the end of each week,each student was asked to produce something which


22 CHAPTER 3 GOING DEEPER

demonstrated what he or she had learnt. At the end ofterm, students self-assessed their work by:

• summarising the standards and criteria, whichthey had been striving to achieve;

• describing their performance in comparableterms; and

• reporting on the process of judgement by whichthey compared their performance with theircriteria and standards.

Asking students to make their assessmentjudgement in this way illustrates an importantcomponent of self-assessment, providing the equalemphasis on process awareness and development aswell as on the rigorous content coverage. Their finalmark was awarded against the outcome of theirsummative judgement, providing that all agreementsin the learning contract had been met. In this way,Cowan saw this example of self-assessment involving:

• the year long experience being predominantlyreflection-in-action;

• the end of term’s assessments being reflect-ion-on-action; and

• the end of term assessments, as they werecompleted, being reflection-for-action.

FORMATIVE ASSESSMENT AND SUMMATIVEASSESSMENT: A ROLE FOR SELF- AND PEER-ASSESSMENT

Formative assessmentSadler (1989) introduced the reader to formativeassessment by considering the adjective formative. Thisimplies the forming or moulding of something, usuallyto achieve a desired end. This is important to bear inmind, as often what is called formative assessment isnothing more than an arrangement of a set of multiplesummative assessment tasks, described as if theirmain function was feedback, but with a considerabledirect influence on the final outcome mark (Cowan2002). Formative assessment contributes to learning byshort-circuiting the randomness and inefficiency of trialand error learning. It is concerned with how judge-ments about the quality of student responses areshaping learning (Sadler, 1989).

Formative assessment is carried out in a seriesof specific stages. Students undertake an assessedassignment. They then receive formative feedback,feedback designed to inform them about their

performance, and the judgement of it. Learning takesplace and student performance is assessed again.Thus feedback is the key element in formative assess-ment, and feedback is often defined in terms of‘information about the gap between the actual leveland the reference level of a system parameter whichis used to alter the gap in some way’ (Ramaprasad,1983). Sadler stressed the active closing of the gap,rather than feedback being by nature for information.For learning to take place, the gap between thestudent performance before and after feedback andthe performance after feedback must close. Sadler(1989) argued that there are three conditions foreffective feedback. Students must be able to:

• monitor their own work;

• appreciate work of high quality; and

• judge (objectively) their product in comparison.

Furthermore, in keeping with good self- andpeer-assessment practice, Sadler stressed theimportance of ownership of a goal (the degree ofperformance or excellence achieved) as playing asignificant part in the voluntary regulation of perform-ance and the involvement of students in using multi-criterion judgements.

A study illustrating the implementation of aformative assessment exercise of histology posters isdescribed by Orsmond et al., (2004). There were anumber of stages involved in this study, the keycomponents of which were.

• Students constructed criteria for marking ahistology poster which had been made by astudent from a previous cohort. This was theexemplar poster.

• Students were then given two criteria by thetutors. Unknown to the students, the tutorshad constructed one ‘worthwhile’ criterionand one ‘ambiguous’ criterion. Studentsworking in pairs or trios were given copies ofboth criteria by tutors and were asked to markan exemplar poster.

• Tutors then marked the poster using bothcriteria.

• Tutors then discussed with students groups: (1)the criteria that the students had constructedearlier in the session, (2) the ‘worthwhile’criterion and ‘ambiguous’ one, (3) the markingprocess, (4) further developments regardingthe concept of marking criteria.


23

• Tutors listened to the student’s perceptions ofmarking criteria and attempted to clarify anymisconceptions.

• Tutors then discussed among themselves theirindividual discussions with student groups.

• Students and tutors wrote down agreed criteriaalong with definitions.

• A week later the tutors and students met againin the same pairs or trios. Students con-structed posters and marked the posters usingthe criteria that had been jointly constructedwith the tutor.

• Students self-assessed their own poster andpeer-assessed those of their colleagues.

In this study students were able to engage informative learning activities which required (1)discussions with their peers and tutors, (2) self-reflection, (3) self- and peer-assessment. Overall thestudents found the process very beneficial with themajority of the students responding in a positive way.

Complex learning can be more readilyaccomplished working with others, where, forexample, alternative interpretation of tasks orsituations is required (Boud, 2000). Working withothers strongly encourages a more formative learningenvironment, incorporating both formative assess-ment and formative feedback. Support by peers isseen as very important because of the autonomyexpected in higher education. There is good evidencethat students help each other, but are not seen asreplacements for staff (Drew, 2001).

For some, a problem for higher education isthat traditional forms of summative assessment arebeing stretched to cover learning outcomes that resistrobust, reliable and affordable summation (Knight,2002). Rather than live with the difficulties, Knight andYorke (2003) argued that greater use be made offormative assessment. Complex learning outcomesare also encountered where, through genuinediscussion, sub-groups of outcomes can begenerated as in claim-making. In making a claim,students ‘claim’ against the expectations set for theprogramme of study and justify these claims withevidence. Claim making encourages reflectivepractice, evaluation of learning and links in well withPDP (Knight and Yorke, 2003).

Summative assessment: the use of self-and peer-assessmentNot everyone perceives the problems of summativeassessment to be as extensive as Knight and Yorke. A

more pressing concern is that because of its extensiveuse, summative assessment may, in some way,suppress the use of formative assessment. Thus it isimportant to maintain a balance between the twoforms of assessment (Boud and Falchikov, 2004) andwhere possible integrate self- and peer-assessment.Taras (2001) advocated an interesting form of self-assessment with a summative end-point. Studentssubmit a summative piece of work, this is marked andgiven written feedback by the tutor. The work isreturned, but the mark withheld. The students workthrough the tutor feedback, through group/classdiscussions, and then self- and possibly peer-assessment. Students are then asked to:

• Judge their work against an agreed criteria.

• Explain how they would improve a comparablepiece of work in future.

• Grade their work.

Tutors collect the students’ comments andgrades. The tutors then feedback how well they thinkthe students have addressed the criteria, andprovide the tutor grade. The process of self-assessment is dependent on tutor and possible peerfeedback. The value of self-assessment with andwithout feedback provides an interesting insight intothe process of assessment and has been furtherdeveloped by Taras (2003).

A detailed study looking at summativeassessment in biosciences is reported by Butcher etal., (1995). Here both the process and product ofgroup projects were assessed. Overall six separateassessments of performance were made. The workis interesting for a variety of reasons. It discussedissues surrounding the arrival of a single studentassessment grade, with particular reference toassessment weightings. It is also a good example ofauthentic assessment practice, in that studentsundertook an assessment task that resembledassignments undertaken by professional biologists,in this case, solving an industrial problem withincertain resource constraints. The case study byBrennan et al., (Case Study 7) also has an emphasison such authentic assessment.

A way of balancing formative and summativeassessment is discussed by Nieweg (2004). Thispaper, relating to learning in physiotherapy,illustrates how self-assessment can be used inconjunction with summative assessments. It alsoprovides an opportunity to consider using form-atively assessed assignments in an authentic,realistic and meaningful way by involving externalclients to give feedback.



SELF-EVALUATION OR SELF-ASSESSMENT:CONSIDERING A RICH DIVERSITY OF APPROACHES

So far in this book we have considered self-assessment within fairly well defined parameters,perhaps too well defined. Klenowski (1995),considered self-evaluation, in terms of ‘the evaluationor judgement of the worth of one’s performance andthe identification of one’s strengths and weaknesswith the view to improving one’s learning outcomes.’Therefore, as Klenowski, explained, ‘self-evaluation isused in a broader sense than student self-assessment because it refers to ascribing value to thelearning experience: first, by the identification of thecriteria used; second, by indicating what is consideredmeritorious; and third, by outlining the implicationsfor future action.’ In other words, students see what isgood in their work and should come to know how tomake it better. Students use the outcomes ofdiscussions or perhaps reflections in their develop-ment and achieve higher learning outcomes. Self-evaluation is being used in ‘a formative context inmeeting a self-development learning function. It is a

process of identifying the value of the teaching andlearning experience for the individual student’.

For some, it may therefore be difficult to seehow self-assessment and self-evaluation differ.Cowan (2002) employed the term ‘evaluation’ todescribe a ‘process’ in which judgements are made bycomparing performances with criteria or standards.Cowan restricted the term ‘assessment’ to evaluationwhich concentrates on an outcome, in the form of agrade or mark or judgement, whether formative orsummative’. Klenowski, used the term ‘evaluation’,and generated a formal grade that was recorded.

This is more than idle banter over words.Searching for meaningful distinctions between termscan provide the opportunity for enquiry into some keyaspects of the assessment process. Claxton (1995),commenting on the Klenowski paper, considered

self-evaluation to open up some interesting questionsabout the nature and function of self-assessment.Claxton had broad sympathy with Klenowski, but hadtwo caveats. Firstly he stated ‘mere clarification ofexternal criteria of assessment does not developlearning acumen, though it may raise attainment’.Claxton seemed to be covering all options with theinclusion of the word ‘mere’. However, the point thatraising attainment is not necessarily linked todeveloping learning acumen is important. The othercaveat, that discussion of criteria needs to beconsidered on ‘the road towards developing an abilitythat is essentially intuitive’, is very important. Intuitivelearning is an often complex and may imply finelytailored understanding.

A comparison of the work of Klenowski, Claxton,Boud, Cowan and Sadler can be very helpful in allowingus to appreciate the rich diversity of thinking andapproaches taken towards students learning, whichmay have implication for those implementing self- andpeer-assessment. Just considering one aspect of theformative assessment process, can show rich diversityof thought, see Table 2 below.

SELF- AND PEER-ASSESSMENT: STUDENTS ANDTUTORS PERCEPTION

Students’ perceptionsStudent perceptions of self-assessment are illustra-ted well by Cowan et al., (1999) who studied how self-assessment can be used in the summative assess-ment of reflective journals and self-assessment. Thepaper is richly embroidered with quotes fromstudents, which readily allow the reader to share theirexperiences over a year of self-assessment. Forexample, in this quote you really feel the exactingdemands being made. ‘I knew instantly that self-assessment was going to be a problem for me. I justdid not know how I was going to devise a criterion tobegin with, let alone assess myself’. However, whilestudents continued to find the self-assessment


25

TABLE 2. A COMPARISON OF ATTITUDES BY KLENOWSKI, CLAXTON AND SADLER IN ONE ASPECTOF COMMUNICATING WITH STUDENTS IN A FORMATIVE SETTING

Klenowski Claxton Sadler

Perceptive, guided thoughtabout each other's work mayincrease 'self-awareness' orprogress in learning. Self-evaluation in relation toidentified criteria is best.

Self-assessment usingexternally specified criteria,can be irrelevant, even counterproductive to learning acumen.Self-evaluation is intuitive andhindered by checklist criteria.

Strictly speaking, all methodsof grading that emphasiseranking or comparison amongstudents are irrelevant forformative purposes.

challenging they were able to comment at the end‘self-assessment did make me realise that I need todevelop myself further into being more critical of mywork. I tend to rely too much on feedback rather thancritically evaluating myself. Self-assessment did giveme the opportunity to question this’. The influence ofpeers was also noted in a positive way ‘having tojustify why I thought some aspects of the journal wasreally good was perhaps more difficult than sayingwhy it was bad. It was useful having a colleaguecommenting on this — even if it was rather unnerving.It has been a very trusting relationship and the needfor confidentiality is essential’. Sometimes theperceptions are not always viewed in a positive light,even if the process does provide a better measure oflearning, such as this quote on alternativeassessment methods; ‘I think it tests you better,because it’s not just testing your memory, it’s testingyour knowledge of the subject. It’s all about ... beingable to interpret and put your own point of view. It’s abit unfortunate, really, isn’t it’ Sambell et al., (1997).In this Guide, comments on student’s perceptionsof peer-assessment are included in the case studyby Rushton et al., (Case Study 3).

Tutors’ perceptionsMaclellan (2001) carried out a study into assessmentfor learning to evaluate the different perceptions ofstudent and tutors. The tutors perceived the primarypurpose of assessment was to grade or rankstudents, but the more developmental purposeswere not discounted. However, the importanceplaced on the developmental formative role is notinternally consistent with other views endorsed bystaff. For example.

• Assessment neither took place at the start, norcould students be assessed when they feltready.

• Self- and peer-assessment were infrequent.

Nevertheless, declarative knowledge per sewas not the sole purpose and functional knowledgesuch as, formulating ideas was assessed. However,the extent to which assessment genuinely focusedon students’ capacity to apply, transform orevaluate the relevance of declarative knowledge indifferent situations could be viewed as question-able, when considering the processes throughwhich assessment information was gathered.

SUMMARY

Very few people have a neutral view on assessment.This is because it is such an emotive issue; asindicated by many of the student quotes throughoutthe book.

In self- and peer-assessment, students mayneed to explore, at different times, rather complexemotions about themselves and what it is that theymay become. The issues, which increasingly tutorsare asking students to engage with are huge; and lotsof students struggle to ‘get their heads round them’.Self- and peer-assessment provide students, as wellas tutors the opportunity to ‘touch base’. At the startof their learning journey students ask their tutorsquestions like ‘is this what you mean?’ By the end ofthe journey, students are asking questions ofthemselves and seeking self justification for what theyhave learned.

The old adage ‘you can take a horse to water but youcan’t make it drink’ seems lost on many in highereducation who spend their time ‘teaching’ studentsrather than allowing them to ‘learn’. As a result, a lotof time and money is spent assessing superficiallearning. Early in Chapter 1, reference was made toRogers’ perception of the goals of education (Rogers,2003). These goals were strongly student-centred,and as Rogers admitted ‘even in our own culturethese are functional goals of very few educators’. ForRogers, writing in the early 1950’s, educationappeared to be operationally based on theassumption ‘you can’t trust the student… the teacherneeds to supply everything’. However, Rogerssteadfastly believed you could trust the student tolearn in a way which will maintain or enhance self.Boud and Falchikov (2004) considered assessmentand write of their dismay at ‘practice inconsistentwith research in higher education and indeedinstitutional policy’.

Sadly, for many, very little seems to havechanged in the intervening years, and we need tobegin to question why this is. For those who havealways had a strong interest in student-centredlearning, the trend of the self remaining central tolearning and assessment persists. This bookdocuments case studies and educational researchby people with a genuine concern for meaningfulassessment, both of and for learning. We continue toseek some reform of many established, but outdatedpractices, as Rogers did; and we need to be strongadvocates for student-centred assessment, whichgoes beyond the superficial.



CLOSING THOUGHTS


27

BIOSCIENCE CASE STUDIES

SThe following section contains a collection of seven bioscience case studies. All the case studies have beenwritten by bioscientists who have implemented self- and/or peer-assessment into their own teaching. The casestudies are organised around common headings ('Background and rationale', 'Advice', 'Troubleshooting', 'Doesit work?' and 'Further Developments'), but each study reflects the author's individual style and preference.

CASE STUDY 1THE EFFECT OF MARKING CRITERIA AND EXEMPLARS ON STUDENTS’ LEARNINGDURING PEER- AND SELF-ASSESSMENT OF SCIENTIFIC POSTERSStephen Merry & Paul Orsmond Faculty of Health and Sciences, Staffordshire University,Stoke-on-Trent ST4 2DE. Email: [email protected]

CASE STUDY 2ON-LINE CALIBRATED PEER-ASSESSMENT — STUDENT LEARNING BY MARKING ASSIGNMENTSVictor Kuri School of Biological Sciences at Seale Hayne (Food Technology), Seale-Hayne Campus,University of Plymouth, Newton Abbot TQ12 6NQ. Email: [email protected]

CASE STUDY 3PEER-ASSESSMENT OF SCIENTIFIC POSTERS — THE LEAGUE FIXTURE APPROACHBrian Rushton School of Environmental Science, University of Ulster,Coleraine, Northern Ireland, BT52 1SA. Email: [email protected]

CASE STUDY 4PEER-ASSESSMENT OF GROUP WORK IN A LARGE CLASS —DEVELOPMENT OF A STAFF AND STUDENT FRIENDLY SYSTEMBarbara Cogdell, Andrea Brown & Ailsa Campbell Institute of Biomedical and Life Sciences,University of Glasgow, Glasgow G12 8QQ. Email: [email protected]

CASE STUDY 5PEER-ASSESSMENT OF PRACTICAL WRITE-UPS USING AN EXPLICIT MARKING SCHEDULEIan Hughes School of Biomedical Sciences, University of Leeds, Leeds LS2 9JT.Email: [email protected]

CASE STUDY 6WRITING AND REVIEWING AN ARTICLE FOR A SCIENTIFIC MAGAZINE —A PEER/SELF-ASSESSMENT EXERCISERob Reed Division of Biomedical Sciences, Northumbria University,Newcastle-upon-Tyne NE1 8ST. Email: [email protected]

CASE STUDY 7PEER-ASSESSED PROBLEM-BASED CASE STUDIESCharles Brennan, Elizabeth Folland, Rick Preston & Nicola BlatchfordSchool of Biological Sciences at Seale Hayne (Food Technology), Seale-Hayne Campus,University of Plymouth, Newton Abbot TQ12 6NQ. Email: [email protected]

It is hoped that these cases studies will provide guidance, inspiration, as well as practical advice on how toimplement self- and/or peer-assessment in the biosciences. There is also a accompanying website to thisguide (http://www.bioscience.heacademy.ac.uk/TeachingGuides/). The website contains a further practicalmaterial to aid the reader in implementing self- and/or peer-assessment. The site includes expanded versionsof the case studies, further bioscience case studies, explicit marking schedules and criteria to download, aswell as video streams of peer-assessment.

http://www.bioscience.heacademy.ac.uk/TeachingGuides/

mailto:[email protected]








28 CASE STUDIES

BACKGROUND AND RATIONALE

The authors of this report are practicing lecturerswith an interest in the influence that assessmentpractices have on the way that students learn. This“case study” is the combination of four studiesinvestigating student and tutor perceptions of postermarking criteria at Level 1 undergraduate moduleswithin the general field of Biology. Self- and peer-assessment exercises of the students’ completedposters together with organised, but informal,formative feedback sessions were used to providedata concerning students’ and tutors’ perceptions ofmarking criteria.

The precise learning outcomes of thecomponent individual studies differed, but overall itcan be stated that, ‘at the end of their participationstudents should be able to’:

• explain the meaning of specific markingcriteria in a professional biological context;

• provide appropriate formative feedback tocolleagues concerning their performance;

• engage meaningfully in the process of peerreview as used by professional biologists;

• reflect more on the assessment process aspart of their own learning and thereby enhancetheir learning.

‘HOW TO DO IT’

The formats of the four studies were similar, but not

identical. The approach described below is acomposite which reflects how we would now run sucha study.

Stage 1(4–6 weeks before the poster assessment exercise)Students were informed that:

• they were required to make a scientific poster,the date of the poster assessment exercise andthe topic of the poster;

• posters are a recognised format in which sci-entific researchers present their results i.e. theassessment was relevant;

• they were required to supply particular mater-ials (i.e. paper headings, adhesives etc) inorder to construct their poster and the size ofthe poster boards;

• they would work in groups of approximately fiveto discuss the marking criteria, but they wouldbe required to produce individual posters;

• more details would be provided at a later date.

Stage 2(3–4 weeks before the poster assessment exercise)Either students were informed what the markingcriteria were (if they were tutor provided) and werethen allowed time (approximately 30 minutes) todiscuss the meaning of the marking criteria in theirgroups with tutors circulating among the groups tocontribute to the discussions.

Or students were allowed time (approximately45 minutes) to work in their groups to both generate,discuss and refine their own poster marking criteriaand agree them with tutors circulating among thegroups during this process.

Students were then informed that:

• they would be required to peer-assess theposters of others and self-assess their ownposters using the marking criteria they had justdiscussed and that tutors would also assessthe posters using the same marking criteria;

• anonymous peer review was a process utilisedby professional scientists which was funda-mental to establishing the credibility ofscientific publications i.e. they were engagingin genuine professional practice;

• they should regard the self- and peer-assess-ment activity as a vehicle for developing

The effect ofmarking criteriaand exemplars onstudents’ learningduring peer- andself-assessment ofscientific postersSTEPHEN MERRY & PAUL ORSMOND

1


29

specific skills such as self reflection andobjective judgement required by professionalbiologists;

• their posters were to be presented anony-mously to reduce any bias in the assessment;

• their self- and peer-assessment would contri-bute to the overall grades awarded for theexercise and that tutors were interested in thequality of comments made by students inaddition to their accuracy of their markingcompared to that of tutors;

• more information would be provided at alater date.

Stage 3(1–2 weeks before the poster assessment exercise)

• Students were given written information con-cerning the meaning of the individual markingcriteria. This information was influenced by thediscussions that had previously taken placebetween tutors and students. It should pointout both the meaning of the criteria and themisconceptions which some students seemedto have.

• Students were also given a copy of the markingform they would be required to use for the self-and peer-assessment of their posters and itsuse was discussed with them, paying particularattention to they types and usefulness of thefeedback comments they might provide to theirpeers.

• Students were given the opportunity to viewexemplar posters and to discuss them in theirgroups and with circulating tutors. They shoulddecide what feedback they would give to theauthor of the poster and what grades theywould award for each marking criterion.

• If student-derived criteria are being used tomark the posters, students should be given, indiscussion with tutors, the opportunity to refinethe criteria; although such changes should beagreed and discussed with the whole classsince changes to the written informationprovided may be required.

• Students were reminded of the date and time ofthe poster construction and assessment exer-cise together with the materials they wouldneed to bring to the session in order to construct

their poster, and the time they have available(i.e. 30 minutes) for poster construction.

Stage 4(The poster assessment exercise)In an initial plenary session students were informed:

• they will be randomly allocated to two rooms;

• they will be given coloured stickers to attach totheir posters;

• they will be given 30 minutes to construct theirposters before the start of the self- and peer-assessment exercise;

• they will be required to self-assess their ownposters and then move to the other roomwhere they are required to peer-assess all theposters having the same coloured sticker astheir own;

• their self- and peer-assessment markingshould be independent, i.e. they should notdiscuss their marks and comments with otherstudents; although tutors were available toprovide guidance regarding the usage of themarking forms;

• tutors were interested in the quality of feed-back comments as much as the gradesawarded;

• in their poster construction they should ensurethat only their student number appears on theposter NOT their name.

At the end of the plenary session students wereallocated to their rooms and given their individualcoloured stickers to attach to their completed posters.

• The format was as described in the intro-ductory plenary;

• Packs of marking forms (one form for eachposter to be assessed) were made available tostudents at the commencement of the self-and peer-assessment exercise;

• At the end of the session tutors collected in thecompleted marking forms and elicited anyinformal feedback on their experience of theassessment process from individual studentsto enable the procedures to be refined forsubsequent cohorts.


30 CASE STUDIES

ADVICE ON USING THIS APPROACH

Tutor discussion with students is the key to thesuccess of the exercise. Tutor discussion shouldprovide to students a) feedback regarding theirinterpretation and use of marking criteria and b)reassurance that they do have the ability to judge thescientific quality of a poster without the backgroundknowledge necessary to judge the accuracy of thefactual detail provided within it.

Posters produced by previous cohorts ofstudents are a good source or exemplars. Studentsshould be given the opportunity to view posters ofdiffering styles and quality. It can help some studentsrealise that attractive posters may, in some cases,have poor scientific depth.

If a sequential allocation of students to roomsand to peer-assessment groups (i.e. a sequentialallocation of different coloured stickers) is adoptedand these are allocated to students in turn as theyleave the plenary session this helps ensure thatfriendship groups (who are likely to leave together)are separated during the assessment process.

In a 1-hour self- and peer-assessmentexercise students should be asked to grade andprovide feedback on a maximum of five posters;including their own. This is to enable them sufficienttime to write meaningful feedback comments. Thenumber of peer-assessment groups, i.e. the numberof different coloured stickers used, should reflect this.

Self- and peer-assessment are skills thatimprove with practice. Furthermore, informalfeedback from students has indicated that they aremore willing to engage with the process at a deeperlevel if they meet it on a second occasion. If possible,repeated self- and peer-assessment exercises shouldbe built into the curriculum rather than beingdelivered as single isolated events.

TROUBLESHOOTING

In some instances students were unwilling to awardlow grades, even if they thought they were deserved,to posters that they recognised as being those of theirclose colleagues. Furthermore, in discussionsapproximately one month after the completion of theexercise individual students have informed tutors thatthey felt other students (to whom they did award a lowgrade) were behaving differently towards them. Toaddress this, tutors need to ensure posters are asanonymous as possible and that friendship groups aredistributed into different peer-assessment groups.

Students do find the process challenging. Thiscan be a surprise if they have initially a superficialview of presenting information in poster format.

Furthermore, some students feel they are not able tojudge the scientific merit of posters whose topic is notstrictly the same as theirs. They need reassuranceand guidance that they can judge the scientific meritof a poster without being able to judge the accuracy ofthe detail present in the content.

Finally some students can treat the process ina rather cavalier fashion. Emphasising at the startthat they are engaging in a practice which is employedby professional biologists to establish the credibility oftheir work helps to dispel this.

DOES IT WORK?

Both formal questionnaires and informal feedbackfrom students have indicated that self- and peer-assessment exercises caused students to reflectmore on the marking criteria and their learning(Orsmond et al., 1996; Orsmond et al., 1997; Orsmondet al., 2000; Orsmond et al., 2002; Orsmond et al.,2004). To this extent the approach does “work”.

The ability of students to mark in an identicalfashion to tutors should not be the sole criteria ofsuccess of self- and peer-assessment , but it canprovide information as to the nature of the learningthat is taking place. Our initial studies (Orsmond et al.,1996; Orsmond et al., 1997) demonstrated an overallagreement between student and tutor grades (r2 = 0.7)comparable to that of other studies (Hughes andLarge, 1993 and Stefani, 1992) with the agreementbeing greater for peer-assessment than for self-assessment. Consideration of the overall mark does,however, mask variations between tutor and studentwith regard to individual marking criteria. Forexample, students over-marked, compared to tutorsfor the criteria “visually effective” and “helpful level ofdetail”, but under-marked for the criterion “clear andjustified conclusion”. The implication is that somestudents had written a clear and justified conclusion,but did not realise that they had done so. Thenecessity for dialogue with students concerningindividual criteria was shown by these findings.

An interesting, and unexpected, outcome wasthat our studies, based on comparison of tutor andstudent grades, have indicated that the nature of thelearning that has taken place differs dependent onwhether the marking criteria are tutor-provided orstudent-derived (Orsmond et al., 2000). The use ofstudent-derived criteria might be expected tocircumvent discrepancies between tutors’ andstudents’ marks for individual criteria since tutors,with their greater experience of interpreting markingcriteria, would be expected to more readilyunderstand student-derived marking criteria than,possibly, students understand marking criteria


31

provided by tutors. In our hands the outcome of usingstudent-derived marking criteria was that students,although having ownership of marking criteria theyhad constructed themselves, were less able todiscriminate between their own individual markingcriteria than between those provided by tutors.Student groups either over-marked or under-markedall their marking criteria compared to tutors such thatoverall agreement between students’ and tutors’marks was not enhanced. It may be that the act ofconstructing their own marking criteria causedstudents to view their posters in a more holisticfashion. An alternative interpretation of the finding isthat students were able to interpret their markingcriteria, but had a poor conception of the subjectstandards, i.e. both students and tutors knew what,for example, the marking criterion “self-explanatory”meant, but, despite the dialogue, they retaineddifferent conceptions of how self-explanatory theposter should be to achieve a particular grade.

Our final published study (Orsmond et al., 2002)indicated that the use of exemplars was able tolargely overcome discrepancies between gradesawarded by students and tutors for student-derivedmarking criteria. The exemplars were posters pro-duced by a previous cohort of students and served asa focus for discussion and application of the markingcriteria. In addition to improving accuracy of markingfor individual criteria, feedback from studentsindicated that the use of exemplars can help students’learning such that higher quality learning outcomes,including reflection, are achieved; althoughexemplars may not necessarily help students in theprocess of poster construction. A recent study(Orsmond et al., 2004) has revealed that peer-assessing students were less able than tutors to writeconstructive feedback comments to the posterauthors. Students’ feedback comments concernedprimarily the quality of the presentation of materialwith little actual mention as to whether the discussedmarking criteria had been met. Tutors’ comments,alternatively, concerned primarily the nature and useof the scientific content of the poster in the context ofthe marking criteria. A possible explanation for this isthat students may focus, when constructing a poster,on the poster itself (i.e. the product of their work)whereas tutors may regard the poster simply as ameans to enable students to demonstrate theunderstanding of science they have developed (i.e. toshow the process that they have undergone).

In summary, the strengths of the approach arethat it causes students to reflect more on their workand their learning, but for this to happen, carefulplanning is required together with the allocation ofclass time for the activities.

FURTHER DEVELOPMENTS

The authors are currently investigating: Howstudents’ perceptions of marking criteria changeduring the course of the six week poster design andconstruction exercise; the type of distractions (i.e.student self-derived individual criteria that aredistinct from the agreed marking criteria) whichinfluence students’ poster design and construction aswell as how students use the feedback provided bytutors to enhance their learning.


One of the first things that markers have to do toevaluate work is to agree a set of standards.Somehow lecturers have to develop an understand-ing of what is a good assignment, what is average andwhat is poor. When I collect a pile of assignments formarking in an area that I have not set before, the firstthing that I do is to try to find some of the extremes,and ‘calibrate’ my marking scheme.

The use of ‘calibrated’ exemplars can helpstudents become competent at peer review andunderstand what makes a good (and bad) assign-ment. I use the web-based Calibrated Peer Review(CPR) system which was developed on a science-based model of peer review (http://cpr.molsci.ucla.edu/). The system is anonymous and could be usedon-line or with printouts during a timed session.

‘HOW TO DO IT’

Prepare an assignment brief, which ideally containsguidelines to set the criteria to which the work isgoing to be marked. It is best suited for text-based

Online calibratedpeer-assessment —student learningby markingassignmentsVICTOR KURI

2

http://cpr.molsci.ucla.edu/


32 CASE STUDIES

assignments, short and well structured. Following preparation and submission of the

text the student proceeds through calibration, peer-assessment, self-assessment and feedback, and results.

CalibrationEach student is presented with an assessmentquestionnaire and one script at the time. They have touse the questionnaire (assessment schedule) toevaluate and mark the script. At some points, they areencouraged to provide feedback. They do this for threescripts in a random order, which were prepared by theinstructor and are standardised to be of low, mediumand high quality. Feedback is provided to the studentsto verify how close they matched the calibrationscripts. There is the facility for students to re-take thecalibration to improve their marking proficiency.

ReviewEach student is presented with a script from one oftheir peers, randomly selected and coded to keep itanonymous. They have to evaluate it and mark itfollowing the questionnaire, where they also providefeedback. This is done for three students (the work ofthis assessor will be correspondingly marked by threerandomly selected reviewers)

Self-assessmentEach student is given the opportunity to mark theirown script following the same criteria. This mark willbe part of the overall mark.

Feedback and resultsThe feedback information is made available to eachstudent (keeping the markers anonymous) and acomposite mark is computed to reflect the effort ofthe participants, considering that marking could betime consuming and challenging.

By the time the students finish they shouldhave understood what was required in the assign-ment, marked seven scripts and have receivedfeedback on their understanding of the assessmentsystem, the requirements and their own compliance.This is a formative exercise which allows students theopportunity to understand and explore the peer-assessment process.

This system had been used with final year BScand MSc students for a range of assignments,including a case study, short practical reports,discussion and conclusions of practicals, virtual posterdisplays and a reflective assignment exploring issuesof food ethics. The briefing may involve instructions forthe students to carry out an activity using a range ofsoftware, calculations, virtual (or laboratory) experi-ments, etc. Students have subsequently to write theoutcome as a text report.


Setting up the method can be time consuming, butonce the assignments are designed, the system iseasy to manage and the assignments can beadministered to large groups with minimum effort.The on-line system does not work with files of webpages by itself, but it is possible to set up a repositoryof files or webpages (i.e student portal in theuniversity intranet, or internet) and ask the studentsto input only the weblink to their work or a code to thefile previously up-loaded by the instructor.

TROUBLESHOOTING

The idea that the lecturer was not marking theassignment was alien to some of the students whofelt uneasy because their peers were going to markthem. Others felt that they were not capable ofmarking assignments. A briefing session wasintroduced to manage students expectations and tomotivate positive participation. Detailed instructionsand a tutorial were set up to help students withlimited IT skills.

One potential problem with the on-line CPRsystem is that the students obtain marks in rangesatypical for the group or university marking scheme.The marks can easily be normalised or the system re-set to provide different weighting for the text and eachone of the tasks. Also, the threshold levels to givemarks after successful completion of each task couldbe modified; i.e. if the self-assessment is less than 1.5points from the reviewer’s average mark (in a scalefrom 0–10), then 10 points are awarded, if it is >1.5and >=2.5, 5 points but if it differs more that 2.5 points,then no points are awarded).

DOES IT WORK?

The overall impact on students can be summarised as:

• Students realise that there are marking sch-emes and that these can help in achievinghigher marks. Marking schemes also helpthem to focus their effort in further coursework.

• Students experience marking their peers andproviding and receiving feedback to and fromtheir peers.

• Some students enjoy being empowered toassess coursework and find it interesting andthe responsibility challenging.


33

• The students learn by marking their peers work.

• Once students were reassured about themechanics of the calibrated peer-assessmentthey understood the relevance of peer review.

• Some students welcomed the change, butmost perceived the calibration and reviewingprocess just as extra work.

Aside from the improved student learning, oneof the key benefits for staff is reduced workload inproviding adequate and timely feedback to students.


Within the Biology degree at the University of Ulster,posters are used in many modules and training isprovided in a Year One Transferable Skills module onthe elements that go to make up a successful poster.One of the learning outcomes of the TransferableSkills module is to develop the students’ criticalabilities and peer-assessment of the posters was thevehicle used for this. Essentially, each student isasked to assess each poster independently usingmarking criteria that had been discussed with theclass beforehand and the marks amalgamated andoverall marks awarded that included an element fromthe tutor.

However, this did not prove entirely satis-factory. The workload for an individual student washigh (typically between 10 and 18 posters) and alsothe students found it difficult marking to an absolutescale. The instructions required that the elements ofthe assessment be scored on a 0–10 scale with 4being a pass mark. The concept of what constituted afail was difficult and even the worst posters weregiven good pass marks.

It was important that the students saw,reviewed and criticised the work of their peers andtherefore an alternative assessment strategy wasintroduced six years ago and has remained littlechanged. It is now used in two other modules as well,another Year One module, Biodiversity and a Year Twomodule, Biological Techniques and Analysis.

‘HOW TO DO IT’

The method currently used in the Transferable Skillsmodule can be summarised as follows:

1. The Year One cohort usually consists of approxi-mately 30 students but this number has been as highas 60+ in recent years. In week five of semester one,the students receive two sessions on the constructionof posters — these focus on presentation and content.At the same time they are given the poster titlestogether with a few starter references. In previousyears these have largely been related to popularissues in biology but this year, in anticipation of theintroduction of Personal Development Planning, theemphasis was switched to careers and placements.Students form groups of three or four and work ontheir posters over the next few weeks and these arethen displayed outside the main teaching laboratoriesduring week eleven.

2. The poster in the Transferable Skills module isworth 10%. In other modules it may be higher (forexample, in the Biodiversity module it is worth 15%with an associated seminar and log book of theprocess being worth a further 10%). A small numberof marks are allocated for how conscientiouslystudents have marked the posters.

3. Bearing in mind the difficulties outlined abovewhen all students marked all posters, the processnow involves:

a. Each student is given a number of markingsheets. The number of sheets depends on thenumber of groups and the number of studentsbut is usually two or three.

b. Each marking sheet bears the titles (ornumbers) of two of the posters on display butnot including the poster of the group to whichthe student belongs.

c. The rest of the marking sheet has a series ofcriteria divided into two categories, present-ation and content; the student is expected toview the two posters and to make positive and

Peer-assessment ofscientific posters —the league fixtureapproachBRIAN RUSHTON

3


34 CASE STUDIES

negative comments under each heading foreach of the two posters. The comments do nottranslate into a numeric score — this waswhere the students found real difficulty whenthey had to score the assessment criteria on a0–10 basis.

d. Instead, after they have made their writtencomments, they have to state which poster isthe best and to justify their decision on thebasis of the individual comments they havemade. This justification is no more than two tothree sentences.

e. They are allowed to state that the two postersare equally good (or bad) if they really find itdifficult to decide between them but they arestrongly encouraged to ‘find a winner’.

4. The marking sheets are designed like the games ina series of sports fixtures so that each poster ‘plays’every other poster — the number of groups and thenumber of students will allow each poster to ‘play’every other poster at least twice. For example if thereare 40 students in groups of four there would be tengroups. For the whole ‘fixture list’ there would be 90group comparisons (or ‘games’) with each studentbeing responsible for two or three comparisons.

5. The ‘winner’ of a comparison is allocated twopoints, the loser none — with ‘drawn’ comparisonsbeing allocated a point each (I haven’t introduced thethree points for a win and onepoint each for a draw system!)and the points totalled foreach poster.

6. The posters are then rankedon the basis of the number ofpoints awarded. At this stage itmay be necessary to modifythe points total if a student orstudents have failed to returnthe mark sheets — there areusually two or three studentswho opt out. Simply calculatingthe average number of ‘points’awarded per ‘game’ is a simplesolution to this problem.

7. I also mark the postersusing my own assessmentcriteria and my marks and thestudents’ points totals areamalgamated to give a finalgrade. I have experimented

with a number of ways of doing this and the methodcurrently used is to rank my marks and those of thestudents separately and add the two ranks togetherand allocate a grade and a percentage mark on theoverall rank. Thus, the highest ranking poster wouldget an A and a mark of (say) 85%. Generally, postersdo not fail!

DOES IT WORK?

Overall, are students able to produce a sensibleassessment of the posters? In Figure 1, the marksderived from the tutor (a percentage score) areplotted against the overall ‘points’ score of the class(adjusted for missing students) — the correlationcoefficient is 0.800 (df = 13, p < 0.01) — suggestingthat overall peer-assessment is remarkably con-sistent with the tutor’s marks. Where there weresignificant discrepancies between tutor marks andthose of the students this was usually for posters thathad excellent presentation and poor content; in thestudents’ minds the visual impact clearly outweighedthe scientific content.

Correlation between tutor marks and thosegiven by students are often low — several examples,drawn from a number of studies spanning a widerange of subject areas are discussed in Griffiths,Houston and Lazenbatt (1995). However, the usualapproach adopted is for students to allocate marks onan absolute scale and this may explain the pooragreement compared with that reported here.

Figure 1. Peer- and tutuor-assessment of scientific posters(correlation coefficient r=0.800, p<0.01); data for one year, 2002

40

50

40

30

20

10

0

Peer

ass

essm

ent (

max

imum

ach

ieva

ble

scor

e =

56)

Tutor marks (%)

50 60 70 80 90 100


35

The students find this method of ‘marking’much easier to cope with as they are simply making ajudgement on which piece of work is best; they are nottrying to use a numerical scale or mark against anabsolute scale. This came out clearly in the moduleevaluation. It also means that they are focussing on asmaller number of posters and therefore more likelyto learn and remember the content. One addedadvantage is that the mark sheets can be useddirectly as feedback to the groups.

WHAT DO THE STUDENTS THINK?

There is little doubt that peer-assessment is avaluable experience and is appreciated by students. Ina previous peer tutoring and assessment exercise(Cook and Rushton, 1995) where Year Two studentstaught information technology skills (MS Word andExcel) to Year One students and then assessed them,the comments of the student tutors were verysupportive of peer-assessment:

“Showed me how lazy and careless people could be withtheir work.”

“Makes me reconsider and reassess my own work andthe way I do it.”

“It taught me ... how much better and assignment canlook and read when more time is spent on it.”

It would seem therefore to be a worthwhileexercise. However, it should not be seen as just analternative to tutor marked assignments but shouldhave clear, non-assessment outcomes — in this case,the development of critical faculties. As Biggs (1999)points out “Peer-assessment [is] not so much anassessment device, but a teaching-learning device.”

ACCOMPANYING MATERIAL

The accompanying website to this guide(http://www.heabioscience.academy.ac.uk/TeachingGuides/) contains an extended version of this casestudy and the following additional material:

• notes on the assessment of posters;

• poster marking sheets.


The first year biology course at the University ofGlasgow is divided into two modules. Part of theassessment (20%) for the module in the second half ofthe year is a “Lifestyle Assignment”. The subjectspecific aims are to investigate and evaluate thelifestyles of (a) species other than humans and (b)humans in other parts of the planet. A portion of theassessment is individual written work, but themajority of the marks are for the group work elementof the Assignment.

There are two tasks for the group work, adebate and the manufacture of a poster. The debate isbased on Darwin’s dilemma. The students arerequired to argue the case for eliminating a species oftheir choice whose lifestyle is too damaging to theplanet. Then they also argue the case for thepreservation of another species chosen by anothergroup. The second task is to produce a poster whichcompares the lifestyle of people in Britain with that ofpeople in another country.

There are between 600 and 700 students takingthe module. They are divided into 14 laboratoryclasses with roughly 48 students in each. Each of thelab classes is further divided into six groups of eightstudents — a total of 84 groups. The students havealready worked together in the lab during the previoussemester as they are always required to sit in thesame lab position and they have already participated ina group discussion exercise. The groups meet both inscheduled lab sessions and in their own study time sothat they can research their topics. Though thescheduled lab sessions are run by members of staff,the staff cannot monitor what happens when thestudents meet outside their lab sessions. The groupsare encouraged to monitor themselves. Therefore they

Peer-assessmentof group work in alarge class —development of astaff and studentfriendly systemBARBARA COGDELL, ANDREA BROWN& AILSA CAMPBELL

4

http://www.heabioscience.academy.ac.uk/TeachingGuides/


36 CASE STUDIES

are required to elect a group leader and he/she isasked to make notes of who attended the sessions andwho did what within the group.

The Lifestyle Assignment replaced a previousgroup work activity which was based on the theme ofAIDS. There were continuous complaints that staff didnot assess the group work that was required for thedebates. Also group members did not like carryingnon-contributors. During the last couple of years ofthe AIDS project, this resulted in a high proportion ofthe students failing to contribute to the debates.

It was decided in the Lifestyle Assignment tomark the group work to ensure motivation. It was alsofelt to be important to introduce a method ofdistinguishing individual contributions, i.e. tointroduce peer-assessment. The method of peer-assessment has evolved over the three years of theAssignment’s existence. Part of this has been enabledby the availability of appropriate technology.

Before the introduction of this peer-assess-ment to our Level 1 course our only experience ofpeer-assessment had been in a Level 2 Biologymodule. This had involved a much smaller number ofstudents, 140 versus 650. Although the method ofpeer-assessment had been very successful it wasconsidered unsuitable to be scaled up to a largergroup. Therefore we looked for a system requiringless administrative time.

YEAR 1 — KEEP THE ADMIN SIMPLE

Two members of staff gave each group a mark out of100 for their debate and poster. The mark wasmultiplied by the number of members in the group.The group was told their total marks and they thenhad to divide the marks between themselves. So if thegroup had eight members and they were given a markof 60, this gives a total mark for the group of 480. Ifthey decided that they had all worked equally hardthey could each get a final mark of 60% for theproject. However if they decided two members of thegroup had worked particularly hard they could havemore marks and if one person had done nothing theycould agree to give that person 0. This might result intwo members of the group getting 90%, five membersgetting 60% and one getting 0. A constraint was putthat nobody could have over 100%.

The students allocated their marks together ina group in a scheduled lab session. Each group wasgiven a single form with the full names and matri-culation numbers of each of the members and aspace to write their marks. At the end of the sessionthe lab leader handed in the completed list of marksas communally agreed. Many groups agreed to sharethe marks equally amongst themselves.

This scheme was fairly simple to run as therewas only one sheet of marks per group for staff toenter into the assessment spreadsheet. Checks weremade to ensure that the students had made correctcalculations. Any queries could be sorted byconsultation with the group leader. Students awardedzero by their group were investigated by staff forextenuating circumstances such as illness.

However the students did not like this schemeat all. They did not like hammering out the marks in agroup setting. They did not like giving low marks tocolleagues face to face. Consequently non-contributors would get the same marks as everyoneelse and the rest of the group would feel resentful.Alternatively the group would mark a member downand this person would complain vociferously. In theworst cases groups split into two or three factions(this only happened on two or three occasions).

It is always to be expected that some groupswill be dysfunctional. However with the large numberof groups involved, and as the mark counts towardstheir final module mark, it is unrealistic to tell thegroups that they should sort things out by themselves.The students have to be given marks, so this schemeresulted in a lot of extra work for the staff trying tomonitor these problems.

YEAR 2 — MAKE THE MARKING CONFIDENTIALAND AUTOMATE THE MARKS CALCULATION

In the second year of the Lifestyle Assignment thedepartment was fortunate to acquire an IntelligentCharacter Recognition (ICR) system. This machinewill read forms with text entries. With the use of thistechnology it became feasible to get each student tosubmit a form with marks for each of the othermembers of their group. The forms are readautomatically and marks entered into a spreadsheet.Then the subsequent calculations can be madeautomatically. Using individual forms meant we couldchange the peer-assessment protocol so that thestudents could give their marks for the othermembers of the group confidentially.

Each student was given a hard copy form withtheir name and matriculation number at the top.Below was a table with the names and matriculationnumbers of the members of their group, not includingthemselves. The forms were generated using the mailmerge function of Microsoft Word and Excel. Extraspaces were provided in case an extra student hadjoined the group without the teaching staff’sknowledge. This could happen if a student was absentwhen the groups were formed or had completelyfailed to get on in an original group.

As before the debate and poster together were


37

given a mark out 100. Again this mark was multipliedby the number of students to give the total groupmarks. The students were asked to enter a mark outof 10 on their forms for each of their colleagues in thegroup. The forms were read by the machine and anaverage peer mark (out of 10) was calculated for eachstudent. All the average marks for the whole groupwere added together to give a sum of peer marks forthe whole group. This was then used to calculate theproportion of peer marks that each student hadobtained. The final mark for each student was thencalculated as this proportion of the total group marks.Although it is possible to get over 100% with thisformula we will cap any one student’s mark to thismaximum. So far this situation has not arisen.

A major advantage of this scheme is that ittakes into account whether the students are harsh orlenient markers.

Obviously the system will not work if studentsfail to return their forms. Consequently the studentswere told that they would get no marks for theirproject if they failed to hand the forms in — there wasa very high return rate of forms.

The students were much happier with thisscheme. The students appreciated being able toreward hard work and penalise freeloaders. Therewere favourable reports from both the end of moduleevaluation questionnaires and the staff-studentcommittee meeting. Some of the students with lowgrades complained but because the group leadershad been instructed to keep attendance registers itwas relatively easy to point out to them that they hadcontributed very little and they usually agreed withoutfurther complaint.

This second scheme solved the problems as faras the students were concerned, but there was still amajor administrative problem for us relating to thereading of the forms. Although the ICR system wasvery efficient it relied on the students using legiblescript and filling all the boxes in correctly. Inparticular problems occurred when a student failed togive an absent student 0 rather than leave the formblank. Each time the forms were illegible or filled inwrongly, they had to be checked by the operator. Withthe large numbers of students involved this becamevery onerous.

YEAR 3 — MOVE THE ADMIN ONLINE

As a result of the problems we have changed thesystem again this year. This time the students arerequired to enter their marks for their colleaguesusing a web-based form. The web forms have built invalidation so that they cannot be submitted with anyblank fields. Each student is sent an email giving

them a unique URL code which has been generatedfrom their matriculation number and name. This URLgives them access to their own individual websitewhich has a web form with a list of their other groupmembers and spaces to enter their marks.

The system is currently working well. It is import-ant to emphasise that we have only been able to copewith running a successful peer-assessment schemefor such a large class, because we have had the assist-ance of a dedicated IT specialist and suitable tech-nology. The programming required for generating theweb forms and using Excel to calculate the marks isnot extremely advanced. It can be done in a number ofways, but does require someone with suitable experience.

FURTHER DEVELOPMENT

One factor that perhaps could be improved is what wetell the students about how their final grade iscalculated. In their instructions for the LifestyleAssignment they are told:

“You will be allocated a mark according to theoverall group performance (i.e. a mark for the posterand the debate) and to how your own group hasassessed your contribution to the group tasks”.

This seems to be perfectly adequate but thereare always a few students who like to know preciselyhow their mark is calculated. On reflection followingwriting up this case study, in future we will use theexplanation given here as information on thestudents’ Level 1 Biology website.


This method of peer-assessment was introduced intoa first year pharmacology programme with 50–160students per year and has also been used with 2ndyear medical students (275). Many of the learning

Peer-assessment ofpractical write-upsusing an explicitmarking scheduleIAN HUGHES

5


38 CASE STUDIES

objectives were particular to the content of each ofthe exercises to which peer-assessment was appliedbut, in addition, some generic problems and learningobjectives were addressed by use of this method ofpeer-assessment:

• Utilization of feedback. There was little evi-dence that students took any notice of (or evenread) the material laboriously written on eachpractical by members of staff. This methodprovides each student with a full explanation ofwhat should have been done. Every studentsgets excellent and timely feedback to which, bythe nature of the process, they must payattention.

• Development of critical evaluation skills.Students have to make judgements about thequality of their work to achieve the standards towhich they aspire and in order to time-managetheir activities. This is not something whichcomes easy to all students and practice withcritical evaluation in the early part of a coursehelps prepare students for what they will needto do later. The ability to be critical of your ownwork and that of others is a valuabletransferable skill. Surveys show graduates infirst employment have to assess the work ofothers surprisingly early in their jobs. Grad-uates are often not prepared for this.

• Better understanding of the material. Students,like everyone else, need a better under-standing to assess something than to produceit. This is particularly true if dealing withsomebody else’s work where the words andtheir order are not those you yourself wouldhave used.

• Improved learning. This method provides asecond look at the material covered. Learningis improved and reinforced by the feedbackresulting from participation in the assessmentprocess.

• Motivation. This method enables students tosee the standard others achieve and wheretheir own work may be improved. This is morepowerful than seeing a ‘perfect answer’ writtenby a member of academic staff (‘of course theycan produce a good answer or they wouldn’t beon the academic staff!’). Seeing your peers aredoing a much better job than you are evenwhen subject to the same pressures is apowerful spur for improvement.

• Developing independence. Students confront the‘personal relationship’ issue and learn to makeassessments independent of any personalrelationship. This requires a very differentattitude to that which many students have onleaving school (“always look after your mates”).

• Significant reduction in marking time. The timeinvolved in marking practical write-ups eachweek was becoming unsustainable as studentnumbers increased. Using peer-assessment250 or more practical write-ups can be markedin one hour.

‘HOW TO DO IT’

The task for the students was to provide a write-up,following a set of instructions, of a scheduledlaboratory practical or computer simulated experi-ment. This practical schedule usually included somequestions to test the students’ understanding of thematerial. Written answers to these questions wererequired as part of the practical write-up.

The write-ups are handed in by a publisheddeadline and there are penalties for being late. Workpresented by the deadline is stamped as beingreceived (this stops students slipping late write-upsinto the marking session). Split groups may havedifferent deadlines providing they are not too farapart. Time is set aside in the timetable (1 hour) for amarking session and it is made clear that attendanceis compulsory, any student missing (without goodreason) the marking session looses half the marksthey are assigned. It is important to be firm about thisas if 200 students do the work and only 120 turn up tothe marking session you have to mark the other 80write-ups! At the marking session, having previouslyexplained the advantages of peer-marking, I distri-bute the write-ups and a record sheet on which themarker fills in their name, the name of the studentbeing marked, the final mark awarded and signs toaccept responsibility. An explicit marking schedule isdistributed. I emphasise the need for silence duringmarking and enforce it. I then go through the markingschedule step by step explaining, with pre-preparedslides or acetates, how things should be done, whatgraphs should look like etc.

Students annotate the write-up they aremarking as appropriate and decide what proportion ofthe marks allocated for each point should be awardedfor the material presented. Students asking if acertain wording is worth x or y marks are told theymust make the decision from the information theyhave. Students total the marks awarded, fill in andsign the record sheet. The write-ups, marking schedule


39

and record sheet are collected so marks can berecorded and then the write-up and marking scheduleare made available for collection by the owner.Students are told that a portion of the write-ups willbe check marked by staff and that any student whofeels they had been marked unfairly could have theirwrite-up re-marked by a member of academic staff(less than 2% do so).


Generally, for students, the process of self-assess-ment is easier to perform than peer-assessment.I often make the first exposure one of self-assess-ment and then progress to peer-assessment. It iseasier to find key words and phrases in work you havedone yourself since you know where everything is.This makes the assessment process easier. How-ever, there is a tendency to assess what you meant towrite rather than what is actually there. In additionthere is a greater potential for cheating as it has beenknown for students to fill in or change material intheir own submitted work while assessing it.However, self-marking does provide an easyintroduction to peer-marking and this can be useful.

Not all practical work is easily amenable to thismethod as it really hinges on the task set. Workresulting from following a practical schedule is readilypeer-assessed. The same measurements have beenmade with similar data obtained and processed thesame way. The write-up needs to follow a specifiedformat that controls the order in which material ispresented and the type of data presentation (e.g.present the data in a table, draw a graph etc.). Thisenables an explicit marking schedule to be providedwith the material broken down into small pieces, eachof which is associated with specific criteria orrequirements for marks to be awarded. Thus, work inyear 1/2 is more likely to fulfil these requirements.

Work resulting from a task like ‘Describe anideal vehicle’ is not easily peer-assessed except at thevery broadest level, since ‘vehicle’ may have been takento mean different things (storage vehicle, transportvehicle, communication vehicle or vehicle in which todissolve something) and ‘ideal’ will depend on wherethe writer is coming from. The marking schedule tomeet all possibilities is either so general as to ignorespecific content or so extensive that it takes too long towrite and is very difficult for students to follow. Finalyear level work, where several completely different butvalid approaches to the task could have been taken, istherefore difficult to peer-assess using the simplemethods described here. Likewise, “Is the work wellpresented?” is not a reasonable question as there areno specific criteria associated with it. Each student may

make a judgement based on different criteria andconsiderable personal preference may come intothe assessment.

The practical work needs to be done by thestudent body over a short period of time so theassessment session can follow in a timely manner. Ifsix weeks elapse between the first student doing thework and the assessment process the students willhave forgotten what it was all about. Work done aspart of a ‘circussed’ set of exercises is therefore notsuitable as the first group cannot be assessed as soonas they have completed the task (or they will pass theanswers on to others) and it may be several weeksbefore all students have done all the tasks, withoutgetting any feedback on their performance.

The task set needs to change from year to year.If an identical task is set each year the markingschedules will get passed on and while studentperformance might improve year on year this is onlybecause they are copying out last year’s markingschedule. I currently have a set of three versions ofeach exercise which I rotate each year and have notyet any evidence that the material gets passed on. Ihave had instances where students handed in a write-up based on last year’s exercise data and thencomplained that I had not warned them that theexercise was different year on year!

TROUBLESHOOTING

Don’t think your students are going to enjoy peer-assessment! Many believe assessment is the job ofthe teacher (“don’t you get paid for this?”), manycomplain that peer-assessment is hard work (“youhave to think and make judgements”), and that it’stiring (“I’m really bushed at the end of a markingsession”). Some find it difficult to concentrate for awhole hour. Some believe student markers are unfairor inaccurate. The reasons for introducing peer- orself-marking need to be explained to students if it isto be introduced without resentment. See Figure 1 fordocumentation that has been used effectively inpreparing students.

Silence in class during the marking process isimperative. Otherwise students will miss your explana-tions, ask for repetitions or misunderstand what wasrequired and the marking session will take forever. Inan ideal world, it might be possible to allow orencourage students to discuss and compare what iswritten in the material they are marking; but when Ihave tried this, the time taken was greatly prolongedand while some students were bored, others demand-ed more time. Not a good idea in practice; unlessthere is only a small amount of material to mark andno absolute deadline to complete the process by.


40 CASE STUDIES

DOES IT WORK?

The published evidence (Hughes, 1995 and 2001)indicates the students on average produced betterwrite-ups when using peer-assessment than they didwhen staff marking was used. The data demonstratethat this is not due to students being easier markers.

Peer-assessment saves an enormous amountof staff time, provides excellent feedback andachieves many of the points bulleted above. Markingaccuracy is often queried but students can alwayscheck their mark against their copy of the markingschedule and appeal to the tutor if they aredissatisfied. To test reproducibility of marking threecopies of the same practical were peer-markedindependently by students as part of the normalmarking session. The marks awarded differed by only3% demonstrating the consistency of the markingprocess. In addition, I have, using the same markingschedule, personally marked several samples ofpeer-marked work. In every case the discrepancy wasless than 5%. Confidence can be placed in peer-generated marks which can therefore be used as partof the marks which contribute to final module grades.External examiners have not objected to the use ofpeer-assessed marks in this way.

Several colleagues have started to utilize thismethod and no new problems or difficulties have beenencountered.



• an explicit peer-marking schedule;

• peer-assessment of oral presentations.

Student Guide to Peer-Assessment of Practicals

Why are we doing this?

You should get several things out of this method of assessment which may be new to you:

1. It is an open marking system; therefore you can see what was required and how to improve yourwork.

2. You see mistakes others make and therefore can avoid them; you also see the standard achievedby others and can set your own work in the spectrum of marks.

3. You get a full explanation of the practical and how you should have processed the data and donethe discussion. Therefore your information and understanding is improved.

4. You get practise in assessing others and their work. You will need this skill quite early in a careerand you will need to come to terms with the problem of bias; someone who is a good friend mayhave done poor work; it can be disturbing to have to give them a poor mark.

5. In assessing others you should acquire the ability to stand back from your own work and assessthat as well. This is an essential ability in a scientist; an unbiased and objective assessment ofthe standards you have achieved in your own work. Once you are away from the teacher/pupilrelationship (i.e. leave university) you will be the person who decides if a piece of work is goodenough to be considered as finished and passed to your boss.

The method of marking adopted in this module is designed with the above factors in mind.

Figure 1. Part of a document used in preparing students for peer-marking, explaining the benefits to them



41


This exercise forms part of a second year module inresearch methods and scientific communication,taught to classes of 60–90 bioscience students.Students can find such topics rather dry and, as aresult, the taught sessions rely heavily on workbooksand worksheets to cover the syllabus, which includes:locating and evaluating sources; primary andsecondary literature; style and layout; the peer reviewsystem and its role in scientific publication; citationand referencing. The assignment requires students toapply the knowledge they have gained in the taughtsessions to a short exercise, to satisfy the followinglearning outcomes:

• Use relevant methods to locate and interpretresearch information in the primary scientificliterature.

• Use appropriate forms of scientific communi-cation, in this module and in other moduleswithin the programme.

‘HOW TO DO IT’

The following steps describe the principal stages:

1. Having come to appreciate the differencebetween a primary and secondary source in theworkshop sessions, students are instructed toselect an interesting, recent paper from theprimary scientific literature (published withinthe last few months, to avoid any possibility ofplagiarism from previous years). Each studentselects a different article (a sign-up sheet onthe notice board enables students to check

which papers have been selected and rewardsthose students who get off to a quick start!).

2. Students make a photocopy or printout of thepaper: this is needed by their peer reviewerand must also be handed in along with theirassignment.

3. Each student then prepares a brief article(400–500 words) about their chosen paper inthe style of the ‘This Week’ section of NewScientist magazine. Students are told that theirarticle should conform in general style andapproach to the examples found in any copy ofNew Scientist (examples are also available fromthe website: http://www.newscientist.com) andthey are given other guidance on layout (e.g.typed double-spaced, 12 point font, to include aword count, a full citation of the primary sourceis required, etc.).

4. Pairs of students then exchange articles andreview each other’s work, using an evaluationsheet very similar in overall style to that usedby scientific journals. The reviewer mustassess the article and (i) decide whether thearticle is acceptable without change or whetherminor/major revision is required (ii) providespecific feedback on any points raised, e.g. bywriting comments on the article, or as anumbered sequence, cross-referenced againstthe article. The reviewer is also given a copy ofthe original article, so he/she can see whetherthere are any omissions, etc.

5. Student reviewers then return the article andevaluation sheet to the original author, who hasthen to consider their response to the review,using a response form. Students must decidewhether to (i) modify their article, where theyfeel that the reviewer’s comments areappropriate and (ii) prepare a written responseto each of the points raised by the reviewer. Inthis way, they are given a hands-onintroduction to a process similar to that usedfor peer review of a primary scientific article.Students are also encouraged to reflect ontheir own work (self-evaluation), especially ifthey feel that their reviewer has been“lightweight” in providing feedback.

6. Students must then hand in for finalassessment (i) the photocopy/printout of theoriginal paper (ii) a copy of their original(unreviewed) article (iii) a copy of their reviewedarticle along with the reviewer’s comments and

Writing andreviewing an articlefor a scientificmagazine — a peer/self-assessmentexerciseROB REED

6

http://www.newscientist.com


42 CASE STUDIES

evaluation sheet (iv) their response to thereview/evaluation and (v) a copy of the finalversion of their article.

7. The exercise is then marked on the followingbasis.

• The quality of the original (unreviewed) versionof the article, as an exercise in presenting keyinformation from the original paper in anappropriate and accessible style, with dueregard for the target audience (generalreadership of New Scientist magazine) — 30%of the overall mark.

• The student’s response to peer review (and/orself-evaluation), as evidenced by (i) thechanges made to the original version inproducing the final version and (ii) the responsesheet, dealing with reviewer’s comments —30% of the overall mark.

• The student’s effectiveness as a peer reviewer,based on (i) written comments on their partner’sarticle and (ii) the evaluation sheet of theirpartner’s article — 40% of the overall mark.


It is essential that students are given clearinstructions in writing at the outset of the exercise, tosupport the oral explanation given during the class. Ihave found it necessary to provide quite detailedguidance (for example, many students didn’t under-stand the concept of double-spacing, thinking thatthis meant having two spaces between each word!).The guidelines now explain that a space equivalent totwo lines is needed in the printed version to givesufficient room for the reviewer to provide hand-written comments, along with step-wise instructionson how to set up MS Word to provide double-spacedtext). I have also found it useful to provide thestudents with a detailed checklist of all of the itemsrequired for submission, since it can be a littleconfusing (they have to realise, for example, that theirwork as a reviewer will be handed in by their partner,and that I will separately assess this aspect of theirwork, and then collate the marks).

It can sometimes be a little difficult keepingtrack of which students are working together — I askthem to sign up in pairs at the outset, and not toswitch partners without informing me. I allow them toselect their own partners, and I tell them that theyshould not regard this in any way as a “soft option”,since I will have oversight of the whole process, and

that students who simply give their partner anundeservedly positive review will score poorly in thataspect of the exercise!

TROUBLESHOOTING

Sometimes students will work in threes, rather thanpairs — in such instances, each person reviews thework of a different person to their own reviewer. Itworks just as well this way, and is an alternativeapproach, avoiding reciprocal peer-assessment.

In occasional instances, there is a problem withone of the team members (e.g. where a student doesnot return the reviewed article by the specified date,or where someone is ill during the programme) —such cases have been dealt with on an individual basisby either (ii) reassigning group members or (ii) askingone student to perform a second (unassessed) review,so that all elements of the process are covered.

It can be a little tricky marking the variousaspects of different people’s work at different times —my approach has been to mark the review (secondperson’s mark) at the same time as the original andfinal versions of the article (first person’s mark) toensure continuity in reading the article, and to use apre-printed feedback sheet with a number of generalcomments to provide overall feedback, as well as amark for each component. This structured approachworks well with a large group of students.

DOES IT WORK?

Student feedback is usually positive for this aspect ofthe programme — students generally regard it as aninteresting exercise, and a welcome change from moretraditional essays and similar written assignments.


It has run successfully in its present form for the pastfive years. To date, the peer/self-assessmentcomponent has been restricted to a broad overallevaluation, based on written feedback, rather than aquantitative numerical mark/grade. One aspect thatcould be introduced relatively easily would be to askstudents to provide a numerical mark for each of theaspects of the process (e.g. self-assessment of (i) theoriginal article and (ii) the final article, and (iii) peer-assessment of their partner’s article. Students wouldthen be able to compare their own assessment markswith those of the lecturer, to see how effectively theycan assess their own work and that of others, usingthe same criteria as those of the teaching staff.


43



• student assignment;

• assignment front sheet;

• peer reviewer's evaluation sheet; and

• author's response to peer reviewer's comment.


Final Year Food Technology students participate in areal-life problem-based case study. Each case studyfocuses on a small problem within a larger graduateresearch project being undertaken by the universitywith an industrial partner. As such, the project tendsto be a blend of the practical use of food technologypilot plant equipment and background theoreticalresearch. Students are allowed to organise their workpattern in order to meet the objectives of theparticular project.

The final assessment of the case study is as agroup, conference-style, oral presentation. Thesepresentations are exclusively peer-assessed. Time istaken within the module to discuss and deviseappropriate marking strategies and descriptors.Thus the students take ownership not only over theirworking time but also in the style of assessmentstrategy, giving them greater understanding oflearning patterns.

'HOW TO DO IT'

During the final week of research activity, studentsare reminded about the mini-conference present-ations which are required as their assessment of thecase study. Guidance is given on presentationtechniques and the use of graphics and IT inpresenting information using MS PowerPoint.Examples of previous conference presentations areprovided as a benchmark. At the same time, themarking strategy is discussed and the elements ofpresentation to be assessed, together with thebalance of marks associated with each element, areagreed within the group. This process is mediated bythe academic; however the students lead thediscussion and formulate the marking criteria.

On the day of the student presentations,evaluation sheets are distributed amongst the groupand the process of peer-assessment is reinforced.The presentation evaluation sheets are graded on ascale 1–9 using the criteria already agreed on. A totalof 10 criteria relating to both product and process areused, such as relevance of information supplied,evidence of sound laboratory practice, evidence ofteamwork, timekeeping, readability of slides andamount of information supplied.

Students are then expected to evaluate eachgroups' performance (according to the criteriaalready laid down), and any additional informationabout a groups' performance is noted on theevaluation form. At the end of the series ofpresentations, all evaluation sheets are collected inby the academic. Evaluation sheets obtained in thisexercise are then scrutinised by the academic and themarks allocated to each group (for every element ofthe assessment) are fed into a database. The finalmark for each specific element of the exercise is givenas the mean awarded to the group by their peers, andthe overall mark is derived according to the markingcriteria as agreed by the students.

Follow-up workshops are used to disseminategood practice to students and to evaluate studentperception of the process.

TIPS/THINGS TO LOOK OUT FOR

Staff need to be willing to explain (openly) how andwhy student assessment criteria are set. Thisfacilitates the students' understanding of developingtheir own marking criteria and leads into the idea ofpeer-assessment. Sometimes the actual idea ofpeer-assessment is so strange to the students thatadditional time needs to be spent in reassuring themof the fairness of such schemes, and the importanceof treating the process professionally.

Peer-assessedproblem-basedcase studiesCHARLES BRENNAN, ELIZABETH FOLLAND,RICK PRESTON & NICOLA BLATCHFORD

7



44 CASE STUDIES

DOES IT WORK?

The use of peer-assessment in this case studybenefits the students. Although there may be a slightreluctance to use peer-assessment for the assign-ment initially (sometimes students express a wishthat the assignment is evaluated by academics,following usual guidelines). However, the students doaccept their roles in the assessment procedure andact responsibly. Through completing the assessmentthey do learn how to reflect on the work of their peers,how to assess and evaluate work separate frompersonal friendships, and how to accept positivecriticisms regarding the quality of their own work.Indeed, it is interesting that the process also allowsthe students to reflect on their own learning stylesand choices of appropriate communication tools.

As such the case study is extremely useful indeveloping critical evaluation of their own com-positions, and a greater autonomy over their workingpractices. This development of self-evaluation, andself-worth, is noteworthy when you also take intoaccount the students' greater awareness of the use oftheir skills and knowledge acquired so far, inproblem-solving real-life situations.


Further developments may be to devise workshopsspecifically aimed at introducing the principles andaims of peer-assessment. This would have theadvantage of reducing student reluctance to par-ticipate in such exercises, and also help with theirunderstanding of assessment marking strategies. Aresult of such could be their ability to better managetheir own assessment achievements in modules.


45

REFERENCES

Adams, C. and King, K. (1995) Towards a framework for self-assessment. Innovations in Education and TrainingInternational, 32(4), 336–343.

Barnett, R. (2000) Supercomplexity and the Curriculum. Studiesin Higher Education, 25(3), 255–265.

Biggs, J. (1996) Assessing learning quality: reconcilinginstitutional, staff and educational demands. Assessment andEvaluation in Higher Education, 21(1), 5–15.

Biggs, J. (1999) Teaching for quality learning at university.Buckingham, UK: Society for Research into Higher Educationand the Open University.

Boud, D. (1986) Implementing student self-assessment. Sydney,Australia: Higher Education Research and DevelopmentSociety of Australia.

Boud, D. (1988) Moving towards autonomy. In Developing StudentAutonomy in Learning, (ed) Boud, D., pp 17–39. London, UK:Kogan Page.

Boud, D. (1989) The role of self-assessment in student grading.Assessment & Evaluation in Higher Education, 14(1), 20–30.

Boud, D. (1995) Assessment and learning: contradictory orcomplementary. In Assessment for Learning in HigherEducation, (ed) Knight, P., pp 35–48. London, UK: Kogan Page.

Boud, D. (2000) Sustainable assessment: rethinking assessmentfor the learning society. Studies in Continuing Education,22(2), 151–167.

Boud, D. and Falchikov, N. (1989) Quantitative studies of studentself-assessment in higher education; a critical analysis offindings. Higher Education, 18, 529–549.

Boud, D. and Falchikov, N. (2004) Beyond formative andsummative assessment: developing a new agenda forassessment for lifelong learning. Paper presented atAssessment 2004 Beyond Intuition, the Second Biannual JointNorthumbria/EARLI SIG Assessment Conference, University ofBergen, Norway, June 23–25 2004.

Brew, A. (1995) How does self-assessment relate to ideas aboutlearning? In Enhancing Learning through Self Assessment, (ed)Boud, D., pp 24–35. London, UK and Philadelphia: Kogan Page.

Brown, S. and Glasner, A. (2003) Assessment Matters in HigherEducation: Choosing and Using Diverse Approaches, 3rdedition. Buckingham, UK: The Society for Research intoHigher Education and Open University Press.

Brown, G., Bull, J. and Pendlebury, M. (1997) Assessing StudentLearning in Higher Education. London, UK: Routledge.

Brown, S., Sambell, K. and McDowell, L. (1998) What do studentsthink about peer assessment? In Peer Assessment inPractice, (ed) Brown, S., 107–112. Birmingham, UK: Staff andEducational Development Association (SEDA).

Butcher, A. C., Stefani, L. A. J. and Tariq, V. N. (1995) Analysis ofpeer-, self- and staff-assessment in group work. Assessmentin Education, 2(2), 165–185.

Candy, P., Harri-Augstein, S. and Thomas, L. (1985) Debriefing inexperience-based learning. In Reflection: Turning Experienceinto Learning, (eds) Boud, D., Keogh, R. and Walker, D.,100–116. London, UK: Kogan Page.

Chanock, K. (2000) Comments on essays: do students understandwhat tutors write? Teaching in Higher Education, 5(1) 95–105.

Cheng, W. and Warren, M. (1997) Having second thoughts:students perceptions before and after a peer assessmentexercise. Studies in Higher Education, 22(2), 233–239.

Cheng, W. and Warren, M. (1999) Peer and teacher assessmentof the oral and written tasks of a group project. Assessment& Evaluation in Higher Education, 24(3), 301–314.

Claxton, G. (1995) What kind of learning does self-assessmentdrive? Developing a ‘nose’ for quality: comments onKlenowski. Assessment in Education, 2(2), 339–343.

Cook, A. and Rushton, B.S. (1995) An information technologycourse for biological science students taught through peertutoring. In Enhancing student learning through peer tutoringin higher education. Section three: Implementing, (eds)Griffiths, S., Houston, K. and Lazenbatt, A. pp. 15–21Coleraine, UK: University of Ulster.

Cowan, J. (2002) On Becoming an Innovative University TeacherReflection in Action 4th Edition. Buckingham, UK: The Societyfor Research into Higher Education and Open University Press.

Cowan, J., Joyce, J., McPherson, D and Weedon, E. (1999) Self -

assessment of reflective journaling — and its effect onlearning outcomes. 4th Northumbria Assessment Conference,1–3 September 1999, University of Northumbria, UK.

Dearing, R. (1997). Higher Education in the Learning Society.Norwich, UK: HMSO.

Dochy, F., Segers, M. and Sluijsman. D. (1999) The use of self-peer and co-assessment in higher education: a review.Studies in Higher Education, 24(3), 331–350.

Drew, S. (2001) Students perceptions of what helps them learnand develop in higher education. Teaching in Higher Education,6(3), 309–331.

Edwards, A. Knight, P. T. (1995) Assessing Competence in HigherEducation. London, UK: Kogan Page.

Ecclestone, K. (2001) ‘I know a 2:1 when I see it’: understandingcriteria for degree classification in franchised universityprogrammes. Journal of Further and Higher Education, 25(3),301–313.

Elwood, J. and Klenowski, V. (2002) Creating communities ofshared practice: the challenges of assessment use inlearning and teaching. Assessment and Evaluation in HigherEducation, 27(3), 243–256.

Falchikov, N. (1995) Peer feedback marking: developing peerassessment. Innovations in Education and TrainingInternational, 32(2), 175–187.

Falchikov, N. (2001) Learning Together. Peer Tutoring in HigherEducation. London, UK: Routledge.

Falchikov, N. (2003) Involving students in assessment. PsychologyLearning and Teaching, 3(2), 102–108.

Falchikov, N. and Boud, D. (1989) Student self-assessment inhigher education: a meta-analysis. Review of EducationalResearch, 59(4), 431–470.

Falchikov, N. and Magin, D. (1997) Detecting gender bias in peermarking of students’ group process work. Assessment andEvaluation in Higher Education, 22(4), 385–396.

Falchikov, N. and Goldfinch, J. (2000) Student peer assessmentin higher education: a meta-analysis comparing peer andteacher marks. Review of Educational Research, 70(3), 287–322.

Freeman, M. (1995) Peer assessment by groups of group work.Assessment and Evaluation in Higher Education, 20(3), 289–299.

Freeman, R. and Lewis, R. (1998) Planning and ImplementingAssessment. London, UK: Kogan Page.

Fry, H., Ketteridge, S and Marshall, S (1999) A Handbook for Teachingand Learning in Higher Education. London, UK: Kogan Page.

Gabb, R. (1981) Playing the project game. Assessment andEvaluation in Higher Education, 6(1), 26–48.

Gielen, S., Dochy, F. and Dierick, S. (2003) New insights intolearning and teaching and their implication for assessment.In Optimising New Modes of Assessment: in Search of Qualitiesand Standards (eds) Segers, M., Dochy, F. and Cascallar, E.,pp. 37–54. London, UK: Kluwer Academic Publishers.

Goldfinch, J. (1994) Further developments in peer assessment ofgroup projects. Assessment and Evaluation in HigherEducation, 19(1), 29–34.

Griffiths, S., Houston, K. and Lazenbatt, A. (1995) Enhancing studentlearning through peer tutoring in higher education. Section three:Implementing. Coleraine, UK: University of Ulster.

Heron, J. (1988) Assessment revisited. In Developing StudentAutonomy in Learning (ed) Boud, D., pp. 77–90. London, UK:Kogan Page.

Heron, J. (1992) The politics of facilitation: balancing facilitatorauthority and learning autonomy. In Empowerment throughExperiential Learning: Exploration of Good Practice (eds)Mulligan, J. and Griffin, C., pp. 66–75. London, UK: Kogan Page.

Higgs, J. (1988) Planning learning experiences to promoteautonomous learning. In Developing Student Autonomy inLearning (ed) Boud, D., pp 40–58. London, UK: Kogan Page.

Hinett, K. (1995) Fighting the assessment war: the idea of assess-ment-in-learning. Quality in Higher Education, 1(3), 211–222.

Hughes, I. E. (1995) Peer assessment of student practicalreports and its influence on learning and skill acquisition.Capability, 1, 39–43.

Hughes, I. E. (2001) But isn’t this what you’re paid for? The prosand cons of self- and peer-assessment. Planet, 2: 20–23.

Hughes, I. E. and Large, B. J. (1993) Staff and peer-groupassessment of oral communication skills. Studies in HigherEducation, 18(3), 379–385.

Jackson, N. and Ward, R. (2004) A fresh perspective on progress


46 REFERENCES

files — a way of representing complex learning andachievement in higher education. Assessment and Evaluationin Higher Education, 29(4) 423–449.

Klenowski, V. (1995) Student self-evaluation process in student-centred teaching and learning contexts of Australia andEngland. Assessment in Education, 2(2), 145–163.

Knight, P. (2001) A Briefing on Key Concepts: Formative andSummative, Criterion and Norm-referenced Assessment.LTSN Generic Centre Assessment Series No. 7. York, UK:Learning and Teaching Support Network.

Knight, P. (2002) Summative assessment in higher education:practices in disarray. Studies in Higher Education, 27(3), 275–286.

Knight, P. and Yorke, M. (2003) Assessment, Learning andEmployability. Berkshire, UK: Open University Press.

Kolb, D. A. (1984) Experiential Learning. Experience as the Sourceof Learning and Development. New Jersey, USA: Prentice Hall.

Kuit, J. A., Reay, G. and Freeman, R. (2001) Experiences ofreflective teaching. Active Learning in Higher Education, 2(2),128–142.

Lejk, M. and Wyvill, M. (2002) Peer assessment of contributionsto a group project: student attitudes to holistic and category-based approaches. Assessment and Evaluation in HigherEducation, 27(6), 569–577.

Li, L. K. Y. (2001) Some refinements on peer assessment ofgroup projects. Assessment and Evaluation in HigherEducation, 26(1), 5–18.

MacDonald, J. (2004) Developing competent e-learners: the roleof assessment. Assessment and Evaluation in HigherEducation, 29(2), 215–226.

Maclellan, E. (2001) Assessment for learning; the differingperceptions of tutors and students. Assessment andEvaluation in Higher Education, 26(4), 307–318.

Magin, D. (1993) Should student peer ratings be used as part ofsummative assessment? Higher Educational Research andDevelopment, 16, 537–542.

Magin, D. (2001) Reciprocity as a source of bias in multiple peerassessment of group work. Studies in Higher Education, 26(1),53–63.

Marton, F. and Saljo, R. (1976) On qualitative differences inlearning: I — outcome and process. British Journal ofEducational Psychology, 46, 4–11.

Miller, J. P (2003) The effect of scoring criteria specificity onself- and peer-assessment . Assessment and Evaluation inHigher Education, 28(4), 383–394.

Moon, J. (2001) Reflection in Higher Education Learning. PDPWorking Paper 4. York, UK: LTSN Generic Centre,http://www.ltsn.ac.uk/embedded_object.asp?id=17305&prompt=yes&filename=PDP012 (accessed 5th August, 2004).

Nieweg, N. (2004) Case study: innovative assessment andcurriculum design. Assessment and Evaluation in HigherEducation, 29(2), 203–214.

Ngu, A. H. H., Shepherd, J. and Magin, D. (1995) Engineering the‘Peers’ system: the development of a computer-assistedapproach to peer assessment. Research and Development inHigher Education, 18, 582–587.

Orsmond, P., Merry, S. and Reiling, K. (1996). The Importance ofMarking Criteria in the Use of Peer Assessment. Assessmentand Evaluation in Higher Education, 21(3), pp 239–250.

Orsmond, P., Merry, S. and Reiling, K. (1997). A study in self-assessment: tutor and students’ perceptions of performancecriteria. Assessment and Evaluation in Higher Education, 22(4),357–369.

Orsmond, P., Merry, S. and Reiling, K. (2000). The use of student-derived marking criteria in peer- and self-assessment.Assessment and Evaluation in Higher Education, 25(1) pp. 23–39.

Orsmond, P., Merry, S. and Reiling, K. (2002) The use ofexemplars and formative feedback when using studentderived marking criteria in peer- and self-assessment.Assessment and Evaluation in Higher Education, 27 (4) 309–323.

Orsmond, P., Merry, S., and Callaghan, A. C. (2004)Implementation of a formative assessment model incorpora-ting peer- and self-assessment. Innovations in Education andTraining International (in press).

Pereira, M. (1999) My reflective practice as research. Teaching inHigher Education, 4(3) 339–354.

Pond, K., Ul-haq, R. and Wade, W. (1995) Peer review: aprecursor to peer-assessment. Innovations in Education and

Training International, 32(4), 314–323.Race, P. (1998) Practical pointers on peer assessment. In Peer

Assessment in Practice, (ed) Brown, S., pp. 113–122.Birmingham, UK: Staff and Educational DevelopmentAssociation (SEDA).

Ramaprasad, A. (1983) On the definition of feedback. BehaviouralSciences, 28, pp. 4–13.

Rogers, C. R. (2003) Client-Centered Therapy. London, UK:Constable and Robinson.

Rushton, C., Ramsey, P. and Rada, R. (1993) Peer assessment ina collaborative hypermedia environment: A case study.Journal of Computer-Based Instruction, 20(3), 75–80.

Rust, C., Price, M. and O’Donovan, B. (2003) Improving students’learning by developing their understanding of assessmentcriteria and processes. Assessment and Evaluation in HigherEducation 28(2), 147–164.

Sadler, D. R. (1989) Specifying and promulgating achievementstandards. Oxford Review of Education, 13, 191–209.

Sambell, K. and McDowell, L. (1998). The construction of thehidden curriculum: messages and meanings in theassessment of student learning. Assessment and Evaluationin Higher Education, 23(4), 391–402.

Sambell, K., McDowell, L. and Brown, S. (1997) “But is it fair?”:an exploratory study of student perceptions of theconsequential validity of assessment. Studies in EducationalEvaluation, 23(4), 349–371.

Schelfhout, W., Dochy, F. and Janssens, S. (2004) The use of self,peer and teacher assessment as a feedback system in alearning environment aimed at fostering skills of cooperationin an entrepreneurial context. Assessment and Evaluation inHigher Education, 29(2), 177–201.

Schön, D. (1983) The Reflective Practitioner: how professionalsthink in action. New York, USA: Basic Books Inc.

Segers, M. and Dochy, F. (2001) New assessment forms inproblem-based learning: the value-added of the students’perspective. Studies in Higher Education, 26(3), 327–343.

Sivan, A (2000) The implementation of peer assessment: an actionresearch approach. Assessment in Education, 7(2), 193–212.

Sluijsmans, D. (2002) Establishing learning effects with integratedpeer assessment tasks, http://www.ilt.ac.uk/1204.asp(restricted access).

Stefani, L. A. J. (1992) Comparison of collaborative self, peer andtutor assessment in a biochemistry practical. BiochemicalEducation, 20(3), 148–151.

Stefani, L. A. J. (1994) Peer, self and tutor assessment: relativereliabilities. Studies in Higher Education, 19(1), 69–75.

Sullivan, K. and Hall, C. (1997) Introducing students to self-assessment. Assessment and Evaluation in Higher Education,22(3), 289–305.

Sutherland, P. (1992) Cognitive Development Today: Piaget and hisCritics. London, UK: Paul Chapman.

Tan, K. H. K. (2004) Does student self-assessment empower ordiscipline students? Assessment and Evaluation in HigherEducation, 29(6), 651–662.

Taras, M. (2001) The use of tutor feedback and student self-assessment in summative assessment tasks: towardstransparency for students and for tutors. Assessment andEvaluation in Higher Education, 26(6), 605–614.

Taras, M. (2003) To feedback or not to feedback in student self-assessment. Assessment and Evaluation in Higher Education,28(5), 549–565.

Topping, K. J. (2003) Self- and peer-assessment in school anduniversity: reliability, validity and utility. In Optimising Newmodes of Assessment in Search of Qualities and Standards,(eds) Segers, M., Dochy, F. and Cascallar, E., pp. 55–87.London, UK: Kluwer Academic Publishers.

Topping, K. J., Smith, E. F., Swanson, I. and Elliot, A. (2000)Formative peer assessment of academic writing betweenpostgraduate students. Assessment and Evaluation in HigherEducation, 25(2) 150–169.

Vygotsky, L. S. (1978) Mind in Society. Cambridge, USA: HarvardUniversity Press.

Wood, D. (1988) How Children Think and Learn, 2nd edition.Oxford, UK: Basil Blackwell Ltd.

Wood, D., Bruner, J. and Ross, G. (1976) The role of tutoring inproblem solving. Journal of Child Psychology and Psychiatry,17, 89–100.

http://www.ltsn.ac.uk/embedded_object.asp?id=17305&prompt=yes&filename=PDP012

http://www.ilt.ac.uk/1204.asp

Paul Orsmond is a Senior Lecturer within the Facultyof Health and Sciences at Staffordshire University,where he has been on the staff since 1992. Hisinterests are in educational development, withparticular reference to assessment. He is a UniversityLearning and Teaching Fellow.

ACKNOWLEDGEMENTS

The Centre for Bioscience would like to thank all those people who contributed case studies and other material to thisguide. Special thanks should also be given to the ‘critical friends’ who provided invaluable advice on the Guide’s content.In addition, the author would like to express his gratitude to Susan and Arthur for lots of interesting discussions.

Centre for BioscienceThe Higher Education AcademyLeeds LS2 9JT

Tel: +44(0)113 343 3001Fax: +44(0)113 343 5894Email: [email protected]://www.bioscience.heacademy.ac.uk

TheHigher

EducationAcademy

Centre forBioscience

http://www.bioscience.heacademy.ac.uk
