Peer Assessment in PBL

8/22/2019 Peer Assessment in PBL

1/18

Peer Assessment in Problem-Based Learning:A Qualitative Study

TRACEY PAPINCZAK*, LOUISE YOUNG and MICHELE GROVESMayne Medical School, School of Medicine, University of Queensland, Herston, 4006, Brisbane,

Queensland, Australia (*author for correspondence, E-mail: [email protected])

Received 26 June 2005; accepted 10 November 2005

Abstract. Peer assessment provides a powerful avenue for students to receive feedback on their

learning. Although student perceptions of peer assessment have been studied extensively in

higher education, little qualitative research has been undertaken with medical students in

problem-based learning (PBL) curricula. A qualitative study of students attitudes to, and per-

ceptions of, peer assessment was undertaken within the framework of a larger study of meta-

cognition with first-year medical students at the University of Queensland. A highly structured

format for provision of feedback was utilised in the study design. Many recommendations from

the higher education literature on optimal implementation of peer-assessment procedures wereput into practice. Results indicated the existence of six main themes: (1) increased responsibility

for others, (2) improved learning, (3) lack of relevancy, (4) challenges, (5) discomfort, and (6)

effects on the PBL process. Five of these themes have previously been described in the literature.

However, the final theme represents a unique, although not unexpected, finding. Students ex-

pressed serious concerns about the negative impact of peer assessment on the cooperative, non-

judgmental atmosphere of PBL tutorial groups. The practical implications of these findings are

considered.

Key words: assessment, peer assessment, peer evaluation, problem-based learning, qualitative

study

Introduction

In recent years, the quality of medical education has attracted considerable

attention, especially in light of the requirement to make education more

relevant to the needs of modern society. Medical education must prepare

students to deal with problems in the future, equipping them with skills

necessary to become active, self-directed learners, rather than passive recip-ients of information (Dolmans and Schmidt, 1996). Recognition of this need

was responsible, in part, for the development of problem-based learning

(PBL) (Barrows and Tamblyn, 1980).

Within the field of medical education, PBL is a curriculum innovation that

involves students in authentic learning activities using ill-defined medical

Advances in Health Sciences Education (2007) 12:169186 Springer 2006

DOI 10.1007/s10459-005-5046-6


2/18

problems as the stimulus and focus for learning (Norman and Schmidt,

1992). The pedagogical appeal of PBL is its perceived capacity to foster,

through these learning processes, enhanced clinical reasoning skills, and the

development of both an adaptable knowledge base to use in professionalsettings and the skills in self-directed learning necessary to become lifelong

learners in that profession (Kelson and Distlehorst, 2000). Four critical

conditions for a deep approach to learning are encompassed within the PBL

approach: a well-structured knowledge base, active learning, collaborative

learner interaction, and a context designed to promote internal motivation

through the provision of pragmatic goals (Margetson, 1994). Evaluation of

student progress in such a student-centred curriculum, however, has re-

mained a challenge (Eva, 2001).Assessment protocols within PBL curricula have sometimes sought to

include participative (or peer) assessment, in which students contribute to the

evaluation of each others work, in order to match evaluation procedures

with the curricular philosophy. Peer assessment helps to develop the acqui-

sition of self-directed learning skills (a key objective of PBL) as students

participate in the assessment experience (Ballantyne et al., 2002). As PBL

emphasizes the development of proficiency in the resolution of clinical

problems, the assessment of student skills, processes and attitudes would take

place most appropriately within the tutorial setting (Eva, 2001). There are

several advantages to employing tutorial-based peer assessment, including:

prolonged interaction between peers for provision of constructive feedback

based on multiple observations of performance; and opportunity to assess

areas of proficiency (such as communication skills, self-directed learning, and

respect for others) not readily evaluated by more traditional forms of

assessment (Eva, 2001). Higher education literature suggests that peer

assessment has many additional benefits, including enhanced metacognitive

skills (Ballantyne et al., 2002), and improved understanding of subject matter(McDowell, 1995). Peer assessment encourages students to reflect on their

own approaches to assessment tasks (Dochy et al., 1999), to develop critical

reasoning skills (Hanrahan and Isaacs, 2001; Searby and Ewers, 1997) and

skills of reflection (Searby and Ewers, 1997; Somervell, 1993). Although

strong support for peer assessment is evident in the literature, difficulties and

limitations have repeatedly been reported.

Students perceptions that peer assessment can be unreliable and unfair

has been raised in several studies in higher education (McDowell, 1995;

Norton and Brunas-Wagstaff, 2000). Sluijmans et al. (2001) confirmed the

existence of bias in peer marking due to interpersonal relationships between

students. Students frequently report feeling uncomfortable carrying out peer

assessment, often because they feel unqualified to make these judgements

(Orsmond and Merry, 1996), lack faith in their own objectivity (Hanrahan

TRACEY PAPINCZAK ET AL.170


3/18

and Isaacs, 2001), or feel the task is too challenging (Mowl and Pain, 1995;

Topping et al., 2000). The time-consuming nature of peer assessment is also

regarded negatively by students (McDowell, 1995; Topping et al., 2000).

While studies in medical education of students attitudes to peer assessmentin PBL have, to date, proven elusive, the results of quantitative studies

suggest that peer ratings are not a reliable means of tutorial-based assessment

in PBL (Reiter et al., 2002; Sullivan et al., 1999). Qualitative studies of peer

assessment within PBL have the capacity to provide insights into the nature

and extent of benefits to learning.

Research into medical education has largely emphasized quantitative ra-

ther than qualitative evaluation, particularly in the area of peer assessment.

Van Rosendaal and Jennett (1992) reported that peer evaluation was viewedas an intrusion into the relationship among medical colleagues. Duffield and

Spencer (2002) surveyed medical students attitudes to assessment procedures

and found meaningful feedback about their progress in the course was

considered very important. Peer assessment provides one avenue for such

feedback to be delivered within the setting of the PBL tutorial.

Given the lack of research into student attitudes to peer assessment within

the PBL tutorial setting, this study has the potential to provide important

insights and to expand upon the findings from studies in non-PBL curricula

reported above. There have also been calls for greater qualitative exploration

of the social and educational aspects of PBL, in particular, ethnographic

accounts of PBL which endeavour to explore the influence of the culture of

the group (Leung, 2002). Such research has the potential to enhance our

awareness of all aspects of the PBL process.

The aim of this study was to explore student attitudes to, and perceptions

of, peer assessment concerning student fulfilment of roles and responsibilities

within their PBL tutorials. The PBL environment, with its emphasis on self-

directed and collaborative learning, provides a unique context in which tostudy students attitudes to, and perceptions of, their self-directed learning

experiences. Qualitative methods allow the researcher to study these dynamic

processes.

Description of the Study

Data were gathered as part of a larger study of metacognitive processes

undertaken with first-year students enrolled in the Bachelor of Medicine and

Bachelor of Surgery (MBBS) Program at The University of Queensland.Only the qualitative results of the peer assessment portion of the study will be

reported here.

Methods associated with participatory action research (Kemmis and

Wilkinson, 1998) and ethnography (Kamil et al., 1985) were adopted to gain

insight into student attitudes to, and perceptions of, peer assessment. Collins

PEER ASSESSMENT IN PROBLEM-BASED LEARNING 171


4/18

(1991) recommends the following epistemological grounding for engaging in

research among adult learners:

Instead of deploying intrusive research designs based on independentlearning project protocols, (we) might invite willing individuals to en-

gage in exploratory conversations, as part of a pedagogical and her-

meneutic process, to discover with what meanings adults endow their

own learning experiences. (Collins, 1991, p. 114)

Ethnographic inquiry endeavours to understand social behaviour from the

participants frame of reference (Kamil et al., 1985). In this context, the

experiences, behaviours, and attitudes of students learning medicine in their

PBL tutorials are observed and documented. The role of this researcher (aPBL tutor) was based on participantobserver inquiry (Bishop, 1999). PBL

tutors are in a fortunate position to be able to observe the culture of the

PBL tutorial and to participate as much or as little as desired in the ensuing

discussion and debate. The context and the participants were used to guide

research questions (Bishop, 1999) the classroom realities, developing

knowledge, and developing perceptions of the group allowed the researcher

to adapt the study design. The resulting design aimed to more fully reflect the

reality of the experience of students in their PBL tutorials.

Action research, involving overlapping cycles of planning, acting and

observing, and reflecting (Kemmis and Wilkinson, 1998), was utilised to

maintain a responsive study design. Data collection and analysis were com-

bined and used to shape ongoing data gathering in order to develop a par-

ticipative-assessment procedure responsive to student priorities and concerns.

The study design drew on literature in the area of peer assessment and

action research, incorporating numerous elements deemed important

including:

1. assessment of process rather than learning outcomes (see Boud et al.,1999);

2. use of assessment criteria targeting five significant areas of performance

in PBL (refer Das et al., 1998; Willis et al., 2002), including assessment of

participation (Dancer and Kamvounias, 2005);

3. use of multiple specific criteria on the assessment instrument, targeting

discrete areas of student performance (as endorsed by Miller, 2003) ra-

ther than a few global indicators;

4. student feedback helping to guide final criteria selection, and explicitdetails of implementation of the peer-assessment process (refer Ballan-

tyne et al., 2002);

5. practice in peer assessment procedures to improve student familiarity (in

keeping with the recommendations of Ballantyne (2002), Sluijmans et al.

(2001), and Sullivan et al. (1999);



5/18

6. provision of examplars of good and/or poor work to students in initial

weeks of orientation (in accordance with the recommendations of Ba-

llantyne et al., 2002; Hanrahan and Isaacs, 2001);

7. constant monitoring of student perceptions in order to guide the actionresearch process (Kemmis and Wilkinson, 1998).

Setting

The MBBS Program introduced a 4-year, graduate-entry PBL curriculum in

1997. First-year students, in small groups of 9 or 10, undertake 5 h of PBL

tutorial time each week for 33 weeks of the year. Working in collaboration

with their small group of peers, students analyse a problem of practice,

formulate hypotheses, and undertake self-directed learning to try to explainthe patients disease process.

Subjects

The study was conducted with 165 first-year medical students and 20 tutors

in two phases over 2 years (refer Table I). In the first year, small student

focus groups were invited to participate in refinement of the study design and

peer-assessment instrument. In the following year (2004), 215 self-selected

subjects, representing 69% of the student group, were recruited for the mainstudy through an information session in the first week of the academic year.

Statistical analysis showed that study subjects were representative of the

entire student cohort on measures of age, gender, and primary degree. As

part of a larger study of metacognition, students were allocated to either

control or intervention groups. Allocation to the intervention cohort, based

on the criterion of total participation of group members within each PBL

Table I. Involvement of two cohorts of first-year medical students in the two phases of the

research

Cohort Timing of

qualitative research

Number of

student participants

Research activities

Cohort 1

(2003)

Phase 1: April & June 20 first-year

medical students 10

Focus group discussions

to identify areas of need

Phase 2: July to

September

10 first-year

medical students

Negotiation of criteria

for peer assessment

10 first-yearmedical students

Trial of peer assessmentinstrument and procedure

and re-negotiation

Cohort 2

(2004)

Phase 3: February to

September

125 first-year

medical students

Implementation of

peer assessment procedure

Student feedback



6/18

tutorial, resulted in 13 intervention (125 students) and 16 control tutorial

groups (90 students). All tutors assigned to intervention tutorial groups

agreed to participate. Intervention students took part in educational activities

within their PBL tutorials, including peer assessment, for a period of fivemonths (from which both qualitative and quantitative data were generated).

PROCEDURE

Ethical approval was obtained from the University of Queenslands

Behavioural and Social Sciences Ethical Review Committee.

Initial exploration of peer-assessment and development of the instrument

Several focus groups from the first cohort of students (2003) were engaged in

an initial exploration of peer assessment options, some negotiation of the

criteria to be included in the assessment instrument (based on the relevant

literature in this field), and a short trial of the peer assessment procedure.

Qualitative data were collected from direct observation and focus group

interviews.

Initially, two different groups of 10 students were consulted within the

PBL tutorial setting in order to test the water regarding student accept-ability of participative assessment. An unstructured interview arrangement,

with the explicit use of group interaction to produce greater insight (Punch,

1998), was adopted. Remarks such as: Marks allocated to PBL would rec-

ognize the amount of work done during this time and help motivate people to

work each week indicated student support for the overall concept. Subse-

quently, another group of ten students were invited to have input into

selection of appropriate marking criteria for participative assessment.

Numerous authors support the involvement of students in discussion andnegotiation of criteria to be used in assessment as it encourages them to

clarify their objectives and think critically about their work. Students were

distributed with a list of criteria proposed by relevant literature such as

criteria for self-evaluation in medical education described by Das et al.

(1998), and student perceptions of motivational influences and assessment

within PBL (Willis et al., 2002). With discussion and negotiation, students

agreed that all characteristics listed were valid and important in describing

students who are good participants, with some reservations concerning self-

awareness items.The negotiated Peer Assessment Instrument (with 19 items) was then

trialled with another group of 10 medical students and rated as easy to use

and understand by all participants. Student dissatisfaction with two items

resulted in their removal from the final version of the instrument. The

resulting scale measures several features of successful adult education, such



7/18

as participation, punctuality, respect for others, effective communication,

and critical analysis. However, the inclusion of items specifically targeting

self-directed learning and self-awareness, core features of PBL, allows it to be

differentiated from others which may be appropriate for open-ended, but lessstudent-centred, approaches - such as case-based instruction (Hay and Ka-

tsikitis, 2001). The instrument consists of 17 items scored on a Likert scale of

one to five across five sub-scores: responsibility and respect, information

processing, communication, critical analysis, and self-awareness. These cri-

teria were designed to address some facets of the important aspects of

learning in PBL, including self-directed learning, effective skills in collabo-

ration and group-work, higher level thinking skills, and the ability to identify

ones weaknesses and strengths.In order to gain some measure of face validity, three experienced PBL

facilitators were asked to indicate whether each of the 17 items on the

instrument was relevant to PBL performance and able to be adequately as-

sessed using the item in question. Unanimous face validity was obtained for

all items in the four sub-scores: responsibility and respect, information

processing, communication, and critical analysis. Some dissent about the

validity of the self-awareness sub-score was evident. Construct validity de-

scribes the degree to which the items used in the instrument define the con-

structs (Pressley and McCormick, 1995). The five constructs or domains of

performance were reported extensively in the medical and nursing education

literature. Each of the three PBL tutors and ten PBL students were asked to

categorize the 17 items into the five specified domains. In all cases, the items

were distributed in accordance with the domains as defined on the instru-

ment. Values for Cronbachs alpha ranged from 0.76 to 0.84, indicating good

internal consistency among the five sub-scores. Acceptable reliability was

found, with Pearson correlation coefficients for peer- and tutor-assessment

ranging from 0.40 to 0.60. Notably, self-awareness items were problematicwith a significant number of students consistently entering not applicable

for those two items.

Unfortunately, time constraints prevented further renegotiation of the

Peer Assessment Instrument or the details of the peer-assessment procedure

with the second cohort prior to the commencement of the main study.

Main study

The main study was undertaken with the following cohort of medical stu-dents, using the instrument and the refined design of the previous cohort.

Written student feedback was collected during, and at the conclusion of, the

main study. This feedback was applied to modify aspects of the larger study

framework, only some of which included the peer-assessment process itself.



8/18

The intervention phase of the study involved participation of intervention

groups, for 20 weeks, in an activity designed to promote metacognition and

enhance learning. As part of this larger framework, students were given

practice in peer assessment for approximately 20 min each week as an inte-gral part of the wrap-up phase of PBL. It was anticipated that repeated

exposure to a system of participative assessment would create a culture of

enthusiastic acceptance, with greater student support for the process, leading

perhaps to greater satisfaction with the PBL process operating within tutorial

groups. Control groups were not involved in this process.

The intervention itself comprised two key components, both of which were

readily integrated into the existing tutorial format with minimal additional

time required from tutors or students:1. Reflection on learning: Each week one student from each intervention

tutorial group was asked to compose a summary of the weeks problem

incorporating the clinical reasoning and collaborative learning occurring

in their PBL tutorial group. One student was encouraged to present the

summary to the group as a concept map or in mechanistic case-dia-

gramming format (see Guerrero, 2001) to give a visual representation of

both the content and the clinical reasoning (a knowledge object (Ent-

wistle and Marton, 1994)).

2. Participative assessment: The student presentation, in association with

fulfilment of PBL roles and responsibilities for that week, was assessed

using the Peer Assessment Instrument (refined through student negoti-

ation with the previous cohort). Both peer- and tutor-assessment took

place concurrently. Scores from these worksheets were compared to look

for correlation between pairs of scores as part of a quantitative study of

peer assessment.

Constant monitoring of student perceptions of, and attitudes to, the

intervention helped to monitor the effects of the dual activities to maximizestudent learning. This is in keeping with action research.

In the first week of the main study (week four of the academic year),

students in the intervention cohort were distributed two documents: a copy of

the Peer Assessment Instrument to enable students to become familiar with

the evaluation criteria, and an exemplar outlining good and poor out-

comes for the criteria. Tutors assigned to each intervention group then led

their group in a practice session, with a tutor-led presentation of a summary

of the previous weeks medical case, in order to establish familiarity with the

instrument. The summary itself was written by the researcher and presented,

with explanation, to each tutor in the week prior to the trial.

In the ensuing weeks (526 of the academic year), intervention tutorial

groups implemented the summarization and peer-assessment activities at the

start of each week as part of the wrap-up of the previous weeks medical



9/18

case. Tutors were encouraged to give concise feedback (based on written peer

comments on the reverse of the assessment sheet) to students as soon as

possible after the complete on of the peer-assessment procedure. Scores ob-

tained from the Peer Assessment Instrument were collated and analysed.These quantitative results are to be reported elsewhere.

Student feedback about the peer-assessment exercise was regularly invited

as part of the action research process. Many groups chose not to reveal their

opinions, therefore more in-depth feedback from selected intervention tuto-

rial groups was obtained using one-on-one interviews. One major criticism of

the peer-assessment process was identified student preference for more

prompt, anonymous, and private feedback from the tutor (based on peer

scores and written responses). This feedback was used to modify the mainstudy design and tutors were advised to provide this form of feedback to

students as much as possible.

The attitudes of a randomly chosen control tutorial group to peer

assessment were also sought at this time. Tutor feedback was also collected at

10-week intervals. During the course of the implementation, three tutorial

groups withdrew from the study. Their justification for withdrawal and

perceptions of their experiences of peer assessment will be reported. The

remaining ten groups in the intervention cohort continued until week 28 of

the academic year. Qualitative data were sought from these groups.

DATA ANALYSIS

Responses to all open questions about peer-assessment and concise obser-

vations of focus groups were collated into a single document. Raw data

themes, in the form of direct quotations, functioned as the essential unit of

analysis. In keeping with the context of qualitative research, inductive data

analysis strategies were used with themes and categories emerging from the

data rather than the data being grouped into pre-determined categories

(Patton, 1990).

Results

The inductive data analysis procedure resulted in the identification of six main

themes. The first two are positive perceptions of peer assessment increased

responsibility for others, and improved learning. The final four themes reflect

negative student perceptions lack of relevancy, challenges, discomfort, andeffects on the PBL process. While the literature supports five of these themes,

the final theme which encompasses the potentially negative consequences of

implementing peer assessment on the PBL process has not previously been

described. However, Eva (2001) hypothesized that this effect may become

evident in PBL tutorial groups when peer assessment is implemented.



10/18

PERCEIVED BENEFITS OF PEER ASSESSMENT

Increased responsibility for others

Concern for the potential effects of their feedback on the learning of theirpeers led students to make comments such as: Peer review is helpful (be-

cause) it keeps me mindful of the fact that all my input is ... important to the

learning of the whole group. Another students comment: It keeps you on

your toes and challenges you to come up with (feedback) that helps others

shows how strongly some students felt about producing quality feedback for

peer benefit in the collaborative learning environment of PBL.

Improved learningIn this research, assessment of participation and leadership, in which the

contribution of students towards the process of learning was evaluated and

rewarded, was undertaken as a meaningful form of assessment of process (as

supported by Armstrong and Boud, 1983). Two main types of advantages were

described: immediate benefits to learning and long-term, sustainable benefits.

Immediate benefits. The positive consequences of constructive criticism

were reported on several occasions by participants. Students felt their peers

were in an appropriate position to judge their performance as functioningmembers of the PBL group: (peer-assessment) gives you a good idea of....

how you could improve and gives lots of ... ideas and it is good to get

feedback from peers so that you know how you are going and where your

strengths and weaknesses are. Through feedback from peers, opportunities

for students to engage in self-evaluation were provided. This assisted students

to identify deficiencies in their understanding and skills that may otherwise

not become readily apparent, thereby enhancing self directed learning.

Long-term benefits. Comments on future benefits to learning from thepractice of peer assessment were made by a small number of participants.

Mindful of a future as a medical professional, one student explained the

benefits of her peer-assessment experience: ... good to learn how to do this

appropriately, as I think we will need to be able to assess our peers per-

formance, as well as our own, throughout our careers. Students made ref-

erences to reflecting on strengths and weaknesses as a means of improving

future performance.

CRITICISMS OF PEER ASSESSMENT

Lack of relevancy

Despite the process of negotiation of assessment criteria with a previous

cohort of students followed by repeated practice at peer assessment,



11/18

perceptions that the exercise lacked relevancy persisted. Two groups of re-

sponses were able to be identified in this category: irrelevant criteria, and

sceptical attitudes.

Irrelevant criteria. Some students failed to see the relationship between

the criteria on the feedback sheet and the aspects of PBL performance they

were expected to assess. One student did not view the activities of PBL as

sufficiently important to their learning as to warrant assessment: Most of the

questions were irrelevant to the learning process. Another respondent was

dissatisfied with the criteria: (I) dont feel that the assessment sheets enable

us to give relevant feedback.

Sceptical attitudes. Frequent comments such as: Not taken too seriously

and Not too much thought goes into the marking reflect a casual attitude tothe peer-assessment process. Some students were overt in their scepticism,

with 23% of the intervention cohort (three groups) withdrawing from the

peer-assessment activity before its completion partly for that reason.

Challenges

The difficult nature of peer assessment was frequently described by students.

Qualitative data analysis of students comments allowed two lower ordercategories to be identified: newness of peer assessment and lack of motivation.

Newness of peer assessment. A common complaint from student asses-

sors was difficulty with the scoring system of the feedback sheet. These

problems were lack of familiarity and difficulties with the Likert scale. In

addition, a small number of comments were made concerning the alien

nature of peer assessment. One student noted: Unless you have a couple of

weeks to get used to it, students dont really know how to. Nevertheless,

students were given 20 weeks to become accustomed to the practice and this

comment was received in the final week.

Lack of motivation. A minority of students expressed a dislike for peer-

assessment attributable to apathy. One individual comment highlighted the

nature of this perception:

I found myself consistently giving fives for most areasIt was easier

than reading through every criteria and grading the personI think Im

not alone in doing this.

Discomfort

Two lower order categories encompassing discomfort were identified: prob-

lems with objectivity and need for greater anonymity.



12/18

Problems with objectivity. A strong reaction to peer assessment was the

widespread perception that this process could be corrupted by bias due to

friendship marking or lack of honesty. Analysis of data from students who

left prematurely from the study indicated this was a very important influenceon the decision to withdraw. The following set of comments sums up the

attitude among many study participants:

I find it difficult to downgrade my peers.

(It is) hard to criticize friends.

Relationships between students can colour opinions.

No one wants to criticise others in PBL.

Most people are too afraid to honestly mark their peers.

An often reported criticism of peer assessment among students in this

study was lack of confidence in the evaluation skills of peers, for example,

Peers are not qualified to mark....They can be biased or picky or too lenient.

One student expressed greater confidence in his own assessment abilities: I

think my judgement is better than that of my peers.

Anonymity. A minority of students expressed concern at the potential for

others to discern the source of individual scores or comments. Two signifi-

cant remarks include: I think (peer assessment) is great but (it) needs to be

done more anonymously and It is very difficult to assess peers within the

confines of the PBL room, especially when we hand the forms in to the tutor

through many sets of hands.

Some students may be more sensitive than others to the negative conse-

quences of anonymity being compromised (an unlikely scenario given the

study design which preserved anonymity to a high degree). It was notable

that one student praised the highly confidential nature of her peer assessment

experience: Its good to be able to do this anonymously a bit too con-

frontational face-to-face.

Effects on the PBL process

Serious reservations about the negative impact of peer assessment on the

integrity and interactive functioning of the PBL tutorial group were dem-

onstrated in this research. Both students and tutors revealed their concerns

that the family atmosphere may be compromised by peer evaluation. One

student commented:

It promotes judgmental attitudes (and) tension, and destroys harmoni-

ous learning. By not having to rate my peers, (we could) learn in a

friendly, non-judgemental environment. We (could) ask questions

without fear of embarrassment or humiliation.



13/18

Within their PBL tutorials, students need to feel free to hypothesize, to ask

questions and request clarification of points raised by others in order to

create a dynamic learning environment (Woods, 1994). Some students rec-

ognized that the collaborative learning within their small group tutorials, sodependent on affable working relationships, could be disrupted by ill-feeling

brought about by negative peer evaluations. This susceptibility was also

recognized by an experienced PBL tutor:

I would not like to see a competitiveness develop in a PBL group.

Everybody should feel comfortable to express their opinions, ask

questions, and generally just have a go without a fear of being assessed.

These potentially negative consequences were not only identified by stu-dents and tutors experiencing peer assessment, but were anticipated by stu-

dents in a randomly selected control group. One student expressed her

expectation:

The lack of a threatening environment is what allows us to discuss ideas

and concepts without fear of being rated or judged. In an environment

like this, feeling safe and respected in your group is paramount... this

would be screwed up by a scheme that asked you to rate your friends.

Discussion

Much of the data is congruent with findings in all disciplines of higher

education. Students have expressed appreciation for the manner in which

peer assessment enhances their learning, both short-term and in anticipation

of future learning. Peer assessment was considered to strengthen the sense of

responsibility that group members have for one another. These positive as-

pects of peer assessment can be regarded as endorsing the PBL approach,with its emphasis on collaborative and meaningful learning. Some students

were enthusiastic and committed to assessment of their peers, endeavouring

to give valid and helpful feedback to support others learning and to gra-

ciously receive feedback from other PBL group members. Initial quantitative

data analysis suggests that these individuals were more accurate peer

assessors.

Several studies undertaken with students in non-medical courses have also

revealed strong positive perceptions of peer assessment encouraging stu-dents to take a more active and constructive role in cooperative learning

(Hanrahan et al., 2001; Orsmond and Merry, 1996), and to develop attitudes

of responsibility for the learning of other group members (Burnett and

Cavaye, 1980; Topping et al., 2000). Furthermore, studies show peer

assessment enhances learning by allowing students to observe how others



14/18

study and write (Ballantyne et al., 2002; Hanrahan and Isaacs, 2001), and by

training students to differentiate high from poor quality work (Race, 1998;

Searby and Ewers, 1997).

Negative perceptions were also strongly apparent in this research. A sig-nificant number of respondents believed that peer assessment was irrelevant

to the important aspects of learning in PBL. Scepticism and feelings of dis-

comfort were very commonly reported. Of all five sub-scores, self-awareness

was the most problematic during negotiation of items for the instrument

and in terms of student and tutor acceptability. Upon reflection, omission of

this sub-score may have improved student confidence in the peer-assessment

process (despite the principal role of self-awareness in PBL). While the PBL

tutorial group offers the students an opportunity to embrace unfamiliarlearning activities, such as collaborative small group learning with its

emphasis on peer-to-peer interaction and communication, peer assessment

may be the final straw for those students struggling to adapt to this learning

environment. It may be that learning environments that are highly dependent

on effective collaboration are not well suited to peer assessment. Alterna-

tively, the nature of peer assessment undertaken in highly collaborative

learning environments may dictate a requirement for different criteria.

Negative perceptions of peer assessment were anticipated in this research

based on the findings of other studies in higher education.

Studies in non-medical courses have found widespread scepticism among

students concerning the validity of peer assessment aligned with beliefs that

assessment should be the sole responsibility of staff (Brindley and Scoffield,

1998; Orsmond and Merry, 1996; Searby and Ewers, 1997). Apprehension

about the arbitrary nature of marking are a major drawback for students

experiencing peer assessment (Ballantyne et al., 2002; Cheng and Warren,

1999). Pond and Rehan (1997) discuss the need to consider the problems of

friendship marking (leading to over-marking) and collusive marking (resultingin lack of differentiation within groups) when considering the introduction of

peer assessment. In his influential article on student attitudes towards assess-

ment, Williams (1992) reported that students viewed peer assessment as criti-

cism of friends. The recommendations made to address this perception, which

included anonymous feedback from groups of peers with staff involvement to

reduce bias, were adopted in this study with little apparent impact.

It is a common finding that students struggle with the newness of peer

assessment as a formal assessment tool (Cheng and Warren, 1999; Sluijmans

et al., 2001). They often have little prior experience with the procedure.

Criteria for peer assessment can create additional problems as some students

have different understandings of individual criteria compared to their peers

and tutor, particularly criteria requiring higher order thinking (Orsmond

et al., 1996,2000). Lack of familiarity may also lead to feelings of



15/18

awkwardness. Students in many studies, including studies of cooperative

learning in postgraduate courses (Divaharan and Atputhasamy, 2002) and

PBL courses in education (Segers and Dochy, 2001; Sluijmans et al., 2001)

describe the discomfort of peer assessment.A significant concern unique to peer assessment in PBL settings (and

apparently fairly unique to this study) was revealed: students perceived that

peer assessment had the potential to negatively impact on PBL. The intimate

quality of relationships within PBL tutorials, which has often been embraced

for its ability to improve self-evaluation and communication, also makes

students reluctant to rate their peers for the purposes of evaluation (Van

Rosendaal and Jennett, 1992). As Eva (2001) comments in his critique of

tutorial-based assessment:

There is a real concern that conducting evaluation in the tutorial setting

may negatively impact on the process itself. So, paradoxically, while the

tutorial is the best place to observe these behaviours, the act of obser-

vation and evaluation may inhibit their demonstration. (p. 244)

There are few studies exploring the effects of peer assessment on the process

of problem-based or case-based learning. Sluijmans et al. (2001) explored

several aspects of peer assessment in a PBL course, collecting mostly quan-

titative data, but also noting that students were aware of the serious conse-

quences of negative critiques within the small group environment. Divaharan

and Atputhasamy (2002) described the unease experienced by students in the

face-to-face assessment of peers within the confines of cooperative small

groups, which had the potential to interfere with peer relationships. Boud

et al. (1999), in their discussion of the limitations and contradictions of peer

assessment, report that formal assessment of others within small group

tutorials can inhibit cooperation, pitting one student against another. How-

ever, peers are a source of valuable and valid feedback which can assistlearners to make judgements about themselves and their learning in PBL.

Conclusion

The results from this study support and extend research in higher edu-

cation on the positive and negative perceptions of students about peer

assessment. However, the qualitative data, gathered through ethnographic

enquiry, presents a more in-depth representation of student attitudes to

the introduction of peer assessment into a PBL course within a medical

school.

The study design adopted in this research followed many guidelines

promoted in the literature. This included, but was not limited to, the use of

student-negotiated criteria for assessment, provision of exemplars of



16/18

performance, and student practice in peer assessment to gain familiarity.

Despite implementation of these recommendations, student perceptions of

their peer-assessment experience remained quite negative. It may be that

students need years of practice in peer assessment in order to become com-fortable with the process. Even though students were aware that peer-

assessment criteria were developed through negotiation with individuals in

the previous cohort, time constraints early in the academic year prevented

participating students from renegotiating criteria. This minimised ownership

of the assessment process, an important consideration in enhancing student

acceptance of participative assessment (Boud, 1995; Strahan and Wilcox,

1996). Student ownership of criteria for peer evaluation may be crucial to its

acceptance as a meaningful activity.The learning advantages offered by peer assessment are quite significant

and warrant further research. Productive benefits may be gained through

both receipt of feedback and provision of meaningful feedback to others.

These types of activities can improve confidence and enhance learning (Black

and Wiliam, 1998; Woods, 1994). Peer-assessment needs to be further ex-

plored with students in PBL and other cooperative small group settings,

including evaluation of fulfilment of the roles and responsibilities of group

members (perhaps utilising criteria reflecting what students value most from

their PBL experience). While the highly structured format of peer assessment

was viewed negatively by many students, an unprompted, anonymous and

open-ended design may be more acceptable. Such an approach may enable

students to give feedback which is relevant to their perceptions of valuable

learning and meaningful evaluation in PBL. Black and Wiliam (1998) found

that an approach to providing student feedback that involved writing com-

ments on work, but not assigning scores, resulted in greater learning.

An integral part of the PBL tutorial process is the willingness and ability

of students to embrace their duties as active members of a group of learners.Principles such as critical analysis, self-awareness, and information process-

ing play an important role in the development of self-direction as a learner.

One of the gauges of quality in tertiary education is the extent to which self-

learning ability is nurtured (Oldfield et al., 1995). A key aspect of this is

developing the ability to judge objectively the quality of their own and others

work.

References

Armstrong, M.T. & Boud, D. (1983). Assessing class participation: An exploration of the issues. Studies in

Higher Education 8(1): 3344.

Ballantyne, R., Hughes, K. & Mylonas, A. (2002). Developing procedures for implementing peer

assessment in large classes using an action research process. Assessment & Evaluation in Higher Edu-

cation 27(5): 427441.



17/18

Barrows, H.S. & Tamblyn, R.M. (1980). Problem-Based Learning: An Approach to Medical Education,

New York: Springer.

Bishop, W. (1999). Ethnographic Writing Research, Portsmouth, NH: Boynton/Cook Publishers.

Black, P. & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education 5(1): 774.

Boud, D. (1995). Enhancing Learning Through Self-Assessment, London: Kogan Page.

Boud, D., Cohen, R. & Sampson, J. (1999). Peer learning and assessment. Assessment & Evaluation in


Brindley, C. & Scoffield, S. (1998). Peer assessment in undergraduate programmes. Teaching in Higher

Education 3(1): 7989.

Burnett, W. & Cavaye, G. (1980). Peer assessment by fifth year students of surgery. Assessment in Higher

Education 5(3): 273278.

Cheng, W. & Warren, M. (1999). Peer and teacher assessment of the oral and written tasks of a group

project. Assessment & Evaluation in Higher Education 24(3): 301314.

Collins, M. (1991). On contemporary practice and research: Self-directed learning to critical theory. In: R.

Edwards, A. Hanson & P. Raggatt (eds.), Boundaries of Adult Learning, London: Routledge/Open

University Press.

Dancer, D. & Kamvounias, P. (2005). Student involvement in assessment: A project designed to assess

class participation fairly and reliably. Assessment & Evaluation in Higher Education 30(4): 445454.

Das, M., Mpofu, D., Dunn, E. & Lanphear, J.H. (1998). Self and tutor evaluations in problem-based

learning tutorials: is there a relationship? Medical Education 32(4): 411418.

Divaharan, S. & Atputhasamy, L. (2002). An attempt to enhance the quality of cooperative learning

through peer assessment. Journal of Educational Inquiry 3(2): 7283.

Dochy, F., Segers, M. & Sluijmans, D. (1999). The use of self-, peer-, and co-assessment in higher

education. Studies in Higher Education 24(3): 331350.

Dolmans, D.H. & Schmidt, H.G. (1996). The advantages of problem-based curricula. Postgraduate

Medical Journal 72(851): 535538.

Drew, S. (2001). Student perceptions of what helps them learn and develop in higher education. Teaching

in Higher Education 6(3): 309331.

Duffield, K.E. & Spencer, J.A. (2002). A survey of medical students views about the purposes and fairness

of assessment. Medical Education 36(9): 879886.

Entwistle, N.J. & Marton, F. (1994). Knowledge objects: Understandings constituted through intensive

academic study. British Journal of Educational Psychology 64(1): 161178.

Eva, K.W. (2001). Assessing tutorial-based assessment. Advances in Health Sciences Education 6(3): 243

257.

Gale, K., Martin, K. & McQueen, G. (2002). Triadic assessment. Assessment & Evaluation in Higher

Education 27(6): 557567.

Guerrero, A.P.S. (2001). Mechanistic case diagramming: A tool for problem-based learning. Academic

Medicine 76(4): 385389.

Hanrahan, S.J. & Isaacs, G. (2001). Assessing self- and peer-assessment: The students view. Higher

Education Research and Development 20(1): 5370.

Hay, P.J. & Katsikitis, M. (2001). The expert in problem-based and case-based learning: Necessary or

not?. Medical Education 35(1): 2228.

Kamil, M., Langer, J.A. & Shanahan, T. (1985). Understanding Reading and Writing Research, Boston:

Allyn & Bacon.

Kelson, A.C.M. & Distlehorst, L.H. (2000). Groups in problem-based learning (PBL): Essential elements

in theory and practice. In: D.H. Evenson & C.E. Hmelo (eds.), Problem-Based Learning: A Research

Perspective on Learning Interactions, Mahwah, NJ: Lawrence Erlbaum Associates.

Kemmis, S. & Wilkinson, M. (1998). Participatory action research and the study of practice. In: B. Atweh,

S. Kemmis & P. Weeks (eds.), Action Research in Practice: Partnerships for Social Justice in Education,

pp. 2136. London: Routledge.

Leung, W. (2002). Why is evidence from ethnographic and discourse research needed in medical education:

the case of problem-based learning. Medical Teacher 24(2): 169172.

McDowell, L. (1995). The impact of innovative assessment on student learning. Innovation in Education

and Training International 32(4): 302313.



18/18

Margetson, D. (1994). Current educational reform and the significance of problem-based learning. Studies

in Higher Education 19(1): 519.

Miller, P.J. (2003). The effect of scoring criteria specificity on peer and self assessment. Assessment &

Evaluation in Higher Education 28(4): 383394.

Mowl, G. & Pain, R. (1995). Using self and peer assessment to improve students essay writing: A case

study from geography. Innovation in Education and Training International 32(4): 324335.

Norman, G. & Schmidt, H.G. (1992). The psychological basis of problem-based learning: A review of the

evidence. Academic Medicine 67(9): 557565.

Norton, L., Brunas-Wagstaff, J. (2000). Students perceptions of the fairness of assessment. Paper pre-

sented at the Institute for Learning and Teaching in Higher Education Annual Conference, College of

Ripon and York, St John.

Oldfield, K.A., MacAlpine, J. & Mark, K. (1995). Peer and self-assessment at tertiary level: An experi-

ential report. Assessment & Evaluation in Higher Education 20(1): 125132.

Orsmond, P. & Merry, S. (1996). The importance of marking criteria in the use of peer assessment.

Assessment & Evaluation in Higher Education 21(3): 239250.

Orsmond, P., Merry, S. & Reiling, K. (2000). The use of student derived marking criteria in peer- and self-

assessment. Assessment & Evaluation in Higher Education 25(1): 2338.

Patton, M.Q. (1990). Qualitative Evaluation and Research Methods, Newbury Park, CA: Sage Publications.

Pond, K. & Rehan, U.H. (1997). Learning to assess students using peer review. Studies in Educational

Evaluation 23(4): 331348.

Pressley, M. & McCormick, C.B. (1995). Advanced Educational Psychology for Educators, Researchers and

Policymakers, New York: Harper Collins College Publishers.

Punch, K.F. (1998). Introduction to Social Research, London: Sage.

Race, P. (1998). Practical pointers on peer-assessment. In: S. Brown (ed.), Peer Assessment In Practice,

SEDA Paper 102, Birmingham: SEDA.

Reiter, H.I., Eva, K.W., Hatala, R.M. & Norman, G.R. (2002). Self and peer assessment in tutorials:

Application of a relative-ranking model. Academic Medicine 77(11): 11341139.

Sambell, K. & McDowell, L. (1998). The value of self and peer assessment to the developing of lifelong

learning. In: C. Rust (ed.), Improving Student Learning: Improving Students as Learners, Oxford: Oxford

Centre for Staff and Learning Development.

Searby, M. & Ewers, T. (1997). An evaluation of the use of peer assessment in higher education: A case

study in the school of music. Assessment & Evaluation in Higher Education 22(4): 371383.

Segers, M. & Dochy, F. (2001). New assessment forms in problem-based learning: The value-added of the

students perspective. Studies in Higher Education 26(3): 327343.

Sluijmans, D.M.A., Moerkerke, G., Merrienboer, J.J.G.van & Dochy, F.J.R.C. (2001). Peer assessment in

problem-based learning. Studies in Educational Evaluation 27(2): 153173.

Somervell, H. (1993). Issues in assessment, enterprise and higher education: The case for self-, peer- and

collaborative assessment. Assessment & Evaluation in Higher Education 18(3): 221233.

Strahan, I.B. & Wilcox, S. (1996). Peer and self-assessment of group work: Developing an effective

response to increased enrolment in a third-year course in microclimatology. Journal of Geography in


Sullivan, M.E., Hitchcock, M.A. & Dunnington, G.L. (1999). Peer and self assessment during problem-

based tutorials. The American Journal of Surgery 177(3): 266269.

Topping, K.J., Smith, E.F., Swanson, I. & Elliot, A. (2000). Formative peer assessment of academic

writing between postgraduate students. Assessment & Evaluation in Higher Education 25(2): 146169.

Van Rosendaal, G.M.A. & Jennett, P.A. (1992). Resistance to peer evaluation in an internal medicine

residency. Academic Medicine 67(1): 63.

Williams, E. (1992). Student attitudes towards approaches to learning and assessment. Assessment &

Evaluation in Higher Education 17(1): 4558.

Willis, S.C., Jones, A., Bundy, C., Burdett, K., Whitehouse, C.R. & ONeill, P.A. (2002). Small-group

work and assessment in a PBL curriculum: A qualitative and quantitative evaluation of student per-

ceptions of the process of working in small groups and its assessment. Medical Teacher 24(5): 495501.

Woods, D.R. (1994). Problem-based Learning: How to Gain the Most from PBL, Waterdown, ON: Donald

R. Woods.


Documents

Peer Assessment in PBL