Peer Review: a Precursor to Peer Assessment

This article was downloaded by: [The University of Manchester Library]On: 11 October 2014, At: 03:47Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Innovations in Education & TrainingInternationalPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/riie19

Peer Review: a Precursor to PeerAssessmentKeith Pond a , Rehan Ul‐Haq a & Winnie Wade a

a Loughborough University of TechnologyPublished online: 09 Jul 2006.

To cite this article: Keith Pond , Rehan Ul‐Haq & Winnie Wade (1995) Peer Review: a Precursorto Peer Assessment, Innovations in Education & Training International, 32:4, 314-323, DOI:10.1080/1355800950320403

To link to this article: http://dx.doi.org/10.1080/1355800950320403

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/loi/riie19

http://www.tandfonline.com/action/showCitFormats?doi=10.1080/1355800950320403

http://dx.doi.org/10.1080/1355800950320403

http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/page/terms-and-conditions

314 IETI 32, 4

Peer Review: a Precursor to Peer Assessment

Keith Pond, Rehan Ul-Haq, Winnie Wade, Loughborough University of Technology

SUMMARY

This paper describes a two-year observation of peer review among first year undergraduates at Loughbor-ough University of Technology. Peer review focuses on the learning benefits of small group work and thedevelopment of critical skills among students as a precursor to peer assessment of tutorial contributions.The paper outlines the methodology, design and pitfalls of the peer review process and offers tentativeinterpretations of some peer generated marks. The findings of the two-year study generally support theavailable literature regarding reliability of peer generated marks and underlines the importance of'ownership'and understanding of review criteria. The exercise has provided the necessary experience anddetailed feedback to harness its more positive lessons for the launch of peer assessment in the future.

INTRODUCTIONInstitutions of higher education are being affectedby enormous changes. Student numbers have risendramatically since 1991 without a concomitant in-crease in resources and the composition of the stu-dent population has also changed considerably.More mature and part-time students are now enter-ing higher education and the proportion of studentswith non-standard entry qualifications has increased.Quality assessment and audit systems have been es-tablished and university departments are being re-quired to assess their teaching.

These changes present the educational system witha significant challenge: How is the quality of teach-ing to be maintained and enhanced while studentnumbers increase and the student population diver-sifies? A response that meets this challenge is a flex-ible approach to teaching and learning. Essentially,flexible learning has two features:

— it provides students with the opportunity to takegreater responsibility for their own learning;

— it enables students to be engaged in learning ac-tivities and opportunities that meet their ownneeds.

Flexible learning encompasses a wide range ofteaching, learning and assessment styles such asgroup work, student led seminars and peer assess-ment, all of which encourage a high level of stu-

dent involvement and which engage students in thelearning process.

Peer assessment

This paper is based on a study that focuses on theenhancement of the quality of student learning intutorials through the processes of peer review andpeer assessment. Brown (1994) stresses that 'peerassessment is not a panacea for the problem of cop-ing with increasing student numbers'. Although stu-dents are developing skills of evaluation and criticalthinking they need a great deal of support in thisdevelopmental process (Brown, 1994).

Assessment is one of the most important factors ina learning environment that influences students toadopt either a deep or a surface approach to learn-ing (Entwistle, 1992). It has also been found thatthe use of peer and self assessment may promotehigher levels of thinking (Falchikov, 1991). In ad-dition, Brown (1992) argues that peer assessment'raises awareness of the importance of group dy-namics as well as tasks in a group setting' (Brown,1992).

Staff reluctance to embark on peer assessment be-cause of fears that 'results may be unreliable' orstudents may feel resentful, (Habeshaw et al, 1993)can be overcome by introducing peer assessment atan early stage with students and by combining it

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 0

3:47

11

Oct

ober

201

4

Peer Review: a Precursor to Peer Assessment 315

with other assessment methods. The students are thusoperating in a safer environment that enables devel-opment of critical, evaluative skills.

Peer assessment criteria should be clearly identifiedand made explicit. Students should not only under-stand what they are being assessed on but also theweighting of the criteria. Criteria can be developedby staff, or by the students in collaboration with staff.If the students feel that they have been involved inthe generation of the criteria they will clearly under-stand what is required in the assessment process andare given some 'ownership' of it.

Assessment methods should be relevant, valid andencourage deep learning. If peer assessment is to besuccessful, it necessitates careful planning and anawareness that it can open up sensitive issues. How-ever, implementing peer assessment will result invery positive benefits for both students and staff.

Rationale for the current study

At Loughborough University Business School, onthe BSc (Hons) banking and finance degree, theundergraduates' learning structure is based aroundthe lecture and the tutorial, the lecture being a guidedtour of pre-read material, and the smaller tutorialbeing where discussion and consolidation of learn-ing is targeted. Assessment is by a mixture of ex-amination and coursework. Typically, examinationaccounts for 75 per cent of marks awarded.

The tutorial is a key element in the learning processbut, historically, has never been part of the assess-ment process. It was therefore decided to focus onthe enhancement of the quality of student learningin tutorials through peer review without initiallycomplicating the issue by also changing assessmentmethods.

The main objectives of the project reported in thispaper were to design and implement a peer reviewprocess in a first year undergraduate module. Byextending the project over two academic years wewere able to test design factors as well as the initialfindings themselves (initial findings, Pond and Ul-Haq, 1994). Having gained greater understandingof the peer review process we now feel confident inextending the concept to embrace peer assessmenttoo. We anticipate that peer assessment will use sim-ilar procedures to peer review, incorporating studenttraining and student led generation of assessmentcriteria. Careful monitoring will also be undertaken

to ensure that the additional factor of student influ-ence on assessed marks reflects the time and effortexpended by individuals.

METHODOLOGYThe first year banking law module is a compulsoryand essential foundation of the banking and financeprogramme. Banking law tutorial exercises repre-sent real life 'in-tray' situations, relating to specifictopic areas and highlighting the legal, commercialand practical aspects of the subject. In this way tuto-rial exercises encourage discussion and assist in pre-paring for end of year examinations too, sincequestions are of a similar format.

The two year study

The peer review study comprised five major stepsin 1993-94 and repeated three of these as the studycontinued into the 1994-95 academic year. Figure 1summarises the procedure that was followed.

During Term 1 of the 1993-94 and 1994-95 aca-demic years the banking law tutorials were run on alow structure basis. Students either worked in sub-groups (more prevalent in 1993-94) or individually.Each subgroup's brief was to prepare, in their owntime, one or more of the pre-set and pre-distributedquestions for presentation and discussion in the fol-lowing week's tutorial. In each tutorial four sub-groups presented their question responses and afollow-up discussion took place.

This approach, outlined above, and its learning out-comes, were evaluated at the start of Term 2(1993—94) using both formal and informal methods.In addition the level of student involvement in thetask was informally gauged by the tutors based onstudent performance in the tutorials.

These enquiries revealed that there was a dual needto encourage involvement in group work of all stu-dents and to improve student learning in tutorials.Several courses of action were suggested; the mostappropriate to the circumstances seemed to be thepeer review of subgroups. This option appeared tooffer the advantages of positive peer pressure, andenhanced opportunities for individuals to becomeinvolved in the learning process.

In Term 2 the Peer Review methodology was intro-duced to students via an additional 'training' tutorial

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 0

3:47

11

Oct

ober

201

4

316 IETI 32,4

1993-94

Term 1(weeks 6 to 10)

Start of Term 2

Week 4 ofTerm 2


Start of Term 3

Activity

Tutorials run on a Mowstructure' basis

Feedback on Term 1tutorials taken.

Criteria generation /training tutorial for eachtutorial group.

Tutorials run on PeerReview basis.


1994-95



Start ofTerm 3

Activity

Tutorials run on a 'lowstructure' basis.

None

None

Tutorials run on PeerReview basis.


Figure 1. Peer review scheme timetable

held in the fourth week of Term 2. Students wereasked to consider the criteria they considered ap-propriate for peer review. After presentation to thewider tutorial group these criteria were prioritized.The findings, across the four tutorial groups (16 sub-groups) showed that students valued the effort putinto research and group discussions over the moreabstract factors such as teamwork or willingness.This information was used to generate a peer reviewform. The criteria generated are shown in Figure 2.

During Term 2, tutorials were run on a structured ba-sis. Subgroup presentations in tutorials were followedby a period of peer review and a subjective review ofthe quality of subgroup output by tutors. At the endof Term 2 a student feedback exercise similar to theone relating to Term 1 tutorials, was performed.

In 1994-1995, the second year of the project, theprocedure was truncated and a number of designfactors were changed to discover their impact on theownership of and learning benefits from the peerreview process. The changes included the pre selec-tion of the review criteria by tutors rather than bystudents and the reduction in the number of obser-vations made by each subgroup (See Figure 2). Prob-lems encountered in 1993—94 were 'designed out'to avoid domination of groups by individuals (nor-mally more able students). Further feedback wastaken at the end of Term 2 (1994-95) together withthe subjective evaluation of sub group output.

The methodology reveals that this project was alearning process in itself and after two years we are

confident that the building blocks of peer assessmentcan be discerned.

STUDY FINDINGS

This section looks at the detailed feedback providedby students with regard to their tutorials in bankinglaw in the 1993-94 and 1994-95 academic years, Itpresents findings based on written feedback, peerreview scoring, subjective quality assessment andtutorial attendance. Tentative interpretations of thefindings are also offered although the authors arefully aware that these are open to alternative con-structions.

Student feedback

The output from the 1993-94 and 1994-95 feedbackmust, necessarily, be seen against the background ofthe novelty of undergraduate studies to the studentsin question. Terms 1 and 2 were, in fact, the first twoterms of undergraduate life that the students had ex-perienced. It could be expected that contribution totutorials would be enhanced in Term 2 regardless ofcourse structure due to increased familiarity with thesubject, the tutors and their peers. The quantitativefeedback appears to follow this logic.

Feedback on Term 1 (1993-94) tutorials gained a63.2 per cent response rate and highlighted studentdissatisfaction with the 'low structure' basis:

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 0

3:47

11

Oct

ober

201

4


1993-94Criteria*

Input

Preparation

AttendanceShareHandout

Teamwork

Presentation

1994-95Criteria**

Input

Preparation

—Share—

Teamwork

—

Basis of Assessment for Individual Students

Input to pre-tutorial discussion and to discussion withintutorialsDelivery of research inputs and considered contributionto pre-tutorial preparationsPhysical attendance at sub-group and tutorial meetingsShare of group tasks undertakenPreparation, design and photocopying of grouphandoutOrganizational skills, especially in developing inputsfrom other team membersTutorial presentation of group findings

Notes: Student generated and agreed.Tutor imposed.

Figure 2. Peer Review Criteria

— Tutorial sessions were considered by some stu-dents as being conducted at too fast a pace.

— There was perceived low commitment to partici-pation in the work of the subgroups.

— Subgroups were considered by some students tobe dominated by individuals.

In addition students felt that tutorials while informalwere relevant and supportive of lectures.

The informal feedback also highlighted:

— The haphazard organization, by subgroup mem-bers, of subgroup work and presentation.

— The lack of definitive notes for each student forrevision purposes.

— The tendency towards short, incomprehensivesubgroup presentations leading to stilted studentdiscussions.

In addition to this more qualitative feedback studentswere asked to record their agreement (or otherwise)with a series of statements about the tutorial processusing a five point 'satisfaction scale'. Of the state-ments relating to the tutorials themselves, (such as'the tutorials related well to the subject objectives'and 'the tutorials were useful in reinforcing lecturematerial') Term 2 (1993-94) saw a significant andconsistent increase in the 'satisfaction score'.

More variation and less certainty are apparent in theresponses to statements relating to the group pro-

cess, however. Broad agreement can be inferred,from the data, that the Term 2 group structure waseffective and useful. This interpretation is supportedby the qualitative feedback regarding the positiveeffects of peer pressure and group synergy.

In the second year of the peer review project 'satis-faction scores' for Term 2 show a marked decline insupport for the tutorial and group/peer process. Stu-dents acknowledged, however, their support for thecontention that group discussions were useful andthat groups worked well together.

Student feedback on their own performance in1993-94 was also enlightening. Term 2 showed sig-nificantly higher ratings than Term 1 for statementssuch as 'I was usually well prepared for tutorials'and 'I participated well in tutorial discussions' al-though variation in scoring was roughly similar. Thelowest score in each term was reserved for the state-ment that addressed student motivation. It seems thatthe subject material is not perceived to be motivat-ing in itself!

Lower scores were also recorded in 1994—95 reflect-ing the decline in student support for the process. Itis interesting to note, however, that the one questionscoring higher than in 1993-94 related to attend-ance. Indeed, attendance levels were, generally,higher than in the previous year. Attendance alone,however, does not necessarily benefit learning.

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 0

3:47

11

Oct

ober

201

4

318 IETI 32,4

Tutorial attendance

Absence from tutorials that have always been pre-sented as a compulsory and integral part of the mod-ule, may have many causes. The interpretation ofabsence data without reference to or measurementof these factors can render analysis fairly meaning-less. Indeed, the factors affecting absenteeism inbanking law tutorials are almost certainly not meas-ured by the peer review or feedback systems. How-ever, the collected data can still be fairly revealing.

The data in Table 1 show a general trend towardsgreater absenteeism towards the end of each termand a lower average attendance under peer reviewin Term 2 of each year. It could be anticipated fromattendance records prior to 1993 that Term 2 attend-ances would be lower and so peer review does notappear to have adversely affected this.

Table 1. Tutorial Attendance (%) (including agreedabsences and illness)

Year Term Week

1993-94N = 571994-95N = 47

1212

6/1698759696

Week7/1791809885

Week8/1895659481*

Week9/1979768894

Week10/20

46658379

Av%82729287

Note: *includes one whole sub-group that failed to attend.

By tracking individual absentees, using the peer re-view grading, it is clear that absence from a tutorialdid not equate with failure to contribute to grouppresentations. There is consistent evidence that in-dividuals who were absent from the tutorial attendedpre-tutorial subgroup meetings and contributed totutorial preparation. The learning benefit of subgroupdiscussion was, therefore, available without tutorialattendance.

Peer review process

In focusing on peer review of group work, rather thanconducting a full exercise in peer assessment, thisproject has enabled the mechanics of the process tobe isolated and examined. Without the burden of al-locating or sharing responsibility for allocating marksto peers it was felt that the nature of the review pro-cess would be more transparent. In addition, the ad-vantage of the development of students' critical skillsthat peer assessment often brings is not lost.

Common problems with peer assessment had to beconsidered for this project too. Relevant literature(Brown and Knight, 1994) and informal conversa-tion with colleagues suggested that the followingcould often be observed:

— 'Friendship marking', resulting in over-marking.— 'Collusive marking', resulting in a lack of differ-

entiation within groups.— 'Decibel marking', where the noisiest get the high-

est marks.— 'Parasite marking', where students 'piggy back'

on group marks.

Any one, or a combination, of these factors couldinvalidate peer generated marks, which could haveserious consequences if they are used in the assess-ment of degree grades. Indeed, the more importantthe level of the assessment, the more the above fac-tors could come into play. Received wisdom sug-gests that such factors can be overcome in peerassessment by careful selection of assessment crite-ria and by requiring evidence to support each markawarded. In the case of the present project the 'own-ership' of review criteria was tested. In both yearsof the project it was made clear that the review marksawarded would have no bearing on formal assess-ment, thus removing much of the motivation to markdishonestly.

In this section of the paper the dimensions of peerreview marking are explored via the weekly reviewforms completed by students following the grouppresentations during tutorial sessions. The actualmarks of individual students are far less important, inthis context, than the apparent influences on review-ers. Where necessary individual's marks have beencompared with the independently checked attendanceregister and comparisons have been made with thetutors' subjective assessment of output quality.

Averaging and equitable allocation of marks

In both years of the project it was decided to use anodd number of review grades (see Figure 3) with1994—95 giving the opportunity for more sensitivemarking. Guidance given to students suggested thatsubgroup marks would centre on a mean of 'B ' in1993-94 or ' C in 1994-95. In both years thesegrades meant 'some contribution'. Peer review markswere awarded for each student against the pre-determined criteria shown in Figure 2.

In 1993—94 the distribution of marks appeared to befairly even for most of the criteria with a distinct

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 0

3:47

11

Oct

ober

201

4


1993-94

A(3) Major contribution

B(2) Some contribution

C(1) Little contribution

1994-95

A(5) Substantial contributionB(4) Significant contributionC(3) Some contributionD(2) Little contributionE(1) No contribution but attended group meetingsF(0) No contribution — did not attend group meetings

Figure 3. Review Grading (with corresponding marks for arithmetical analysis)

bias for higher scores in the important 'input to dis-cussion' (input), 'preparation and research' (prep-aration) and 'Attendance at subgroup and tutorialmeetings' (attendance) criteria. In these categoriesgroups recorded average marks of two and above in60 per cent of cases (67 per cent for attendance). The'share of work' (share) and 'teamwork' (teamwork)criteria were more evenly marked with only 53 percent scoring two or above as shown in Table 2.

The scoring for the remaining criteria of 'handoutpreparation' (handout) and 'presentation in tutorial'(presentation) were not marked as highly with only33 per cent of subgroups averaging a score of twoand above. This would suggest that these two taskswere performed on a rota basis and individualsmarked with an 'A' if it was their turn and with a' C if it was not. Perhaps certain individuals domi-nated these particular tasks because they possessedparticular skills or had more ready access to a com-puter. There is certainly evidence to support both ofthese possibilities within the peer review feedback.

The findings on the administrative criteria of hand-

out, presentation and attendance gave support forthe redesign of the 1994-95 scheme. In 1994-95students were told to ensure that handout prepara-tion and tutorial presentations were arranged on arota basis. By naming the handout preparer and thepresenter each week on the peer review sheet theeffectiveness of the informal rota could be checked.In practice the informal rota system worked well withall but one subgroup sharing the tasks around thegroup.

The distribution of subgroup average marks in1994-95, however, paints a very different picture tothe previous year. Despite the expanded grading scalegiving the opportunity for more sensitive marking,marks were skewed significantly towards the upperend of the scale. Marks were higher, on average, andmore polarized for three of the four criteria. In bothyears there was a greater tendency towards highermarks as the term progressed.

Marks awarded to individuals within subgroupsbroadly support the above findings in both years.Table 3 summarizes individuals' average scores over

Table 2. Distribution of Sub-Group Average Marks (% of sub-groups)

Criteria

InputPreparationAttendanceShareHandoutTeamworkPresentation

Lower93-94

40403347674767

94-95

1717—8—25—

Middle93-94

1371460206

94-95

00—17—0-

Higher93-94

47535347333327

94-95

4883—75—75—

Note: 1993-94 Middle score = 2 (B); 1994-95 Middle score = 3 (C). Both scores related to 'some contribution'.

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 0

3:47

11

Oct

ober

201

4

320 IETI 32,4

Table 3. Distribution of Individuals Average Marks (% of students)

Criteria


Lower93-94

24433230563544

94-95

1715—15—19—

Middle93-94

2891124112624

94-95

611—8—2—

Higher93-94

48485746333932

94-95

7774—77—79—

Note: 1993—94 Middle score = 2 (B); 1994—95 Middle score = 3 (C). Both scores related to 'some contribution'.1993—94 Individual scores exclude two students who left the course before the end of the term.

the five weeks of tutorials, again showing a fairlybalanced picture for 1993-94 and a more polarizedone for 1994—95. Findings generally, however, un-derline the ability of students to differentiate betweenlevels of contribution. In 1994-95 the full range ofmarks above three were used.

In 1993—94 higher marks for individuals could beexplained by the friendship marking phenomenondespite subgroup selection having been on a morerandom basis in order to reduce the effects of thisparticular bias. (Tutors also need to be aware of theindividual students who sMxacXparasite marking&ndprovide no other contribution to the group.) What iscertain, however, is that where the handout is to beused for group or peer assessment purposes, groupsneed to be advised to vet the output to ensure that allcontributions are reflected. Such vetting and originalinput drafts could also provide valuable evidence tosupport individual grading under peer review or peerassessment.

The 1994-95 data show less differentiation betweenreview criteria and similar patterns of high marks andpolarization as the subgroup data. The removal of thethree administrative criteria, however, appears to haveendowed the teamwork and share criteria with addi-tional attributes. Where individuals prepared the hand-out or presented the group's findings their teamworkand share marks were higher. Since these tasks wererotated the effect was averaged out.

This gives greater support to the view that lack ofownership of criteria in 1994-95 allowed somemisinterpretation of these criteria.

Marking consistency

By calculating a standard deviation statistic in re-spect of the marks awarded by students some meas-ure of the variability in marking, both for groupsand for individuals as the term progressed, could beestimated. From the plentiful literature on peer as-sessment, outlined above, it was thought that someconsistency in marking would be established for theimportant input and preparation criteria as the sameindividuals contributed to these areas each week.Greater variability could, therefore, be expected inthe handout and presentation criteria.

The low standard deviation can be interpreted as a formof collusive marking, the uniform marking of eachgroup member. Where this is associated with a loweraverage mark students have possibly failed to rewardthe contributions of more active group members.

Table 4 shows that in 1993-94, Week 17 showedmost groups marking in a uniform way across nearlyall of the criteria, creating an upward 'blip' on anupward trend in group marks. As the term progressedstudents appeared to become less able to make dis-tinctions between their peers. 1994-95 saw a similarlack of variability in marks but with a wider set ofgrades to distribute the standard deviations should,arguably, have been higher. A reduction in the pro-portion of subgroups with low variation in markswas seen as the term progressed but collusive mark-ing is still evident.

These figures are given greater weight when the to-tal lack of variability (SD = Nil) is reviewed. In1994-95 roughly half of those subgroups with stand-ard deviations below 0.5 (see Table 4) actually re-

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 0

3:47

11

Oct

ober

201

4


Table 4. Variation in Weekly Sub Group Marks (% of Groups per week with mark Standard Deviation <= 0.5)

Criteria


Week93-94

' 5 6444444

37.56931

1694-95

4267—

58.5—75—

Week93-94

75757575

62.57550

1794-95

58.550—50—67—

Week93-94

5637.55050

37.55631

1894-95

58.558.5

—58.5

—50—

Week93-94

73.56067

73.540

73.540

1994-95

6767—75_

83.5—

Week93-94

47.553.547.560

33.560

26.5

2094-95

5025—

33.5—50—

Notes: N = 16 in Weeks 16-18 and 15 in Weeks 19 and 20 (1993-94)N= 12 (1994-95)

turned standard deviation of nil. The data do notshow, however, if the collusion was due to inde-cisiveness or design.

There were, of course, a number of groups that couldmake firmer judgements. The review sheets revealthat the groups returning uniform marks each weekwere, largely, the same ones. In each year one ortwo whole tutorial groups, were particularly badlyaffected. In some weeks these groups awarded 'A'grades to almost everyone, collusive marking beingstrongly associated with higher grading.

The variability in marks awarded to individual stu-dents was also estimated using the standard devia-tion statistic. In 1993-94 few groups had a standarddeviation above 1.0 in any of the five weeks of tuto-rials. This was not the case, however, for individualmembers of the groups.

Combined with a review of the level at which markswere given it becomes clear that some groups con-tained individuals who contributed nothing and wereawarded nothing by way of marks (these are smallin number but coincide with absenteeism at formaltutorial sessions), giving the impression that studentswould not tolerate a lack of contribution. It also be-comes clear from 1993-94 data that a number of groupswere consistently 'led' or 'dominated' by one or twoof their members. These individuals achieved consist-ently high marks in most of the assessed criteria.

Leadership and domination of groups

One criticism of the more informal tutorial regimeused in Term 1 1993—94, apparent through student

feedback, was the 'leadership' or 'domination' oftutorial groups by individuals. It was felt that thepeer review process would give all group membersthe opportunity to contribute to discussions andhandouts and go some way to reducing the imbal-ance within groups. Student feedback relating toTerm 2 (1993-94) showed that this had been effec-tive, to a large extent, but an analysis of the peerreview marks shows that some groups were still look-ing to a small number of members to lead them.

The 1993-94 data show that, while student feedbackno longer felt that individual students were taking aleading role within subgroups, peer review markingrevealed consistent high scoring for a small numberof group members. On average this effect was no-ticed in almost half of the subgroups and could, per-haps, be interpreted as Decibel marking.

The changes made to the project design for 1994—95should not, by themselves, have masked domination,should it be apparent. The vital input and prepara-tion criteria were still available and the handout cri-teria was placed on an informal rota basis. Asmentioned before only one sub group saw this ac-tivity 'dominated'. However, the high average gradesand massive collusive marking shown in Tables 2and 3 have made identification of 'dominators' in1994-95 impossible.

Performance assessment

Although the tutorial presentations did not form partof the formal assessment process, and the peer re-view, by design, addressed only learning criteria, asubjective assessment of output quality was made

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 0

3:47

11

Oct

ober

201

4

322 IETI 32,4

by the two tutors involved. Because of the largelyunstructured way in which tutorial presentationswere made in Term 1 of each year no specific qual-ity assessment was made. However, the tutors in-volved believed that the quality achieved in Term 2was just as good as that achieved in previous years.

The subjective assessment was made along tradi-tional lines that took into consideration (in order ofimportance) content of the answer, structure of theanswer and written and verbal presentation. Marksawarded for subgroups ranged from 40 per cent fora basic answer that showed no evidence of researchbeyond lecture notes to 90 per cent for a well-researched and well-considered answer that was alsopresented neatly and in a logical and clear manner.

The marks awarded were not revealed to students inany formal way. Tutors gave guidance on areas whereanswers could be improved and commented on over-all quality. The decision was made not to releaseactual marks to students as this could have detractedfrom the learning objectives of the peer review pro-cess itself.

CONCLUSIONOne of the most positive features of the peer reviewexercise was the nature of the feedback from par-ticipating students. Peer review, in itself, appears toact as a strong impetus towards greater individualparticipation in tutorials and, potentially, to greaterlearning benefits, if student ownership of the pro-cess is high.

As a precursor to peer assessment the project ap-pears to have highlighted problem areas such as re-liability of marking and student motivation and hassuggested some ways in which these might be miti-gated. This follows the work of Brown et al (Brown,Rust and Gibbs, 1994).

The formal and informal feedback from students andthe performance of subgroups in tutorial sessions addweight to the contention that the peer review proc-ess improved learning outcomes and encouraged stu-dents to develop a deep learning approach to thesubject. This could be further enhanced by studentinvolvement in the assessment process too.

The peer review exercise was designed to respondto the student-held perception (echoed by tutors) thatsome individuals did no work in tutorials either bydesign or as a result of not possessing the necessary

learning skills. The peer pressure that the exercisehoped to generate was acknowledged as a prime fac-tor in encouraging participation. Some individualswere still able to dominate or lead groups in sometasks, perhaps inevitably. The exercise also providedquality data on those who failed to attend tutorials.

While peer review is not a viable or desirable strat-egy for all modules or subject areas, even wheregroup work is emphasized, it can add a formal di-mension to out of class activities that are alwaysnotoriously difficult to monitor.

This exercise has shown, rather predictably, that peer-generated marking is not always reliable and can beadversely influenced by group collusion, friendship(or hostility) and can allow, and even encourage, domin-ation by individuals. Students are, however, willingto penalize peers who do not contribute but often failto differentiate between higher levels of effort. Clearunderstanding of criteria and of the assessment pro-cedure is necessary for effective marking together withevidence to support the marks awarded.

Another interpretation might be that first year un-dergraduates do not possess the necessary groupwork skills that would allow objective assessmentsof colleagues' contributions to be made. Althoughthe lack of negative feedback would indicate thatthe process was well understood some training orguidance on peer review is a necessary part of theprocess.

The exercise has given huge encouragement to har-ness its more positive lessons for future years, espe-cially with semesterization on the horizon. The peerreview process could be converted to peer assess-ment by allowing students to allocate the percent-age of the total marks awarded by the tutor to theindividual group members. We hold the view that apeer assessment framework should enhance the posi-tive benefits of peer review and learn from its short-comings while mitigating the negative elements ofpeer assessment.

REFERENCES

Brown G. A. and Pendlebury, M. (1992) AssessingActive Learning, Sheffield: CVCP/USDU.

Brown, S. and Dove, P. (eds) (1991) Self and PeerAssessment, Birmingham, SCED Paper 63.

Brown, S. and Knight, P. (1994) 'Assessing Learn-ers in Higher Education', in J Stephenson (ed),

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 0

3:47

11

Oct

ober

201

4

Teaching and Higher Education Series, London:Kogan Page.

Brown, S. Rust, C. and Gibbs, G. (1994) Strategiesfor Diversifying Assessment in Higher Education,OCSD, Oxford Rewley Press.

Entwistle, N. J. (1992) The Impact of Teaching onLearning Outcomes, Sheffield: CVCP, USDU.

Falchikov, N. (1991) Group Process Analysis. InBrown, S. and Dove, P. (eds) Self and Peer Assess-ment, Birmingham: SCED Paper 63.

Habeshaw, S., Gibbs, G and Habeshaw, T. (1993)53 Interesting Ways to Assess your Students,Melksham: The Cromwell Press.

Pond, K. and Ul-Haq. R. (1994) Peer Review: AnEffective Method of Enhancing Student Learning inTutorials? Loughborough University BusinessSchool Research Series Paper 1994:11.

BIOGRAPHICAL NOTES

Keith Pond has been a lecturer in banking andeconomics in Loughborough University Businessschool since 1991. He was also Midland Bank GroupVisiting Fellow at Loughborough University from1986-89 during which time he completed an MPhilon the subject of personal insolvency. He gainedbanking experience with Midland Bank between


1978 and 1991 including: branch manage-ment; project management; computer systemsconsultancy and corporate 'intensive care'. Keith'smain research interests are in the fields of banking,credit, corporate rescue and insolvency. Keithis married to Christine, has four children, fourgoldfish and is parent governor of a local primaryschool.

Rehan Ul-Haq has held an ESRC ManagementTeaching Fellowship at Loughborough UniversityBusiness School and also an Associate Member ofthe Banking Centre since 1993. He was previouslya banker working in the corporate, international andtreasury areas. Main research interests include strate-gic alliances in the UK and European banking sec-tor, long range development issues in Pakistan,strategic management and the enhancement of stu-dent learning.

Winnie B Wade co-ordinates the Flexible LearningInitiative at Loughborough University of Technol-ogy and is directing the HEFC national dissemina-tion programme for the 'Flexibility in CourseProvision' projects. She has extensive teaching andstaff development experience in higher education andhas published widely.

Address for correspondence: Rehan Ul-Haq, ESRCManagement Teaching Fellow, Loughborough Uni-versity Business School, Loughborough, Leicester-shire LE11 3TU. E-mail: [email protected]

Dow

nloa

ded

by [

The

Uni

vers

ity o

f M

anch

este

r L

ibra

ry]

at 0

3:47

11

Oct

ober

201

4

Documents

Peer Review: a Precursor to Peer Assessment