Implementation of a formative assessment model incorporating peer and self-assessment

This article was downloaded by: [University of Connecticut]On: 29 October 2014, At: 05:08Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office:Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Innovations in Education and TeachingInternationalPublication details, including instructions for authors and subscriptioninformation:http://www.tandfonline.com/loi/riie20

Implementation of a formativeassessment model incorporating peer andself‐assessmentPaul Orsmond a , Stephen Merry a & Arthur Callaghan aa Staffordshire University , UKb Department of Biological Sciences , School of Sciences, StaffordshireUniversity , College Road, Stoke on Trent Staffs, ST4 2DE, UK E-mail:Published online: 17 Feb 2007.

To cite this article: Paul Orsmond , Stephen Merry & Arthur Callaghan (2004) Implementation of aformative assessment model incorporating peer and self‐assessment, Innovations in Education and TeachingInternational, 41:3, 273-290

To link to this article: http://dx.doi.org/10.1080/14703290410001733294

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”)contained in the publications on our platform. However, Taylor & Francis, our agents, and ourlicensors make no representations or warranties whatsoever as to the accuracy, completeness, orsuitability for any purpose of the Content. Any opinions and views expressed in this publicationare the opinions and views of the authors, and are not the views of or endorsed by Taylor &Francis. The accuracy of the Content should not be relied upon and should be independentlyverified with primary sources of information. Taylor and Francis shall not be liable for anylosses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilitieswhatsoever or howsoever caused arising directly or indirectly in connection with, in relation to orarising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any substantialor systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, ordistribution in any form to anyone is expressly forbidden. Terms & Conditions of access and usecan be found at http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/loi/riie20

http://dx.doi.org/10.1080/14703290410001733294

http://www.tandfonline.com/page/terms-and-conditions

Innovations in Education and Teaching International,Vol. 41, No. 3, August 2004

ISSN 1470–3297 (print)/ISSN 1470–3300 (online)/04/030273–18© 2004 Taylor & Francis LtdDOI: 10.1080/14703290410001733294

Implementation of a formative assessment model incorporating peer and self-assessment

Paul Orsmond

*

, Stephen Merry and Arthur Callaghan

Staffordshire University, UK

Taylor and Francis LtdRIIE41304.sgm

10.1080/14703290410001733294Innovations in Education and Teaching International1470-3297 (print)/1470-3300 (online)Original Article2004Taylor & Francis Ltd413000000August 2004PaulOrsmondDepartment of Biological SciencesSchool of Sciences, Staffordshire UniversityCollege RoadStoke-on-Trent StaffsST4 [email protected]

This paper reports a study designed to enhance students’ ability to implement assessment marking criteria andto develop the role of dialogue in student learning. Pairs or trios of undergraduate biology students were askedto complete a poster on the theme of histology using student/tutor-constructed marking criteria. The studyshowed: (1) students can effectively implement a variety of marking criteria after formative discussion withtutors and peers; (2) there is a distinction between students being able to implement marking criteria and beingable to transfer that understanding of marking criteria to written comments; and (3) formative assessment mod-els allow discussion to focus on student learning and for feedback to be given within the same or differentmodules or courses. The study supports previous work which showed peer and self-assessment are extremelyuseful in helping students reach their learning goals.

Introduction

For many years there have been a number of pressures on academics to either review or changetheir assessment practice (Balla & Boyle, 1994). A specific reason for change is the need to makeassessment processes more ‘transparent’. To address this requirement higher education isincreasingly adopting an outcomes-based assessment (OBA) programme, with the integralrequirements of student-centred learning (Ecclestone, 1999). Two developments for makingassessment aims more transparent are meaningful formative feedback and a criterion-referencedassessment scheme. In order to meet these requirements the correct assessment practice needsto be employed.

*

Corresponding author: Department of Biological Sciences, School of Sciences, Staffordshire University,College Road, Stoke on Trent ST4 2DE, UK. Email: [email protected]

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

5:08

29

Oct

ober

201

4

274

P. Orsmond

et al

.

Assessment practice

Fallows and Chandramohan (2001) differentiate between ‘assessment task’, ‘assessmentapproach’ and ‘assessment process’. The assessment task is the item of work that the student isasked to undertake. The assessment approach refers to the mechanisms through which gradingjudgement is made, for example, peer, self or tutor judgements. The assessment process encom-passes all stages from setting of the assessment task through to confirmation of the grade andprovision of feedback to the student. Assessment grading judgements are either summative orformative. Summative assessment is often seen as an endpoint assessment providing formalaccountability of knowledge, but it may have a role in student learning. Briggs (1998), using thelearning portfolio as an example, argues that summative assessment defines the parameters forformative assessment.

Formative learning

Fundamental to a formative learning process is the use of feedback. Ramaprasad (1983) definedfeedback as information about the gap between the actual level and the reference level which isused to alter that gap. If information is simply stored in memory it is not feedback. For formativelearning to occur effective feedback needs to be given to students and for feedback to be effectivestudents need to be aware of certain parameters. They need to have: (1) a concept of their learn-ing goal; (2) the ability to compare actual and desired performances; and (3) the ability to act insuch a way as to close the gap (Sadler, 1989). In order to encourage formative learning, forma-tive feedback must meet and address these concepts.

Formative assessment

Wiliam and Black (1996) have stated there is no common understanding of the term ‘forma-tive assessment’ in the literature. Such meanings as ‘classroom evaluation’, ‘curriculum-basedassessment’, ‘feedback’ and formative evaluation’ have been used. The form in which feed-back is to be given can vary. Miller

et al

. (1998) indicate that formative assessment requires amark to be allocated, which is then fed back to students providing information on the level ofthe students performance in relation to teacher expectations. Black (1995) maintains thatformative assessment must be pursued for its main purpose of feedback into the learningprocess; it can also produce information, which can be used to meet summative purposes.Black and Wiliam (1998), without using the word feedback, appear to be enigmatic whendefining assessment as formative only when comparison of the actual and reference levelsyields information, which is then used to alter the gap. The implication being, although notstated, is that Black and Wiliam see formative assessment just as feedback. Cowie and Bell(1999) define formative assessment as: ‘the process used by teachers to recognize, andrespond to student learning in order to enhance that learning, during learning’. Harlen andJames (1997) do not perceive formative assessment in terms of marks but in relation tomaking judgements of a pupil’s work or progress taking into account such things as the partic-ular context of the pupil’s work and the progress that the pupil has made over time. Further-more, with respect to the pupil–context influence, Harlen and James make the important

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

5:08

29

Oct

ober

201

4

Implementation of a formative assessment model

275

observation that the judgement of a piece of work, and what is fed back to the pupil, willdepend on the pupil and not just on the relevant criteria. Formative assessment, therefore,may have no need of assessment criteria.

Implementing formative assessment practice

In order to implement formative assessment, Cowie and Bell (1999) articulate two kinds offormative assessment processes, ‘planned’, where an arrangement has been made with studentsto undertake a specific activity and ‘interactive’ where no specific activity is undertaken, theinteractive assessment having arisen out of the learning activity. Both these forms of formativeassessment can be linked. This study uses the planned process.

Peer and collaborative assessment

Peer and collaborative assessment both provide a vehicle for undertaking formative assessmentexercises. Peer assessment is the process whereby individuals or groups of students assess thework of their peers. This assessment may take the form of either awarding marks or giving awritten or oral comment on the work. Collaborative assessment occurs when students andtutors combine to determine the criteria for assessment. Both assessment processes arediscussed in Somervell (1993) and Dochy

et al

. (1999). Although Cowie and Bell (1999) donot specifically advocate peer and collaborative assessment within the formative assessmentprocesses they describe, both these assessment methods do lend themselves to a formativeassessment model.

Using marking criteria to enhance student learning

Outcomes-based assessments require assessable learning outcomes; this necessitates explicitassessment criteria. In general terms assessment criteria specify how student performance inrespect of the module’s learning outcomes are to be recognized. The links between learningoutcome, assessment and assessment criteria are detailed in Gosling and Moon (2001).Students find criterion-referenced schemes a ‘good idea’ and one which helps clarify assessmentrequirements although there are specific shortcomings, for example, multiple interpretation ofcriteria, assessment standards and subjectivity (O’Donovan

et al.

, 2001). Attempts have beenmade to address these identified shortcomings and to illustrate how the type of criteria mayinfluence student learning outcomes.

Orsmond

et al

. (2000) have shown that students need guidance in distinguishing between theprocesses of constructing marking criteria and marking using those criteria. Students need to beclear in their minds that there is a difference. To address this problem, this present study wasdesigned to make students aware of how to apply marking criteria to a specific task. This is seenas a first step to helping them understand how marking criteria can be useful in their subjectlearning, and hence begin to value the difference between a good and a bad criterion. Themethod used in this study was a modified version of the planned formative assessment processcharacterized by the three stages of eliciting, interpreting and acting on assessment informationadapted from Cowie and Bell (1999). The reason for using this particular process was that it

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

5:08

29

Oct

ober

201

4

276

P. Orsmond

et al

.

allowed a structured exercise to be carried out that gained information from a whole group aboutthe method of implementing marking criteria. It also allowed tutors to gain student feedback andother information that would inform their teaching.

The aims of this study were twofold:

1. To develop effective formative assessment techniques to enhance students ability to imple-ment assessment marking criteria.

2. To develop and evaluate the role of dialogue in student learning.

Method

The assessment was carried out by level 1 undergraduates studying a skills-based module enti-tled ‘Work Experience and Personal Development’. None of the students had taken part in peerassessment exercises at Staffordshire University prior to this study. In total there were 16students who worked in pairs or trios. The students were told at the start of the module (i.e. sixweeks before the posters were to be presented) that as part of their course work a scientific posterwould be produced on the topic of Histology. The study consisted of two separate assessmentevents, formative and summative. The formative assessment exercise was undertaken in twosessions. Firstly, session 1 was a formative assessment workshop. There were detailed contribu-tions from tutors and the atmosphere was conducive to discussion. Session 2 involved theconstruction of the poster and peer marking. The session was formative such that it allowedstudents to reflect on the peer comments and marks given to their work. The summative aspectof the exercise, the tutors marking of the work, is not the focus of this paper and is only discussedin the context of formative assessment.

Session 1

Stage 1. Eliciting information as part of a formative assessment

Part 1.

One week before the posters were to be displayed students undertook a formativeassessment exercise. Students were given information on: (1) the role of marking criteria as usedby tutors when marking work; (2) the benefits to their learning of understanding what criteriaare used to mark their work; and (3) the need for both tutor and student to have a commonunderstanding of the meaning of the terms used within the marking criteria.

Students were given information on peer assessment. It was stressed that this assessmentprocess was about guidance, sharing feedback and thinking about their own learning and notcriticizing and upsetting friends. Students were told that they would carry out a peer assessmentexercise on displayed posters.

Part 2.

Students in pairs or trios were asked to construct marking criteria for a poster. Theywere then given two sets of criteria by tutors (see below). A poster made by a member of a previ-ous cohort on the theme of histology was then shown to the students. This is referred to as theexemplar poster. They were then asked to mark this poster in pairs or trios first using criteria 1and then criteria 2. Tutors also marked the poster according to criteria 1 and 2. The tutor marks

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

5:08

29

Oct

ober

201

4


277

throughout the study were agreed and used as a baseline. The class and tutor marks werecollected and displayed. Students were then told by the tutors that criteria 1 were regarded as‘worthwhile criteria’ and criteria 2 as ‘ambiguous’. The criteria were defined as ‘worthwhile’ or‘ambiguous’ as in the opinion of the tutors (see discussion) and the authors accept that theseterms are open to a variety of interpretations. For each marking criterion both tutors andstudents used a 1–5 marking scale: 1 = very poor to 5 = excellent. The marking scale is shownin Appendix 1.

Worthwhile criteria 1 given to students in Session 1, Stage 1, Part 2

●

Clear logical progression.

●

Appropriate scientific content.

●

Clarity.

●

Appropriate conclusion.

●

Quality of presentation.

Ambiguous criteria 2

●

Colourful.

●

Simple to understand.

●

Clear title.

●

References.

●

Conclusion.

Stage 2. Interpreting as part of a formative assessment

Tutors and student pairs or trios discussed the criteria constructed by the students earlier inthe sessions. Then considering the ‘worthwhile’ and ‘ambiguous’ criteria, tutors further devel-oped the concept of criteria construction and usage. Tutors listened, ascertaining whetherstudents displayed learning related to the role and to the construction of marking criteria.Tutors also attempted to clarify misconceptions over either terminology used or conceptsdeveloped in discussion with the students. In these discussions with students regarding crite-ria, tutors made reference to the exemplar poster and the biology contained within it. Allstudents were asked whether they were aware, before being told, of the differences betweenmarking criteria 1 and 2.

Stage 3. Acting on formative assessment

Tutors discussed among themselves the outcomes of the discussions with students. Theydiscussed ways of improving the student-constructed criteria made earlier in the session. On thebasis of this, tutors and students then reconvened into the earlier groups. Each group wrotedown agreed definitions of the marking criteria. Students were informed that they were going toimplement the student/tutor-constructed criteria to construct a poster on the theme of Histologyin session 2 and to mark according to the tutor-constructed marking scale. Individual posters

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

5:08

29

Oct

ober

201

4

278

P. Orsmond

et al

.

were to be made by each student within the group, using the same group criteria. The groupswere: Groups A with three students, and Groups B, C, D, E, F and G, all with two students.

Session 2

Students constructed their posters. Typed versions of the individual group marking criteriawere attached to the front of the relevant posters. Feedback comment sheets were alsoattached. These were blank sheets of paper on which students were encouraged to writefeedback. Students self-assessed their own poster and peer-assessed those posters made bytheir peers using the criteria appropriate to that group. For example, students from Group Awould mark posters made by Group B using the criteria constructed by Group B. Theassessment took the form of both a grade and comment(s) on their impression of the poster.

Tutors then marked the posters giving both a grade and making comments. Tutors onlymarked the posters of students with whom they had discussed marking criteria. Students wereasked to complete a feedback questionnaire on the end of the sessions (see Appendix 2 for a copyof the questionnaire).

Regression lines were fitted to scatter plots (Sokal and Rohlf, 1981). Significance was assumedwhen

p

< .05.

Results

Marks comparisons

Student and tutor marking using worthwhile and ambiguous marking criteria.

Table 1 shows thepercentage difference between student peer and tutor marking of the exemplar poster using theambiguous marking criteria. The mean percentage difference (with standard error) of the sevenposter marks was 8.54 (3.76).

Table 1. Comparison of tutor and student marking of a provided poster using the ambiguous marking criteria

Student group Student mark Tutor mark

Difference intutor and

student mark

% Difference in tutor and

student mark

A 15 15 0 0.0B 15 15 0 0.0C 13 15 2 13.3D 14 15 1 6.6E 19 15 4 26.6F 15 15 0 0.0G 13 15 2 13.3

Sixteen students worked in pairs or trios to generate student group marks. Three tutors generated an agreed tutor mark. The mean % difference and SE is 8.54 (3.76).

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

5:08

29

Oct

ober

201

4


279

Table 2 shows the percentage difference between student peer and tutor marking of the exem-plar poster using the worthwhile marking criteria. The mean percentage difference (with stan-dard error) of the seven poster marks was 63.4 (6.72).

Student peer assessment and tutor marking of completed student posters.

Figure 1 shows a scatterplot for the relationship between individual student peer assessment marks and tutor marks. The

F

value for the regression line was 12.17 (

p

= .0007), thus the regression was significant.

Figure 1. Relationship between student individual peer assessment marks and tutor marks, in judging a range of posters using tutor-student agreed criteria

Student self-assessment and tutor marking of completed student posters.

Figure 2 shows a scatterplot for the relationship between individual student self-assessment marks and tutor marks. The

F

value for the regression line was 4.77 (

p

= .048), thus the regression was significant.

Figure 2. Relationship between student assessment marks for their own posters and tutor marks, using tutor-student agreed criteria

Group poster marking criteria for assessment of posters

Groups C, F and G:

●

Content at the appropriate level.

●

Relevant content.

●

Originality.

●

Organized layout.

●

Appropriate use of diagrams and text.

Groups A, D and E:

●

Visually effective.

●

Content understandable.

●

Quality of production.

●

Structure.

●

Originality.

Table 2. Comparison of tutor and student marking of a provided poster using the worthwhile marking criteria

Student group Student mark Tutor mark

Difference intutor and

student mark

% Difference in tutor and student

mark

A 16 9 7 77.7B 13 9 4 44.4C 12 9 3 33.3D 15 9 6 66.6E 16 9 7 77.7F 15 9 6 66.6G 16 9 7 77.7

Sixteen students worked in pairs or trios to generate student group marks. Three tutors generated an agreed tutor mark. The mean % difference and SE is 63.4 (6.72).

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

5:08

29

Oct

ober

201

4

280

P. Orsmond

et al

.

Group B:

●

Logical development.

●

Self-explanatory.

●


●

Clarity.

●

Meaningful conclusion.

Written tutor and student comments on poster completed by Group B

The following set of comments are typical of all the tutor and student written comments on themarked posters.

Tutor comments on Group B’s poster

●

Logical development.

Yes read well and there was a logical progression through the differentstages. But what use is made of the picture of the microscope. How does that fit into the ‘logi-cal development’?

●

Self-explanatory.

Mostly. The microscope picture is well used here. A clear focus point andthe text is self-explanatory, but those cells should have a label and are those magnifications

Figure 1. Relationship between student individual peer assessment marks and tutor marks, in judging a range of posters using tutor–student agreed criteria

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

5:08

29

Oct

ober

201

4


281

right? Do you explain what ‘objective’ means? Also perhaps you could be more informativeabout the parts of a microscope.

●


Not so good here. Is it really

×

10 and

×

40, or

×

100 and

×

400.Have you taken eyepiece magnification into account? How would you measure the cells,remember qualitative and quantitative information.

●

Clarity.

Yes clarity is good. Not too much information, just right.

●

Meaningful conclusion.

Could have given a little more information. Try and link the conclu-sion to the introduction, this is where giving just a little more detail in the introductioncould help. For example, what is magnification and how does it help to allow you to seethe cells.

Student comments on Group B’s poster

●

The poster is very clear and understandable. I like it, it is very logical and with good depth.

●

Very clear to follow.

●

Easy to follow and neat in appearance.

●

Very neat and good guidelines.

●

Clear, logical, easy to follow.

●

Lovely to look at, a lot of time taken over project. Good use of colours. Well structured. Inter-esting.

●

Satisfactory.

Figure 2. Relationship between student assessment marks for their own posters and tutor marks, using tutor–student agreed criteria

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

5:08

29

Oct

ober

201

4

282 P. Orsmond et al.

Student feedback to questionnaire

The following sets of comments are typical of student responses to the feedback questionnaire.Eleven questionnaires (69%) were returned. The number in italics at the end of each questionrepresents the number of student responses to that question.

1. Did you find the session on explaining marking criteria helpful? If so why? (11) ● Yes. It helped to clarify what others (teachers and peers) considered important, for a crite-

ria. It was good talking in bigger groups about opinions.● I found it helpful because it gives us an idea of what the person marking your poster is

looking for, so we could design towards a kind of plan.● I found it helpful because it gives me a view of the way my poster and work is marked. It

shows me how people had different ideas about the same thing.● No because I am quite aware of criteria and how they work.

2. Did understanding the marking criteria help you in preparing your poster? If so how? (11) ● I don’t think it made a lot of difference. It made me think about why I’m putting certain

things in but there wasn’t a lot of input from other members in the groups so there wasn’ta lot of others views and ideas to consider.

● Yes, I knew how not to overcrowd the poster, keep it simple and informative.● It did because it made me aware of what needs to be included in the poster.● It did in a small way, however I still think I went about doing the poster in the same way

just occasionally glancing at the criteria to see if I had done everything it asked for.● Yes knowing specifically what to aim for was a real advantage.● Yes because it helped me be to know how to be precise with the information I got.

3. All of you were able to recognize the worthwhile criteria and ambiguous criteria but you gaveboth the same marks. Why do you think this was? (8) ● Personal taste.● This was because we haven’t used them (criteria) ourselves in the past, we’re not used to

marking criteria.● Inexperience at using criteria and working with others and marking other people’s work.● We are not familiar with criteria marking or really how it works.● In the ambiguous criteria marks that were lost in one part of the criteria area were gained

in another due to having broad areas to mark.● Not truly knowing the relevance of criteria.

4. What difficulties do you think you still have in understanding and using marking criteria?(10) ● I don’t think I shall have any more difficulties in using and understanding marking criteria.

As long as I can understand what it’s asking for, it should be easy to follow.● Being accurate and not too generous.● That your own definitions of what is required may be different from the markers.● How specific the marking criteria were. Would we lose marks if we were just slightly

wrong?● People have different ideas of what things actually mean. I found it hard to always inter-

pret several words (not very specific).

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

5:08

29

Oct

ober

201

4

Implementation of a formative assessment model 283

5. In your groups did you discuss the marking criteria when preparing your poster? (9) ● Yes we went through it in some detail.● We had a quick glance at the criteria during the construction of the poster.● Not before but during.● The marking criteria was what the poster was mainly designed towards so when designing

the poster we talked so the design had relevance to the criteria.● Yes every now and again during construction we referred back to the criteria.

6. Will you consider the marking criteria when undertaking your next assignment? If so, in whatway? (10) ● Yes, in some ways. But we never compared all the class’s criteria to see who put what. I

would not use my criteria because it could be missing a major factor that is essential togetting a better mark.

● Yes because it gives some indication as to how your work will be marked.● Yes because I will take into account what needs to be done to get a good mark.● Yes why revise or put something in a poster that is not going to be marked.

7. Do you think understanding the marking criteria can help you in your learning? Please givereasons for your answer. (10) ● Yes it makes you identify what you’re doing and why you’re doing it. It draws your atten-

tion to key aspects of your work and focuses on improving them.● Yes because it helps in logically thinking and differentiating between important material

and (possibly) trivia.● Yes, because it gives you an idea of what your work needs to contain.● Yes, if you do not know what the examiner is looking for, you would most likely not

include it in your work.● Not in the actual learning, however it helps in achieving better work and shows what things

should be included in the future.● Yes, it enables you to think more about what the audience will want to see/read, and put

your view forward.● Yes because I will know to learn the relevant information.

Discussion

This study has (1) investigated the effectiveness of formative assessment techniques inenhancing student’s ability to implement assessment marking criteria and (2) evaluated therole of dialogue in student learning. It develops work undertaken using formative feedbackand exemplars to enhance the student learning environment (Orsmond et al., 2002). In orderto reduce the number of variables the tutor marking was considered the baseline mark,although the authors are aware that concern about lecturers’ mark distribution has beenexpressed (Yorke et al., 2000).

Marking profiles

Marking profiles for ambiguous and worthwhile marking criteria. As part of this study studentswere asked to mark an exemplar poster using two sets of criteria given them by the tutors. One

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

5:08

29

Oct

ober

201

4


set of criteria was deemed ambiguous and the other worthwhile. While these criteria are open tointerpretation the authors do see specific differences. For example, ‘clear title’ is ambiguous; itdoes not mean, although it may imply, correct title or appropriate title. ‘Clarity’ is deemedworthwhile as expressing quality of communication to the viewer. Furthermore, the worthwhilecriteria in order to be correctly addressed require the student to have greater understanding ofsubject knowledge compared with the ambiguous criteria. For example, ‘appropriate conclu-sion’ requires greater engagement with the subject, compared to the criterion ‘conclusion’.Tables 1 and 2 show how well students implemented the ambiguous and worthwhile markingcriteria in comparison to tutor marking.

It appears that students have no difficulties in implementing ambiguous criteria but were lesswell able to implement the worthwhile criteria successfully. They have shown the skill requiredto implement the criteria effectively only when using the ambiguous marking criteria. This mayreflect their inadequate understanding of subject knowledge. In order to implement the worth-while criteria effectively the student needs to understand the science contained within the poster.A worthwhile criterion engages the student with the subject, and does not just focus on theconstruction of a poster. Thus, a student’s struggle with the worthwhile criteria may indicatetheir lack of subject knowledge. The ambiguity of the criteria may result from the high degreeof subjectivity contained. Earlier studies (Orsmond et al., 1996, 1997) have considered individ-ual criteria showing that students might have difficulties when addressing subjective criteria like‘visually effective’. Furthermore, these earlier studies identified students’ difficulties in inter-preting more complex criteria, for example, ‘clear and justified conclusion’. While not the focusof this present study, and so not reported here, consideration of each individual criterion for theworthwhile criteria shows over- and under-marking trends as reported in Orsmond et al. (1996,1997).

Students’ peer and self-assessment of completed student posters. Following the formative assess-ment exercise and tutor/student dialogue, students were asked to construct a poster accordingto the group’s individual criteria. The students were then asked to peer and self-assess their post-ers using the particular criteria applicable to each poster. Figure 1 shows the data from this peerassessment exercise. Using regression analysis there seems to be a close relationship betweenpeer and tutor marking (p = .0007). Poster Groups G (mark range 11–19), F and C (mark range10–19) have on average higher student marks when compared to the tutor. For poster GroupsD (mark range 10–19), E (mark range 13–22), B (mark range 15–20) and A (mark range 12–22) student marks are, on average, similar or slightly lower compared to the tutor mark.

Figure 2 shows the data from the self-assessment exercise. Again, there is a statistically signif-icant relationship (p = .048) between student and tutor marks, although the significance is lessthan that for peer marking. Poster Group G gave self-marks of 5 and 8 compared with the tutormark of 11. Poster Groups C and F gave self-marks of 16 and 17 and 14 and 16, respectively,compared to the tutor mark of 13. Poster Group D gave self-marks of 17 and 20 compared to atutor mark of 15. Poster Groups E, A and B gave self-marks of 9 and 12, 17 and 18 (×2), and19 (×2), respectively, compared to the tutor mark 17. Therefore students’ self-mark for posterGroups A, B, C, D and F are slightly higher than the tutor whereas Groups G and E’s aredistinctly lower.

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

5:08

29

Oct

ober

201

4


Our data show Group G to have an average peer mark of 15 (compared to tutor mark 11) andGroup E have an average peer mark of 17, the same as the tutor mark. If Groups G and E self-marks are compared with their peer marks, Group G thought their poster deserved a lower markthan that given by tutors, but their peers thought the work deserved a higher mark. Group Ethought their work deserved a lower mark compared to the tutor, but their peers marked theposter similar to the tutors. This may indicate that in self-assessment Groups G and E are unableto make a valid assessment of their work, but using the criteria derived by Group E peers wereable to mark more accurately, which they were unable to do when using the criteria used byGroup G. Poster Groups C, F and G all had the same marking criteria and in Figure 1 all peermarks are higher for these groups compared to the tutor mark. Groups A, D and E had the samecriteria and in Figure 1 peer marks are more closely aligned to the tutor mark. This may indicatethat Groups A, D, E and B may be using less ambiguous criteria overall compared to the criteriaused by poster Groups C, F and G. Another interpretation may be that groups A, D, E and Bmay have achieved expression of their criteria more successfully compared to Groups C, F and G.

A further interpretation of the lesser significance obtained in Figure 2 compared to Figure 1may relate to the degree of ownership students feel over their own constructed criteria. Orsmondet al. (2000) showed that students appeared less able to discriminate between individual criteriawhen considering self-constructed criteria compared to provided criteria. The slope of theregression line in Figure 2 approximates 1 and the intercept approximates 0. This contrasts withthe data shown in Figure 1. Students are able to provide an overall mark of their own work, butnot that of their peers, in a similar fashion to tutors. It is outside the scope of this paper toprovide a definite explanation of this difference; however, factors such as peer pressure or unfa-miliarity of the criteria may provide possible answers.

Student/tutor poster marking criteria. There is similarity between the criteria used in this studycompared to the three earlier studies (Orsmond et al., 1996, 1997, 2000). This may reflect thefact that the tutors involved in this study were also involved in these earlier studies. However,this should not mask the contribution from the students in the formative feedback discussiongroups. The discussion between students and tutor was open-ended and the tutors did not havean agenda to direct the students to their own favoured criteria. In discussion with their tutorstudents were, in general, keen to construct criteria that demonstrated varying degrees ofcomplexity. Students demonstrated an understanding for criteria that would require highercognitive ability, for example, in Group B ‘logical development’. The criteria constructed by theindividual groups also showed the breadth and depth of a tutor-provided criteria. The studentsappear to have identified a number of apparently highly subjective criteria, for example, inGroups A, D and E ‘visually effective’ and in Groups C, F and G ‘originality’. This does notseem to have presented their peers with any marking difficulties (see Figure 1). These subjectivecriteria may have been more problematic in the self-marking as Groups E, D and G have notmarked as meaningfully when compared to the tutor.

Written students’ comments on the poster completed by Group B

The complete set of student comments is given allowing comparisons and contrasts to be drawn.For example, in response to question 6 the comment ‘yes why revise or put something in a poster

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

5:08

29

Oct

ober

201

4


that is not going to be marked’ implies some reflection on the process, but the comment ‘yesbecause it gives some indication as to how your work will be marked’ suggests that the studentwants to be told what to do. Both comments give an indication of student’s attitudes towardslearning. While students (see ‘Results: Marks comparisons’) seem to have demonstrated anincreased understanding of the role of marking criteria and are able to implement the criteriasuccessfully by awarding meaningful marks, they apparently have not transferred that under-standing of marking criteria to their written comments (see ‘Results: Student feedback to ques-tionnaire’). They have not addressed individual criteria making relevant comments as the tutorsdid. Nor have they displayed relevant content knowledge when writing their comments.However, a number of the comments do in a more general way appear to address the criteria.For example, ‘very logical’, with ‘good depth’, ‘good guidelines’ and ‘easy to follow’ appear toaddress criteria such as ‘logical development’, ‘self-explanatory’ and ‘clarity’. The more cogni-tively developed criteria such as ‘appropriate scientific content’ and ‘meaningful conclusion’have not been addressed. This does not mean that the students have not recognized a ‘meaning-ful conclusion’. It is implied that they have by the relationship between student and staff marksshown in Figure 1. However, the students’ comments do not reflect this. Recognizing somethingand commenting on something requires different skills. A further study could consider differentskills development within a formative assessment session.

Student feedback questionnaire

Students appear to have found the formative assessment exercise of benefit and overall theirresponses were positive. The answers to questions 1, 2 and 6 are functional and relateunderstanding marking criteria to improving individual marks which may indicate a betterunderstanding of the subject. Answers to question 3 were helpful to the tutors involved in theformative assessment session. The comments made are pertinent to the views expressed in theprevious section of this discussion. Students were able to perceive a difference in the types ofcriteria but their lack of experience in working with criteria meant they were not able to imple-ment the two kinds of criteria differently. Answers to question 4 show some ambiguity regardingthe perceptions of marking criteria. Again students seem to fixate on the marks aspect of thecriteria rather than regarding understanding criteria as a way to understand the subject. Answersto question 5 were encouraging. The majority of respondents did use the criteria in the construc-tion of their poster. Finally, the answers to questions 7 show how, by understanding the markingcriteria, the students were helped to learn. A number of students referred to ‘learning’ in termsof thinking and understanding, but the greater proportion still appear to closely relate learningto achieving a better mark.

Evaluation of formative assessment model

Issues relating to problems of communicating assessment feedback have been previously raised.Higgins et al. (2001) perceive the difficulty with feedback as not being related to whetherstudents take notice of the feedback, or whether it relates to specific criteria, or whether thequantity is sufficient. Rather the issue is how tutors construct the feedback, how the studentunderstands the feedback and how they use the feedback to make sense of the assessment and

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

5:08

29

Oct

ober

201

4


the learning context. This study has attempted to implement a model for formative assessmentand evaluate how meaningful the feedback the students received was in their learning. Toconsider how tutors construct their feedback, and students understand and utilize it, the Cowieand Bell (1999) model was adapted to involve tutors eliciting information from students, inter-preting that information and then acting on the interpretation. Both quantitative and qualitativemeasurements were made.

Constructing feedback. Tutors found that in order to construct meaningful feedback it wasimportant to elicit relevant information from students about their understanding of the assess-ment task. This study has allowed more tutor understanding of assessment tasks. At the start ofthe formative assessment exercise the construction of their initial criteria by students providedthe initial focus for the feedback. Tutors were then able to collect information as to how studentsmarked according to the worthwhile and ambiguous criteria and so become aware that at thisstage the students were able to distinguish a difference between the criteria but were unable toimplement the criteria differently. After the posters had been constructed tutors were able toelicit information from both student self- and peer marking and from student written commentson the individual posters. Information was also collected on the process of using marking criteriaas a way of helping students to understand subject-specific information, in this case histology.Using this variety of information sources it was possible for tutors to discuss and constructappropriate feedback to be given to the students.

Delivering and receiving feedback. As the formative assessment exercise had been planned,opportunities existed for tutor and students to discuss the ongoing learning taking place withinthe individual components of the study. For example, in reviewing the student-constructedmarking criteria for the posters, tutors were able to direct specific comments to the studentsbased on their previous use of marking criteria within the exercise. Furthermore, the tutorsused information collected within this study as the basis for some of the feedback given to thestudents in the following weeks of the module and, indeed, within other modules. For exam-ple, one tutor discussed subject-specific aspects of their poster with the students in Group Bwithin a subsequent practical module. Because of their previous experience the feedbackdiscussion was more meaningful. In this fashion the formative assessment model employedattempts to begin addressing some of Sadler’s (Sadler, 1998) requirements to improve thequality of feedback.

Implications of this study

The study was designed to develop subject-specific knowledge and generic skills throughgreater understanding of poster marking criteria. It became evident during the two sessionshow important it was to have a subject-specific examplar available to facilitate meaningfuldiscussions. This enabled the exercise to focus on using the poster to demonstrate an under-standing of, in this case, histology rather than using the poster only to demonstrate the abilityto present information clearly in poster format. Students were clearly working with the

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

5:08

29

Oct

ober

201

4


subject content in order to develop a better understanding of histology as shown by theirresponses to question 7 of the questionnaire; although their endeavours were only partiallysuccessful. The written comments of students regarding their peers’ posters did not addressspecific criteria, but were very general; not surprising, perhaps, for level 1 students. What hasbeen produced does, however, give a foundation for further learning; a foundation which wasrecognized by both tutor and student. Excepting this, students were able to demonstrate therequired subject-specific understanding and generic skills to implement a variety of assess-ment marking criteria effectively. The model used allowed effective dialogue between tutorand student, and also provided opportunities for reflection. The outcomes of this studysupport those of Cooper (2000) where the emphasis of student actions moves from redoing torethinking work. In that study students were required to rewrite assignments and the need tofocus students on the use of feedback was instrumental in redirecting the use of classroomtime to include such things as critical thinking.

Conclusion

The outcomes of this study may be of interest to tutors in designing their teaching and learningsessions. The following points are considered of worth in this respect:

● Using formative assessment models allows for both the learning of subject-specific knowledgeand the introduction/development of skills.

● Formative assessment models allow for meaningful discussions between students which canfocus students in their learning.

● Formative assessment models allow for formative feedback to be given within the same or indifferent modules or courses.

Acknowledgements

The authors would like to thank the students of the Department of Biological Sciences for theirco-operation. The authors would like to thank the referees for their helpful comments.

Notes on contributors

Paul Orsmond is a senior lecturer within the Department of Biological Sciences at StaffordshireUniversity. He is a University Learning and Teaching Fellow and has an interest in assess-ment issues.

Stephen Merry is a senior lecturer within the Department of Biological Sciences at StaffordshireUniversity. He teaches cell biology and immunology and is interested in how student learn-ing and the learning environment may be improved.

Arthur Callaghan is a retired principal lecturer in Biology. He is a Research Associate within theDepartment of Biological Sciences at Staffordshire University and his interests are focusedon the study of the ecology of Conidiobolus and Basidiobolus, both from the order Entomoph-thorales

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

5:08

29

Oct

ober

201

4


References

Balla, J. & Boyle, P. (1994) Assessment of student performance: a framework for improving practices, Assess-ment and Evaluation in Higher Education, 19(1), 17–28.

Black, P. (1995) Can teachers use assessment to improve learning, British Journal of Curriculum and Assessment,5(2), 7–11.

Black, P. & Wiliam, D. (1998) Assessment and classroom learning, Assessment in Education, 5(1), 7–74.Briggs, J. (1998) Assessment and classroom learning: a role for summative assessment?, Assessment in Educa-

tion, 5(1), 103–110.Cooper, N. (2000) Facilitating learning from formative feedback in level 3 assessment, Assessment and Evalua-

tion in Higher Education, 25(3), 279–291.Cowie, B. & Bell, B. (1999) A model for formative assessment, Assessment in Education, 6(1), 101–116.Dochy, F., Segers, M. & Sluijsmans, D. (1999) The use of self-, peer and co-assessment in higher education: a

review, Studies in Higher Education, 24(3), 331–350.Ecclestone, K. (1999) Empowering or ensnaring?: the implications of outcomes-based assessment in higher

education, Higher Education Quarterly, 53(1), 29–48.Fallows, S. & Chandramohan, B. (2001) Multiple approaches to assessment: reflections on use of tutor, peer

and self assessment, Teaching in Higher Education, 6(2), 229–245.Gosling, D. & Moon, J. (2001) How to use learning outcomes and assessment criteria (London, Southern England

Consortium for Credit Accumulation and Transfer).Harlen, W. & James, M. (1997) Assessment and learning: differences and relationships between formative and

summative assessment, Assessment in Education, 4(3), 365–379.Higgins, R., Hartley, P. & Skelton, A. (2001) Getting the message across: the problem of communicating

assessment feedback, Teaching in Higher Education, 6(2), 269–274.Miller, A. H., Bradford, W. I. & Cox, K. (1998) Student assessment in higher education (London, Kogan Page).O’Donovan, B., Rust, C. & Price, M. (2001) The student experience of criterion-referenced assessment

through the introduction of a common criteria assessment grid, Innovation in Education and Teaching Inter-national, 38(1), 38–43.

Orsmond, P., Merry, S. & Reiling, K. (1996) The importance of marking criteria in peer assessment, Assess-ment and Evaluation in Higher Education, 21(3), 239–249.

Orsmond, P., Merry, S. & Reiling, K. (1997) A study in self-assessment: tutor and students’ perceptions ofperformance criteria, Assessment and Evaluation in Higher Education, 22(4), 357–369.

Orsmond, P., Merry, S. & Reiling, K. (2000) The use of student derived marking criteria in peer and selfassessment, Assessment and Evaluation in Higher Education, 25(1), 23–38.

Orsmond, P., Merry, S. & Reiling, K. (2002) The use of exemplars and formative feedback when usingstudent-derived marking criteria in peer and self assessment, Assessment and Evaluation in Higher Educa-tion, 27(4), 309–323.

Ramaprasad, A. (1983) On the definition of feedback, Behavioural Science, 28, 4–13.Sadler, D. R. (1989) Formative assessment and the design of instructional systems, Instructional Science, 18,

119–144.Sadler, D. R. (1998) Formative assessment: revisiting the territory, Assessment in Education, 5(1), 77–84.Sokal, R. R. & Rohlf, F. J. (1981) Biometry (San Francisco, CA, W. H. Freeman and Company).Somervell, H. (1993) Issues in assessment, enterprise and higher education: the case for self-, peer and collab-

orative assessment, Assessment and Evaluation in Higher Education, 18(3), 221–233.Wiliam, D. & Black, P. (1996) Meanings and consequences: a basis for distinguishing formative and summa-

tive functions of assessment?, British Educational Research Journal, 22(5), 537–548.Yorke, M., Bridges, P. & Woolf, H. (2000) Mark distribution and marketing practices in UK higher education,

Active Learning in Higher Education, 1(1), 7–27.

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

5:08

29

Oct

ober

201

4


Appendix 1. Marking form for poster assessment

The ability to fulfil the criteria should be graded using the following scale.Marking scale: 1. Very poor; 2. Poor; 3. Satisfactory; 4. Good; 5. Excellent.

Appendix 2. Feedback on marking criteria exercise

1. Did you find the session on explaining marking criteria helpful? If so why?2. Did understanding the marking criteria help you in preparing your poster? If so how?3. All of you were able to recognize the worthwhile criteria and ambiguous criteria but in both

cases you gave same mark. Why do you think this was?4. What difficulties do you think you still have in understanding and using marking criteria.5. In your groups did you discuss the marking criteria when preparing your poster?6. Will you consider the marking criteria when undertaking your next assignment? If so, in what

way?7. Do you think understanding the marking criteria can help you in your learning? Please give

reasons for your answer.

Poster ACriteria 1… 2… 3… 4… 5…Poster BCriteria 1… 2… 3… 4… 5…Poster CCriteria 1… 2… 3… 4… 5…Poster DCriteria 1… 2… 3… 4… 5…Poster ECriteria 1… 2… 3… 4… 5…Poster FCriteria 1… 2… 3… 4… 5…Poster GCriteria 1… 2… 3… 4… 5…

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

5:08

29

Oct

ober

201

4

Documents

Implementation of a formative assessment model incorporating peer and self-assessment