GIVING FEEDBACK ON STUDENT WRITING IN THE SCIENCES…sydney.edu.au/stuserv/documents/learning_centre/Feedback_Handboo… · science giving feedback on student writing in the sciences:

SCIENCE

GIVING FEEDBACK ON STUDENT WRITING IN THE SCIENCES: MAKING IT WORK

REPORT AND HANDBOOK OF GOOD PRACTICELORRAINE RYAN, LEARNING CENTRE, THE UNIVERSITY OF SYDNEY

This document is an outcome of the project funded by the University of Sydney Teaching Improvement and Equipment Scheme:

Feedback in the Sciences: what is wanted, what is given, and how it can be improved amongst a diverse student population

Project Team:

Leader: Meloni Muir, Discipline of Physiology

Team members: Miriam Frommer, Discipline of Physiology; Fiona White, School of Psychology; Vanessa Gysbers, School of Molecular Bioscience; Lorraine Ryan, Helen Drury, Learning Centre.

Research Assistants: Isabel Arnaiz, Andrew Howe, Leonardo Gabales

FEEDBACK: MAKING IT WORK

– It wastes my time to write the same comment on almost every student’s work.

– I don’t think I know enough about giving feedback.

– It would be good to have more training.

– It’s a challenge to give clear, concise and constructive feedback.

– I try not to just correct students’ work but then it’s difficult and takes time to explain why something is wrong.

– Should I give general or specific feedback or both?

WHAT MARKERS SAY ABOUT GIVING FEEDBACK

– Should I give feedback on both language and content?

– If a student is from a nonEnglish speaking background, should I consider this when I’m marking and giving feedback?

– If I understand the content, isn’t that enough?

– How can students use my feedback if they only get the assignment back at the end of semester?

– I wonder if students read and learn from my comments …

– Do students understand my feedback?

– There isn’t enough time to give students individual written or oral feedback

– I’m a casual staff member, so I don’t have enough paid time to give the feedback students want.

SCIENCE

Produced by the University of Sydney, 2013. The University reserves the right to make alterations to any information contained within this publication without notice. ABN 15 211 513 464 CRICOS 00026A

FOR MORE INFORMATION CONTACT

Meloni M. MuirE [email protected]/science

REDUCING FEEDBACK TIME

– Discuss assignment guidelines and criteria with students before the assignment

– Hold a report/essay writing tutorial for students when they are working on their assignment

– Provide exemplars of similar past assignments for students

– Use audio-files instead of written feedback

– Use feedback statement banks

EXPRESSING FEEDBACK – When possible, start feedback with

a positive statement

– Be constructive: note mistakes, but also explain how to improve

– Focus individual feedback on major points, e.g. on two/three themes/concepts

– Give timely, generic feedback on an assignment to the whole class asap after submission

– Markers provide general feedback to coordinator to compile and put online

WHAT COORDINATORS AND MARKERS CAN DO TO IMPROVE FEEDBACKCoordinators helping markers:

– Align assessment objectives with marking criteria and course outcomes

– Involve markers in assessment design and marking criteria development*

– Hold a marker session prior to marking to discuss expectations, feedback provision and grade criteria (e.g. P and HD)*

– Organise and manage a blog for markers’ questions

– Hold meetings to standardise marks before assignments returned*

– Hold post-marking meeting re problems encountered with marking scheme/rubric/criteria etc.*

* Note funding implications for sessional staff

GIVING FEEDBACK ON LANGUAGE

– Ensure that students know that English expression forms part of the assessment mark

– Coordinators and markers agree on the amount and type of literacy feedback, e.g. choose two problem areas and give one example of better phrasing for each

– Coordinators and markers agree on how non-English speaking background students’ work will be marked

HELPING STUDENTS TO USE FEEDBACK

– Set a small, low stakes assessment early in semester so that students can build on feedback in later assignments

– Approve a draft essay/report plan and/or give feedback on a draft and minimal comments on final assignment

– Give students experience as markers by holding an assessment tutorial using exemplars, then students give peer feedback before submission

– Students self-assess their work on submission

– Return assignments with feedback only; students rate themselves, with reasons, before mark given

– Students explain how they have used prior feedback in current assignment

1

Table of Contents Introduction: Using the Feedback Handbook ................................................................. 3

List of Tables ................................................................................................................. 2

Part A: Perspectives on Feedback .............................................................................. 5 1.0 The Feedback Project: Student Perspectives ................................................................ 5

1.1 Project Overview ............................................................................................................ 5 1.2 What the students say/want .......................................................................................... 5

1.2.1 Summary of outcomes from student questionnaire ............................................................. 5 1.2.2. Student Focus groups ........................................................................................................... 8

2.0 The Feedback Project: Markers’ Perspectives ............................................................. 13 2.1 Marker Interviews ......................................................................................................... 13

2.1.1 Problems with written feedback ......................................................................................... 13 2.1.2 Marker comments re dealing with diverse language backgrounds of students ................. 13

2.2 Marker workshops ........................................................................................................ 14 2.2.1. Current practices and problems ............................................................................................. 14

3.0 The Feedback Project: Suggested Solutions to Feedback Issues and Challenges .......... 16 3.1 Coordinator-‐centred issues and suggestions ................................................................ 16 3.2 Marker-‐centred issues and suggestions ....................................................................... 16

3.2.1 Solutions suggested in Marker Interviews .......................................................................... 16 3.2.2 Solutions suggested by Workshop participants .................................................................. 17

3.3 Suggestions from the literature to bring students into ‘the community of assessment practice’ .................................................................................................................................. 18

Part B: What the literature says: ‘Suggested Solutions’ Themed Annotated Bibliography (non-‐exhaustive) ..................................................................................... 20

4.1: From the literature: Overview of Feedback ................................................................ 23 4.1.1 General concerns ................................................................................................................ 23 4.1.2 Purposes of feedback .......................................................................................................... 24 4.1.3 Problems with the transmission view of feedback .............................................................. 25 4.1.4 Student understanding of feedback comments .................................................................. 25 4.1.5 Student engagement with feedback ................................................................................... 26 4.1.6 Student use of feedback ...................................................................................................... 26 4.1.7 Student perceptions of tutors’ written feedback ................................................................ 27 4.1.8 Effectiveness of feedback .................................................................................................... 27 4.1.9 Importance of ‘tacit knowledge’ ......................................................................................... 29 4.1.10 University Feedback Websites (see others under ‘Using Technology to provide peer and marker feedback’, below) ................................................................................................................ 29 4.1.11 University of Sydney Feedback webpages ....................................................................... 30

4.2 Suggested Solutions from the Literature ...................................................................... 31 4.2.1 Feedback practice ............................................................................................................... 41 4.2.2 Examples of feedback responses markers can use ............................................................. 43 4.2.3 Student engagement with feedback ................................................................................... 45 4.2.4 Dialogic approaches to feedback ........................................................................................ 49 4.2.5 Using technology to provide peer and marker feedback .................................................... 50 4.2.6 Dealing with large classes .................................................................................................... 54 4.2.7 Feedforward ........................................................................................................................ 55

5.0: Glossary of key terms – definitions from the literature ............................................. 57 References .......................................................................................................................... 59

2

List of Tables Table 1: From the literature: Overview of Feedback – Guide………………………… 20 Table 2: From the literature: Suggested Solutions – Guide……………………………..32 Table 3: Suggested Solutions – Feedback practice………………………………………….36 Table 4: Suggested Solutions – Student engagement with feedback………………37 Table 5: Suggested Solutions – Using technology to provide peer and marker feedback…………………………………………………………………………………………………………39 Table 6: Suggested Solutions – Dealing with large classes………………………………40

3

Introduction: Using the Feedback Handbook In university Science teaching contexts in Australia, markers are under pressure to provide timely, quality feedback to large and diverse student cohorts. We designed this Handbook to help markers in their provision of written feedback on student writing. Of course, there are no ‘easy answers’ to all the situations and questions/issues related to the giving and receiving of written feedback, so all suggestions need to be considered in the light of your own teaching context. You can refer to the Handbook for tips and practical suggestions. Lecturers and unit co-‐ordinators responsible for designing curriculum and assessment can also use it in staff training sessions, particularly those that address assessment, marking and giving feedback. The Feedback Flyer (you will find this at the beginning of the Handbook) would also be useful for these sessions and for discussions about using feedback with students. The main focus of the Handbook is on the practical aspects of giving feedback. We know that markers are often sessional staff, so we wanted to provide information in an easily accessible form which we think can improve practice with little or no extra time commitment. The Handbook has two parts. Part A reports on a project carried out with a large cohort of Science students and their markers at the University of Sydney. There are three sections to Part A.

• In Section 1, after a brief overview of the project, you can read short summaries of students’ perspectives on feedback. Since the University cohort is multicultural and multilingual, we wanted to find out if students’ perspectives varied according to their facility with English and other languages. So some information is provided from students who are native English speakers only (monolingual) or fluent in English and another language (bilingual), or whose language competence in English is reasonable but still developing (non-‐English speaking background or NESB students).

• In Section 2, you will find summaries of markers’ experiences and their

problems with giving feedback.

• Section 3 presents markers’ suggestions for improving feedback or solutions to the problems they raised in Section 2.

In Part B, you will find a selection from the literature on feedback. As you can imagine, there has been a huge amount published in this area so we have limited our selection to those publications with practical advice, generally published after 2000.

4

Part B is divided into 2 sections:

• The Annotated Bibliography in Section 4 provides general information on

feedback, and then focuses on suggested solutions for addressing some of the concerns or problems with giving effective feedback. To help you to find what you want in the literature, there is a summary table of the relevant publications at the beginning of each sub-‐section. If you want more detail, you can find it after each table.

• If you need to check the meaning of any of the terms used in the area of feedback, there is a Glossary in Section 5 and also a full bibliography at the end of the Handbook.

If you would like to give us feedback on this Feedback Handbook, please email [email protected]

5

Part A: Perspectives on Feedback

1.0 The Feedback Project: Student Perspectives Section 1 contains a summary of the outcomes of the project entitled : Feedback in the Sciences: what is wanted, what is given, and how it can be improved amongst a diverse student population.

1.1 Project Overview

The project involved the administration of a questionnaire to second year University of Sydney students in the Faculty of Science (Physiology, Medical Science, Psychology and Molecular Biology) to ascertain the types of written feedback they found useful. The questionnaire, ‘Your experience of receiving feedback on your writing’ was completed by 419 students; their mean age was 20.2 years, and 62.6% were female. Of the students who responded to the question concerning language spoken at home, 77 % named English, 14% Chinese, and 9%, ‘Other’. The questionnaire was followed by six student focus group sessions, involving native speakers of English, both monolingual and bilingual, and non-‐native speakers (non-‐English speaking background: NESB): these sessions were designed, in part, to gain deeper insight into the impact of diversity on student perspectives regarding feedback. Then a number of semi-‐structured marker interviews were held in order to identify current feedback practices. After this, a series of marker workshops was held: these workshops had two purposes. The first was to inform markers of student perspectives and of the perspectives reflected in the current literature regarding best practice for providing effective feedback. The second purpose was to assist markers in identifying and implementing changes to their own feedback practices to improve student learning. The following sections summarise some of the findings from each of these aspects of the project.

1.2 What the students say/want

1.2.1 Summary of outcomes from student questionnaire The questionnaire included four general questions, each with 11 to 15 related statements which the students were asked to rate from 1 to 4 as ‘strongly disagree’ to ‘strongly agree’, or ‘not important at all’ to ‘very important’ in the case of the last of these questions. They were asked to circle the number that most closely reflected their ‘experience in second year’ for the first question, their ‘University experience’ for the second and third, and their ‘opinion regarding the importance of different types of written feedback’ for the last. There were also five open-‐ended questions asking students for their perceptions of feedback. These included ‘What kind of feedback is most helpful to you?’ and ‘Do you use feedback on written assignments when writing future assignments? If yes, how? If no, why not?’

6

Responses to Question 1: How do you experience feedback? • Most participants indicated that the written feedback they received was

related to the assessment criteria, course objectives and the marks given, but was too brief.

• Most students considered negative feedback to be constructive. • Most students reported receiving written but not verbal feedback on their

work. • The students who did receive verbal feedback were almost equally divided on

whether they remembered it or not. Responses to Question 2: What do you do with feedback?

• The majority reported carefully reading and using feedback to go over the current assignment or revise work, as well as to improve future assignments.

• More than two-‐thirds of the students used the feedback even if a high grade was achieved.

• The students did not ignore negative feedback or report negative reactions, such as feeling demoralised or angry, in response to it.

• Positive feedback was reported to boost confidence. • Students reported that receiving only a mark was unhelpful. • The majority of students found markers to be consistent in their application

of assessment criteria; however, there was sizeable minority who did not find markers consistent.

Responses to Question 3: How useful do you find feedback?

• Students found feedback helpful for improving their work, identifying gaps in their knowledge and understanding the course content.

• Students indicated that feedback encouraged reflection on what they had learned, prompted revision and encouraged them to improve.

• As regards feedback given to the whole class, however, opinion was divided, with nearly equivalent numbers finding this helpful and unhelpful.

Responses to Question 4: What type of feedback is important?

• Almost all types were useful except that which provided general comments. • There was some division among students regarding feedback on mistakes in

the use of language. • The data here suggest that students do not regard general feedback as highly

as more detailed/specific feedback. The comparison of item endorsement for the statements within each question showed that, overall, participants most strongly

• sought to use feedback to benefit their learning; • felt that negative feedback was beneficial and not detrimental; • regarded feedback as useful for improving their understanding of both course

content and course requirements; and • regarded feedback that explains mistakes and specifies how to improve

written work as the most important of all feedback types mentioned. As regards students from a NESB, analysis of the data suggests that, overall

• non-‐English at home speakers were, on average, more satisfied with the quality and quantity of feedback than their English at home peers;

7

• those who self-‐rated their fluency in English as ‘reasonable’ or ‘poor’ (the categories being ‘native speaker’, ‘near native’, ‘reasonable’, ‘poor’) placed greater importance on detailed, written feedback than those who rated themselves as more fluent.

Other results suggest that

• higher achieving, more confident students are more desirous of feedback than their lower achieving counterparts.

Answers to the 5 open-‐ended questions in Student Questionnaire: 1. What does ‘feedback’ mean to you?

• Most students referred to o information about weaknesses and strengths, and o suggestions for improvement

• Other common answers were o constructive criticism o personalised comments/advice/suggestions o explanations of why answer is not correct o examples of how they could have done better o comments on

§ what’s missing/what more could have been done § writing technique, and § student’s work in relation to that of rest of class

2. What kind of feedback is most helpful to you?

• Answers generally mirror those for Q1, i.e. related to o formative feedback

§ strong and weak points § suggestions for improvement § specific, constructive criticism § examples of good/correct responses

• Other common answers: o written o verbal, one-‐on-‐one o detailed o concise o specific o general o timely

3. What kind of feedback is least helpful to you?

• Answers specified o the mark only o ticks and crosses without explanation o short comments, especially negative feedback, without explanation o ambiguous/vague comments

• Other common answers:

o group/generalised o marking rubric with no comments

8

o none! 4. Do you use feedback on written assignments when writing future assignments? If yes, how? If no, why not?

• YES answers dominated: o Most students referred to using feedback by

§ trying not to make the same mistakes § implementing any suggestions § keeping what they did well

o Some students mentioned a preference for general comments, others a preference for specific comments.

• There was a small proportion of NO answers: o Most students gave the following reasons:

§ forgetting to look back § feedback on one assignment not relating to next § not enough feedback given to use on next assignment § different marking styles of tutors § not receiving feedback in time

5. What aspect(s) of feedback given on your written assignments at University has/have been most useful to you in helping to improve your writing? Many students seem to have interpreted this question as ‘What kind of feedback would be most helpful to you?’

• Most students referred to feedback on o what they did wrong/badly o why this was wrong o what they can do to improve o what they need and do not need to include next time o what parts of the assignment are good/well interpreted o writing style o misunderstandings of subject matter (with explanation) o structural deficiencies o faults in essay argument o how to improve referencing o sentence structure and use of language

• Other common answers were o written, specific feedback o precise, relevant criticism o individualised, verbal feedback o examples of better ways of proceeding o feedback on drafts

1.2.2. Student Focus groups General information:

• 16 students attended the voluntary focus group sessions o 7 monolinguals o 6 bilinguals o 3 NESBs

• 6 sessions were held o 2 with monolinguals only

9

o 1 with bilinguals only o 2 with monolinguals and bilinguals o 1 with NESBs only

• The open-‐ended question answers of the four cohorts were reflected in the individual comments of the student focus group participants

• In some cases, the comments evidenced disagreement among group members about the usefulness of oral vs. written and whole class vs. individual feedback

Native Speakers: Monolinguals and Bilinguals 1. What kind of feedback is most helpful to you?

• Specific, detailed, targeted, personalised o If it’s detailed feedback and it’s directed at your problem areas, then it’s

definitely good feedback. • Improvement oriented:

o … I went really well but I still lost marks. So … how could I improve? Because that’s the main thing you want to be knowing. It’s constructive criticism, you did this wrong but okay, so what can I do better?

o 10% more – would have been nice to know how to have achieved that. • Written:

o Oral at the time is useful, but … it’s a lot more useful to have written to go back to.

• Oral: o Then they’re going to go into more detail and you can understand it

better. o If it’s verbal, you can actually ask questions if you’re confused.

• Constructive negative feedback: o I really like it if, within the criticism, there might be some good points in

there, some positive things. • Timely:

o Especially when you can use [the feedback] for the next assessment. • Also:

o Whole class oral and written: § It is helpful because then you know how the people who did well,

what they did that was good, and then you also found out, well, I could do it better the next time.

§ how we went in relation to other students (spread of marks). o Model answers:

§ This is what a good paper looks like, and then we can … do the self-‐learning, actually, we don’t need to be told, we can absorb a lot of information just from seeing what excellence looks like.

o Comprehensive feedback (on all aspects of the report): § Content, as well as … how we should approach the assignment or

report. What we should emphasise … all the different sections and what you should include. What is the normal approach.

10

2. What kind of feedback is least helpful to you?

• Mark only and/or ticking of boxes on a marking rubric/grid, with no written comments on assignment itself – no way of knowing where or why marks were lost:

o No explanation … you don’t know what to think of it. o Like, where did I go wrong? … I thought I understood everything, I

thought I’d answered everything quite well; got 6 out of 10, so I don’t know …

• Non-‐constructive/only negative: o … if the feedback is very negative and doesn’t really give you a

springboard to be proactive and improve where you are going. o If all they say is where you went wrong, then it’s just criticism rather

than helpful feedback.

• Impersonal, general comments: o Directed at the whole class o A general comment at the end of the assignment o Comments without suggestions for improvement, such as ‘You need to

elaborate on this’ • Too specific:

o So specific it’s unusable elsewhere, in any other sort of framework … You can’t really transfer it.

• Received after the next assignment or after the exam: o … so you can’t improve from one to the other. So that demotivates me to

go and process it because I needed it earlier. • Advice on writing the report of the experiment not given in lab: genuine

questions on writing the report not answered o They’ll say ‘… you can completely find everything you need’, but then you

need some little help, especially if like you know you’ve got to write a report on it, you want to at least feel confident going to attempt the report, and if you … leave the place going ‘Well, that was fun …’, you’re really not going to do stuff for the report.

3. Do you use feedback on written assignments when writing future assignments? If yes, how? If no, why not?

• Yes: When it can be used for another assignment o Return to feedback when writing next report o Return to feedback sometimes even years later: good because if reread

essay then, § you’re coming from the tutor’s perspective

• No: When it’s so specific it’s unusable elsewhere • Inconsistency in marking:

o One report, you know, tick, tick, tick, tick, the next report I do exactly the same thing, but a different tutor says, ‘No, you shouldn’t have done that, you shouldn’t have done that’ …

4. What aspect(s) of feedback given on your written assignments at University has/have been most useful to you in helping to improve your writing?

• Individual, written, timely, informative:

11

o I’ve had …[demonstrators] who just, you know, … feedback straight away, ... write down exactly where you lost the marks, what the answer is, why, so ‘If you need explaining, let me know’. Just absolutely helpful.

• Individual/group oral: o Demonstrators for most pracs have been fantastic in giving their own

feedback, and … if we’ve got any questions, there’s no problems for us to ask them questions, which is good.

o In our little desk with our tutor, … we get some good, like just non-‐formal feedback from her … She often takes us into the tute room to go over like the questions and the discussions … with us, just to make sure we’ve really understood it …

• Improvement-‐oriented: o Things that help you improve the next time, that you can utilise in the

next assessment. o [When] they give us back the actual quiz paper and also the model

answers ... I could actually go through and see where I’d gone wrong. • All feedback is useful (even a small amount):

o I did terribly in first year reports and I have significantly improved this year … We weren’t given too much feedback, but …what little we were given was useful. If we were given even more, then obviously we would have done even better.

• Experience of acting as a marker (one student) NESB students

1. What kind of feedback is most helpful to you? • Specific:

o On what they can do to improve next time, but if too specific – won’t be relevant to next task

o In whole sentences rather than phrases o On their strengths:

… like my idea is good but only problem is like my grammar or structure like that. I want to know what I’ve good, so I can keep that and change the wrong part.

• Improvement-‐oriented with examples: o It’s better with some examples how to do it, that tell us what’s wrong

with that and what we should do … in the next time. • Written:

o Summary: student can read it again later, won’t forget it. If he doesn’t understand the comments, he’ll make an appointment to discuss it.

• Oral: o Summary: sometimes written feedback is too short and he doesn’t

understand it. He asks tutor for oral feedback in pracs and tutorials and sometimes makes a note of it: It’s more interactive in oral feedback.

• Feedforward is the best: o Better before we do it!

2. Feedback: least helpful; future use; most helpful feedback at university so far

• Least helpful: o Confusing

§ if the feedback makes me confused, what I did the wrong … yeah, that’s kind of the detriment.

12

o In challenging handwriting o Marking rubrics: helpful with structuring the report, but without

feedback comments on theory or content, do not know how to improve • Future use:

o Students read feedback when marked assignment received and again before the next assignment

• Most helpful feedback so far: o Written feedback o Talking with tutor after written feedback

3. Feedback on language matters:

• Feedback on grammar: o Most students wanted specific feedback in the language area,

especially grammar: § I need to know specific where I wrong in English, because I’m

learning … I just need to know the biggest problems … o One student said tutors should not have to correct grammar: he liked

comments pointing out areas where he could improve, e.g. the structure of an assignment, bringing in the other side in a debated area, but not comments where he felt he could not improve:

§ hard to change, like for example, grammar or writing skills. • Markers’ response to grammar problems:

o Markers draw attention to grammar/expression problems, but don’t point out what’s wrong or correct grammar:

§ they write ‘Poor structure’, ‘Poor grammar’, but they didn’t point out where is the wrong.

• Prior experience of English language preparation programs/Foundation programs

o At (Uni Prep) Taylor’s College, much more feedback on grammar and word choice issues than at uni

o At uni, most feedback concerned with content and structure issues, what’s right and what’s wrong (e.g. theoretical mistakes).

Thus, the main findings from the student questionnaire and the focus groups showed that students were generally positive about the importance of good feedback, stating that feedback was more helpful if it was personalised and timely.

13

2.0 The Feedback Project: Markers’ Perspectives In Section 2, you will find summaries of markers’ experiences and their problems with giving feedback.

2.1 Marker Interviews

2.1.1 Problems with written feedback • Time taken writing feedback on each student assignment: no real payment for

time spent giving detailed, specific, individualised feedback • Time taken when giving more elaborated individual oral feedback • Timeliness: feedback irrelevant if received at the end of semester/after exam • Feeling that students are not going to read feedback, and if they do, are not

going to act on it/learn from it • Possibility of student misinterpretation of feedback comments • Frequent repetition of same comment • Fatigue leading to more generic and less specific comments • Student misinterpretation of seriousness of flaws if comments phrased in

relatively positive terms • Students simply reading comments without then reconsidering the related

problem areas in their work

2.1.2 Marker comments re dealing with diverse language backgrounds of students Comments from markers who take language background into consideration:

• Some commented on different feedback given: o If there are language problems, and if a non-‐English speaking

background (NESB) is suspected/known, § they might phrase their feedback about these problems

differently from the way they would with English speaking background students, i.e., it might be more gentle;

§ they make sure their feedback is easily understandable. • Some engaged in self-‐questioning:

o If NESB students are not marked down for poor phrasing, and/or if this aspect of their work is not commented on, is this fair?

Comments from markers who try not to take language background into consideration:

• Some said o they ignore language problems unless these impede understanding of

content; o they award no ‘sympathy points’ because of language background and

mark on the merit of the work only; o they give a lower mark to all students whose papers are written in less

coherent English, irrespective of their background; o they have difficulty sometimes distinguishing between students with

poor language skills and those who have made no effort (they want to reward effort);

14

o if a student’s written expression is not clear, § it is difficult to know whether they have fully mastered the

concept; § and also difficult to articulate to them in detail specifically

where their writing is not clear.

2.2 Marker workshops

Three Marker workshop sessions were held, with the following focuses and titles: • ‘Current practices and problems’ • ‘Feedback practices from the literature’, and • ‘An opportunity for improvement: Applying new approaches’.

26 markers attended from the areas of Physiology, Psychology, Biological Sciences, Molecular Bioscience and Chemistry. Online trigger questions were completed before Sessions 1 and 3, and each participant read a journal article on feedback before Session 2. The first session included

• small group cross-‐disciplinary discussion of markers’ current practices and problems

• a general sharing of the main points of these discussions • a summary of the online trigger question responses • summaries of the quantitative and qualitative data outcomes

o from the student questionnaires in the Feedback Project, and of o the data from the student focus groups.

The second session included • small group critical discussion of the feedback research articles the markers

had read, with o presentations of the summaries of group ideas

• presentation of the marker interview data • the linking of this data, the Session 1 discussions and the responses to Session

1’s first trigger question to the wider literature. In the final session,

• the related trigger question responses were summarised • discipline-‐based groups discussed the feasibility of feedback

strategies/changes, and • the markers committed to 1-‐2 ways of changing practice at the micro-‐level.

The workshops were then evaluated by the attendees, and they were asked to complete a post-‐workshop online survey summarising and evaluating the changes they attempted when providing feedback on students’ written assignments.

2.2.1. Current practices and problems Trigger Question 1: ‘What are the top 3 problems you encounter when providing feedback on students’ writing?’ and the first workshop discussion identified the main problems the markers encountered when providing feedback on students’ written work.

15

The first main problem area identified was the actual feedback provided, i.e. the difficulty of

• expressing their feedback o appropriately, clearly and specifically, and o objectively, concisely and constructively

• dealing with the poor expression of ideas • articulating ways of improving • finding something positive to say • editing vs. identifying ways to improve • dealing with literacy matters vs. discipline-‐specific content • dealing with the work of NESB students • deciding whether to provide specific or general feedback, or both, and in how

much detail The second problem area identified was time, i.e. the mismatch between the time taken to write comprehensive feedback and the time allocated by the payment system. Other important concerns were the difficulty of

• dealing with the constraints of standardized feedback sheets • marking very poor papers • ensuring equity across all students • dealing with ‘my students’ vs. others, i.e. thinking a good student understands

the material, even if this hasn’t been articulated • dealing with a lack of quality control: no models provided, little training, lack

of discussion re marking criteria Trigger Question 2: ‘What do you think students want out of the feedback they receive?’ The responses included knowing

• what was done well • what was done poorly/what they can improve on • how they can improve • how the mark was arrived at • some encouragement

Trigger Question 3: ‘Do you consider student diversity in your marking practice? If so, in what ways?’.

• Most responded ‘No’, o stressing that all students needed to be marked in the same way, o some adding that they tried to focus on the content rather than

‘perfectly good English’ but that if the language was poor, the students would have more difficulty expressing themselves and so would be penalised.

• Others responded ‘Yes’, o explaining that they would not penalise grammar or spelling mistakes

as long as they could understand what the student was saying; o that is, they would focus on the concepts ‘rather than worrying about

exactly how they expressed them’.

Thus, marker interviews and marker workshops raised similar issues: the time needed for good feedback and the difficulty related to giving appropriate and good quality feedback, particularly for students from diverse language backgrounds.

16

3.0 The Feedback Project: Suggested Solutions to Feedback Issues and Challenges

3.1 Coordinator-centred issues and suggestions

• Discipline/School/Faculty financial support to coordinators o to develop/revise/adapt assessments to align with the University’s

Assessment Policy o to develop resources for markers and students on how to provide and

use feedback o to develop resources for sessional marker training

• Encourage markers to provide feedback on assessment design, marking rubric, marker training, etc.

• Have pre-‐ and post-‐marking meetings for all markers • Feed back to lecturers concepts students find difficult or misunderstand as

evidenced in their assignments • Integrate into the curriculum how to use feedback for students to build skills

in interpreting & using feedback • Engage in coordinating assessment design across a program aligning with

feedback to feed forward in the development of discipline specific knowledge and skills and graduate attributes

3.2 Marker-centred issues and suggestions

3.2.1 Solutions suggested in Marker Interviews

• The provision of clear criteria – some are too vague, others too broad • The holding of a discussion session with lecturer/coordinator prior to

marking re what students might have written • The provision of specific guidelines concerning what to give marks for and

what not to give marks for • The provision of specific guidelines concerning what is expected of all

markers, like ‘Give qualitative feedback; don’t just tick the rubric boxes’ • The provision of a summary of the components of a good response to each

essay question or a detailed rubric for each question • The provision of a sample HD essay for each topic • The provision of feedback statement banks • When questions are raised by individual markers, the emailing of Q and

answer to all markers • The holding of a post-‐marking meeting re problems encountered and changes

that need to be made to marking scheme for following semester • The provision of a marker training course • More communication between coordinator and all markers • Encouraging markers to email coordinator with general feedback re

assignments which can be posted online • Marker ‘stars’ best answers and these are posted up outside their office so

that students see what a good answer entails

17

• Hearing what students have to say about the marking process • Getting students to think like markers: inviting them to a voluntary

assessment workshop. This involves o the use of good and poor examples of past students’ work (exemplars), o the discussion of why these are good or poor, o the discussion of marking criteria, o student group discussion and the marking of exemplars, using criteria, o the discussion of groups’ marks, and o the comparison with marks given and discussion of reasons why.

3.2.2 Solutions suggested by Workshop participants: Strategies suggested in discussions:

• Better quality control: institution of ‘blind’ marking as standard • Report writing tutorial for students as standard • Tutorial on ‘good/bad’ report examples, with students marking these • Student writing of a 500-‐word final Psychology essay plan: this has to be

approved before work on essay goes ahead • The swapping and marking of draft Biology essays with peers in a workshop • Tutor pointing out of ‘the obvious’ in assignment guidelines!

Strategies suggested in the responses to trigger questions Question 1: ‘Identify at least one strategy or change you could implement that would address one or more of the problems you identified when providing feedback on students’ writing’. Responses included

• using an audio file instead of written feedback • focussing feedback on major points • outlining how to improve • Starting comments with ‘Next time, …’ • providing more comprehensive, clear and positive feedback • going over the marking criteria with students • having model reports • student self-‐assessment of work prior to submission • focus groups to discuss tutor feedback effectiveness • the identification of common problems that students encounter

Question 2: ‘What help do you think could be given to markers to support them in the marking and feedback process?’ Responses here included:

• the allocation of more paid time per assignment • guidelines from coordinators detailing specific criteria that should be met for

a student to receive a particular grade • better/clearer marking criteria • a review process to ensure consistency in marking standards • marker training • listening to student feedback on the assessment and feedback process

Question 3: ‘Could assessments be improved in any way to help you to give students better feedback? If yes, please briefly explain how’. Responses included:

• the draft submission of assessments

18

• more than one assessment testing the same skills, allowing students to apply feedback

• the alignment of the objectives of assessment with marking criteria and course requirements and outcomes

• early simple essay questions, with a gradual increase in difficulty • clear, detailed and specific marking criteria to reduce inconsistency among

markers • outlining that English expression will form part of the assessment

Strategies markers implemented after the workshop and found useful:

• Drawing up a list of the most common comments made on assessments and using these to identify the most common problem areas in written assignments;

• Rather than thinking of the positive and negative aspects of what the student wrote, focusing on what the student should change in order to produce a better outcome in the next assessment (i.e. implying strengths and weaknesses this way);

• Providing audio files for each student, in addition to brief, written comments; • Providing specific, direct feedback; • Giving students a clear marking scheme before submission and using this as

the feedback sheet; • Providing positive feedback; • Being more concise in feedback comments, and focussing on a few major

points: at the end of student report, brief written feedback about what was done well and what could be improved on;

• Being more positive in comments made, starting with a positive statement before moving to comment on mistakes; being constructive: noting the mistake, but also explaining how to improve;

• When giving essay feedback, drawing on a pool of generic comments that can be copied from a Word document; additional individualised feedback also given.

Other marker-‐suggested but unimplemented strategies:

• as well as annotating individual assignments, providing students with a photocopied sheet of longer explanations for the most frequently encountered problems;

• getting students to write a draft report and giving feedback on this; students rewrite report, integrating the feedback given.

3.3 Suggestions from the literature to bring students into ‘the community of assessment practice’

• Before/at the time of final submission: o Peer feedback using assessment criteria before final draft written: this

can be organised via webCT o Student written self-‐assessment according to assessment criteria when

handing in a piece of work o Draft, then feedback from marker, then final submission with mark

only given

19

o Student required to explain (in writing) how they’ve used prior feedback in current assignment

• After submission: o Digital audio feedback: using voicefiles

§ see the ‘Sounds Good’ website at: http://sites.google.com/site/soundsgooduk/

o Marker feedback given on work submitted, then student comments on this: no mark given until student comments received by marker

o Work returned, with feedback and criteria: students asked to rate themselves with reasons, before mark given

o Early generic feedback to whole class rather than specific individual feedback which is available too late

o Limiting the number of feedback comments

20

Part B: What the literature says: ‘Suggested Solutions’ Themed Annotated Bibliography (non-‐exhaustive): NOTE: This Bibliography is limited to documents published, in almost all cases, from 2000 on. Part B is divided into two main sections, each providing summary tables and an annotated bibliography. Section 4.1 presents an overview of recent literature on feedback, giving first some relevant background information, then focusing on the purposes and effectiveness of feedback, and students’ understanding and use of it. Section 4.2 summarises the suggested solutions found in the literature, focusing on good feedback practice, the wording of feedback comments, student engagement, dialogic approaches, using technology, dealing with large classes, and feedforward. These solutions are then developed in the annotated bibliography which follows. Section 5 provides a glossary of key terms.

21

Table 1: From the literature: Overview of Feedback – Guide Sources G

eneral concerns Purposes of feedback

Problems w

ith transm

ission view of

feedback

Student understanding of feedback com

ments

Student engagement

with feedback

Student use of feedback

Student perceptions of tutors’ w

ritten feedback

Effectiveness of feedback

Importance of ‘tacit

knowledge’

Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education, 5(1), 7-74. doi: 10.1080/0969595980050102 ✓ Carless, D. (2006). Differing perceptions in the feedback process. Studies in Higher Education, 31(2), 219-233. doi: 10.1080/03075070600572132 ✓ Chanock, K. (2000). Comments on Essays : Do students understand what tutors write? Teaching in Higher Education, 5(1), 95-105. ✓ Gibbs, G., & Simpson, C. (2004). Conditions Under Which Assessment Supports Students ’ Learning. Learning and Teaching in Higher Education, 1(1), 3-31. ✓ ✓ Glover, C., & Brown, E. (2006). Written Feedback for Students : too much , too detailed or too incomprehensible to be effective ? Bioscience Education eJournal, 7(May).

✓

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112. doi: 10.3102/003465430298487 ✓ Higgins, R., Hartley, P., & Skelton, A. (2001). Getting the message across: The problem of communicating assessment feedback. Teaching in Higher Education, 6(2), 269-274. doi: 10.1080/13562510120045230

✓

Higgins, R., Hartley, P., & Skelton, A. (2002). The Conscientious Consumer : Reconsidering the role of assessment feedback in student learning. Studies in Higher Education, 27(1), 53-64. doi: 10.1080/03075070120099368

✓ ✓ ✓

Holmes, L. E., & Smith, L. J. (2003). Student evaluations of faculty grading methods. Journal of Education for Business, 78(6), 318-323. ✓ Hounsell, D., McCune, V., Hounsell, J., & Litjens, J. (2008). The quality of guidance and feedback to students. Higher Education Research & Development, 27(1), 55-67. doi: 10.1080/07294360701658765

✓

Hyland, F., & Hyland, K. (2001). Sugaring the pill: Praise and criticism in written feedback. Journal of Second Language Writing, 10, 185-212. ✓ Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self‐regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218. doi: 10.1080/03075070600572090

✓ ✓ ✓ ✓ ✓

22

Sources General concerns

Purposes of feedback

Problems w

ith transm

ission view of

feedback

Student understanding of feedback com

ments

Student engagement w

ith feedback

Student use of feedback

Student perceptions of tutors’ w

ritten feedback

Effectiveness of feedback

Importance of ‘tacit

knowledge’

Orsmond, P., & Merry, S. (2011). Feedback alignment: Effective and ineffective links between tutors’ and students’ understanding of coursework feedback. Assessment & Evaluation in Higher Education, 36(2), 125-136. doi: 10.1080/02602930903201651

✓

Orsmond, P., Merry, S., & Reiling, K. (2005). Biology students' utilization of tutors' formative feedback: A qualitative interview study. Assessment & Evaluation in Higher Education, 30(4), 369-386. doi: 10.1080/02602930500099177

✓ ✓

Price, M., Carroll, J., O'Donovan, B., & Rust, C. (2010). If I was going there I wouldn't start from here: a critical commentary on current assessment practice. Assessment & Evaluation in Higher Education. Retrieved from doi:10.1080/02602930903512883

✓ ✓ ✓

Price, M., Handley, K., Millar, J., & O'Donovan, B. (2010). Feedback : all that effort, but what is the effect? Assessment & Evaluation in Higher Education, 35(3), 277-289. doi: 10.1080/02602930903541007

✓ ✓

Rust, C. (2007). Towards a scholarship of assessment. Assessment & Evaluation in Higher Education, 32(2), 229-237. doi: 10.1080/02602930600805192 ✓ Rust, C., Price, M., & O'Donovan, B. (2003). Improving students' learning by developing their understanding of assessment criteria and processes. Assessment & Evaluation in Higher Education, 28(2), 147-164. doi: 10.1080/02602930301671

✓

Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119-144. ✓ ✓ University of Sydney feedback webpages ✓ ✓ University websites: Open University & Sheffield Hallam, Oxford Brookes, Strathclyde, Edinburgh; bioassess website ✓ ✓ Vardi, I. (2012). Effective feedback for student learning in higher education:

HERDSA. ✓ Weaver, M. R. (2006). Do students value feedback? Student perceptions of tutors’ written responses. Assessment & Evaluation in Higher Education, 31(3), 379-394. doi: 10.1080/02602930500353061

✓

23

4.1: From the literature: Overview of Feedback

4.1.1 General concerns For Hattie and Timperley (2007), feedback ‘is one of the most powerful influences on learning and achievement’; however ‘this impact can be either positive or negative’ (p. 81). The main purpose of feedback is ‘to reduce discrepancies between current understandings and performance and a goal’ (p. 86). They summarise the results of their previous (1999) synthesis of meta-‐analyses as showing that ‘the most effective forms of feedback provide cues or reinforcement to learners; are in the form of video-‐, audio-‐, or computer-‐assisted instructional feedback; and/or relate to goals’ (p. 84). The authors then identify the conditions that ‘maximize the positive effects on learning’, using a model of feedback as a framework. They argue that effective feedback needs to answer the questions: ‘Where am I going? (What are the goals?), How am I going? (What progress is being made toward the goal?), and Where to next? (What activities need to be undertaken to make better progress?)’, these questions corresponding to the concepts of ‘feed up, feed back and feed forward’ (p. 86). Their model also distinguishes between four levels of feedback: ‘the task, the processing, the regulatory, and the self levels’ (p. 102). The authors then address a number of ‘commonly debated issues’, namely ‘the timing of feedback, the effects of positive and negative feedback, the optimal classroom use of feedback, and the role of assessment in feedback’ (p. 98). They conclude that ‘a feedback intervention provided for a familiar task, containing cues that support learning, attracting attention to feedback-‐standard discrepancies at the task level, and void of cues that direct attention to the self is likely to yield impressive gains in students’ performance’ (p. 104). Gibbs and Simpson’s (2004) paper centres on ‘how to design assessment that supports worthwhile learning’, arguing that ‘assessment works best … when a series of conditions are met’ (p. 4). These conditions refer to two categories: ‘the influence of the design of assessment systems and assignments on how much students study, what they study and on the quality of their engagement’ and ‘the influence of feedback on learning’ (p. 12). The latter conditions (seven of the ten given) are that: ‘Sufficient feedback is provided, both often enough and in enough detail’; ‘The feedback focuses on students’ performance, on their learning and on actions under the students’ control, rather than on the students themselves and on their characteristics’; ‘The feedback is timely in that it is received by students while it still matters to them and in time for them to pay attention to further learning or receive further assistance’; ‘Feedback is appropriate to the purpose of the assignment and to its criteria for success’; ‘Feedback is appropriate, in relation to students’ understanding of what they are supposed to be doing’; ‘Feedback is received and attended to’; and ‘Feedback is acted upon by the student’ (pp. 16-‐25). Nicol and Macfarlane-‐Dick (2006), based on a synthesis of the research literature, identify seven principles of good feedback practice that support student self-‐regulation of their learning. Self-‐regulation refers to ‘the degree to which students can regulate aspects of their thinking, motivation and behaviour during learning’, and is manifested in, e.g., ‘the setting of, and orientation towards, learning goals’, and ‘reactions to external feedback’ (p. 199). The principles are that good feedback practice: ‘1) helps clarify what good performance is (goals, criteria, expected standards); 2) facilitates the development of self-‐assessment (reflection) in learning; 3) delivers high quality information to students about their learning; 4) encourages

24

teacher and peer dialogue around learning; 5) encourages positive motivational beliefs and self-‐esteem; 6) provides opportunities to close the gap between current and desired performance; 7) provides information to teachers that can be used to help shape teaching’ (p. 205). The authors go on to specify teacher strategies to assist self-‐regulation for each of these principles, including, e.g., for the first principle, ‘involving students in assessment exercises where they mark or comment on other students’ work in relation to defined criteria and standards’ (p. 207), and for the third principle, ‘providing corrective advice, not just information on strengths/ weaknesses’ (p. 210). Black and Wiliam’s (1998) literature review on classroom formative assessment, although largely concerned with school situations, makes the point that ‘the quality of the feedback provided is a key feature in any procedure for formative assessment’ (p. 36). They add that ‘feedback interventions that cue individuals to direct attention to the self rather than the task appear to be likely to have negative effects on performance. Thus praise, like other cues which draw attention to self-‐esteem and away from the task, generally has a negative effect’ (p. 49). Gibbs and Simpson (2004, p. 9) comment that ‘Black and Wiliam’s (1998) comprehensive review of formative assessment emphasizes the extraordinarily large and consistent positive effects that feedback has on learning compared with other aspects of teaching.’ Hounsell, McCune, Hounsell & Litjens (2008) report on the findings of a large survey (questionnaires and group interviews) of students’ experiences of the provision of guidance and feedback. They present a ‘guidance and feedback loop’ with six interrelated steps: ‘Students’ prior experiences of assessments’, ‘Preliminary guidance’, ‘Ongoing clarification’, ‘Feedback on performance/ achievement’, ‘Supplementary support’ and ‘Feed-‐forward’ (p. 60). They also identify potential troublespots, e.g. ‘feedback dwells on shortcomings & demotivates’, and ‘feedback lacks transparency’ (p. 66), and explain how a shortcoming in one step could affect the following steps in the loop, either positively or negatively. Rust (2007) argues for the articulation and establishment of a ‘scholarship of assessment’ as a vital part of the scholarship of teaching and learning, as assessment ‘seems to be the part of the learning and teaching process that has been especially criticised’ (p. 229), and the area of feedback the ‘worst of all’. So the scholarship of assessment should include ‘explicit guidelines on giving effective and prompt feedback’ (p. 230).

4.1.2 Purposes of feedback Price, Handley, Millar and O’Donovan (2010) identify five roles attributed to feedback: ‘correction, reinforcement, forensic diagnosis, benchmarking and longitudinal development (feed-‐forward)’. The authors see these roles as acting as a ‘nested hierarchy’, i.e., ‘each building on information provided by the previous category’ (p. 278). They argue that the first two roles are not really relevant for much feedback in higher education, which ‘has limited scope to “correct” complex work’, but, citing Sadler (1989), that the forensic role, ‘diagnosing problems with the work’, links with the benchmarking role ‘where feedback identifies a gap between what is understood/has been demonstrated and the standard of performance expected’. Whether feedback can help to fill that gap depends on the nature of the gap:

Where the gap relates to the curriculum content, the feedback may be able to specify the knowledge that needs to be understood. However, where the gap

25

identifies the need for development of, for example, academic or cognitive skills, feedback may not be able to be specific in its remedy for filling the gap due to the slowly learnt nature of knowledge needed to address the gap. (p. 278)

(For feedforward, see below.) Feedback is also used by markers to justify the grade given. Price, Carroll, O’Donovan and Rust (2010), discussing current assessment practice, argue that this, rather than to develop learning, tends to be the purpose when feedback ‘is concentrated on summative work’, because of the ‘emphasis on reliability of marking’ (p. 3) in the higher education sector.

4.1.3 Problems with the transmission view of feedback Nicol and Macfarlane-‐Dick (2006) argue that in higher education, ‘formative assessment and feedback are still largely … seen as the responsibility of teachers; and feedback is still generally conceptualised as a transmission process, even though some influential researchers have recently challenged this viewpoint (Sadler, 1998; Boud, 2000; Yorke, 2003)’ (p. 200). They identify a number of problems with this transmission view, including that of how students ‘can become empowered and develop the self-‐regulation skills needed to prepare them for learning outside university and throughout life’, and the problem of the assumption that teachers’ feedback information is ‘easily decoded and translated into action’ (p. 201).

4.1.4 Student understanding of feedback comments Higgins, Hartley and Skelton (2001) state that feedback comments ‘convey a message based on an implicit understanding of particular academic terms, which in turn reflect a much more complex academic discourse, which in turn may be only partially understood by students’ (p. 272). According to the authors, Hounsell (1997) and McCune (1999) suggest that ‘HE students may struggle to access the particular discourses underpinning tutors’ comments’. Chanock (2000) carried out a study on the common marker comment on essays in the Humanities of ‘Too much description; not enough analysis’, asking tutors and students what they thought this comment meant. She found that ‘almost half of the students who responded did not interpret this comment in the way their tutors intended it’ (p. 95). Possible reasons for this include the ambiguous nature of the term ‘analysis’ and its discipline-‐specific meaning. She concludes that ‘marking comments need to be carefully explained with examples from the discourse of lectures, tutorials and readings in the disciplines’ (p. 95). Hyland and Hyland (2001) offer a detailed analysis of ‘the written feedback given by two teachers to ESL students over a complete proficiency course’, considering especially its interpersonal aspects, namely ‘its functions as praise, criticism, and suggestions’. Criticisms and suggestions were often softened by praise, ‘hedging devices, question forms and personal attribution’; however, this sometimes resulted in lack of student understanding of these indirect comments. So such indirectness ‘carries the very real potential for incomprehension and miscommunication’ (2001, p. 185).

26

Holmes and Smith (2003) report on a study of student problems with grading methods, including that of too little feedback from markers. They refer to the research of Speck and Jones (1998) on the use of codes, symbols and checkmarks within papers ‘as “minimalist grading” that saves faculty members time but may not enhance learning’ (p. 319). Putting question and exclamation marks and ‘Yes’ or ‘No’ in margins ‘assumes that students will understand what the teacher is referencing’, while using codes and symbols ‘assume[s] that students … will look up what the symbols mean and improve their work in following assignments’, assumptions that ‘may be erroneous’. They add that marginal comments ‘should be accompanied by closing comments that give a summative picture of the student’s work’, addressing strengths and weaknesses. They add that ‘giving positive comments along with suggestions for improvement, using language that is respectful, and giving evidence of the fairness of the grades are good practices supported by the literature’ (p. 319). Higgins, Hartley and Skelton (2002, p. 56), referring to Hounsell (1987), discuss the failure of some students to understand ‘the taken-‐for-‐granted academic discourses which underpin assessment criteria and the language of feedback’. They point out that teachers ‘may recognise a good performance, yet struggle to articulate exactly what they are looking for because conceptions of quality usually take the form of tacit knowledge’. They refer to a body of literature (from 2000 and before) which echoes the view ‘that students often experience problems interpreting the academic language underpinning assessment.’ See also Carless (2006).

4.1.5 Student engagement with feedback According to Price, Carroll, O’Donovan and Rust (2010), student engagement with feedback is most effective ‘where: the purpose of the feedback is clear; it has utility and can be used in future work; it accounts for the transfer of tacit as well as explicit knowledge about standards and disciplinary knowledge; and the student perceives some sort of relationship with the marker’ (p. 4). Nicol and Macfarlane-‐Dick (2006) report that ‘if students do not share (at least in part) their teachers’ conceptions of goals (and criteria and standards), then the feedback information they receive is unlikely to “connect” (Hounsell, 1997)’ (p. 206). This makes it difficult for them to ‘evaluate discrepancies between required and actual performance’.

4.1.6 Student use of feedback Orsmond, Merry and Reiling’s (2005) study of Biology students’ use of formative feedback showed that feedback was used to ‘enhance motivation, enhance learning, encourage reflection and clarify … progress’ (p. 373). Some students also used feedback to ‘enrich their learning environment’ (p. 374). For the students interviewed, motivation resulted from either positive or negative comments, though some comments were ‘upsetting’; and they saw learning as ‘developmental and progressive’ (p. 375). Feedback also allowed ‘clarification of tutors’ general expectations of students’ performance in assignments’ (p. 376). Among the authors’ recommendations for both biology and other discipline areas are that ‘previous student feedback could be linked to “feedforward” discussions and so could provide a focal point for assignment preparation’, and that ‘heightened awareness by tutors

27

of how students implement feedback could positively influence how tutors write feedback’ (p. 382).

4.1.7 Student perceptions of tutors’ written feedback The research project of Higgins, Hartley and Skelton (2002) focused on (Business and Humanities) students’ understandings of feedback. The research suggests that the students were ‘eager to read feedback comments’, and believed they deserved feedback. This links to ‘a perception that higher education is a service and, as such, it is also the tutor’s “duty” to offer feedback’. The authors conclude that ‘it is … likely that many of today’s students have a “consumerist awareness” reflected in a focus on achieving a grade alongside intrinsic motivations’ (p. 61). Weaver’s (2006) small-‐scale study of Business and Art & Design students’ perceptions of feedback showed that ‘feedback is valued’, but that ‘tutor comments could be more helpful’ (p. 379). Four main themes of feedback considered unhelpful for improving learning were uncovered: ‘comments which were too general or vague, lacked guidance, focused on the negative, or were unrelated to assessment criteria’. The results suggest that ‘providing a better balance between positive and critical feedback’ is ‘an important step towards making feedback more effective’ (p. 392). Orsmond and Merry (2011) discusses the misalignment between student conceptions of the role of feedback and the focus of tutors when giving feedback. The authors make a number of recommendations to tutors, including that they should ‘write feedback with variation in it and draw particular attention to students’ development as professional biologists and the acquisition of lifelong learning skills’, ‘make greater use of exemplars and whole-‐class feedback to show variation’, and ‘encourage a greater awareness in students concerning the value of self-‐assessment’ (p. 134).

4.1.8 Effectiveness of feedback Sadler (1989) outlines a theory of formative assessment (see Glossary) which applies ‘wherever multiple criteria are used in making judgments about the quality of student responses’ (p. 119). In these ‘complex learning settings’, even where teachers provide ‘regular, accurate feedback’, students often show ‘little or no growth or development’. Sadler argues that ‘students … need more than summary grades if they are to develop expertise intelligently’ (p. 121):

… the learner has to (a) possess a concept of the standard (or goal, or reference level) being aimed for, (b) compare the actual (or current) level of performance with the standard, and (c) engage in appropriate action which leads to some closure of the gap.

He distinguishes between feedback and self-‐monitoring, explaining that formative assessment includes both, and it is the transition from feedback to self-‐monitoring which is at the heart of closure of the gap. This transition can only be made when the above three conditions are satisfied. Sadler contends that criteria sheets do not necessarily deal adequately with the qualities of a piece of work because there are both manifest and latent criteria involved in evaluation, and that students need to develop their knowledge of both in order to be able to monitor their own performance, i.e. they need to ‘be given adequate evaluative experience themselves’ (p. 134) to develop ‘an appropriate body of tacit knowledge’ (see Glossary). This can

28

be done by engaging them in ‘evaluative and corrective activity on other students’ work’ (p. 140). Price, Handley, Millar and O’Donovan (2010) point out that feedback can only be effective ‘when the learner understands the feedback and is willing and able to act on it’ (p. 279). However, in higher education, most feedback requires interpretation by the learner, and students can choose whether to act on it or not. This choice may depend on the student’s positive or negative response to the feedback, on its timing, and on the student’s level of engagement with it, the latter depending in part on the student-‐tutor relationship. The authors report on a three-‐year study of student engagement with feedback, revealing the difficulty of measuring the effectiveness of feedback because of its multiple purposes (see above), ‘its temporal nature and the capabilities of evaluators’ (p. 277), i.e. staff, students, and/or external observers. They conclude that ‘the pedagogic literacy of students is key to evaluation of feedback and feedback processes’ (p. 277). Gibbs and Simpson (2004) argue that writing comments on assignments ‘remains a major component of teachers’ workload in higher education’, and, as a result of increased class sizes, teachers ‘can find themselves spending much of their time marking’. They ask ‘Is all this effort worthwhile?’ and ‘How well does feedback actually work?’ (p. 10). Nicol and Macfarlane-‐Dick (2006, p. 213) refer to the second of Yorke’s (2003) questions regarding external feedback, namely whether feedback leads to changes in student behaviour, citing Boud:

The only way to tell if learning results from feedback is for students to make some kind of response to complete the feedback loop (Sadler, 1989). This is one of the most often forgotten aspects of formative assessment. Unless students are able to use the feedback to produce improved work, through, for example, re-‐doing the same assignment, neither they nor those giving the feedback will know that it has been effective. (Boud, 2000, p. 158)

Orsmond, Merry and Reiling (2005) report on Orsmond et al. (2002) which showed that effective feedback ‘resulted from the use of exemplars as a focus for discussion between tutor and student allowing a common understanding of terms to be developed’, and from the student and tutor’s ‘common understanding of how this feedback might be implemented’ (p. 370). Glover and Brown (2006) present a ‘detailed analysis’ of the effectiveness of written feedback in Biosciences and Physical Sciences at two UK universities. They found that the conditions leading to effective feedback (Gibbs & Simpson, 2004) ‘are not met as frequently as originally believed’, and that although there may be ‘plenty’ of feedback, ‘it is not always received in time to be of use to students’, and ‘may be misunderstood in relation to assessment criteria’. Other findings are that the prime function of the feedback given is ‘to inform the students about their past achievement rather than looking forward to future work’, and most serves primarily ‘to justify the grade’. The authors suggest that ‘far less emphasis’ needs to be placed on marks, and ‘far more on achievement’; that ‘teaching, learning outcomes and assessment’ be ‘constructively aligned’; and that ‘main weaknesses’ be focused on and explained in depth.

29

Vardi (2012) is an excellent (HERDSA) Guide to ‘Effective feedback for student learning in Higher Education’. The author makes an overview of the literature on feedback, and provides practical lists of what improves and what impedes student performance and confidence. She then discusses practical ways of putting theory into action in a university context. These ways include planning a learning program which embeds the provision of opportunities for teacher feedback, both ‘quick and immediate’ in-‐class and ‘more detailed delayed’, and for ‘students to act and build on that feedback’ (p. vii). She describes, and gives examples of, ‘how to give powerful feedback immediately in class situations such as lectures, laboratories and tutorials and their online equivalents’, as well as ‘how to give good quality written teacher feedback on assignments’, this feedback feeding forward to future student work. She then focuses on students as recipients and providers of feedback, detailing practical ways ‘to improve student awareness and receptiveness to feedback’, providing ‘useful tips’ on helping them identify, seek and use feedback effectively, and explaining ‘how to support peer feedback for maximum success’ (p. viii).

4.1.9 Importance of ‘tacit knowledge’ (see Glossary) Rust, Price and O’Donovan (2003) argue that this kind of knowledge is experience-‐based and ‘can only be revealed through the sharing of experience-‐socialisation processes involving observation, imitation, dialogue and practice (Nonaka, 1991; Baumard, 1999)’:

So over time discussion and shared experiences of marking and moderation among staff enable the sharing of tacit knowledge, resulting in more standardised marking (Saunders and Davis, 1998). It follows that inviting students into this shared experience should also enable more effective knowledge transfer of assessment processes and standards to them. (p. 152)

See also: Higgins, Hartley and Skelton (2002), Price, Carroll, O’Donovan and Rust (2010), Sadler (1989), and Nicol and Macfarlane-‐Dick (2006), above.

4.1.10 University Feedback Websites (see others under ‘Using Technology to provide peer and marker feedback’, below) The Open University and Sheffield Hallam University: FAST project The FAST project, a collaboration between The Open University and Sheffield Hallam University, ‘was designed to change assessment so as to support student learning more effectively’. The project methodology ‘involved a two year cycle of diagnostic evaluation, innovation, and evaluation of change followed by one year of wider dissemination to science teachers in other institutions’. The project aimed to address Gibbs and Simpson’s (2004) eleven conditions under which assessment supports student learning, which ‘relate to the quantity, distribution and quality and level of student effort; the quantity, timing and quality of feedback; and the student response to that feedback.’ Two major outcomes of the project were the generation of Case Studies (the website provides the outcomes of this activity) and ‘the establishment of a single “Special Interest Group”, looking at formative assessment in science. See also Brown (n.d.) below. http://www.open.ac.uk/fast/FASTProject/Project.htm Oxford Brookes University’s ASKe (Assessment Standards Knowledge exchange) website:

30

This website makes available series of '1,2,3' leaflets ‘which highlight some practical ways in which teaching staff can improve their students' learning’; there are also leaflets for students. Each leaflet ‘focuses on a piece of assessment-‐related research and clearly states how that research can be applied to teaching practice in three easy steps’. Relevant leaflets for tutors are: ‘How to make your feedback work in three easy steps’, ‘Making Peer Feedback work in three easy steps’, ‘Using Generic Feedback Effectively’, ‘Improve your students’ performance in 90 minutes!’, and ‘Face-‐to-‐face feedback’; and for students, ‘Advice for students: Feedback – Make it work for you!’ http://www.brookes.ac.uk/aske/resources/ University of Strathclyde: ‘Feedback is a dialogue’ The resources on this site include the ‘Feedback is a dialogue Staff Leaflet’ and the ‘Feedback is a dialogue Student Leaflet’. http://www.strath.ac.uk/learnteach/teaching/staff/assessfeedback/ University of Edinburgh: ‘Enhancing Feedback’ site This site has information for staff and students, featuring, for staff, ‘Time-‐friendly ways to boost feedback’, and ‘Ideas, strategies and case examples’, among other topics. The links in the first, e.g., include those to ‘assessment dialogues’, ‘engaging with criteria and standards’, ‘elective feedback’, ‘self-‐generated feedback’, and ‘the use of exemplars’. http://www.enhancingfeedback.ed.ac.uk/ See especially the ‘Feedback in your subject’ pages’. The links take you to an annotated bibliography of relevant documents in various subjects, including, from Science and Engineering, Biological Sciences and Chemistry; from Medicine and Veterinary Medicine, Medicine; and from Humanities and Social Science, Psychology. http://www.enhancingfeedback.ed.ac.uk/staff/subjectfeedback.html There is also a useful feedback resource available at the bioassess website (Harris, Krause, Gleeson, Peat, Taylor, & Garnett, 2007). Harris, K-‐L., Krause, K., Gleeson, D., Peat, M., Taylor, C. & Garnett, R. (2007). Enhancing Assessment in the Biological Sciences: Ideas and resources for university educators. Available at: www.bioassess.edu.au

4.1.11 University of Sydney Feedback webpages The Institute for Teaching and Learning website features, in ‘Projects and Planning’ – ‘Assessment Resources’, a webpage on Feedback which includes links to further resources such as their own ‘Teaching Insights – 4’ on ‘Giving Feedback’, consisting of a set of guidelines (e.g. ‘Clarify your assessment standards’, ‘Use exemplars’). http://www.itl.usyd.edu.au/ The First Year Experience (FYE) website is a resource providing support for coordinators of first year programs and units of study. In its ‘Teaching and Learning’ pages, the site features a ‘Giving quality feedback’ section, the latter giving an overview of the factors involved in making feedback effective for learning, of methods for giving feedback, including innovative approaches, and using online feedback. There are also links to related journal articles and websites, as well as

31

related University of Sydney links (e.g. The Business School’s page on feedback for students in large classes). sydney.edu.au/staff/fye

4.2 Suggested Solutions from the Literature

In Section 3 of this Handbook, a division was made between ‘University/ Coordinator-‐centred Suggestions’ and ‘Solutions suggested by Markers’. In the following ‘Suggested Solutions – Guide’, for practical reasons concerning overlap, the suggestions relate to markers more generally, and include those which will depend on decisions and action by Coordinators. Vardi’s (2012) HERDSA Guide (Effective feedback for student learning in Higher Education), sections of which are referred to in the ‘Guide’ below, gives much good advice to Coordinators. Two examples of this advice follow. Preparing markers to give feedback Vardi argues that tutors ‘often stand at the heart of a university teaching program’, and that ‘students need to be able to trust their judgement’ (p. 32). To ensure that this trust is present, Vardi lists some useful ways of supporting staff ‘to provide powerful feedback’. These include: ‘Induct your staff thoroughly into all aspects of the unit.’ ‘Develop or improve your performance expectations in collaboration with your teaching staff.’ ‘Provide your tutors with a feedback tip sheet outlining what they need to do.’ Evaluating and improving the effectiveness of feedback Vardi provides a model for evaluation which includes, firstly, ‘whether or not you put into place the right mechanisms for a powerful context, with powerful feedback and responsive students’ (p. 41), and gives a table of questions (‘Is there …’) and a corresponding ‘Action list’. The second part consists of ‘check[ing] how well each component is working’, and she again provides a table with ‘Check …: and ‘Collect data …’ headings. The third part of the model concerns the relationship of the context, feedback and student responsiveness to institutional data, and the last, to the improvements that can be made and how these could be addressed.

32

Table 2: From the literature: Suggested Solutions – Guide Sources

Feed

back

pr

actic

e

Exam

ples

of

actu

al

feed

back

re

spon

ses

Stud

ent

enga

gem

ent

with

fe

edba

ck

Dia

logi

c ap

proa

ches

to

feed

back

Usi

ng

tech

nolo

gy

for p

eer a

nd

mar

ker

feed

back

Dea

ling

with

la

rge

clas

ses

Feed

forw

ard

Ballantyne, R., Hugher, K., & Mylonas, A. (2002). Developing procedures for implementing peer assessment in large classes using an action research process. Assessment & Evaluation in Higher Education, 27(5), 427-441. doi: 10.1080/0260293022000009302

✓

Basnet, B., Brodie, L., & Worden, J. (2010). Enhanced Feedback - Does Peer Assessment achieve this goal? Paper presented at the AaeE 2010: Past, Present, Future - the 'Keys' to Engineering Education Research and Practice.

✓

Bridgeman, A., & Rutledge, P. (2010). Getting personal: Feedback for the masses. Synergy (Institute for Teaching and Learning, University of Sydney)(30), 61-68. ✓ ✓ Brown, E. (n.d.). Effective feedback. Formative Assessment in Science Teaching (FAST) Retrieved 22.03.12 ✓ Brown, E., & Glover, C. (2006). Evaluating written feedback. In C. Bryan & K. Clegg (Eds.), Innivative assessment in Higher Education (pp. 81-91). London: Routledge. ✓ ✓ Carless, D. (2006). Differing perceptions in the feedback process. Studies in Higher Education, 31(2), 219-233. doi: 10.1080/03075070600572132 ✓ Carless, D., Salter, D., Yang, M., & Lam, J. (2011). Developing sustainable feedback practices. Studies in Higher Education, 36(4), 395-407. doi: 10.1080/03075071003642449 ✓ ✓ Cartney, P. (2010). Exploring the use of peer assessment as a vehicle for closing the gap between feedback given and feedback used. Assessment & Evaluation in Higher Education, 35(5), 551-564. doi: 10.1080/02602931003632381

✓

Case, S. (2007). Reconfiguring and realigning the assessment feedback processes for an undergraduate criminology degree. Assessment & Evaluation in Higher Education, 32(3), 285-299. doi: 10.1080/02602930600896548

✓

Covic, T., & Jones, M. K. (2008). Is the essay resubmission option a formative or a summative assessment and does it matter as long as the grades improve? Assessment & Evaluation in Higher Education, 33(1), 75-85. doi: 10.1080/02602930601122928

✓

Denton, P., Madden, J., Roberts, M., & Rowe, P. (2008). Students' response to traditional and computer-assisted formative feedback: A comparative case study. British Journal of Educational Technology, 39(3), 486-500. doi: 10.1111/j.1467-8535.2007.00745.x

✓

Duncan, N. (2007). 'Feed-forward': improving students' use of tutors' comments. Assessment & Evaluation in Higher Education, 32(3), 271-283. doi: 10.1080/02602930600896498

✓

33

Sources

Feed

back

pr

actic

e

Exam

ples

of

actu

al

feed

back

re

spon

ses

Stud

ent

enga

gem

ent

with

fe

edba

ck

Dia

logi

c ap

proa

ches

to

feed

back

Usi

ng

tech

nolo

gy

for p

eer a

nd

mar

ker

feed

back

Dea

ling

with

la

rge

clas

ses

Feed

forw

ard

Gibbs, G., & Simpson, C. (2004). Conditions Under Which Assessment Supports Students Learning. Learning and Teaching in Higher Education, 1(1), 3-31. ✓ Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112. doi: 10.3102/003465430298487 ✓ Hendry, G. D., Bromberger, N., & Armstrong, S. (2011). Constructive guidance and feedback for learning: The usefulness of exemplars, marking sheets and different types of feedback in a first year Law subject. Assessment & Evaluation in Higher Education, 36(1), 1-11. doi: 10.1080/02602930903128904

✓ ✓

Hepplestone, S., Holden, G., Irwin, B., Parkin, H. J., & Thorpe, L. (2011). Using technology to encourage student engagement with feedback: A literature review. Research in Learning Technology, 19(2), 117-127. doi: 10.1080/21567069.2011.586677

✓

Higgins, R., Hartley, P., & Skelton, A. (2001). Getting the message across: The problem of communicating assessment feedback. Teaching in Higher Education, 6(2), 269-274. doi: 10.1080/13562510120045230

✓ ✓

Higgins, R., Hartley, P., & Skelton, A. (2002). The Conscientious Consumer : Reconsidering the role of assessment feedback in student learning. Studies in Higher Education, 27(1), 53-64. doi: 10.1080/03075070120099368

✓

Hounsell, D., McCune, V., Hounsell, J., & Litjens, J. (2008). The quality of guidance and feedback to students. Higher Education Research & Development, 27(1), 55-67. doi: 10.1080/07294360701658765

✓

Lunsford, R. F. (1997). When less is more: Principles for responding in the disciplines. In P. Elbow & M. D. Sorcinelli (Eds.), Writing to learn: Strategies for assigning and responding to writing across the disciplines (pp. 91-104). San Francisco: Jossey-Bass.

✓

Lunt, T., & Curran, J. (2009). ‘Are you listening please?’ The advantages of electronic audio feedback compared to written feedback. Assessment & Evaluation in Higher Education, 35(7), 759-769. doi: 10.1080/02602930902977772

✓

Merry, S., & Orsmond, P. (2008). Students' attitudes to and usage of academic feedback provided via audio files. Bioscience Education, 11. Retrieved from http://www.bioscience.heacademy.ac.uk/journal/vol11/beej-11-3.pdf

✓

34

Sources

Feed

back

pr

actic

e

Exam

ples

of

actu

al

feed

back

re

spon

ses

Stud

ent

enga

gem

ent

with

fe

edba

ck

Dia

logi

c ap

proa

ches

to

feed

back

Usi

ng

tech

nolo

gy

for p

eer a

nd

mar

ker

feed

back

Dea

ling

with

la

rge

clas

ses

Feed

forw

ard

Nicol, D. J. (2009). Assessment for learner self-regulation: Enhancing achievement in the first year using learning technologies. Assessment & Evaluation in Higher Education, 34(3), 335-352. doi: 10.1080/02602930802255139 Nicol, D. J. (2010). From monologue to dialogue: improving written feedback processes in mass higher education. Assessment & Evaluation in Higher Education, 35(5), 501-517. doi: 10.1080/02602931003786559

✓

Nicol, D. J. (2010). From monologue to dialogue: improving written feedback processes in mass higher education. Assessment & Evaluation in Higher Education, 35(5), 501-517. doi: 10.1080/02602931003786559

✓

Nicol, D. J. (2011). Developing students' ability to construct feedback. Paper presented at the QAA Enhancement Themes Conference, Heriot-Watt University. ✓ Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self‐regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218. doi: 10.1080/03075070600572090

✓ ✓ ✓


✓

Pearce, J., Mulder, R., & Baik, C. (2009). Involving students in peer review: Case studies and practical strategies for university teaching. Retrieved from http://www.cshe.unimelb.edu.au/

✓ ✓

Price, M., Handley, K., Millar, J., & O'Donovan, B. (2010). Feedback : all that effort, but what is the effect? Assessment & Evaluation in Higher Education, 35(3), 277-289. doi: 10.1080/02602930903541007

✓

Price, M., O'Donovan, B., & Rust, C. (2007). Putting a social-constructivist assessment process model into practice: Building the feedback loop into the assessment process through peer review. Innovations in Education and Teaching International, 44(2), 143-152.

✓

Providing Helpful Feedback, 2012, e-learning, University of Sydney. ✓ Rotheram, B. (2007). Using an MP3 recorder to give feedback on student assignments. Educational Developments, 8(2), 7-10 ✓ Rotheram, B. (2009). Practice tips on using digital audio for assessment feedback Retrieved 31 January, 2012, from http://www.engsc.ac.uk/events/using-technology-to-enhance-learning-teaching-assessment-and-feedback

✓

Rust, C. (2001). A briefing on assessment of large groups. Generic Centre Assessment Series No. 12, from http://www.bioscience.heacademy.ac.uk/resources/themes/assessment.aspx

✓

35

Sources

Feed

back

pra

ctic

e

Exam

ples

of a

ctua

l fe

edba

ck re

spon

ses

Stud

ent e

ngag

emen

t with

fe

edba

ck

Dia

logi

c ap

proa

ches

to

feed

back

Usi

ng te

chno

logy

for p

eer

and

mar

ker f

eedb

ack

Dea

ling

with

larg

e cl

asse

s

Feed

forw

ard

Rust, C., Price, M., & O'Donovan, B. (2003). Improving students' learning by developing their understanding of assessment criteria and processes. Assessment & Evaluation in Higher Education, 28(2), 147-164. doi: 10.1080/02602930301671

✓

Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119-144. ✓ Sadler, D. R. (2008). Transforming holistic assessment and grading into a vehicle for complex learning. In G. Joughin (Ed.), Assessment, learning and judgement in higher education (pp. 49-64). Dordrecht: Springer.

✓

Sadler, D. R. (2010). Beyond feedback: Developing student capability in complex appraisal. Assessment & Evaluation in Higher Education, 35(5), 535-550. doi: 10.1080/02602930903541015

✓

Sandhu, S. (2004). The use of statement banks as a means of assessing large groups in Accounting. Investigations in University Teaching and Learning, 2(1), 49-55. ✓ University websites: Sydney University: ITL Assessment Resources: Feedback Leeds Metropolitan University: ‘Sounds Good’ project Nottingham University: ‘Enhancing student feedback through the use of technology’ project Sheffield Hallam University: ‘Technology, Feedback, Action!’ project

✓

Vardi, I. (2012). Effective feedback for student learning in higher education: HERDSA. ✓ ✓ Yorke, M. (2003). Formative assessment in higher education: Moves towards theory and the enhancement of pedagogic practice. Higher Education, 45, 477-501. ✓

36

Table 3: Suggested Solutions – Feedback practice

Sources

Tuto

rs’ a

ctiv

e en

gage

men

t with

as

sess

men

t crit

eria

Goo

d qu

ality

fe

edba

ck fr

om

mar

kers

Impr

oved

crit

eria

sh

eets

Lim

iting

the

num

ber

of fe

edba

ck

com

men

ts

Giv

ing

gene

ric

feed

back

on

the

wor

k of

the

clas

s as

a

who

le

Mul

ti-st

age

assi

gnm

ents

: re

draf

ting

afte

r tu

tor/p

eer f

eedb

ack

Res

ubm

issi

on o

f w

ork

Brown, E., & Glover, C. (2006). Evaluating written feedback. In C. Bryan & K. Clegg (Eds.), Innivative assessment in Higher Education (pp. 81-91). London: Routledge. ✓ Carless, D., Salter, D., Yang, M., & Lam, J. (2011). Developing sustainable feedback practices. Studies in Higher Education, 36(4), 395-407. doi: 10.1080/03075071003642449 ✓ Covic, T., & Jones, M. K. (2008). Is the essay resubmission option a formative or a summative assessment and does it matter as long as the grades improve? Assessment & Evaluation in Higher Education, 33(1), 75-85. doi: 10.1080/02602930601122928

✓

Hendry, G. D., Bromberger, N., & Armstrong, S. (2011). Constructive guidance and feedback for learning: The usefulness of exemplars, marking sheets and different types of feedback in a first year Law subject. Assessment & Evaluation in Higher Education, 36(1), 1-11. doi: 10.1080/02602930903128904

✓

Higgins, R., Hartley, P., & Skelton, A. (2001). Getting the message across: The problem of communicating assessment feedback. Teaching in Higher Education, 6(2), 269-274. doi: 10.1080/13562510120045230 ✓ Lunsford, R. F. (1997). When less is more: Principles for responding in the disciplines. In P. Elbow & M. D. Sorcinelli (Eds.), Writing to learn: Strategies for assigning and responding to writing across the disciplines (pp. 91-104). San Francisco: Jossey-Bass.

✓

Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self‐regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218. doi: 10.1080/03075070600572090

✓ ✓ ✓ ✓


✓

Rust, C., O'Donovan, B., & Price, M. (2005). A social constructivist assessment process model: how the research literature shows us this could be best practice. Assessment & Evaluation in Higher Education, 30(3), 231-240. doi: 10.1080/02602930500063819

✓ ✓ ✓

37

Suggested Solutions – Examples of feedback responses markers can use Providing Helpful Feedback, 2012, e-learning, University of Sydney (Dr Meloni Muir) Examples of helpful feedback comments on Psychology essays (Dr Fiona Hibberd) Vardi, I. (2012). Effective feedback for student learning in higher education: HERDSA. Table 4: Suggested Solutions – Student engagement with feedback

Sources

Stud

ent r

eque

sts

for k

inds

of

feed

back

Stud

ent i

dent

ifica

tion

of a

reas

of

diffi

culty

Stud

ent s

elf-a

sses

smen

t bef

ore

wor

k su

bmitt

ed

Giv

ing

feed

back

but

no

grad

e w

hen

wor

k in

itial

ly re

turn

ed

Stud

ents

as

mar

kers

: exe

mpl

ars

and/

or p

eer a

sses

smen

t/fee

dbac

k

Dev

elop

ing

stud

ents

’ abi

lity

to s

elf-

asse

ss

(Impr

ovem

ent i

n pe

rfor

man

ce a

fter

peer

mar

king

exe

rcis

e)

Hel

ping

cha

nges

in fe

edba

ck

prac

tice

to w

ork

Basnet, B., Brodie, L., & Worden, J. (2010). Enhanced Feedback - Does Peer Assessment achieve this goal? Paper presented at the AaeE 2010: Past, Present, Future - the 'Keys' to Engineering Education Research and Practice.

✓

Cartney, P. (2010). Exploring the use of peer assessment as a vehicle for closing the gap between feedback given and feedback used. Assessment & Evaluation in Higher Education, 35(5), 551-564. doi: 10.1080/02602931003632381

✓

Gibbs, G., & Simpson, C. (2004). Conditions Under Which Assessment Supports Students ’ Learning. Learning and Teaching in Higher Education, 1(1), 3-31.

✓

Hendry, G. D., Bromberger, N., & Armstrong, S. (2011). Constructive guidance and feedback for learning: The usefulness of exemplars, marking sheets and different types of feedback in a first year Law subject. Assessment & Evaluation in Higher Education, 36(1), 1-11. doi: 10.1080/02602930903128904

✓

Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self‐regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218. doi: 10.1080/03075070600572090

✓ ✓ ✓ ✓

Nicol, D. J. (2011). Developing students' ability to construct feedback. Paper presented at the QAA Enhancement Themes Conference, Heriot-Watt University.

✓

38


✓

Price, M., O'Donovan, B., & Rust, C. (2007). Putting a social-constructivist assessment process model into practice: Building the feedback loop into the assessment process through peer review. Innovations in Education and Teaching International, 44(2), 143-152.

✓

Rust, C., O'Donovan, B., & Price, M. (2005). A social constructivist assessment process model: how the research literature shows us this could be best practice. Assessment & Evaluation in Higher Education, 30(3), 231-240. doi: 10.1080/02602930500063819

✓ ✓ ✓

Rust, C., Price, M., & O'Donovan, B. (2003). Improving students' learning by developing their understanding of assessment criteria and processes. Assessment & Evaluation in Higher Education, 28(2), 147-164. doi: 10.1080/02602930301671

✓

Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119-144. ✓ Sadler, D. R. (2008). Transforming holistic assessment and grading into a vehicle for complex learning. In G. Joughin (Ed.), Assessment, learning and judgement in higher education (pp. 49-64). Dordrecht: Springer. ✓ Sadler, D. R. (2010). Beyond feedback: Developing student capability in complex appraisal. Assessment & Evaluation in Higher Education, 35(5), 535-550. doi: 10.1080/02602930903541015 ✓ Vardi, I. (2012). Effective feedback for student learning in higher education: HERDSA. ✓ ✓

Yorke, M. (2003). Formative assessment in higher education: Moves towards theory and the enhancement of pedagogic practice. Higher Education, 45, 477-501.

✓

Suggested Solutions – Dialogic approaches to feedback Carless, D. (2006). Differing perceptions in the feedback process. Studies in Higher Education, 31(2), 219-233. doi: 10.1080/03075070600572132 Carless, D., Salter, D., Yang, M., & Lam, J. (2011). Developing sustainable feedback practices. Studies in Higher Education, 36(4), 395-407. doi:

10.1080/03075071003642449 Nicol, D. J. (2010). From monologue to dialogue: improving written feedback processes in mass higher education. Assessment & Evaluation in Higher Education,

35(5), 501-517. doi: 10.1080/02602931003786559 Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self‐regulated learning: A model and seven principles of good feedback practice. Studies in

Higher Education, 31(2), 199-218. doi: 10.1080/03075070600572090

39

Table 5: Suggested Solutions – Using technology to provide peer and marker feedback Sources

Onl

ine

lear

ning

task

s

Use

of a

udio

file

s

Peer

revi

ew u

sing

onl

ine

tool

s

Com

pute

r ass

iste

d fe

edba

ck,

incl

udin

g st

atem

ent b

anks

Rel

ated

Uni

vers

ity w

ebsi

tes

Bridgeman, A., & Rutledge, P. (2010). Getting personal: Feedback for the masses. Synergy (Institute for Teaching and Learning, University of Sydney)(30), 61-68. ✓ Case, S. (2007). Reconfiguring and realigning the assessment feedback processes for an undergraduate criminology degree. Assessment & Evaluation in Higher Education, 32(3), 285-299. doi: 10.1080/02602930600896548 ✓ Denton, P., Madden, J., Roberts, M., & Rowe, P. (2008). Students' response to traditional and computer-assisted formative feedback: A comparative case study. British Journal of Educational Technology, 39(3), 486-500. doi: 10.1111/j.1467-8535.2007.00745.x

✓

Hepplestone, S., Holden, G., Irwin, B., Parkin, H. J., & Thorpe, L. (2011). Using technology to encourage student engagement with feedback: A literature review. Research in Learning Technology, 19(2), 117-127. doi: 10.1080/21567069.2011.586677 ✓ Lunt, T., & Curren, J. C. (2009). ‘Are you listening please?’ The advantages of electronic audio feedback compared to written feedback. Assessment & Evaluation in Higher Education, 35(7), 759-769. doi: 10.1080/02602930902977772 ✓ Merry, S., & Orsmond, P. (2008). Students' attitudes to and usage of academic feedback provided via audio files. Bioscience Education, 11. Retrieved from http://www.bioscience.heacademy.ac.uk/journal/vol11/beej-11-3.pdf ✓ Nicol, D. J. (2009). Assessment for learner self-regulation: Enhancing achievement in the first year using learning technologies. Assessment & Evaluation in Higher Education, 34(3), 335-352. doi: 10.1080/02602930802255139 ✓ Pearce, J., Mulder, R., & Baik, C. (2009). Involving students in peer review: Case studies and practical strategies for university teaching. Retrieved from http://www.cshe.unimelb.edu.au/ ✓ Rotheram, B. (2007). Using an MP3 recorder to give feedback on student assignments. Educational Developments, 8(2), 7-10. ✓ Rotheram, B. (2009). Practice tips on using digital audio for assessment feedback Retrieved 31 January, 2012, from http://www.engsc.ac.uk/events/using-technology-to-enhance-learning-teaching-assessment-and-feedback ✓ University websites: Leeds Metropolitan University: ‘Sounds Good’ project Nottingham University: ‘Enhancing student feedback through the use of technology’ project Sheffield Hallam University: ‘Technology, Feedback, Action!’ project

✓

40

Table 6: Suggested Solutions – Dealing with large classes Sources Feedback with

large classes

Peer assessment in large classes

Use of statement banks with large classes

Ballantyne, R., Hugher, K., & Mylonas, A. (2002). Developing procedures for implementing peer assessment in large classes using an action research process. Assessment & Evaluation in Higher Education, 27(5), 427-441. doi: 10.1080/0260293022000009302

✓

Bridgeman, A., & Rutledge, P. (2010). Getting personal: Feedback for the masses. Synergy (Institute for Teaching and Learning, University of Sydney)(30), 61-68. ✓ Rust, C. (2001). A briefing on assessment of large groups. Generic Centre Assessment Series No. 12, from http://www.bioscience.heacademy.ac.uk/resources/themes/assessment.aspx

✓

Sandhu, S. (2004). The use of statement banks as a means of assessing large groups in Accounting. Investigations in University Teaching and Learning, 2(1), 49-55.

✓

Suggested Solutions – Feedforward Brown, E. (n.d.). Effective feedback. Formative Assessment in Science Teaching (FAST) Retrieved 22.03.12, 2012 Brown, E., & Glover, C. (2006). Evaluating written feedback. In C. Bryan & K. Clegg (Eds.), Innivative assessment in Higher Education (pp. 81-91). London: Routledge Duncan, N. (2007). 'Feed-forward': improving students' use of tutors' comments. Assessment & Evaluation in Higher Education, 32(3), 271-283. doi:

10.1080/02602930600896498 Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112. doi: 10.3102/003465430298487 Higgins, R., Hartley, P., & Skelton, A. (2001). Getting the message across: The problem of communicating assessment feedback. Teaching in Higher Education, 6(2),

269-274. doi: 10.1080/13562510120045230 Hounsell, D., McCune, V., Hounsell, J., & Litjens, J. (2008). The quality of guidance and feedback to students. Higher Education Research & Development, 27(1), 55-

67. doi: 10.1080/07294360701658765 Price, M., Handley, K., Millar, J., & O'Donovan, B. (2010). Feedback : all that effort, but what is the effect? Assessment & Evaluation in Higher Education, 35(3), 277-

289. doi: 10.1080/02602930903541007

41

4.2.1 Feedback practice Key Points

• Tutors’ active engagement with assessment criteria • Good quality feedback from markers • Improved criteria sheets • Limiting the number of feedback comments • Giving generic feedback on the work of the class as a whole • Two-‐stage (or more) assignments involving redrafting after tutor or peer

feedback • Resubmission of work

Tutors’ active engagement with assessment criteria For Rust, O’Donovan and Price (2005, p. 233), active engagement with assessment criteria by both markers and students is the key to ‘achieving meaningful understanding of assessment’. For tutors, the process takes the form of discussion of the criteria, followed by some marking using the criteria, then further discussion, and then discussion in moderation sessions after the completion of marking. (See the two (staff and student) parallel, interrelated and dynamic cycles of desirable practice.) Good quality feedback from markers According to Nicol and Macfarlane-‐Dick (2006), there is little consensus in the research literature about what good quality (external, as opposed to students’ own internal) feedback is. They define it as ‘information that helps students troubleshoot their own performance and self-‐correct: that is, it helps students take action to reduce the discrepancy between their intentions and the resulting effects’ (p. 208). Higgins, Hartley and Skelton (2002, p. 62) offer the following advice to tutors/ coordinators: ‘Firstly, … timely feedback is vital; … interim feedback on a first draft or an essay plan might also be productive’. Secondly, … misconceptions need to be explained and improvements for future work suggested.’ The authors explain that students ‘may not view comments on “surface” aspects of their work as particularly relevant or useful’, and that ‘fostering “higher order” critical skills may have more long-‐term educational value’. See also Brown and Glover (2006) below. Improved criteria sheets One of the strategies suggested by Nicol and Macfarlane-‐Dick (2006) which have ‘proved effective in clarifying criteria, standards and goals’ for students is the provision of ‘better definitions of requirements using carefully constructed criteria sheets and performance-‐level definitions’ (p. 207). Limiting the number of feedback comments Lunsford (1997) examines the written feedback comments on student writing of 12 experienced composition theory markers, and presents guidelines for responding to this writing. He refers to his previously published ‘seven principles for response’, which include the writing of ‘well-‐developed and text-‐specific comments’, a focus on ‘global, not local, concerns’, and the framing of comments in ‘nonauthoritative modes’, and categorises these principles in terms of ‘Development and Specificity’, ‘Purposeful Commenting’, ‘Correctness’, and ‘Extra-‐textual Response’ (pp. 91-‐92).

42

Under ‘Purposeful Commenting’, Lunsford explains that ‘less is more’ if students are going to improve their writing. The markers studied averaged ‘just over three issues per paper marked’, i.e. they ‘arrive[d] at a set of themes for their responses’. They marked two or three examples of the focus problem, and provided ‘one reasonably fully developed comment [which] might be two or three sentences long, allowing the teacher to give the student a mini-‐lesson on the issue’ (1997, p. 93). He also develops the idea of the making of nonauthoritative comments and of comments which respond to what the student writers have to say, rather than to errors or form more generally. The results of the small-‐scale study by Hendry, Bromberger and Armstrong (2011) (see below) show that students perceive feedback in the form of both ‘a limited number of personalised, concrete comments’ focussing on performance improvement and a ‘standards-‐based, “checklist-‐like”’ type, as helping them ‘target their learning efforts effectively’ (p. 9). Orsmond, Merry and Reiling (2005, p. 370) report, from a study by Brockbank and McGill (1998), that ‘the quantity of both positive and negative feedback may overwhelm the student so that they are unable to take it in overall’. Giving generic feedback on the work of the class as a whole Rust, O’Donovan and Price (2005, p. 235) suggest that giving only generic feedback which ‘highlight[s] common mistakes, things that were generally done well, etc.’ means a short time lag between submission and feedback. Students can then be encouraged to ‘try and identify which of the generic feedback applies to them, and from that identify what they need to [do] differently next time’. They add that this approach could be linked to initial withholding of the grade. Two-‐stage (or more) assignments involving redrafting after tutor or peer feedback Rust, O’Donovan and Price (2005) presents the authors’ ‘social, constructivist approach’ to assessment, in which active engagement with assessment criteria by both markers and students is the key to ‘achieving meaningful understanding of assessment’ (p. 233). The authors argue that one way of achieving active engagement with feedback on the part of students is two-‐stage (or more) assignments, i.e. those with a first draft and then a redraft after tutor or peer feedback. The example they mention involves: peer discussion of feedback received using a checklist; then writing about their feedback using a pro-‐forma, this writing including ‘deriv[ing] conclusions about what to do differently in future’; and benchmarking their work against an ‘A’ grade assignment (p. 235). The assessment design strategy which Carless et al. (2011, p. 398) see as encouraging dialogic feedback involves two-‐stage or multi-‐stage assignments: ‘Such assignments facilitate sustainable feedback when required standards are becoming increasingly transparent and dialogic feedback processes support students in self-‐monitoring their work while it is being developed’. Nicol and Macfarlane-‐Dick (2006, p. 213) view closing the ‘gap between current performance and the performance expected by the teacher’ as being, in part, about supporting the act of production of a piece of work: this requires ‘the generation of concurrent or intrinsic feedback that students can interact with while engaged in an

43

assessment task’. This feedback could come from peers in a group task, or the task might have a series of components which each led to feedback, or the sub-‐task feedback might be ‘automatically generated’ (i.e. electronically) to support student engagement. Resubmission of work Nicol and Macfarlane-‐Dick (2006, p. 213) argue that closing the learning gap (Sadler, 1989) is, in part, about ‘providing opportunities to repeat the same “task-‐performance-‐external feedback cycle” by, for example, allowing resubmission … (Boud, 2000)’. Covic and Jones’ (2008) study provided a group of 54 students with the opportunity to submit an essay and receive ‘detailed written feedback’ as well as a grade, and to have the option of modifying the essay according to this feedback and to resubmit it at the end of two weeks. 26 students resubmitted their essays, and 25 of these essays received higher marks, one receiving the same mark. 12 of those who did not resubmit cited ‘workload and time constraints’ as the reason, while 9 were ‘happy with their mark’ on the first submission. Nine students provided comments on the feedback received, and all comments were positive. However, the findings suggest that ‘for a number of students, the resubmission option was viewed as a ‘safety net’ rather than an opportunity to learn and apply that knowledge to improving the essay through resubmission’ (p. 82), while the assessment strategy was a time-‐consuming exercise for the marker. The authors suggest that

meaningful formative feedback can be integrated into assessment without too much additional time investment, for example, breaking up the essay into separately marked components, such as … an introduction section that with feedback will inform, if utilized, the rest of the essay. (p. 84)

4.2.2 Examples of feedback responses markers can use In Physiology, Dr Meloni Muir was the Project Leader in the production of the Providing Helpful Feedback website (available via Blackboard). The project involved collaboration between the University of Sydney’s discipline of Physiology and e-‐learning. The site contains advice, examples and exercises for markers on giving feedback on authentic written reports. Currently access is limited to markers in Physiology; however, it is possible that this resource could be made available to markers who need to give feedback on reports in the Faculty of Science. An example of the way the site works follows. In the ‘Introduction’ section (other sections include ‘Method’, ‘Results’, and ‘Discussion’), Sample 1 features a student text with a button stating ‘unhelpful feedback’, and an instruction ‘To reveal feedback comments click on each arrow in turn’. The unhelpful comments revealed, directly related to highlighted sections of the Introduction text are ✓ Watch your spacing Incorrect grammar This section well done but justify predictions! By clicking on a ‘show info’ button, the marker receives an explanation as to why the feedback is unhelpful, and also examples of helpful feedback. For example, the unhelpfulness of ✓ is explained thus: Tick marks are ambiguous; need to specify what the tick means/refers to. and the helpful feedback comment is

44

Overall Intro is well written & paragraphs are well organised. The unhelpfulness of Incorrect grammar is explained as Punctuation errors need rule explained and correction suggested. and the helpful feedback comment is A semi-‐colon is required before “however” in the middle of a sentence. The unhelpfulness of This section well done but justify predictions! is explained as Need to explain/identify what has been done well and to spell out what is missing and the helpful feedback comment is Good: aim has been clearly distinguished from hypotheses. However insufficient background in the Intro to support predictions made in hypotheses, e.g. relationship between PCO2 and drive to breathe. In Psychology, Dr Fiona Hibberd (interviewed for this project), as Coordinator, provides her markers with a list of sample comments to help them give their students constructive and tactful/sensitive feedback. She specifies that the aim ‘is to provide appropriate, legible and careful brief comments that make clear the strengths and weaknesses of each piece of work in sufficient detail to allow the student to understand the grounds for the mark they received and improve their performance on subsequent assignments’ (from her document ‘Providing feedback comments on written work’). Examples of these comments are: The Introduction to this essay is excellent (or could be improved). It provides (fails to adequately provide) some background to the topic, and explains to the reader the scope of the topics to be covered in the essay. This is an excellent point in that … The point you make is flawed in that … The essay is well structured and flows logically and smoothly. The wording of this paragraph could be improved by … This section is vague and general – aim to be precise and to be more specific. Vardi’s (2012) HERDSA Guide (Effective feedback for student learning in Higher Education) provides helpful examples of immediate and delayed (written) feedback. An example of immediate – evaluative, instructive, and encouraging, affirming and motivating – feedback responses (in lecture, tutorial, laboratory or fieldwork situations) is: That summarises the facts of the case well. Now, how could you analyse that? (p. 26). The delayed feedback example (written) responses are divided by Vardi into ‘Direct, prescriptive and specific’ and ‘Direct, explanatory and specific’. An example of the first is: This called for the use of the X method of analysis and of the second: That presupposes … However, if one takes into account … (p. 28). Vardi also provides examples of ‘global’ and ‘local’ feedback, describing the first as ‘a reflection on the assignment as a whole’, as including the holistic mark/grade, and as justifying the mark/grade given ‘by highlighting the performance levels achieved’ (p. 29). Vardi comments that ‘Simply augmenting your global evaluation with the types of global feedback listed has the potential to significantly impact on future student learning and achievement in subsequent related assessment tasks.’ An example is: ‘2. Explain what they specifically needed to have done in order to achieve the next higher level of performance.

45

In order to achieve a Distinction, you needed to address and research … in the question. This will help you deepen your analysis by … and interpret this in relation to … You also need to address …’ Local feedback is that provided ‘at specific points in the assignment’. Vardi recommends that this be of a ‘limited amount’. (p. 31). She comments that ‘students like to have some indication that you have read their work, and would like to know where they were on track and where they faltered’. She then provides a list of ‘powerful local feedback’. An example is: ‘2. Focus on the conclusions drawn in each section. This can significantly impact on the level of critical thought applied. You would benefit from a conclusion to this section that compares … This conclusion neatly draws together … However, it omits …’

4.2.3 Student engagement with feedback Key points

• Student requests for the kinds of feedback they would like when they hand in work

• Student identification of where they are having difficulties when they hand in assessed work

• Student self-‐assessment before work submitted • Developing students’ ability to self-‐assess • Giving students feedback but no grade when the work is initially returned • Students as markers, using exemplars and/or peer assessment/feedback • (Demonstrating improvement in performance as a result of a peer marking

exercise) • Helping changes in feedback practice (to a more student-‐centred model) to

work Student requests for the kinds of feedback they would like when they hand in work This is one of the strategies to improve student self-‐regulation of learning suggested by Nicol and Macfarlane-‐Dick (2006). Student identification of where they are having difficulties when they hand in assessed work This is one of the strategies to improve student self-‐regulation of learning suggested by Nicol and Macfarlane-‐Dick (2006). Student self-‐assessment before work submitted Nicol and Macfarlane-‐Dick (2006, p. 208) suggest student identification of the strengths and weaknesses in their own work against criteria or standards before submission as an example of ‘structured reflection and self-‐assessment’. They also refer to research showing that ‘self-‐assessment can lead to significant enhancements in learning and achievement’ (p. 207). One study by Taras (2001, 2002, 2003), e.g., revealed that ‘self-‐assessment with integrated tutor feedback helped students identify and correct more errors … than self-‐assessment prior to peer or tutor feedback’. Developing students’ ability to self-‐assess

46

Vardi’s (2012) (HERDSA) Guide (Effective feedback for student learning in Higher Education) builds on the peer feedback section to focus on ‘Developing students’ ability to self-‐assess’. She argues for the incorporation of self-‐assessment into learning programs by asking students to, e.g. ‘1. Identify what they have learnt or understood. What did you learn from …? 2. Identify the gaps in learning. What are you unsure about? What haven’t you been able to do? … 4. Compare what they have done to what experts do or successful models. Compare your work with the exemplar. 5. Set goal or plan for how they will improve. Where do you need to improve …? What are you going to do about this? Giving students feedback but no grade when the work is initially returned For Rust, O’Donovan and Price (2005), active engagement with assessment criteria by both markers and students is the key to ‘achieving meaningful understanding of assessment’ (p. 233). The authors argue that one way of achieving active engagement with feedback on the part of students is that they be given comments only on their returned work, and be asked to ‘identify from reading the comments what grade they think … has been awarded’, the actual grade being communicated only after they do this (p. 235). Students as markers, using exemplars and/or peer assessment/feedback Rust, Price and O’Donovan (2003) report on a structured intervention with a group of 300+ first year undergraduates in a Business module, ‘an intervention that placed increased emphasis on the tacit knowledge transfer processes of practice and imitation to achieve shared understanding’ (p. 152). The students, whose attendance at the workshop was voluntary, engaged in activities involving ‘discussion of exemplars and marking criteria, marking exercises and self-‐assessment’. The study found that workshop participants achieved ‘significantly better results’ in their assessed coursework than their non-‐participant peers, and one year later, were still showing significantly better results (p. 156). Rust, O’Donovan and Price (2005) argue that active engagement with assessment criteria by both markers and students is the key to ‘achieving meaningful understanding of assessment’ (p. 233). For students, this takes the form of involvement in a marking exercise in which they use the criteria to mark sample texts and then discuss their evaluations with peers and tutors, and/or of peer marking of student work. See below for discussion of two-‐stage assignments. Sadler (1989) (see above) contends that criteria sheets do not necessarily deal adequately with the qualities of a piece of work because there are both manifest and latent criteria involved in evaluation, and that students need to develop their knowledge of both in order to be able to monitor their own performance, i.e. they need to ‘be given adequate evaluative experience themselves’ (p. 134) to develop ‘an appropriate body of tacit knowledge’ (see Glossary). This can be done by engaging them in ‘evaluative and corrective activity on other students’ work’ (p. 140). Sadler (2008) argues for a student induction process in the appraisal of ‘open-‐response’ assignments in order to develop students’ evaluative expertise and enable them to self-‐monitor the development of their own assignments. He specifies that they need to develop a concept of what constitutes quality, and that this can be done

47

by making evaluative activity ‘the primary pedagogical vehicle for teaching a considerable proportion of the substantive content of a course’ (p. 59). Sadler (2010) points to a gap between marker feedback and student ‘appreciation of its practical import’ (p. 541), and suggests that learners be provided with teacher-‐like assignment appraisal experience. He advocates the use of ‘purposeful peer assessment’ as part of the teaching design, and identifies ‘task compliance, quality and criteria’ as making up ‘the explicit agenda to be addressed’ (p. 542). Nicol and Macfarlane-‐Dick (2006), referring to Rust et al. (2003) and other studies, report on the difficulty of making assessment standards and criteria explicit to students using verbal descriptions: ‘Most criteria for academic tasks are complex, multidimensional (Sadler, 1989) and difficult to articulate: they are often “tacit” and unarticulated in the mind of the teacher’ (p. 206). Referring to Orsmond, Merry and Reiling (2002), they argue that providing students with exemplars of performance is an approach which ‘has proved particularly powerful in clarifying goals and standards’ (p. 206): ‘exemplars are effective because they make explicit what is required, and they define a valid standard against which students can compare their work’. The authors also suggest the involvement of students in peer-‐assessment exercises in relation to ‘defined criteria and standards’. Providing students with opportunities to evaluate and provide feedback on the work of peers ‘help[s] develop the skills needed to make objective judgements against standards, skills which are transferred when students turn to producing and regulating their own work (Boud et al,. 1999; Gibbs, 1999)’ (p. 208). Hendry, Bromberger, & Armstrong (2011) report on a small study exploring first year Law students’ ‘perceptions of the usefulness of exemplars and different types of feedback for guiding them in completing assessments’ (p. 1). Their results show ‘clearly’ the greater usefulness to students of the marking and discussion of exemplars in class compared to the usefulness of a marking sheet. For Hendry et al., exemplars provide ‘concrete illustrations of the style of language and structure expected and not expected in a good assignment’, while the marking sheet is a more ‘abstract representation of teacher expectations’ (p. 8). Their results also show that students perceive feedback in the form of both ‘a limited number of personalised, concrete comments’ focussing on performance improvement and a ‘standards-‐based, “checklist-‐like”’ type, as helping them ‘target their learning efforts effectively’ (p. 9). Pearce, Mulder and Baik (2009), in their study of peer review, address three questions: ‘What is student peer review? Why incorporate student peer review in subjects/courses? How can student peer review be implemented effectively in subjects/courses?’ (p. 2). They provide an overview of the literature, focussing on the benefits and potential impediments to implementation of peer review, and present four case studies of how peer review was implemented in teaching, including in Departments of Zoology and Information Systems. In each case, student perceptions of the peer review process are summarised, and there is a ‘Lessons learned’ concluding section. A ‘reflective discussion’ of ‘two academics’ experiences in facilitating and implementing peer review’ and an examination of students’ perceptions of the experience follow. The student evaluations suggest that ‘peer review ‘is generally perceived to be effective in improving learning’ (p. 17). The authors then provide some ‘key considerations in implementing student peer review’, such as ‘Dealing with variation in review quality’, ‘Disparities between reviewer feedback and assessment outcomes’, and ‘Plagiarism’.

48

Nicol (2011, p. 2) makes a ‘case’ for peer feedback, arguing that it ‘can add significantly to the amount and variety of feedback students receive, without a corresponding increase in teacher workload’, and can also ‘address the timeliness issue’. He explains that students ‘cannot avoid engagement [with feedback] if they are required to produce commentaries on the work of others’, that is, they have to ‘actively engage with assessment criteria and standards’. They also ‘learn about different approaches and tactics to the same assignment’, ‘realise that quality is not a fixed attribute but can be produced in different ways’, and are encouraged to ‘take more responsibility for learning, both their own and that of others.’ He then details ways of implementing peer feedback, including that ‘peer review should begin in the first year with some simple review tasks, with complexity and depth being enhanced in later years.’ He explains how to ‘address academic integrity and circumvent plagiarism, i.e. by ‘having students review assignments produced by peers and then comment on their own assignment, but without having the opportunity to rewrite.’ (p. 3). Vardi (2012, pp. 39-‐40) gives advice on ensuring that students are ‘informed providers of feedback’, i.e. ‘informed evaluators’, not just giving their opinion. She lists a number of strategies, examples of which follow: ‘1. Place peer feedback activities appropriately in the learning program. Ensure that students have already undertaken the necessary work (e.g. readings, activities) prior to an activity incorporating peer feedback. 2. Provide a clear focus for the feedback. Focus the students’ attention: evaluation of the use of an analysis technique, evaluation of a draft assignment. … 5. Provide the students with the necessary training. Before students provide feedback, ensure that they thoroughly understand a. the criteria, frameworks or measures they are to use b. how to give feedback. Provide students with practice in giving feedback using exemplars.’ See also: Cartney (2010). Demonstrating improvement in performance as a result of a peer marking exercise Yorke (2003) cites Gibbs’ (1999, p. 43ff.) example of Engineering students who, six times a year, were required to grade their peers’ work anonymously one hour after submission: ‘the improvement in students’ end-‐of-‐course outcomes was very marked, and attributable to aspects of good learning’. Among these aspects were the promptness of the feedback, and ‘the stimulus to self-‐regulation regarding the standard of submitted work’ (p. 479). Gibbs and Simpson (2004) summarise a (Forbes and Spence (1991)) study of assessment on an engineering course which demonstrates that periodic peer-‐assessment of problem sheets ‘as a course requirement but without the marks contributing’ was followed by a ‘dramatic[] increase’ in the students’ exam marks, i.e. ‘to a level well above that achieved previously when the lecturers did the marking’. They comment that ‘what achieved the learning was the quality of student engagement in learning tasks, not teachers doing lots of marking’ (p. 8). (Cf.:

49

Price, O’Donovan & Rust (2007): failure to demonstrate any tangible improvement as a result of the intervention, but ‘both students and assessors evaluate[d the latter] positively with both believing it to be beneficial’ (p. 150). Orsmond, Merry and Reiling (2005, p. 371) reporting on Orsmond et al. (2004): ‘actively engaging students in a formative assessment exercise centred on marking criteria’ resulted in their being able to ‘both construct and interpret detailed marking criteria and to mark assignments … accurately compared to tutors’. However, the students were unable to write ‘academically meaningful comments … addressing specific criteria’, i.e. ‘the formative feedback provided had failed to promote demonstrable learning’. Basnet, Brodie and Worden (2010, p. 205) reports on students’ evaluation of peer assessment of assignments, finding that while a narrow majority of respondents found the peer feedback to be useful, about 26% ‘considered peers’ feedback of little use’. A number commented that their peers ‘lacked sufficient knowledge to be able to provide useful feedback’, while other suggested ‘providing the choice of a set of specific comments’ so that peers would not write ‘“offending” comments’ as feedback.) Helping changes in feedback practice (to a more student-‐centred model) to work For Rust, O’Donovan and Price (2005), active engagement with assessment criteria by both markers and students is the key to ‘achieving meaningful understanding of assessment’ (p. 233). The authors make a number of suggestions to tutors to try and prevent a negative reaction from students who might think that what they are being asked to do is ‘the tutor’s job’. These suggestions include the following (p. 235):

• Always explain clearly why the approach is being implemented, i.e. what are ‘the pedagogic reasons behind the process’;

• Where this is possible, ‘share [the] research … evidence of its effectiveness’; and

• ‘Consider attaching assessment incentives to the activities, if appropriate’.

4.2.4 Dialogic approaches to feedback Nicol and Macfarlane-‐Dick (2006) argue for the conceptualisation of feedback as ‘dialogue rather than as information transmission’. Conceptualising feedback as ‘a transmission process involving “telling”’ disregards ‘the active role the student must play in constructing meaning from feedback messages, and of using this to regulate performance’ (p. 210). If teacher dialogue is difficult because of large class sizes, feedback can be increased by peer dialogue, e.g. by asking students in tutorials to discuss the feedback comments they have received and suggest strategies for improved performance in the next assignment, or asking students to discuss examples of feedback comments they have found useful and how they helped. Carless (2006, pp. 230-‐231) argues for ‘assessment dialogues’ between tutors and students, that is, ‘discussions related to the assessment process as a general concept, but not related to the specifics of subject matter or what students need to do for a particular assignment’. He identifies some of the areas in which assessment dialogues might be useful, including ‘unpacking assessment criteria or involving students in generating or applying criteria’. Carless, Salter, Yang and Lam (2011) argue that the feedback process needs to be reconceptualised, and that part of this reconceptualisation relates to enhancing the

50

student role in the feedback process. Thus the authors place the development of student self-‐regulation at the heart of feedback processes, arguing that feedback needs to be viewed ‘more as dialogue than information transmission’ (p. 396). Student involvement in assessment is related to the concept of ‘sustainable feedback’, which Carless et al. define as ‘dialogic processes and activities which can support and inform the student on the current task, whilst also developing the ability to self-‐regulate performance on future tasks’ (p. 397). This concept is at the opposite end of the feedback practices continuum from ‘conventional’ practices, which involve strategies such as written feedback on drafts or final versions of assignments. So student self-‐regulation is ‘a central aspect of sustainable feedback’ (p. 398). The assessment design strategy which Carless et al. see as encouraging dialogic feedback involves two-‐stage or multi-‐stage assignments (see below). Peer-‐review, blogs engaging students ‘in dialogues around learning’ as part of the class participation grade, and exploratory rather than directive feedback in PBL in class to promote self-‐directed learning, are all examples of involving students in activities promoting self-‐evaluation. Nicol (2010) ‘suggests ways in which the nature and quality of feedback dialogue can be enhanced when student numbers are large without necessarily increasing demands on academic staff’. It ‘concludes with a conceptual discussion of the merits of taking a dialogical approach when designing feedback’ (p. 501).

4.2.5 Using technology to provide peer and marker feedback Key points

• Online learning tasks • Use of audio files • Peer review using online tools • Computer assisted feedback, including the use of statement banks • Related University websites

Online learning tasks Nicol (2009) explores ‘how formative assessment and feedback might be used to enhance the first-‐year experience and enable students to develop the skills needed for self-‐regulated learning’ (p. 335). The paper also demonstrates ways in which technology can be used to provide feedback in large classes. The author understands formative assessment [see Glossary] as including the involvement of students in ‘evaluative judgements about their own work and the work of their peers’ (p. 338). In the first case study of a redesigned course, the online learning tasks involved collaborative writing, peer dialogue and feedback, the use of model answers, and general, usually motivational feedback from the course leader via the class discussion board. Nicol comments that the redesign did not increase staff workloads, and also that there was a significant increase in (self and peer) feedback opportunities. Use of audio files Merry and Orsmond (2008) report on a small-‐scale study of biology students’ and tutors’ attitudes to receiving and giving formative feedback on written work using audio files, and on the students’ use of this feedback. The student response was very positive, the audio feedback being judged as of good quality because ‘it was easier to understand, had more depth and was more personal’. A large majority of the

51

students listened to the audio feedback more than once, and used it ‘to annotate or alter their original work as they listened’. Tutors found that while ‘providing audio feedback did not save them time, … it might do so with more practice’, and that its main benefit was its ‘higher quality’. Merry and Orsmond compared the audio and written comments for assignments marked in both ways, and found that there was a ‘statistically significant … preponderance’ of written feedback for ‘identifying errors’ and one of audio feedback for ‘demonstrating best practice’. Guidelines are provided for tutors wishing to provide audio-‐file feedback. Lunt and Curran (2009) report on a small-‐scale study of the use of virtual learning environment (VLE)-‐ and email-‐delivered audio feedback from the perspectives of two markers and 26 students. Both tutors ‘found the software easy to download, understand and use’ (p. 761). The use of MP3 (as opposed to WAV) files solved the problem of emailing large files. The average time taken to record the audio feedback was five minutes for a 2000-‐word assignment, and was shown to be significantly quicker than for handwriting or typing the same comments. The students who responded to the survey after receiving their audio feedback were very positive about it, the feedback being judged as ‘more detailed than written comments’, and 65% agreeing or strongly agreeing that they ‘would like to receive [their] feedback using the audio method’ (pp. 763-‐764). Lunt and Curran provide a number of suggestions for tutors interested in using the audio method. Rotheram (2009) provides some ‘practice tips on using digital audio for assessment feedback’ under four headings: saving time, technical matters, administration, and feedback structure. These tips include the following: ‘after some practice, audio may allow you to give noticeably more feedback without spending much longer on the task’; ‘MP3 is a very useful audio format: compact and widely playable. … You can make MP3 recordings with a … handheld digital audio recorder [which can record directly to MP3]’; ‘Make sure key administrative and quality-‐assurance staff accept that you are giving audio rather than written feedback. Letting people listen to a few examples usually helps to convince them of the educational value’, and ‘Keep the files short – don’t “overdo it”’. Rotheram helpfully includes ‘an outline of [his] most recent practice when commenting on a typical essay or report via an MP3 recording’. The link to his complete project report can be accessed via the Sounds Good website (www.soundsgood.org.uk). See also Rotheram (2007) for some background on using MP3 recordings to give feedback on student assignments. Peer review using online tools Pearce, Mulder and Baik (2009), in part, present ‘an overview of online tools that are currently available to academics for managing student peer reviews’ (p. 2). They discuss ‘well-‐established and accessible platforms (for which good documentation exists), and … highlight their capabilities, strengths and weaknesses’ (p. 23). These software packages are: CPR, PRAZE (the University of Melbourne’s in-‐house program), SPA (Blackboard), SWoRD, and Turnitin, which are compared in terms, e.g., of ‘Primary use/niche’, ‘Main advantages’, ‘Main drawbacks’, ‘Features’ and ‘Availability/Cost’. Pearce et al. argue that ‘the best choice of software for a particular subject or assignment will depend on the level of individual customisation that is required …, and the administrator’s level of experience and/or willingness to experiment …’ (p. 24). Computer assisted feedback, including the use of statement banks

52

Denton, Madden, Roberts and Rowe (2008) report on a study which compared students’ responses to ‘traditional’ handwritten feedback on an extended lab report with their responses to computer-‐assisted feedback (an MS Office application, Electronic Feedback freeware). Tutors selected 11 statements from a bank of 64 ‘criterion-‐style’ comments and up to 14 comments which had been inputted, in addition to making their general comments, and then generated a report for each student which was emailed out automatically. The findings were that students generally showed a preference for the electronic feedback, which Denton et al. explain as being due to the ‘clear structure of the feedback and its integration with the marking scheme’, and the greater amount of feedback given as a result of the use of a statement bank (p. 495). The markers reported that they took 2-‐3 minutes’ less time to mark each script, but ‘this time has to be offset against the effort required to write the initial bank of feedback statements’ (p. 497). Case (2007) reports on the implementation of a reconfigured system of assessment feedback with undergraduate students, a system which he describes as addressing the crucial aspects of explicit engagement with assessment criteria and constructive feedback (p. 287). The reformulation process involved the devising of an electronic template feedback form which incorporated ‘the assessment criteria relevant to each student’s grade as a table’. Students were provided with this table, as well as ‘recommendations for how to use it’. Markers were provided with an electronically-‐stored pre-‐formulated statement bank, ‘chosen to save time and effort reiterating comments on scripts relating to common student weaknesses …, mistakes …, recommended improvements …, and strengths’ (p. 289), and they were also able to add their own comments. Students were offered a follow-‐up, face-‐to-‐face meeting with their marker. The reconfigured system resulted in ‘significant improvements in assessment grades’, and student feedback suggested that it promoted ‘a strategic approach to learning’. It also streamlined the feedback procedure for lecturers, resulting in ‘reduced marking time …, fewer student queries …, and an efficient student feedback system’ (p. 292). And it seems that students are ‘engaging positively’ with the feedback provided, and are ‘beginning to understand the reasons behind assessment decisions’ (p. 294). Hepplestone, Holden, Irwin, Parkin and Thorpe (2011) present a literature review on ‘Using technology to encourage student engagement with feedback’. In this review, the authors focus on research exploring ‘the publication of feedback and grades online, the automated adaptive release of grades (or how technology can support encouraging student engagement through disengaging the grade from feedback), and the use of technology to automatically generate feedback aligned to assessment criteria through pre-‐populated statement banks’ (p. 119). See also Bridgeman and Rutledge (2010) in 3.2.6 – Dealing with large classes – below. Related University websites Sheffield Hallam University: ‘Technology, Feedback, Action!’ project The project ‘Technology, Feedback, Action!: The impact of learning technology upon students’ engagement with their feedback’ was undertaken in 2008/2009. In the final short report on this project (2010), Hepplestone, Parkin, Irwin, Holden and Thorpe explain that it ‘evaluated how a range of technical interventions might encourage students to engage with their feedback and formulate actions to improve future learning’. These interventions included the online publication of feedback and

53

grades, disengaging the grade from the feedback, and linking the feedback given to assessment criteria (p. 4). The first two of these were found to ‘significantly enhance students' engagement with their feedback’. The third, however, ‘while effective in enabling students to identify strengths and weaknesses at a glance and helping to identify learning targets, was less effective in terms of enhancing engagement with feedback’. The materials also include the full final report and an ‘Annotated Bibliography’, where the literature is set out on a grid specifying which documents relate to eight topic areas. The latter include ‘Key source/research on feedback’, ‘Technology-‐enabled feedback’, ‘Linking feedback to learning outcomes/comment banks’, and ‘Feedback as dialogue’. They also include ‘A best practice guide for academic staff’, ‘A student guide to using feedback’, and ‘A ten minute guide for senior managers’. http://evidencenet.pbworks.com/w/page/19383525/Technology,%20Feedback,%20Action!%3A%20Impact%20of%20Learning%20Technology%20on%20Students%27%20Engagement%20with%20Feedback Leeds Metropolitan University: ‘Sounds Good’ project Final Report (Rotheram, 2009, http://sites.google.com/site/soundsgooduk/) This project (Jan 2008 – March 2009) explored the use of digital audio to give assessment feedback. Its main aim was ‘to test the hypothesis that using digital audio for feedback can benefit staff and students by saving assessors’ time (speaking the feedback rather than writing it), and providing richer feedback to students (speech is a richer medium than written text)’. Five key questions were explored, including ‘(Without reducing the amount of feedback) in what circumstances can using digital audio save assessors’ time?’ and ‘What do assessors think of digital audio as a medium for providing feedback to students?’ (p. 2). The answers revealed that the most favourable circumstances included the assessor being ‘comfortable with the technology’, and the availability of ‘a quick and easy method of delivering the audio file to the student’. They also revealed that most staff said they intended to continue using digital audio feedback, several commenting that ‘they were able to give more, and higher-‐quality, feedback using audio, which they felt was worthwhile’ (p. 2). Brown (n.d.) presents an overview of best practice in feedback, including the use of oral feedback, statement banks, computer-‐assisted assessment, electronic feedback, and self and peer assessment. (See also: Brown, Sally (2006) Powerpoint ‘Using Formative Assessment to promote student learning’ www.ldu.leeds.ac.uk/news/events/documents/BrownPowerPoint.pdf) Nottingham University: ‘Enhancing student feedback through the use of technology’ project This project (‘Enhancing student feedback through the use of technology: trialling audio and video e-‐feedback on students’ work in the School of English’), which began in September 2011 with a one-‐day conference at Nottingham, ‘aims to improve feedback to students and the student academic experience by trialling the use of audio and video feedback in the School [of English]’. It ‘will test out the effectiveness of these technologies with both undergraduate and postgraduate students and in the “live” and distance learning environments’. http://www.nottingham.ac.uk/english/teachinglearning/enhancing-‐feedback-‐report.aspx

54

4.2.6 Dealing with large classes Key points

• Peer assessment and feedback in large classes • Use of statement banks for feedback with large classes

Feedback with large classes Rust (2001) argues that, as class sizes go up, the amount and quality of feedback students receive is likely to become ‘a major casualty’ (p. 4). Strategies for addressing this problem include giving general rather than individual class feedback, peer assessment, using feedback sheets, and using statement banks. Bridgeman and Rutledge (2010) report on a their ‘open-‐source software’ ‘Fast & Personal e-‐Feedback system (FPF)’, a new approach which allows ‘the rapid, efficient delivery of personalised feedback to large student cohorts’ (p. 61), and which is now used in First and Second Year Chemistry units at the University of Sydney. The program ‘processes marks files to produce individual reports and emails for every student in a class, whatever the class size’ (p. 62). All email correspondence is personalised, and the messages are ‘short and highly pertinent’. The system has proved to be ‘enormously popular with students’ as the FPF software writes a report for each student on the regular quizzes they complete, and this report is attached to their email as a pdf file. The report provides a full solution or explanation for each question answered incorrectly, and full solutions to all questions are made available online. Where a series of incorrectly answered linked questions indicates a weakness in a certain area, the student’s personal feedback report lists resources that can be used to ‘remedy their shortcoming’ (p. 64), the feedback thus feeding forward to future assessments. Rapidity is another ‘key feature’ of the FPF, as computer marking enables emails to be sent out ‘almost immediately’ after the task has been completed, thus maintaining student interest. The system also provides feedback for teachers, and exam feedback for students. Peer assessment in large classes: Ballantyne, Hughes and Mylonas (2002) reports on the development of peer assessment procedures for use in large classes. The development of procedures ‘is discussed in relation to assessment tasks, assessment criteria, anonymity, procedural guidelines, distribution systems, marking procedures and tutor remarking’. The study suggest that ‘the specific difficulties associated with the use of peer assessment in large classes … are outweighed by the learning benefits for students’, and recommendations are made ‘for ways in which peer assessment might be successfully applied in large classes’ (p. 427). Use of statement banks with large classes: Sandhu (2004) presents a case study which evaluates the effectiveness of statement banks in the assessment of large groups of students in Accounting. The author found that tutors were able to give more formative feedback (the markers could also add in further comments), that, since the process of marking was ‘speed[ed] up’, the students received ‘more timely feedback’, and that the use of the statement banks ‘enhance[d] marker consistency’. Students were able to ‘recognise their strengths and weaknesses and thus understand how their final grade was established’ (p. 51). Sandhu makes a number of recommendations concerning the feedback form used, e.g. it had to ‘show both positive and negative comments [and ] in detail where the student may have mis-‐understood the question being asked’, and to ‘be flexible and

55

adaptable if it was to be developed over time’, and includes a copy of the feedback form used in this study.

4.2.7 Feedforward (See Glossary) Higgins, Hartley and Skelton (2001, p. 274), in an article dealing with feedback as a communication process, suggest in their last lines that the emphasis may need to be moved from feeding back to ‘feeding forward’ into a piece of student work, i.e. ‘equip[ping] students with a better appreciation of what is expected of them’, and ‘develop[ing] their understandings of academic terms and appropriate practices before or as they begin to write’. In Hattie and Timperley’s (2007) feedback model (a ‘framework to understand why particular kinds of feedback promote learning effectively and why some others do not’ (p. 86)), the feedforward question is ‘Where to next?’:

The power of feedback … can be used to specifically address this question by providing information that leads to greater possibilities for learning. These may include enhanced challenges, more self-‐regulation over the learning process, greater fluency and automaticity, more strategies and processes to work on the tasks, deeper understanding, and more information about what is and what is not understood. This feedforward question can have some of the most powerful impacts on learning. (p. 90)

Duncan (2007) reports on a small-‐scale research project developed in response to students’ non-‐use of feedback due to ‘the lack of appreciation that comments on one essay could help achievement in a later assignment’ (p. 272). The project involved 16 student volunteers in one module of study submitting eight or more feedback sheets from previous assignments, and tutor analysis of the key issues ‘that appeared as repeated difficulties for each participant’. Then module tutors ‘interpreted the criteria for the next assignment [of which they were to be the markers] in the light of the comments on students’ previous work’ in order to provide feedforward to the individual students, who were also offered a one-‐on-‐one session for needs analysis (D. Hounsell et al., 2008) and the formulation of ‘an individual learning plan for the next, summative, assignment’, and a draft-‐reading service (p. 273). The statistical outcomes ‘showed a small gain in the grades achieved against those who did not participate’ (p. 271). For Hounsell, McCune, Hounsell & Litjens (2008), feedforward is the last of the six steps of their guidance and feedback loop, and its potential can only be realised if feedback is timely. Price, Handley, Millar and O’Donovan (2010) identify five roles attributed to feedback (see ‘Purposes of feedback’ above): ‘correction, reinforcement, forensic diagnosis, benchmarking and longitudinal development (feedforward)’. As regards the latter, the authors argue that the increasing emphasis on feedforward means ‘not only including feedback directed at supporting improvements in the next assignment but also providing advice and guidance that supports slowly learnt literacies (Knight and York 2004) and coming to understand threshold concepts (Meyer and Land 2006)’ (p. 279).

56

Brown (n.d.) argues that formative feedback can be defined as ‘feedback that feeds forward’, in other words, that feedback ‘must be relevant to future work’. For Brown and the other members of the Formative Assessment in Science Teaching (FAST) project team, most written feedback received by students ‘serves to justify their grades for the current work by exposing (mainly) their weaknesses, without necessarily explaining clearly the bases of the weaknesses or enabling students to improve their future performance’. The author refers to a number of ‘strategies for change’, one such strategy involving the avoidance of any reference to marks in the feedback provided in order to encourage students to focus on the areas for improvement identified by the marker. Brown and Glover (2006) report on a study which first ‘classif[ied] systematically the different types of teacher [written] comments that constitute feedback so that the quality of feedback [could] be analysed’ (p. 82). This classification system has five main categories: comments ‘about the content of a student’s response’, comments ‘that help a student to develop appropriate skills’, comments ‘that actively encourage further learning’, comments ‘that are motivational’, and comments ‘that may de-‐motivate’ (p. 83). The authors identify three levels of feedback, namely that which ‘acknowledge[s] a weakness: i.e., ‘acknowledge[s] a performance gap exists (level 1)’, that which ‘provide[s] correction: i.e., gives the student the information needed to close the gap (level 2)’, and that which ‘explain[s] why the student’s response is inappropriate/why the correction is a preferred response: i.e., enable[s] the student to use the information to close the gap (level 3)’ (p. 85). This classification system was then used to analyse tutor feedback. The changes made as a result of the identification of weaknesses in the type and quality of feedback include ‘encouraging tutors to highlight aspects of the student’s strengths and weaknesses that have relevance for future work’, and ‘providing exemplars (specimen answers ) for students with explanatory notes that stress skills development and the relevance to future work’ (p. 89).

57

5.0: Glossary of key terms – definitions from the literature: Explicit knowledge

• that part of knowledge which ‘can be expressed in formal and systematic language’ (Nonaka et al., 2002, p. 43, cited in Rust et al., 2003).

Feedback defined in terms of its informational content:

• ‘is … information about how successfully something has been or is being done … Most [skills] require practice in a supportive environment which incorporates feedback loops. This usually includes a teacher who knows which skills are to be learned, and who can recognize and describe a fine performance, and indicate how a poor performance can be improved’ (Sadler, 1989, p. 120).

defined in terms of its effect: • ‘is information about the gap between the actual level and the reference level

of a system parameter which is used to alter the gap in some way’ (Ramaprasad, 1983, p. 4, cited in Sadler, 1989, p. 120).

Main audiences: the teacher and the student • ‘Teachers use feedback to make programmatic decision with respect to

readiness, diagnosis, and remediation. Students use it to monitor the strengths and weaknesses of their performances, so that aspects associated with success or high quality can be recognized and reinforced, and unsatisfactory aspects modified or improved’ (Sadler, 1989, pp. 120-‐121).

Feedforward

• refers to feedback that has the potential to feed forward into a subsequent assignment or assessment (Hounsell et al., 2008) . For Hounsell et al., it represents the last of the six steps in their guidance and feedback loop.

Formative assessment

• ‘refers to assessment that is specifically intended to generate feedback on performance to improve and accelerate learning’ (Nicol & Macfarlane-‐Dick, 2006, p. 199).

• ‘is concerned with how judgments about the quality of student responses (performances, pieces, or works) can be used to shape and improve the student’s competence by short-‐circuiting the randomness and inefficiency of trial-‐and-‐error learning’ (Sadler, 1989, p. 120).

Qualitative judgement(s)

• ‘one made directly by a person, the person’s brain being both the source and the instrument for the appraisal’ (Sadler, 1989, p. 124).

• are those made in a context where ‘student development is multidimensional rather than sequential, and prerequisite learnings cannot be conceptualised as neatly packaged units of skills or knowledge’, and where ‘it is more appropriate to think in terms of the quality of a student’s response or the degree of expertise than in terms of facts memorized, concepts acquired or content mastered’ (Sadler, 1989, p. 123).

• have five characteristics: o ‘multiple [interlocking] criteria are used in appraising the quality of

performances’;

58

o ‘at least some of the criteria used in appraisal are fuzzy [with ‘continuous gradation from one state to another] rather than sharp’ [e.g. correct or incorrect];

o ‘the teacher may find it impossible to specify all the relevant criteria in advance, or may find that a fixed set of criteria is not uniformly applicable to different student responses … to the same task’;

o ‘the final court of appeal is to another qualitative judgment’; o ‘the final decision is never arrived at by counting things’ – if … marks …

are used, they are assigned after the judgment has been made’ (Sadler, 1989, pp. 124-‐125).

Summative assessment

• ‘is concerned with summing up or summarizing the achievement status of a student, and is geared towards reporting at the end of a course of study especially for purposes of certification (Sadler, 1989, p. 120).

Tacit knowledge

• that part of [a teacher’s] knowledge which is difficult to articulate and can go ‘missing’. It is ‘highly personal and hard to formalise’ (Rust et al., 2003, pp. 151-‐152).

• Deeply rooted in action and often in an individual’s commitment to a profession, tacit knowledge consists partly of technical skills based on professional experience, and in a more cognitive dimension, in our ingrained mental models, beliefs and perspectives (Nonaka et al., 1991, cited in Rust et al., 2003).

59

References Ballantyne, R., Hugher, K., & Mylonas, A. (2002). Developing procedures for

implementing peer assessment in large classes using an action research process. Assessment & Evaluation in Higher Education, 27(5), 427-‐441. doi: 10.1080/0260293022000009302

Basnet, B., Brodie, L., & Worden, J. (2010). Enhanced Feedback -‐ Does Peer Assessment achieve this goal? Paper presented at the AaeE 2010: Past, Present, Future -‐ the 'Keys' to Engineering Education Research and Practice.

Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education, 5(1), 7-‐74. doi: 10.1080/0969595980050102

Bridgeman, A., & Rutledge, P. (2010). Getting personal: Feedback for the masses. Synergy (Institute for Teaching and Learning, University of Sydney)(30), 61-‐68.

Brown, E. (n.d.). Effective feedback. Formative Assessment in Science Teaching (FAST) Retrieved 22.03.12, 2012

Brown, E., & Glover, C. (2006). Evaluating written feedback. In C. Bryan & K. Clegg (Eds.), Innivative assessment in Higher Education (pp. 81-‐91). London: Routledge.

Carless, D. (2006). Differing perceptions in the feedback process. Studies in Higher Education, 31(2), 219-‐233. doi: 10.1080/03075070600572132

Carless, D., Salter, D., Yang, M., & Lam, J. (2011). Developing sustainable feedback practices. Studies in Higher Education, 36(4), 395-‐407. doi: 10.1080/03075071003642449

Cartney, P. (2010). Exploring the use of peer assessment as a vehicle for closing the gap between feedback given and feedback used. Assessment & Evaluation in Higher Education, 35(5), 551-‐564. doi: 10.1080/02602931003632381

Case, S. (2007). Reconfiguring and realigning the assessment feedback processes for an undergraduate criminology degree. Assessment & Evaluation in Higher Education, 32(3), 285-‐299. doi: 10.1080/02602930600896548

Chanock, K. (2000). Comments on Essays : Do students understand what tutors write? Teaching in Higher Education, 5(1), 95-‐105.

Covic, T., & Jones, M. K. (2008). Is the essay resubmission option a formative or a summative assessment and does it matter as long as the grades improve? Assessment & Evaluation in Higher Education, 33(1), 75-‐85. doi: 10.1080/02602930601122928

Denton, P., Madden, J., Roberts, M., & Rowe, P. (2008). Students' response to traditional and computer-‐assisted formative feedback: A comparative case study. British Journal of Educational Technology, 39(3), 486-‐500. doi: 10.1111/j.1467-‐8535.2007.00745.x

Duncan, N. (2007). 'Feed-‐forward': improving students' use of tutors' comments. Assessment & Evaluation in Higher Education, 32(3), 271-‐283. doi: 10.1080/02602930600896498

Gibbs, G., & Simpson, C. (2004). Conditions Under Which Assessment Supports Students ’ Learning. Learning and Teaching in Higher Education, 1(1), 3-‐31.

Glover, C., & Brown, E. (2006). Written Feedback for Students : too much , too detailed or too incomprehensible to be effective ? Bioscience Education eJournal, 7(May).

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-‐112. doi: 10.3102/003465430298487

Hendry, G. D., Bromberger, N., & Armstrong, S. (2011). Constructive guidance and feedback for learning: The usefulness of exemplars, marking sheets and

60

different types of feedback in a first year Law subject. Assessment & Evaluation in Higher Education, 36(1), 1-‐11. doi: 10.1080/02602930903128904

Hepplestone, S., Holden, G., Irwin, B., Parkin, H. J., & Thorpe, L. (2011). Using technology to encourage student engagement with feedback: A literature review. Research in Learning Technology, 19(2), 117-‐127. doi: 10.1080/21567069.2011.586677

Higgins, R., Hartley, P., & Skelton, A. (2001). Getting the message across: The problem of communicating assessment feedback. Teaching in Higher Education, 6(2), 269-‐274. doi: 10.1080/13562510120045230

Higgins, R., Hartley, P., & Skelton, A. (2002). The Conscientious Consumer : Reconsidering the role of assessment feedback in student learning. Studies in Higher Education, 27(1), 53-‐64. doi: 10.1080/03075070120099368

Holmes, L. E., & Smith, L. J. (2003). Student evaluations of faculty grading methods. Journal of Education for Business, 78(6), 318-‐323.

Hounsell, D., McCune, V., Hounsell, J., & Litjens, J. (2008). The quality of guidance and feedback to students. Higher Education Research & Development, 27(1), 55-‐67. doi: 10.1080/07294360701658765

Hyland, F., & Hyland, K. (2001). Sugaring the pill: Praise and criticism in written feedback. Journal of Second Language Writing, 10, 185-‐212.

Lunsford, R. F. (1997). When less is more: Principles for responding in the disciplines. In P. Elbow & M. D. Sorcinelli (Eds.), Writing to learn: Strategies for assigning and responding to writing across the disciplines (pp. 91-‐104). San Francisco: Jossey-‐Bass.

Lunt, T., & Curran, J. (2009). ‘Are you listening please?’ The advantages of electronic audio feedback compared to written feedback. Assessment & Evaluation in Higher Education, 35(7), 759-‐769. doi: 10.1080/02602930902977772

Merry, S., & Orsmond, P. (2008). Students' attitudes to and usage of academic feedback provided via audio files. Bioscience Education, 11. Retrieved from http://www.bioscience.heacademy.ac.uk/journal/vol11/beej-‐11-‐3.pdf website:

Nicol, D. J. (2009). Assessment for learner self-‐regulation: Enhancing achievement in the first year using learning technologies. Assessment & Evaluation in Higher Education, 34(3), 335-‐352. doi: 10.1080/02602930802255139

Nicol, D. J. (2010). From monologue to dialogue: improving written feedback processes in mass higher education. Assessment & Evaluation in Higher Education, 35(5), 501-‐517. doi: 10.1080/02602931003786559

Nicol, D. J. (2011). Developing students' ability to construct feedback. Paper presented at the QAA Enhancement Themes Conference, Heriot-‐Watt University.

Nicol, D. J., & Macfarlane-‐Dick, D. (2006). Formative assessment and self-‐regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-‐218. doi: 10.1080/03075070600572090

Orsmond, P., & Merry, S. (2011). Feedback alignment: Effective and ineffective links between tutors’ and students’ understanding of coursework feedback. Assessment & Evaluation in Higher Education, 36(2), 125-‐136. doi: 10.1080/02602930903201651

Orsmond, P., Merry, S., & Reiling, K. (2005). Biology students' utilization of tutors' formative feedback: A qualitative interview study. Assessment & Evaluation in Higher Education, 30(4), 369-‐386. doi: 10.1080/02602930500099177


61

Price, M., Carroll, J., O'Donovan, B., & Rust, C. (2010). If I was going there I wouldn't start from here: a critical commentary on current assessment practice. Assessment & Evaluation in Higher Education. Retrieved from doi:10.1080/02602930903512883

Price, M., Handley, K., Millar, J., & O'Donovan, B. (2010). Feedback : all that effort, but what is the effect? Assessment & Evaluation in Higher Education, 35(3), 277-‐289. doi: 10.1080/02602930903541007

Price, M., O'Donovan, B., & Rust, C. (2007). Putting a social-‐constructivist assessment process model into practice: Building the feedback loop into the assessment process through peer review. Innovations in Education and Teaching International, 44(2), 143-‐152.

Rotheram, B. (2007). Using an MP3 recorder to give feedback on student assignments. Educational Developments, 8(2), 7-‐10.

Rotheram, B. (2009). Practice tips on using digital audio for assessment feedback Retrieved 31 January, 2012, from http://www.engsc.ac.uk/events/using-‐technology-‐to-‐enhance-‐learning-‐teaching-‐assessment-‐and-‐feedback

Rust, C. (2001). A briefing on assessment of large groups. Generic Centre Assessment Series No. 12, from http://www.bioscience.heacademy.ac.uk/resources/themes/assessment.aspx

Rust, C. (2007). Towards a scholarship of assessment. Assessment & Evaluation in Higher Education, 32(2), 229-‐237. doi: 10.1080/02602930600805192

Rust, C., O'Donovan, B., & Price, M. (2005). A social constructivist assessment process model: how the research literature shows us this could be best practice. Assessment & Evaluation in Higher Education, 30(3), 231-‐240. doi: 10.1080/02602930500063819

Rust, C., Price, M., & O'Donovan, B. (2003). Improving students' learning by developing their understanding of assessment criteria and processes. Assessment & Evaluation in Higher Education, 28(2), 147-‐164. doi: 10.1080/02602930301671

Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119-‐144.

Sadler, D. R. (2008). Transforming holistic assessment and grading into a vehicle for complex learning. In G. Joughin (Ed.), Assessment, learning and judgement in higher education (pp. 49-‐64). Dordrecht: Springer.

Sadler, D. R. (2010). Beyond feedback: Developing student capability in complex appraisal. Assessment & Evaluation in Higher Education, 35(5), 535-‐550. doi: 10.1080/02602930903541015

Sandhu, S. (2004). The use of statement banks as a means of assessing large groups in Accounting. Investigations in University Teaching and Learning, 2(1), 49-‐55.

Vardi, I. (2012). Effective feedback for student learning in higher education: HERDSA. Weaver, M. R. (2006). Do students value feedback? Student perceptions of tutors’

written responses. Assessment & Evaluation in Higher Education, 31(3), 379-‐394. doi: 10.1080/02602930500353061

Yorke, M. (2003). Formative assessment in higher education: Moves towards theory and the enhancement of pedagogic practice. Higher Education, 45, 477-‐501.

Addendum: Update of Feedback Bibliography 4.1.5 Student engagement with feedback Burns, C. A., & Foo, M. (2012). Evaluating a formative feedback intervention for international students. Practitioner Research in Higher Education, 6(1), 40-‐49. Burns, C., & Foo, M. (2013). How is feedback used?–The international student response to a Formative Feedback Intervention. The International Journal of Management Education, 11(3), 174-‐183. Handley, M., & Williams, L., (2009). From copying to learning: using exemplars to engage students with assessment criteria and feedback. Assessment & Evaluation in Higher Education, 36(1), 95-‐108. 4.1.6 Student use of feedback Tian, M., & Lowe, J. (2013). The role of feedback in cross-‐cultural learning: a case study of Chinese taught postgraduate students in a UK university. Assessment & Evaluation in Higher Education, 38(5), 580-‐598. 4.1.7 Student perceptions of tutors’ written feedback Agius, N. M., & Wilkinson, A. (2014). Students' and teachers' views of written feedback at undergraduate level: A literature review. Nurse education today, 34(4), 552-‐559. Ali, N., Rose, S., & Ahmed, L. (2014). Psychology students' perception of and engagement with feedback as a function of year of study. Assessment & Evaluation in Higher Education. DOI: 10.1080/02602938.2014.936355 Dowden, T., Pittaway, S., Yost, H., & McCarthy, R. (2013). Students’ perceptions of written feedback in teacher education: Ideally feedback is a continuing two-‐way communication that encourages progress. Assessment & Evaluation in Higher Education, 38(3), 349-‐362. Murdoch-‐Eaton, D., & Sargeant, J. (2012). Maturational differences in undergraduate medical students’ perceptions about feedback. Medical education, 46(7), 711-‐721. Pears, A., Harland, J., Hamilton, M., & Hadgraft, R. (2013, March). What is Feedback? Connecting Student Perceptions to Assessment Practices. In Learning and Teaching in Computing and Engineering (LaTiCE), 2013 (pp. 106-‐113). IEEE. Strong, J., Hughes, C., Wilson, W., Arnott, W., Isles, R., & Bennison, A. (2012). Perceptions of feedback among undergraduate and postgraduate students of four health science disciplines. The Internet Journal of Allied Health Sciences and Practice, 10(4).

4.1.8 Effectiveness of Feedback Colthorpe, K., Liang, S., & Zimbardi, K. (2013). Facilitating timely feedback in the Biomedical Sciences. International Journal of Innovation in Science and Mathematics Education (formerly CAL-‐laborate International), 21(3). DiBattista, D. (2013). The Immediate Feedback Assessment Technique: A learner-‐centered multiple-‐choice response form. Canadian Journal of Higher Education, 35(4), 111-‐131. Duijnhouwer, H., Prins, F. J., & Stokking, K. M. (2012). Feedback providing improvement strategies and reflection on feedback use: Effects on students’ writing motivation, process, and performance. Learning and Instruction, 22(3), 171-‐184. Fernández-‐Toro, M., Truman, M., & Walker, M. (2013). Are the principles of effective feedback transferable across disciplines? A comparative study of written assignment feedback in Languages and Technology. Assessment & Evaluation in Higher Education, 38(7), 816-‐830. Field, D., Pulman, S., & Whitelock, D. (2014). The pragmatics of margin comments: An empirical study. ACL 2014, 43. Gul, R., Lakhani, A., & Rizvi, N. F. (2014). Teachers’ Accounts of Their Perceptions and Practices of Providing Written Feedback to Nursing Students on Their Assignments. International Journal of Higher Education, 3(3), p70. Kishbaugh, T. L., Cessna, S., Horst, S. J., Leaman, L., Flanagan, T., Neufeld, D. G., & Siderhurst, M. (2012). Measuring beyond content: a rubric bank for assessing skills in authentic research assignments in the sciences. Chemistry Education Research and Practice, 13(3), 268-‐276. Jones, O., & Gorra, A. (2013). Assessment feedback only on demand: Supporting the few not supplying the many. Active Learning in Higher Education, 1469787413481131. Kerrigan, M. J., Walker, S., Gamble, M., Lindsay, S., Reader, K., Papaefthimiou, M. C., ... & Saunders, G. (2012). The Making Assessment Count (MAC) consortium maximising assessment and feedback design by working together. Research in Learning Technology, 19. Li, J., & De Luca, R. (2014). Review of assessment feedback. Studies in Higher Education, 39(2), 378-‐393. MacWilliam, T., & Malan, D. J. (2013, July). Streamlining grading toward better feedback. In Proceedings of the 18th ACM conference on Innovation and technology in computer science education (pp. 147-‐152). ACM. Maggs, L. A. (2014). A case study of staff and student satisfaction with assessment feedback at a small specialised higher education institution. Journal of Further and Higher Education, 38(1), 1-‐18.

Molloy, E., & Boud, D. (2013). Changing conceptions of feedback. Feedback in higher and professional education: Understanding it and doing it well, 11-‐33. Nicol, D. (2012). Resituating feedback from the reactive to the proactive. Feedback in Higher and Professional Education: Understanding it and Doing it Well, 34. Price, M., Handley, K., O’Donovan, B., Rust, C., & Millar, J. (2013). Assessment feedback. Reconceptualising Feedback in Higher Education: Developing Dialogue with Students, 41. Ross, P. M., & Tronson, D. A. (2012, November). Providing quality feedback-‐Where to from here?. In Proceedings of The Australian Conference on Science and Mathematics Education (formerly UniServe Science Conference) (Vol. 11). Sutton, P. (2012). Conceptualizing feedback literacy: knowing, being, and acting. Innovations in Education and Teaching International, 49(1), 31-‐40. Tang, J., & Harrison, C. (2011). Investigating university tutor perceptions of assessment feedback: three types of tutor beliefs. Assessment & Evaluation in Higher Education, 36(5), 583-‐604. Tett, L., Hounsell, J., Christie, H., Cree, V. E., & McCune, V. (2012). Learning from feedback? Mature students’ experiences of assessment in higher education. Research in Post-‐Compulsory Education, 17(2), 247-‐260. Tuck, J. (2012). Feedback-‐giving as social practice: teachers’ perspectives on feedback as institutional requirement, work and dialogue. Teaching in Higher Education, 17(2), 209-‐221. Venning, J., & Buisman-‐Pijlman, F. (2013). Integrating assessment matrices in feedback loops to promote research skill development in postgraduate research projects. Assessment & Evaluation in Higher Education, 38(5), 567-‐579. 4.2.1 Feedback practice Barker, M., & Pinard, M. (2014). Closing the feedback loop? Iterative feedback between tutor and student in coursework assessments. Assessment & Evaluation in Higher Education, DOI: 10.1080/02602938.2013.875985. Burke, D., & Pieterick, J. (2010). Giving students effective written feedback. Maidenhead: Open University Press/McGraw Hill. Burke, D. (2011). Now I’ve got the feedback, what do I do with it? Strategies for students to get more out of tutor feedback. Practice and Evidence of Scholarship of Teaching and Learning in Higher Education, 6(1), 43-‐60. Cathcart, A., Greer, D., & Neale, L. (2013). Learner-‐focused evaluation cycles: facilitating learning using feedforward, concurrent and feedback evaluation. Assessment & Evaluation in Higher Education, 790-‐802. DOI: 10.1080/02602938.2013.870969

Dube, C., Kane, S., & Lear, M. (2012). The effectiveness of students redrafting continuous assessment tasks: the pivotal role of tutors and feedback. Perspectives in Education, 30(3), 50-‐59. East, M., Bitchener, J., & Basturkmen, H. (2012). What Constitutes Effective Feedback to Postgraduate Research Students? The Students' Perspective. Journal of University Teaching and Learning Practice, 9(2), 7. Freeman, R., & Dobbins, K. (2013). Are we serious about enhancing courses? Using the principles of assessment for learning to enhance course evaluation. Assessment & Evaluation in Higher Education, 38(2), 142-‐151. Hughes, G., Wood, E., & Kitagawa, K. (2014). Use of self-‐referential (ipsative) feedback to motivate and guide distance learners. Open Learning: The Journal of Open, Distance and e-‐Learning, 29(1), 31-‐44. Jonsson, A. (2012). Facilitating productive use of feedback in higher education. Active Learning in Higher Education, 1469787412467125. Lilly, J., Richter, I. U., & Macias, B. R. (2009). Feeding Back, Feeding Forward: An Exchange of Ideas. Networks, 77. Nordrum, L., Evans, K., & Gustafsson, M. (2013). Comparing student learning experiences of in-‐text commentary and rubric-‐articulated feedback: strategies for formative assessment. Assessment & Evaluation in Higher Education, 38(8), 919-‐940. Scoles, J., Huxham, M., & McArthur, J. (2013). No longer exempt from good practice: using exemplars to close the feedback gap for exams. Assessment & Evaluation in Higher Education, 38(6), 631-‐645. 4.2.3 Student engagement with feedback Boud, D., Lawson, R., & Thompson, D. G. (2013). Does student engagement in self-‐assessment calibrate their judgement over time?. Assessment & Evaluation in Higher Education, 38(8), 941-‐956. Boud, D., & Molloy, E. (2013). Rethinking models of feedback for learning: the challenge of design. Assessment & Evaluation in Higher Education, 38(6), 698-‐712. Cartney, P. (2010). Exploring the use of peer assessment as a vehicle for closing the gap between feedback given and feedback used. Assessment & Evaluation in Higher Education, 35(5), 551-‐564. Donovan, P. (2014). Closing the feedback loop: physics undergraduates’ use of feedback comments on laboratory coursework. Assessment & Evaluation in Higher Education, (ahead-‐of-‐print), 1-‐13. Fleckhammer, L., & Wise, L. (2011). The role of tutors in facilitating online student engagement. Changing Demands, Changing Directions. Proceedings ascilite Hobart, 392-‐397.

Giles, T. M., Gilbert, S., & McNeill, L. (2014). Do students want and need written feedback on summative assignments? Engaging students with the feedback process-‐a topic review activity. ergo, 3(1). McMahon, T. (2010). Using peer-‐assessment to generate high-‐quality and proximate feedback and feed-‐forward on students’ work. ICERI2010 Proceedings, 3422-‐3423. Nicol, D. (2013). Assessment and Feedback In the hands of the student. JISC e-‐Learning Programme Webinar. Nicol, D., Thomson, A., & Breslin, C. (2014). Rethinking feedback practices in higher education: a peer review perspective. Assessment & Evaluation in Higher Education, 39(1), 102-‐122. Robinson, S., Pope, D., & Holyoak, L. (2013). Can we meet their expectations? Experiences and perceptions of feedback in first year undergraduate students. Assessment & Evaluation in Higher Education, 38(3), 260-‐272. Rubin, R. F., & Turner, T. (2012). Student performance on and attitudes toward peer assessments on Advanced Pharmacy Practice Experience assignments. Currents in Pharmacy Teaching and Learning, 4(2), 113-‐121. 4.2.4 Dialogic approaches to feedback Blair, A., Curtis, S., Goodwin, M., & Shields, S. (2013). What feedback do students want? Politics, 33(1), 66-‐79. Blair, A., & McGinty, S. (2013). Feedback-‐dialogues: exploring the student perspective. Assessment & Evaluation in Higher Education, 38(4), 466-‐476. Blair, A., Curtis, S., Goodwin, M., & Shields, S. (2013). The significance of assignment feedback: from consumption to construction. European Political Science, 12(2), 231-‐244. Blair, A., Wyburn-‐Powell, A., Goodwin, M., & Shields, S. (2014). Can dialogue help to improve feedback on examinations? Studies in Higher Education, 39(6), 1039-‐1054. Carless, D. (2013). Trust and its role in facilitating dialogic feedback. Feedback in higher and professional education, 90-‐103. Merry, S., Price, M., Carless, D., & Taras, M. (Eds.). (2013). Reconceptualising Feedback in Higher Education: Developing Dialogue with Students. Routledge. Yang, M., & Carless, D. (2013). The feedback triangle and the enhancement of dialogic feedback processes. Teaching in Higher Education, 18(3), 285-‐297. 4.2.5 Using technology to provide peer and marker feedback Allan, R., & Bentley, S. (2012). Feedback mechanisms: efficient and effective use of technology or a waste of time and effort? eprints.hud.ac.uk

Alvarez, I., Espasa, A., & Guasch, T. (2012). The value of feedback in improving collaborative writing assignments in an online learning environment. Studies in Higher Education, 37(4), 387-‐400. Attenborough, J., Gulati, S., & Abbott, S. (2012). Audio feedback on student assignments: boon or burden? Learning at City Journal, 2(2). openaccess.city.ac.uk. Auld, G., Ridgway, A., & Williams, J. (2013). Digital oral feedback on written assignments as professional learning for teacher educators: A collaborative self-‐study. Studying Teacher Education, 9(1), 31-‐44. Bayerlein, L. (2014). Students’ feedback preferences: How do students react to timely and automatically generated assessment feedback? Assessment & Evaluation in Higher Education, DOI: 10.1080/02602938.2013.870531. Berlanga, A. J., Van Rosmalen, P., Boshuizen, H. P. A., & Sloep, P. B. (2012). Exploring formative feedback on textual assignments with the help of automatically created visual representations. Journal of Computer Assisted Learning, 28(2), 146–160. doi: 10.1111/j.1365-‐2729.2011.00425. Borup, J., West, R. E., Thomas, R., & Graham, C. R. (2014). Examining the impact of video feedback on instructor social presence in blended courses. The International Review of Research in Open and Distance Learning, 15(3). irrodl.org Bourgault, A. M., Mundy, C., & Joshua, T. (2013). Comparison of audio vs. written feedback on clinical assignments of nursing students. Nursing Education Perspectives, 34(1), 43-‐46. nlnjournals.org. Brearley, F. Q., & Cullen, W. R. (2012). Providing students with formative audio feedback. -‐ Bioscience Education, 20, 22-‐36. Carruthers, C., McCarron, B., Bolan, P., Devine, A., & McMahon-‐Beattie, U. (2014). Listening and Learning: Reflections on the use of Audio Feedback. An Excellence in Teaching and Learning Note. Business and Management Education in HE, 1(1), 4-‐11. Carruthers, C., McCarron, B., Bolan, P., Devine, A., McMahon-‐Beattie, U., & Burns, A. (2014). ‘I like the sound of that’–an evaluation of providing audio feedback via the virtual learning environment for summative assessment. Assessment & Evaluation in Higher Education, DOI: 10.1080/02602938.2014.917145. Chang, N. (2012). Pre-‐Service Teachers' Views: How Did E-‐feedback through Assessment Facilitate Their Learning? Journal of the Scholarship of Teaching and Learning, 11, 16-‐33. josotl.indiana.edu Chang, N., Watson, A. B., Bakerson, M. A., Williams, E. E., McGoron, F. X., & Spitzer, B. (2012). Electronic feedback or handwritten feedback: What do undergraduate students prefer and why? Journal of Teaching and Learning with Technology, 1(1), 1-‐23. Crook, A., Mauchline, A., Maw, S., Lawson, C., Drinkwater, R., Lundqvist, K., ... & Park, J. (2012). The use of video technology for providing feedback to students: Can it

enhance the feedback experience for staff and students?. Computers & Education, 58(1), 386-‐396. Edwards, K., Dujardin, A. F., & Williams, N. (2012). Screencast Feedback for Essays on a Distance Learning MA in Professional Communication. Journal of Academic Writing, 2(1), 95-‐126. Ferrell, G., & Sheppard, M. (2013). Supporting assessment and feedback practice with technology: a view of the UK landscape. jiscdesignstudio.pbworks.com Frank, B. M., & Hamgini, B. B. (2014). Collaborative cloud-‐based documents for real-‐time bi-‐directional feedback in large lecture activities. 121st ASEE Annual Conference & Exposition. Gleaves, A., & Walker, C. (2013). Richness, redundancy or relational salience? A comparison of the effect of textual and aural feedback modes on knowledge elaboration in higher education students' work. Computers & Education, 62, 249-‐261. Gould, J., & Day, P. (2013). Hearing you loud and clear: student perspectives of audio feedback in higher education. Assessment & Evaluation in Higher Education, 38(5), 554-‐566. Harper, F., Green, H., & Fernandez-‐Toro, M. (2012, September). Evaluating the integration of Jing® screencasts in feedback on written assignments. In Interactive Collaborative Learning (ICL), 2012 15th International Conference on (pp. 1-‐7). IEEE. Hennessy, C., & Forrester, G. (2014). Developing a framework for effective audio feedback: a case study. Assessment and Evaluation in Higher Education, 39(7), 777-‐789. Hepplestone, S., Holden, G., Irwin, B., Parkin, H. J., & Thorpe, L. (2011). Using technology to encourage student engagement with feedback: a literature review. Research in Learning Technology, 19(2). Jones, N., Georghiades, P., & Gunson, J. (2012). Student feedback via screen capture digital video: stimulating student’s modified action. Higher Education, 64(5), 593-‐607. Laughton, D. J. (2013). Using audio feedback to enhance assessment practice–an evaluation of student and tutor perspectives. Student Engagement and Experience Journal, 2(2). Lee, K. (2012). Technology-‐mediated feedback. Quick Hits for Teaching with Technology: Successful Strategies by Award-‐Winning Teachers, 80. Malau-‐Aduli, B. S., Assenheimer, D., Choi-‐Lundberg, D., & Zimitat, C. (2013). Using computer-‐based technology to improve feedback to staff and students on MCQ assessments. Innovations in Education and Teaching International, 51(5), 510-‐522. Marriott, P., & Teoh, L. K. (2012). Using screencasts to enhance assessment feedback: students’ perceptions and preferences. Accounting Education, 21(6), 583-‐598.

Mathisen, P. (2012). Video Feedback in Higher Education–A Contribution to Improving the Quality of Written Feedback. Nordic Journal of Digital Literacy, 7(02), 97-‐112. McGregor, K., Merchant, A. R., & Butler, M. (2012). Making feedback more immediate. International Journal of Innovation in Science and Mathematics Education (formerly CAL-‐laborate International), 16(1). Morrissey, J. (2012). Podcast steering of independent learning in higher education. AISHE-‐J: The All Ireland Journal of Teaching and Learning in Higher Education, 4(1). Mulder, R. A., Elgar, M. A., & Brady, D. (2012, October). APRES: Electronically managed student feedback via peer review. In Proceedings of The Australian Conference on Science and Mathematics Education (formerly UniServe Science Conference) (Vol. 11). Parkin, H. J., Hepplestone, S., Holden, G., Irwin, B., & Thorpe, L. (2012). A role for technology in enhancing students’ engagement with feedback. Assessment & Evaluation in Higher Education, 37(8), 963-‐973. Ribchester, C., France, D., & Wheeler, A. (2007). Podcasting: a tool for enhancing assessment feedback. Education in a Changing Environment. Roberts, S. (2008). Pod casting feedback to students: Students’ perceptions of effectiveness. Innovations in Practice, 44. Shannon, S., Francis, R., & Torpey, G. (2012, November). Barriers to adoption of blended learning and online feedback and assessment by sessional staff. In Paper accepted for ASA 2012 Conference, Griffith University, Gold Coast (pp. 14-‐16). Singh, R., Gulwani, S., & Solar-‐Lezama, A. (2013, June). Automated feedback generation for introductory programming assignments. In ACM SIGPLAN Notices (Vol. 48, No. 6, pp. 15-‐26). ACM. Thompson, R., & Lee, M. J. (2012). Talking with students through screencasting: experimentations with video feedback to improve student learning. The Journal of Interactive Technology and Pedagogy, 1(1). Tong, V. C. (2014). Towards technology-‐and research-‐enhanced education (TREE): electronic feedback as a teaching tool in geoscience. In Geoscience Research and Education (pp. 199-‐212). Springer Netherlands. Upson-‐Saia, K., & Scott, S. (2013). Using iAnnotate to enhance feedback on written work. Journal of Teaching and Learning with Technology, 2(2), 43-‐59. Walet, N., & Birch, M. (2012, May). Using online assessment to provide instant feedback. In Proceedings of the HEA-‐STEM Annual Conference, London.

Walkington, H. (2012). Developing dialogic learning space: The case of online undergraduate research journals. Journal of Geography in Higher Education, 36(4), 547-‐562. Wallace, I., & Moore, C. (2012, January). Providing in-‐depth and personalized feedback to online students using audio recording. In 3rd International Conference on e-‐education, e-‐Business, e-‐Management, and e-‐Learning. Willis, L., & Webb, A. (2010). Enhancing feedback for engineering students. © Higher Education Academy Engineering Subject Centre, Loughborough University. Wong, S. H. S., & Beaumont, A. J. (2012). A quest for helpful feedback to programming coursework. engineering education, 7(2), 51-‐62. 4.2.6 Dealing with large classes Kulkarni, C., Wei, K. P., Le, H., Chia, D., Papadopoulos, K., Cheng, J., ... & Klemmer, S. R. (2013). Peer and self assessment in massive online classes. ACM Transactions on Computer-‐Human Interaction (TOCHI), 20(6), 33. Rienties, B., Willis, A., Alcott, P., & Medland, E. (2013). Student Experiences of Self-‐Reflection and Peer Assessment in Providing Authentic Project-‐Based Learning to Large Class Sizes. In Facilitating learning in the 21st century: leading through technology, diversity and authenticity (pp. 117-‐136). Springer Netherlands. 4.2.7 Feedforward Backstrom, M. (2012). Enhancing opportunities to feed forward; developing problem solving skills using web based scenarios. eprints.qut.edu.au Baker, D. J., & Zuvela, D. (2013). Feedforward strategies in the first-‐year experience of online and distributed learning environments. Assessment & Evaluation in Higher Education, 38(6), 687-‐697. Bird, F. L., & Yucel, R. (2014). Feedback codes and action plans: building the capacity of first-‐year students to apply feedback to a scientific report. Assessment & Evaluation in Higher Education, DOI: 10.1080/02602938.2014.924476 Bennett, S., & Barker, T. (2013). Helping students self monitor through EVS based feed-‐forward exercises. In e-‐Learning and e-‐Technologies in Education (ICEEE), 2013 Second International Conference (pp. 1-‐6). IEEE. Deyi, S. (2011). Feedback or feed-‐forward?. Changing Spaces: Writing Centres and Access to Higher Education, 47. Duncan, N. (2007). ‘Feed-‐forward’: improving students' use of tutors' comments. Assessment & Evaluation in Higher Education, 32(3), 271-‐283. Fladd, N. (2011). Getting Feedback to Feed Forward: Incorporating revision into upper-‐year English papers. Teaching Innovation Projects, 1(2), 3.

Greaves, L. (2012). Feed-‐forward for Informed Learning (FfIL): learner-‐generated materials for personalised learning. Enhancing Learning in the Social Sciences, 4(3). Moore, C., & Wallace, I. P. Personalizing Feedback for Feed-‐Forward Opportunities Utilizing Audio Feedback Technologies for Online Students. Moore, S. A. (2010) Does a Feedforward Approach Improve Students’ Assignment Grades? Innovations in Practice, 2(2), 112-‐122. Morrell, L. J. (2013). Use of Feed-‐forward Mechanisms in a Novel Research-‐led Module. Bioscience Education, (0). Murtagh, L., & Baker, N. (2009). Feedback to feed forward: Student response to tutors’ written comments on assignments. Practitioner research in higher education, 3(1), 20-‐28. Orsmond, P., Maw, S. J., Park, J. R., Gomez, S., & Crook, A. C. (2013). Moving feedback forward: theory to practice. Assessment & Evaluation in Higher Education, 38(2), 240-‐252. Osado, A. G., Merlo, L. S. B., & del Campo, M. H. D. (2013). Formative assessment in a Chemistry course through feedback andfeed-‐forward strategies. INTED2013 Proceedings, 3852-‐3860. Pears, A., Harland, J., Hamilton, M., & Hadgraft, R. (2014, April). Four Feed-‐Forward Principles Enhance Students' Perception of Feedback as Meaningful. In Teaching and Learning in Computing and Engineering (LaTiCE), 2014 International Conference on (pp. 272-‐277). IEEE. Quinton, S., & Smallbone, T. (2010). Feeding forward: using feedback to promote student reflection and learning–a teaching model. Innovations in Education and Teaching International, 47(1), 125-‐135. Rodway-‐Dyer, S., & Dunne, E. (2009). Technology enhanced feed-‐forward for learning. HEA Final Report. Robson, S., Leat, D., Wall, K., & Lofthouse, R. (2012). Feedback or feed forward? Supporting master’s students through effective assessment to enhance future learning. Cross cultural teaching and learning for home and international students: Internationalisation, Pedagogy and Curriculum in Higher Education: Internationalisation of Pedagogy and Curriculum in Higher Education, 53. Taylor, C. (2007). Integrating Feedforward on Academic Writing into an Undergraduate Science Course. Assessment design for learner responsibility 29-‐31 May 07 http://www.reap.ac.uk Vardi, I. (2013). Effectively feeding forward from one written assessment task to the next. Assessment & Evaluation in Higher Education, 38(5), 599-‐610.

Walker, S., & Hobson, J. (2014). Interventions in teaching first-‐year law: Feeding forward to improve learning outcomes. Assessment & Evaluation in Higher Education, 39(3), 326-‐338. Wimshurst, K., & Manning, M. (2013). Feed-‐forward assessment, exemplars and peer marking: evidence of efficacy. Assessment & Evaluation in Higher Education, 38(4), 451-‐465. Arts and Social Sciences: Wakefield, C., Adie, J., Pitt, E., & Owens, T. (2014). Feeding forward from summative assessment: the Essay Feedback Checklist as a learning tool. Assessment & Evaluation in Higher Education, 39(2), 253-‐262. Walker, S., & Hobson, J. (2014). Interventions in teaching first-‐year law: Feeding forward to improve learning outcomes. Assessment & Evaluation in Higher Education, 39(3), 326-‐338.

SCIENCE

CONTACT Meloni M. Muir, PhD Discipline of Physiology Sydney Medical School The University of Sydney NSW 2006 E [email protected]

Documents

GIVING FEEDBACK ON STUDENT WRITING IN THE SCIENCES…sydney.edu.au/stuserv/documents/learning_centre/Feedback_Handboo… · science giving feedback on student writing in the sciences: