Improving the quality of feedback through the use of electronic marking – a journey.

Maureen Readle, Jak Radice, Neil McKeown

Centre for Educational Development

University of Bradford

2

• Background issues.• University of Bradford policies.• Present practices.• Development of eFeedback and eMarking• Experience of eMarking• Issues encountered• The way forward?

An eMarking Journey

3

Background issues

• How to improve the quality of feedback?

• Why isn’t feedback valued?

NSS 2012

Feedback on my work has been prompt 64%I have received detailed comments on my work 63%

4

University Policies

Policies that affect assessment and marking:• Anonymous marking.• 20 day turnaround time for completion of

marking process.• Electronic submission of coursework.• Typed feedback.

Drivers for eMarking

5

Marking and Feedback

• Emphasis on summative assessment marked anonymously.

• Some good practice related to formative feedback.

6

Development of eFeedback

No consistent policy or approach:• Audio feedback.• Word comments.

– Returned via Blackboard or email.• TurnitinUK GradeMark:

– Ad hoc by individual academics.

7

Development of eMarking

Development of stand alone whole-school approaches:

• SOM, SCIM, EDT;• Web-based assessment criteria grids /

rubrics.• Outside institutional framework, e.g.

– Not integrated with centrally-supported tools such as VLE

8

Development of eMarking

Development of whole-school approaches using centrally supported tools:

TurnitinUK GradeMark:• QuickMarks:

– Trialled with SOHS‘s IPE module;– Use by 2 Schools + lecturers in other schools.

• Rubric: – Qualitative rubric [one school].

9

eMarking Requirements include

• Anonymous marking.• On-line and off-line marking.• Rubric (assessment criteria).• Comments:

– In text; – About each criteria;– General comments.

• Second marking / moderation;– Blind double marking.

• Audit trail for External Examiner.• Check for unoriginal content.• Marking process hidden from students.• Spell checker.

etc. etc.

10

Implementation:

TurnitinUK GradeMark

1. Trialled with IPE module (2010-11):– School of Health Studies interdisciplinary

module;– 145 3rd year students.

2. Whole of School of Health Studies (2011-12)– With Anonymous Marking feature used.

11

Students’ Experience of Using TurnitinUK

… you can use that positive criticism towards hopefully getting a better mark …

… if you look at your feedback you are able to make improvements and see where you are failing or not doing so well …

… it makes a difference, knowing specifically where you went wrong…

… You can see exactly what you did well and where you didn’t do so well …when you next come to write an essay …you’ve learnt from previous attempts …

… that was really good, just one click, mark, feedback, done, that was it…

… You can submit it anytime you like…

12

Staff Experience of Using TurnitinUK [1]

… I always feel I always have to justify the mark I award them …

… You have to justify the mark that you’ve given, through your feedback, so that students can understand how to improve  …

… provides us as lecturers with an ‘aide memoire’ … when the students come and challenge us…

… I did find it quite easy to annotate the script using this system so I did a bit more  …

13

Staff Experience of Using TurnitinUK [2]

… I didn’t tend to use [QuickMark] … it’s not personal, it’s not valuable to the student because it’s not about their work, it’s about academic work generally…

…you get more insight into how other people are marking because you can see their comments …

… there are certain things that I find myself saying over and over again to the students, but if I find myself writing [a QuickMark] then I’d save it in my list so I could use it again and again …

…we can second mark it very quickly …

14

Blackboard Rubrics

Development of whole-school approaches:

Blackboard interactive rubrics:• 2012-13:

– Used by 1 school (School of Health Studies).– One module’s coursework marked and

moderated.• Staff generally positive about:

– Ease-of-use of rubrics;– Speed of marking;– Rubric features (spell checker!).

15

Issues Encountered

• Drivers: – University policies relating to the administration of

assessment;– Not centred on pedagogy.

• eMarking exposes flexible processes and practices relating to manual submission.

• Requires a change to attitude and practice:– Health & safety fears.

• Learning curve.• Technical, including:

– Staff digital literacy or lack of!– Off-line anonymous marking.

16

Way forward?

Proposal for the University:

• Formative personalised feedback in advance of summative assessment.

• Summative assessment marked anonymously against assessment criteria with general comments.

17

Question time

M.Readle@Bradford.ac.uk

N.M.McKeown@Bradford.ac.uk