View
224
Download
0
Category
Preview:
Citation preview
The Effectiveness of Feedback to
Students and Scoring Rubrics
Craig Taylor
Feedback
What is feedback?
Information that is used to alter the gap between the
students actual level of performance and some
reference level. (Derived from Ramaprasad (1983)).
So information that is provided but in a way that can’t be
used by students (e.g. because students do not
understand it) is not feedback by this definition.
Breakdown of survey responses
Discipline n %
English 68 9.4
Creative writing 42 5.8
Languages 38 5.2
Drama 38 5.2
Philosophy 12 1.7
Education 62 8.6
Law 130 17.9
Theology 16 2.2
Biology 319 44.0
Total 725 100.0
Statistical Analysis of Results
Schools and Humanities Disciplines were compared
using Chi-squared tests.
A p value of < .05 indicates a significant difference.
Specific comments I will make are an interpretation of
this difference. Others are possible.
Philosophy and Theology were excluded as the sample
sizes were too small to be statistically significant.
Frequency of use of feedback for future
assessed work
0%
10%
20%
30%
40%
50%
60%
All of the time Most of the time Sometimes Rarely Never
Pe
rcen
tag
e
Able to use feedback in time
Humanities
Education
Law
Biology
Kinds of feedback received by school
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Written comments Completed marking
templates
Individual verbal comments General verbal comments
Perc
en
tag
e
Types of feedback received
Humanities
Education
Law
Biology
Usefulness of kinds of feedback by school
0%
10%
20%
30%
40%
50%
60%
Written comments Completed marking
templates
Individual verbal comments General verbal comments
Pe
rce
nta
ge
Most useful type of feedback
Humanities
Education
Law
Biology
Satisfaction with kinds of feedback
received
0%
10%
20%
30%
40%
50%
60%
70%
Strongly agree Moderately agree Neutral Moderately disagree Strongly disagree
Pe
rce
nta
ge
Happy with kinds of feedback
Humanities
Education
Law
Biology
Most common individual comments
1. Need to improve handwriting for written comments
2. More detailed comments not just ‘ticks’ and ‘nos’
3. Explanation of where student has gone wrong
4. More specific comments about how to improve work, not just comments about where student went wrong
5. Individual ‘one on one’ verbal feedback with students
6. A better indication of where marks have been lost
7. Providing an ideal essay as a guide to what is expected for top marks
8. Workshopping ideal essay
9. More comments on structure and content, less on just spelling and grammar
10. Comments throughout essay not just at end
11. Using examples to illustrate a better answer to question
12. More positive feedback
13. Letting students know what they have done well
14. Marker not simply expressing their frustration in written comments
15. Avoiding shorthand notation/symbols that student does not understand
Marking Templates – or Scoring Rubrics
Survey results do not show that no useful place for marking
templates
Perhaps there is an overreliance on this form of feedback
But equally more careful attention to the design of templates
might overcome some of the concerns indicated in the
survey
A challenge here is that there is a quite limited literature of
template or rubric design.
Varieties of Scoring Rubrics
Here it is important to note the great variety in these instruments that reflects the different purposes for their use.
Some are mere checklists designed either for ease or efficiency of marking or (in large classes with multiple markers) consistency of marking.
But others may provide students with substantial information about their level of performance, and in some cases may include inserted verbal comments such as those available through i-innotate.
Perceptions of rubrics
As Reddy and Andrade (2010) note perceptions of
rubrics vary for students as compared with instructors.
• Students frequently referred to them as serving the
purpose of learning and achievement
• While instructors focussed on the role of a rubric in
quickly, objectively and accurately assigning grades.
(Ibid p. 439)
Rubric design
Two question proposed by Allen and Tanner (2006) that are important to ask before designing rubrics are:
• What do I want students to know and be able to do?
• How will I know when they know it and can do it well?
As they go on to say, not only important for rubric design but also for the choice of assessment task as most relevant if one is to ‘collect evidence about how outcomes are met.’ (ibid p. 198)
Holistic versus analytic rubrics
An important distinction in the literature is between holistic and analytic rubrics.
Holistic rubrics assess discreet assessment criteria on a point scale with the number of points achieved on that scale being holistically described. For example on a three point scale high medium and low achievement.
Analytic rubrics on the other hand defines much more specifically what would in fact count as say high medium and low achievement.
Example of a holistic rubric
Consider an example from Allen and Tanner (2006): Suppose one is assessing a students explanation of how plants get their food from the soil, and in particular the following criterion of assessment:
Student demonstrates an understanding that food can be thought of as carbon-rich molecules including sugars and starches.
An holistic rubric might give 2 points for complete understanding of the concept; 1 point for incomplete understanding; and 0 points for not addressing the concept at all.
Example of an analytic rubric
But now an analytic rubric might further define these levels of achievement as follows:
2 points: Defines food as sugars, carbon skeletons or starches and glucose.
1 point: Attempts to define food but does not include sugars, carbon skeletons or starches.
0 points: Does not address what could be meant by food.
Challenges for rubric design
Obviously analytic rubrics provide more detailed feedback.
They attempt to define very specifically what ‘language and knowledge’ the student must show to demonstrate competency (Allen and Tanner (2006), p. 198).
But as Allen and Tanner also point out we can think of this as ‘an attempt to make discreet what is fundamentally fuzzy.’ (ibid)
This is perhaps a significant challenge in using these kinds of rubrics in many of the disciplines in this Faculty. What scope is there for creative or innovative answers?
Challenges for rubric design
But there are as Allen and Tanner note ways of making rubrics more effective.
• Describe the levels of quality and proficiency for separate criteria.
• Avoid here using words like ‘good’ or ‘creative’ as these are so general as to be fairly useless for guiding the learner.
• Consider how many levels of mastery learners can make any sense out of. Some educators argue that three levels (basically ‘there’, ‘somewhat there’, and ‘not there yet’) is the most that learners can really process.
Providing rubrics to students
Looking at our survey and qualitative comments:
A concern many students have is perhaps with knowing what is expected of them in a given assessment task.
Students are much more likely to find rubrics useful if handed out with the relevant assignment rather than just afterwards with their grade (Schneider (2006)).
A concern though: students may focus too much on satisfying the criteria in the rubric and not enough directly on the actual assessment task. Which may, for example, inhibit creative answers not anticipated by instructors.
Further innovation in rubric design
Clearly there is much scope for further research and
innovation for more effective rubric design.
We noted a wide variety of different rubric designs
provided to students who took our survey.
One innovation worth noting was a rubric used by
Biology that required students to incorporate feedback
from a previous assessment exercise in a current
assessment task.
Further innovation in rubric design
The survey results again: Biology students were more
likely than students in other schools to use feedback on
previously assessed work in subsequent assignments.
So there is at least prima facie evidence that this
innovation has been effective in encouraging students
to use feedback in this way.
References
• Allen, D., & Tanner, K. (2006). Rubrics: Tools for Making
Learning Goals and Evaluation Criteria Explicit for Both
Teachers and Learners. CBE–Life Sciences Education 5: 197–
203.
• Ramaprasad, A. (1983). On the Definition of Feedback.
Behavioral Science 28, no.1: 4–13.
• Reddy, Y.M., & Andrade, H. (2010) A Review of Rubric use in
higher education. Assessment & Evaluation in Higher Education
35, no. 4: 435–48.
• Schneider, J.F. (2006) Rubrics for teacher education in
community college. The Community College Enterprise 12, no.
1: 39–55.
• Taylor, C., & Burke da Silva K. (forthcoming) An analysis of the
effectiveness of feedback to students on assessed work. Higher
Education Research and Development
Recommended