Upload
vinny
View
56
Download
0
Embed Size (px)
DESCRIPTION
TMA feedback: can we do better? Mirabelle Walker. Centre for Open Learning of Mathematics, Science Computing and Technology (COLMSCT). FAST project findings (1). Quantity and timing of feedback Sufficient feedback is provided, often enough and in enough detail - PowerPoint PPT Presentation
Citation preview
TMA feedback: can we do better?
Mirabelle Walker
Centre for Open Learning of Mathematics, Science Computing and Technology (COLMSCT)
FAST project findings (1)
• Quantity and timing of feedback– Sufficient feedback is provided, often enough and in enough detail– The feedback is provided quickly enough to be useful to students
• Quality of feedback– Feedback focuses on students’ performance and learning, and on actions
under their control– Feedback is appropriate to the purpose of the assignment and to its criteria
for success– Feedback is appropriate in relation to students’ understanding of what they
are supposed to be doing
• Student response to feedback– Feedback is received and attended to– Feedback is acted upon by the student
Gibbs & Simpson (2004–5)
FAST project findings (2)
What students said about their TMA feedback• Received plenty of it• Motivated by praise and encouragement• Mainly received within three weeks• Read feedback but rarely acted on it
FAST project findings (3)
Category of comments
0102030405060
Per
cen
tag
e
FAST project findings (4)
Type of content comment
0
10
20
30
40
50
60
70
errors omissions irrelevancies clarifications
Per
cen
tag
e
Idea of ‘depth’ of comment
• Depth 1 – acknowledges e.g. ‘more needed here’; ‘good’
• Depth 2 – corrects / amplifies e.g. ‘you needed to mention xxxx’; ‘a good introduction’
• Depth 3 – explains e.g. ‘you needed to mention xxxx because …’; ‘a good introduction because …’
FAST project findings (5)
Depth of comments
0
10
20
30
40
50
60
70
depth 1 depth 2 depth 3
Per
cen
tag
e
FAST project findings (6)
Significant weaknesses in current practice:• Too much emphasis on justifying the grade• Lack of shared understanding of assessment criteria
(students & CT)• ALs good at articulating students’ weaknesses;
explaining strengths problematic• Lack of holistic assessment of students’ work
My project
• Replicate some of FAST investigations in Technology – results similar?– Courses chosen: T173, T209 & T224– Analysis of feedback on sample TMAs– Telephone interviews with students
• Follow-up action with ALs, monitors, CTs, STs as appropriate – and review
Progress so far
• Feedback analysed on all three courses• Some results circulated to T209 & T224 ALs• Some comparisons with Science made• Telephone interviews conducted on T209 & T224 –
T173 next month• Some analysis of telephone feedback done – awaiting
T173 to finalise
Category percentages compared – Technology & Science
0
10
20
30
40
50
60
Tech
Science
Category percentages compared – Technology courses & Science
0
10
20
30
40
50
60
Perc
en
tag
e T173
T209
T224
Science
Category percentages compared – Technology (not T209) & Science
0
10
20
30
40
50
60
T173 & T224
Science
Type of content comment compared – Technology & Science
0
10
20
30
40
50
60
70
errors omissions irrelevancies clarifications
Per
cen
tag
e
Tech
Science
Type of content comment compared – Technology courses & Science
0
10
20
30
40
50
60
70
errors omissions irrelevancies clarifications
Per
cen
tag
e T173
T209
T224
Science
Type of content commentcompared within T173
T173 Questions 1 & 4
0
0.5
1
1.5
2
2.5
errors omissions irrelevancies clarifications
no
. p
er s
tud
ent
Qn1
Qn 4
Depth percentages compared – Technology & Science
0
10
20
30
40
50
60
70
80
Tech Science
Per
cen
atg
e depth 1
depth 2
depth 3
Depth percentages compared – Technology courses & Science
0
10
20
30
40
50
60
70
80
90
T173 T209 T224 Science
Per
cen
tag
e depth 1
depth 2
depth 3
Further comparison of depths
0%
20%
40%
60%
80%
100%
Per
cen
tag
e
depth 3
depth 2
depth 1
Number of comments per student
Mean Median Lowest Highest
T173 29 26 4 56
T209 36 35 9 (twice) 81
T224 17 15 8 (twice) 48
Science 39
Action taken on T209 and T224
• Document ‘Using TMA comments to good effect’ prepared for each course– Explains ideas of ‘depth’ and ‘feed forward’– Contains course-specific examples of depth 2 & depth
3 comments• Sent out in first tutor mailing (2006) with commendation
from Course Chair• Monitors briefed accordingly
Telephone surveys
• T209 and T224 complete – carried out immediately after end of course
• Examined– student’s perception of usefulness/helpfulness of
feedback– whether (& how) student had used the feedback in
any future TMA / the ECA / the exam– student’s preferences regarding placing of comments:
on PT3, script, (pro-forma)
Preliminary findings (1)
• Students are eager to receive their marked TMAs and do read the feedback …
• … but they do not necessarily use the feedback again in the rest of the course (approx 20% said they never used it)
• T209 students were likely to make more use of the feedback later in the course than T224 students
• T209 students particularly mentioned using skills development feedback
Preliminary findings (2)
• Not all students found that tackling the TMA, and the feedback they subsequently received, encouraged them to study the rest of the course
• A small number of students said they were disappointed with the quality of the feedback (but some were surprisingly accepting)
Preliminary findings (3)
• Overwhelming majority of students value comments on the script the most:– ‘where lost marks made clear’– ‘tells me exactly where the mistake is’– ‘very specific’– ‘more evidenced against actual text’– ‘easier seeing my work with comments relating to it’
Aside about feedback and eTMAs
• We can’t assume that students read the PT3 first – or even at all
• We can’t assume that students find and read a separate marking document sent back with the marked TMA
• Some (most?) students find juggling documents on the screen awkward/difficult
• It’s easy for tutors to place comments exactly where they apply
• Turn-around times longer: symptom of a problem?
Action taken on T209 and T224
• T209 and T224:Students reminded to look for and read the PT3
• T209:Separate pro-forma dropped for eTMAs; tutors asked to copy and paste grids at end of questions
FAST project findings in Technology?
Significant weaknesses in current practice:
• Too much emphasis on justifying the grade
• Lack of shared understanding of assessment criteria (students & CT) varied
• ALs good at articulating students’ weaknesses; explaining strengths problematic
• Lack of holistic assessment of students’ work
Good feedback in Technology?
• Quantity and timing of feedback– Sufficient feedback is provided, often enough and in enough detail In
general– The feedback is provided quickly enough to be useful to students In general
• Quality of feedback– Feedback focuses on students’ performance and learning Too much biased
towards performance on this TMA– Feedback is appropriate to the purpose of the assignment and to its criteria
Purpose and criteria often implicit– Feedback is appropriate in relation to students’ understanding of what they
are supposed to be doing Sometimes, but more explanations would be helpful
• Student response to feedback– Feedback is received by students, and attended to Yes– Feedback is acted upon by the student Not sufficiently
Implications for Course Teams
• Different sorts of questions and criteria elicit different types of feedback
• (or) To elicit particular feedback, write the question and criteria accordingly – maybe even write the course material accordingly
More implications for CTs
• Be explicit with ourselves and with ALs and students about what an assignment’s purpose and criteria are
• Don’t assume that ALs will instinctively know what sort of feedback we’re hoping for – be explicit in the marking guide (or elsewhere)
Implications for Associate Lecturers
• Need for shift towards emphasis on supporting student’s learning and progress through course, rather than just explaining what was wrong in this particular TMA
• That implies more student-centred feedback – and more holistic feedback on PT3s
• It may also imply giving as much (more?) emphasis to feedback as to marks
Implications for monitors
• Need to shift emphasis from ‘Was the mark OK?’ towards ‘Was the feedback OK?’
• Need to encourage appropriate forms of feedback (and discourage non-appropriate ones?)
References
• FAST presentations given at Open University, 10 February 2005
• Gibbs, G & Simpson, C (2004–5) ‘Conditions under which assessment supports students’ learning’ Learning and teaching in higher education 1(1) pp 3–31; available via http://www.open.ac.uk/science/fdtl/pub.htm
Centre for Open Learning of Mathematics, Science Computing and Technology (COLMSCT) The Open UniversityWalton HallMilton KeynesMK7 6AA
http://cetl.open.ac.uk/colmsct