Embed Size (px)
Citation preview
Retrospective review of e-Assessment data to enhance student feedback
Dr D. Rhys [email protected]
Dr Karen L. [email protected]
www.cems.uwe.ac.uk/dewis
Senior Lecturers & UWE L&T FellowsDept. of Engineering Design & MathematicsUniversity of the West of England, Bristol
Retrospective marking
• A second stage of marking at the end of the assessment period.
• Marks typically not altered.• Facilitates:
– Detection of common student errors (CSEs).– Improved targeted feedback to the students,– Evaluation of the e-Assessment,– Improvements to the next run.
Example of a common student error (mal-rule)
Common Student Error Analysis
• Analysis of loss-less data to look for possible CSEs
• Retrospective marking then searches the whole data for all instances of this potential CSE.
• If triggered, this detection is reported to the student and explains why their approach is incorrect.
Overview of the e-Assessment
• 30 credit module: Stress & Dynamics (250 first year students)
• Stress: four e-Assessments worth a total of 12.5%
• Concentrate on the first question of the first e-Assessment here
Randomly generated parameters
Two inputs (closed form decimals)
Work through of the question
Feedback to Question 1:
Current feedback gives a work through of the question
Can we point out where the student has gone wrong?
Searching for CSEs from the e-Assessment data
DEWIS• Loss-less data e-Assessment system• Extensive reporting
Performance flags
-1: not answered 0: incorrect 1: correct
DEWIS reporter (Question 1)
Flag value = 1, question correct
Flag value = 0, question incorrect
First input incorrect,Second input correct
Flag value = -1, question not answered
Total of 360 attemptsOnly a selection shown here
Values for RA and RD entered the wrong
way round
35 students answered both parts of Qn 1 incorrectly.
Only a selection shown here.
Performance Flag search
Common student error 1
• Values for RA and RD entered the wrong way round.
• Introduce a 3rd flag for this question:= 1 if made this mistake
= 0 did not make this mistake
• Re-mark for all students’ attempts
10 students made this error
If ( flag3 == 1 ) then
“We notice that you have entered your reaction forces the wrong way round. Remember that taking moments about A allows you to find the reaction force at D”
Common student error 27 students made this error
Award continuation marks?
The student has RA + RD = F1 + F2 Is this worth 1 mark?
True for 29 out of 35 students
E-Assessment Evaluation/Results(Question 1)
Overview of error Numbers
1. Interchange RA and RD 10 students
2. Incorrect distance for AC used 7 students
3. Used/muddled first attempt data 4 students
4. Wrong value of L used 2 students
5. Rounding error 1 student
6. Forgot to divide by L 1 student
7. Unexplained 10 students
Conclusions
• Loss-less data, algorithmic e-Assessment system facilitates the process.– DEWIS: www.cems.uwe.ac.uk/dewis
• Enhanced feedback is beneficial for all future students taking the tests.
Future work
• Extend to other e-Assessments
• Student views
