An Evaluation of Medical Students’ Responses to Structured Exam

Feedback from Formative E-Assessments

Terese Bird, Educational Designer, Leicester Medical SchoolASME Scientific Meeting 6 Jul 2016

Belfast, Northern Ireland, UK

Leicester Medical School Context

Groupwork Sessions: Learning by Enquiry (Cox et al., 2010)

• Students divided into Belbin groups• Same groups meet for every

module• Groups work on questions in

iPad-workbooks • Answers given the week after or

never• Sound process of thinking

through to the answer is the point• Junior doctors are “expert

guides”

Student study issues• Lecture notes +

workbook “answers” = holy canon of notes on iPad• “Revision” happens

in the weeks before exams• Concern that study

is mainly “remember”

Bloom’s Taxonomy in study

Feedback issues• Feedback is single highest-impact factor in learning

(Rowntree, 1987).• NSS feedback scores• End of term is too late

Our approach:“Little and often” formative assessments via ExamSoft on iPads

ExamSoft £30 / student / year

Instant answers with additional information

Students log into portal to check progress

Research questions• Do students who engage with the regular formative feedback in the GI

unit change their study strategy as a result, and if so in what way(s)? • Do they look at other material they would not have looked at if they hadn’t used the

exams? • What new study techniques, if any, did they employ?

• Do students who engage with regular formative exam feedback improve learning in “thinking levels” associated with higher elements of Bloom’s Taxonomy?

• Does student engagement with regular formative exam feedback enable students’ learning benefits in ways that do not occur when the regular formative exams are not given? • Do students perceive learning benefits enough to engage with structured formative

assessment when it is optional?

Examsoft formatives Gastro-Intestinal Unit Autumn 2015

Exam Type When Content Circumstance

Session-by-session exams

Weekly, from Oct through Dec 2015

Material studied in the past week

Exam done in groupwork class sessions, not under exam conditions

Christmas 1Christmas 2New Year 1New Year 2

During Christmas break Dec ember 2015–Jan 2016

All questions from session-by-session exams, put together and randomised

Four separate exams with the same exam content, offered 4 times throughout Christmas break for self-study whenever students wanted.

Mixed-methods research

N=237 Year 2Gastro-Intestinal Module

How did the formatives change students’ study if at all?• “A mini-revision before the proper revision” – forced earlier study• Uncovered trouble spots which caused them to seek

solutions• Back-and-forth communication = feedback loop• Active study

Thinking and learning on higher levels of Bloom’s Taxonomy• If questions were written higher on Bloom’s,

students learnt higher on Bloom’s – constructive alignment (Biggs, 1996)

• Single Best Answer led to Evaluation and Analysis

• Students reacted with Create to using exams in revision• Created document of info on uncovered trouble spots• Created answers when they tested themselves

without the multiple-choice

“I’m looking for… an explanation for why an answer is the answer. It’s Single Best Answer, there are two that are right, but one’s a better answer.”

Student engagement with formative assessments• 85% of students completed exams in class• 77% of students did at least one outside-of-class exam

• Students engaged despite the fact these exams contributed nothing to their official marks

“It’s not clear that I performed better on summatives because of these exams, as the summatives are integrated, but they helped me to learn GI better.”

Key words used by students describing the formative exams in module evaluation Winter 2016

Recommendations• Learning objectives and assessments should

constructively align “higher on the Bloom’s” (Biggs, 1996)• Explore ways encouraging students to act on feedback

received• Advise students regarding their own study:• Active study• Little and often• Throughout the term• Less dependence on writing “definitive” sets of notes

References• Biggs, J. (1996) ‘Enhancing teaching through constructive alignment’,

Higher Education, [online] Available from: http://edukologija.vdu.lt/en/system/files/ConstrutivismAligment_Biggs_96.pdf.• Cox, B., Calder, M. and Fien, J. (2010) Teaching and Learning for a

Sustainable Future, [online] Available from: http://www.unesco.org/education/tlsf/mods/theme_d/mod23.html.• Rowntree, D. (1987) Assessing Students: How shall we know them?,

Revised Ed. London, Kogan Page.

Questions? Email me t.bird@le.ac.uk @tbirdcymru on Twitter