Formative Quizzes and their Contribution to the Understanding of Computer Programming by Level One Students

Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and Learning The Open University is incorporated by Royal Charter (RC 000391), an exempt charity in England & Wales and a charity registered in Scotland (SC 038302).

Michael Isherwood

Associate Teaching Fellow, COLMSCT

Presentation Summary

• Project Background• Indicators of Success?• For the Future

Background• Students find getting to grips with programming difficult• This leads to frustration and dropping out of M150 (and

therefore further computing courses)

Background - TMA02/03 Marks

A Solution - iCMAs• A series of Interactive Computer Marked Assessments• Optional but Accessible through M150 Website• But

– Also other initiatives – podcasts and video files– Ian Cooke’s Project– Course Team Initiatives on Assessing Programming

Dropout and TMA03 Skipping

No of Students engaging

% drop out after TMA01

% drop out after TMA02

% skipping TMA03

2008B 1262 9.1 13.4 7.8

2008J 2213 9.8 9.9 2.7

2009B 1486 8.5 9.6 3.8

All Three 4961 9.2 (7.6) 10.7 (10.5) 4.3 (5.3)

Bracketed figures for pre iCMA presentations: 2006B – 2007B

• 2008J and 2009B has iCMAs accessible from M150 Website• Fewer skipping TMA03 than previously (2008B aberrant)• Marginally fewer dropping out after TMA02 • Final figures for completions of 2008J and 2009B not available as resits pending

Logical AND Question

• Student needs to know form of expression and have the correct Boolean operator (AND) as well as the correct conditions (bracketed parts)

• OpenMark identifies each of these components, recognises errors and gives targeted feedback on these before the student has a second attempt

CP2B A steel bar of nominal length 220mm needs to be less than 1mm different from this length. To be within tolerance, its length must therefore be between 219mm and 221mm. Write a compound condition that evaluates to true for being within tolerance, using len for length (in mm).

Free text answer is: (len > 219) AND (len < 221)

Learning on Similar Questions• A series of questions has been set using the logical operators

AND and OR• In Quiz 2 there were two similar questions. The logical operator

was AND in the first case and OR in the second case. (The Qs were of the same format)

• In Quiz 3 there was a similar question requiring selection of either AND or OR

• There was more than described here to each of the questions/% AND OR Other1 62 19 19

2 28 70 2

3 25 74 1

• First question has many still learning the requirement• By third question still improvement

Units in Conditional Statements• The two Quiz 2 Logical Operator questions incorporated

units in the wording of the question. • The units depending on variant were g mm nm and oC• In the first question 10% of students incorporated the

unit and in the second, it was 12% - no improvement!

Learning through Attempts• Students have 3 attempts per question• After each attempt, targeted feedback is given based on

the student’s answer

1st Try 2nd Try 3rd Try Still Wrong

% 32 15 8 45

OR question used earlier

• It is evident when looking at an individual student’s attempts that most learn on their 2nd and 3rd attempts• However this is often not true for those who are wrong after the 3rd attempt

Weak Students – 3 Wrong Attempts• 37% with wrong Boolean operator – of these 75%

correct on 2nd attempt BUT 25% don’t despite pretty explicit feedback

• 13% have all 3 attempts identical and a further 38% have two attempts the same

• 10% had little or no idea with 3 “garbled” attempts and 2% left completed blank

The Student View• Feedback from Students indicates that most find the

quizzes both enjoyable and useful• Many Students also wanted more• There is more on this in Poster Presentation

For the Future• Need many more questions available for students who

need to develop their skills• Needs routes through quizzes that are determined by a

combination of that student’s performance and that student’s choice – differentiated learning

• Tutors need to be able to monitor progress and, if necessary, intervene

• Much more time needed for analyses of student performance and needs

Thank you for Listening