The use of diagnostic software in teaching a mathematics module for computer science students

Neil GordonDepartment of Computer Science

University of Hull, Hull HU6 7RX England

n.a.gordon@hull.ac.ukhttp://www.hull.ac.uk/php/cssnag/

Brief plan to the talk

Identify the need for mathematics and formalism in computing

Establish the basis of the problem in pre-university mathematics that is creating an issue for computing departments

Consider one approach to dealing with this based on using diagnostic formative assessment to drive student learning

Introduction

The close relationship between mathematics and computing as disciplines is well known.

Recent changes in English mathematics teaching and assessment, combined with a decline in the basic mathematical skills of students arriving at universities is leading to growing difficulties for computer science.

Whilst focussing on the situation in English H.E., much of this is relevant in a wider context e.g. the problem has been identified internationally over several years, for example in the U.S.

The role of mathematics in computing

Mathematics is naturally occurring in science subjects, especially computing. The role of mathematics as a key tool has been noted over the years.

Regarding computing, mathematics is identified in the subject benchmark, and is specified by many professional accrediting bodies (e.g. the BCS).

Successful teaching of mathematics for computing requires that students are able to cope with the language and methods of various mathematics topics.

Hence the joint ICS/MSOR meeting on “Mathematics for Computing”

Admissions requirements

Historically many computing departments required A-level Mathematics for entry

This is no longer the case, with a wide variety of admissions requirements (see ICS network survey results)

However, the perception of mathematics as an indicator of computing ability persists

Weak mathematics = weak computing?

A particular mathematics topic which seems to have suffered in recent times is basic algebraic manipulation - which is crucial to computing.

Evidence of the importance of these skills in computing is provided by the identification of Mathematics and Formalism Education as one of the grand challenges facing computing at the 2004 U.K. Grand Challenges in computing meeting.

The Gap between university expectations and students’ maths skills

staff in Higher Education departments that rely on students having mathematical skills have been identifying problems with students’ grasp and fluency in basic maths

Topics such as algebra, trigonometry and basic mathematical manipulation have seemed to be more and more problematic for students entering Higher Education.

At Hull we have used diagnostic testing to assess these skills for incoming students. Analysis of this over the last four years, has revealed a measurable decline

Declining maths skills?

0%

10%

20%

30%

40%

50%

60%

70%

2000 2001 2002 2003

Entire cohort

GCSE

AS levels

A levels

Average overall mark for new students on our mathematics diagnostic test.

Interpreting these changes

These results do not mean students are less able mathematically

However, they identify a growing mismatch between University expectations and requirements and students own knowledge and skills

N.B. Mathematics allows objective measurements of this discrepancy – measured here using a diagnostic computer package – Diagnosys

Mathematics Difficulties

These are caused by a number of distinct isuses Widening participation larger cohort sizes pre-university mathematics experience - particularly for students

who have only done GCSE mathematics - can lead to mathematical illiteracy [a lack of familiarity with the notations and key methods of mathematics]

“Mathphobia” i.e. the fear of mathematics. [can lead to mathematical illiteracy, or possibly be a consequence of it]

Dyscalculia - is used for those who have a difficulty with mathematics due to a learning disability similar to dyslexia [also known as developmental arithmetic disorder, and affects up to 6% of children]. Dyscalculia is also a possible cause of mathphobia

Diagnostic testing in practice

Overall average grades in this test are now down to 37% students who have no advanced mathematics grades getting an

average of only 19% since 2000, results for students with A-level mathematics have

improved slightly, but they take longer to complete the test For all other groups of students, attainment in the test is

declining. This may be explained by the fact that the choices provided via

AS-levels means that students who are weaker at mathematics have the option to drop it before A-level.

The effectiveness of the diagnostic test can be seen by the general correlation between diagnostic score and incoming mathematics grades

Scatter plot of incoming mathematics grades against diagnostic grade (scaled between 0 to 20 – where 20 is A in GCSE mathematics, and A in A-level mathematics, 10 would be GCSE and AS maths)

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0 5 10 15 20

Incoming Maths grade

Dia

gn

osti

c R

esu

lts a

t in

du

cti

on

Diagnostic test with Diagnosys

A computer based mathematics diagnostic environment. Uses an adaptive skills net to test skills efficiently and

quickly Provides group profiles, with overall grades etc. and

number of students able to demonstrate various mathematical skills

Supported by the Study Advices Services See http://www.staff.ncl.ac.uk/john.appleby/diagpage/diagindx.htm

Use of diagnostic results

Diagnostic tests such as this really require post test support.

Evidence indicates that immediate remedial support can help, but upon removal grades generally again slip

Diagnosys gives profiles of individual students (which can be returned to them via tutors with supporting advice)

Also provides profiles on the entire class – allowing for alteration of lecture material etc. and inclusion of items where there were common problems.

Diagnostic tests of Hull computing students: 2000-2003

Qualifications Year 2000 2001 2002 2003

Entire cohort Mark 43% 45% 41% 37%

Average time taken 44 minutes 45 minutes 43 minutes 49 minutes

GCSE-only Mark 24% 22% 21% 19%

Average time taken 37 minutes 39 minutes 31 minutes 44 minutes

AS-level Mark 61%♠ 46% 46% 43%

Average time taken 47 minutes 52 minutes 52 minutes 54 minutes

A-level Mark 57% 57% 57% 59%

Average time taken 47 minutes 49 minutes 51 minutes 53 minutes

♠Note: the AS mathematics in 2000 was very different to the new AS levels introduced in 2001.

Skills against percentage able to do them at induction

Diagnosys Results: 2003 CS intake

0

20

40

60

80

100

120

Topics tested

% s

uccessfu

l

2: successful ontopic (total)

3: qual{A-level}

4: qual{AS-level}

6: qual{GCSE}

7: qual{Vocational(eg BTec GNVQ)}

9: qual{Accesscourse/FoundationYear}

A framework for supporting mathematics learning

Pre-module diagnostic test AND appropriate action and support within lectures

Lectures (informed by diagnostic results) and closely linked with the main subject

Workshops – with formative assignments to develop skills Online support materials

> Lecture notes> Interactive worksheets (MathCAD)> Links to other support sites

Liaison and work with Study Advice Services> Worksheets/special support booklet> Organised

Summative coursework to assess progress and encourage students engagement with material

Final exam to assess learning outcomes

The framework in practice

Applied to a first year (level 4) quantitative methods for computing module

Taken by students with no advanced mathematics (so potentially GCSE C)

supports a range of modules and degrees in computing, ranging from databases in IE, to the formalizations required in SE.

Subjects include: set theory; relations and functions; logic; algebra; trigonometry; finite state machines; vectors and matrices.

Teaching on the module

Content influenced by results of diagnostic test

Include relevant applications of notions in computing context – essential for students who are less “secure” in their maths

Use workshops to provide practice (like programming, you need to learn by application)

Computer resources

Diagnosys test (available for students to retake)

Module website (lecture notes etc) Interactive (MathCAD) worksheets to allow

interaction with live mathematics Links to external support sites (e.g. mathtutor) Usual dept. support – email/forums/study

advice resources

Example MathCAD worksheet. Students are encouraged to explore and experiment with the mathematics

Assessment Program

assessment designed to differentiate between abilities - to provide a challenge for those already skilled and able in mathematics, as well as encouraging and be accessible to those who have less developed mathematical skills.

Weekly formative assessment covers the main concepts met in lectures, with supporting workshops

model solutions released week after, allowing more chance for feedback in that weeks workshops.

module has been run using 3 summative assignments, designed to encourage students to engage with the material. 2 as exercise sheets, 1 as a class test

Final end of module exam (60% of module)

Evidence of success?

There are a variety of indicators of success: Negative indicators include: poor attendance at workshops; take up

of the formative worksheets is low; attendance at the extra-departmental workshops became so low that these were stopped; algebraic and numerical skills in concurrent modules showed concerning gaps in students grasp of basic mathematical techniques.

Positive indicators include student feedback that many enjoy the material. In fact, several asked about studying more mathematics outside of the module.

Exam results mirror other modules in department and shows good student achievement with acceptable pass rates and average for the module being around 50% module mark, with 80% of the class passing.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

0.00% 20.00% 40.00% 60.00% 80.00% 100.00%

Module Mark

Dia

gn

osti

c R

esu

lt

Diagnostic Test

-Scatter plot of diagnostic score at entry against module mark -shows that there is little correlation between incoming maths skills and final marks in this module -indicates that the module is successful in providing students with sufficient maths to overcome any initial barriers to success in their computer science.

Conclusions: the way ahead?

We have considered a framework for supporting students in studying mathematics for computing.

Along with identification of the wider context and problems that affect the learning of the subject, we have some possible approaches to help students

Further development of a suitable framework include > embedding the diagnostic test as a formal part of assessment> Encourage students to attend workshops by embedding coursework into

weekly worksheets> Formal evaluation of the approach

Since mathematics skills are crucial to a complete computing education, teaching these to modern computing students is an important but continually changing task.