1

Using Computer Technology to Enhance the Teaching & Learning of Engineering Mathematics

MARTIN HARRISON, ARUNA PALIPANA, DAVID PIDCOCK and JOE WARD

The Mathematics Education Centre, Loughborough University, Loughborough, LE11 3TU, UK Email:helm@lboro.ac.u k

The HELM (Helping Engineers Learn Mathematics) project was a major three-year curriculum development project undertaken by a consortium of five UK universities and sponsored by UK government funding from October 2002-September 2005. It used the expertise from consortium partners and computer technology to enhance the mathematical education of engineering undergraduates through the development of a range of flexible learning resources in the form of Workbooks and web delivered interactive courseware elements together with an integrated web-delivered CAA implementation. This paper first describes the HELM learning resources. These consist of Workbooks, Computer-Aided Learning (CAL) courseware and Computer-Aided Assessments (CAA). As well as covering the mathematics essential for engineering undergraduates in the first two years of their degrees, the 50 Workbooks include engineering examples and case studies, a Student’s Guide and a Tutor’s Guide. The CAL courseware, consisting of on-line interactive lessons to aid understanding, is web-delivered and based on many of the more elementary Workbooks. An extensive CAA regime, which facilitates the regular testing of large numbers of students, is used to drive student learning. It takes two forms, either an integrated web-delivered version or an alternative stand-alone CD-based version. Its implementation and its use for both formative and summative assessment of engineering students learning mathematics are outlined. The CAA regime powerfully encourages students to engage more in their own learning and has been essential to the success of the project. Finally the viability of adopting the HELM learning resources and implementing the CAA regime at other institutions is briefly examined. Use of these resources can have a positive impact on the learning experience of students. Key Words curriculum development; mathematics teaching; learning technology; Computer Aided Assessment. 1. Introduction Engineering undergraduates need mathematical skills in order to progress and succeed. Yet there are well-documented challenges, particularly in respect of (lack of) mathematical skills and knowledge, with which many UK universities are faced [1]. Loughborough University’s Mathematics Education Centre (MEC) [2] has been a key player in several high-profile externally-funded national projects which attempt to address some of these challenges. Progress on all of these has been significant and they have helped to raise the profile of the Centre and the University as a UK leader in the mathematical education and mathematics support of non-specialists. In 2005 the MEC achieved Centre for Excellence status [3]. The HELM (Helping Engineers Learn Mathematics) project [4] is one of a number of major projects with which MEC staff have been very closely involved. It was a three-year curriculum development project, led by Loughborough and undertaken by a consortium of five UK universities from October 2002-September 2005. Its resources were launched at a major teaching and learning conference held at the University in September 2005 [5].

2

The HELM project has developed a strategy for addressing some of the practical difficulties currently encountered in the teaching of mathematics to UK engineering undergraduates. This comprises the production and dissemination of high quality extensive teaching and learning materials, complemented by web-delivered CAL (Computer Aided Learning) courseware and supported by a comprehensive Computer Aided Assessment (CAA) testing regime. Some of the developmental work undertaken during the first two years of the project 2002-04 has been reported previously [6, 7]; this paper overviews the completed project and describes how the HELM project team have used computer technology to enhance the teaching and learning of engineering mathematics.

2. An overview of the HELM project

The HELM team consisted of staff at five UK universities: Loughborough (the lead institution), Hull, Manchester, Reading and Sunderland. The HELM project’s output consists of Workbooks, CAL courseware and a CAA regime which is used to help drive the student learning. Sample materials can be seen at the HELM website at http://helm.lboro.ac.uk. With the emphasis on flexibility, the Workbooks may be integrated into existing engineering degree courses either by selecting isolated stand-alone units to complement other materials provided or by creating a complete scheme of work for a semester, a year or two years, by selecting from the large set of Workbooks available. The Workbooks may be used to support lectures or for independent learning, or a mixture. CAL courseware segments provide online interactive lessons for topics covered in typical UK first year undergraduate engineering mathematics courses, covered by the first 20 workbooks. The banks of CAA questions developed by the HELM project contain around 5000 questions. 3. The HELM project workbooks The main student learning resources are the HELM Workbooks and accompanying Guides, which are available in electronic format. A full list is given in Appendix 1 and samples are available via the HELM website. There are 46 high quality Student Workbooks written specifically for the typical engineering student with mathematical and statistical topics, worked examples, exercises and related engineering examples. Workbooks are subdivided typically into four or five manageable Sections. Each Section begins with statements of prerequisites and desired learning outcomes. As far as possible, each Section is designed to be a self-contained piece of work that can be attempted by the student in a few hours. In general, a whole Workbook represents about two to three weeks’ study at university. A Section consists of an introduction and the presentation of mathematical concepts, simply explained, interspersed with examples and tasks. Although the examples are mostly purely mathematical, a significant effort was made to ensure that the Workbooks were student focused and contained some examples of engineering applications of the mathematics. A fully worked solution is provided immediately following each example. Each Section includes student tasks which include space for students to write their working and answers, and, where appropriate, guide them through problems in stages. Additional exercises are included, usually at the ends of Sections, with answers but not usually worked solutions. Workbook 47 contains a miscellany of Mathematics and Physics related problems and Workbook 48 covers more in depth Engineering Case Studies from various engineering contexts and disciplines. Both can be used to obtain supplementary material to help motivate

3

the learning of mathematics for engineering students. A Student’s Guide provides advice, commentary, formulae and a comprehensive index. A Tutor’s Guide provides commentary on each Workbook and associated CAL and CAA resources. This focuses on the modes of pedagogic usage and implementation strategies of the HELM learning resources based on experience from the consortium and triallists. 4. The HELM project CAL materials The project has 77 CAL courseware segments developed using Macromedia Authorware [8] which consist of online interactive lessons linked to the more elementary Workbooks. These typically cover Year 1 mathematics of a UK undergraduate engineering degree. They are web–based versions of small parts of the Workbooks containing interactivity, audio and animations, which can enhance student interest. Revision exercises with randomly generated questions are provided for the benefit of students working independently. These CAL segments have been found to be especially useful for supporting students of moderate mathematical ability, and for revision. They can be used for independent learning allowing students to work at their own pace. They are also useful for illustrating lectures. 5. The HELM project assessment regime 5.1 CAA Assessment is normally an integral part of learning, and CAA is one means of encouraging students to manage their own learning. The HELM assessment strategy uses CAA to encourage formative self-assessment, which many students neglect, to verify that the appropriate skills have been learned. Our philosophy is that assessment should be at the core of any learning and teaching strategy and CAA is both a convenient and a practical way to do this, especially with large numbers of students, and for promoting self-learning. HELM provides an integrated web-delivered CAA regime, for both self-testing and formal assessment, which is fully described in [6] and [7]. Briefly, students are typically tested four or five times each semester with questions delivered over the web. They are encouraged to engage in their own learning by allowing them unlimited practice (formative) tests before taking a one-attempt summative test. At Loughborough, each summative test is typically worth from 5% to 10% of the module mark; this ‘carrot’ motivates students to keep up with their studies, thereby improving achievement and progression. QuestionMark Perception (QMP) [9] is used at Loughborough University to deliver tests over the web; this is integrated with a central database so that students can submit their completed tests for processing and aggregation of marks. An alternative implementation based on CDs has also been developed so that students without an internet connection can still do the required work and complete the formative tests. There are around 5000 HELM CAA questions designed to match particular mathematical concepts in support of the topics covered by the Workbooks. The questions relevant to each mathematical (or statistical) concept have been structured into two sets, one nominally designated formative and the other summative. In most cases, each of the two sets contains 10 questions cloned from a designated master question, thereby ensuring a comparable level of difficulty is maintained; questions are selected randomly from each set for test purposes. A

4

complete test can be set up by choosing the relevant Workbook topics and then choosing appropriate questions chosen randomly from the relevant set of question banks (libraries). It is interesting to consider how many tests on average, a student needs to take before all the questions have appeared at least once. It has been considered in detail in [10]. It is not as many as you might think and the results for the case of a bank of 10 questions appear in Table 1.

Table with a bank of 10 alternatives for each question No. of test questions 1 2 3 4 5 No. of tests required 29 35 39 41 43

Table 1: Average number of tests required for all questions in a bank (library) to have appeared at least once. Question feedback is an important pedagogical aid, which drives student learning, and is provided as an option in both formative and summative tests. Ideally this will be the specific worked solution, but sometimes an example solution or a generic solution is provided instead. This is particularly popular with students. 5.2 HELM CAA question types HELM uses a number of different question types: multiple choice questions (MCQs), single numeric input questions (the majority), multiple numeric input questions and staged questions. MCQs and single input questions are straightforward to construct. A single numeric input question is also simple to construct and allows for the easy generation of clones with which to populate the relevant question bank (library). However, there are disadvantages too. Students may understand and complete a question correctly but then fail to round their numeric answer correctly and, as a consequence, receive no marks for that question. This proved to be a common occurrence so we now allow for numeric input responses within a given tolerance where rounding is required, for example, ±0.01 for questions requiring 2 d.p. accuracy, and give full or partial marks accordingly. Where marks are reduced, this is indicated in the feedback. Single numeric input questions may be inappropriate in other situations. For example, in the case of finding the roots of a quadratic equation, it may be useful to mark individually for each root which means allowing for the entry of two separate numbers, in either order, as the answer as shown in Figure 1, which requires two inputs. This is far better than using an MCQ approach, simply listing a number of alternatives. It also eliminates guesswork. With multi-input questions computerised marking is clearly more complicated and the complexity increases dramatically as the number of inputs increases. Automated marking conditions need to mark the answers collectively but be flexible enough to allow students to input the answers in the input boxes in any order, as in the example in Figure 1. A clear disadvantage inherent in single numeric input questions is that a wrong response does not earn credit for any correct working done by learners prior to submission of their answer. To address this situation we have investigated the use of questions in a multi-stage format, whereby partial credit is given for a correct response at each of several stages within a question (see example in Appendix 2). The learner, even after submitting an incorrect answer at an intermediate stage, is subsequently presented with sufficient information at the commencement of the next stage to allow him or her to continue, thus giving the opportunity to gain partial credit.

5

FIGURE 1. HELM CAA QUESTION REQUIRING TWO NUMERIC INPUTS (IN EITHER ORDER) 5.3 Assessment Typically students are given a Workbook for a mathematics topic for two weeks’ work. followed by a summative CAA test in the following week. They can access a practice version of this test on the web during the week before the summative test, at any time they wish, as often as they want and from wherever they have internet access. Server logs show that most students access the practice test a number of times demonstrating their formative use of it. Student feedback confirms their enthusiasm with this part of the assessment and their engagement with the learning process. The summative test is then available for two days only. Students may access it at anytime within this period and from anywhere but they are only allowed one attempt. The formative and summative tests have identical form, questions being selected randomly from previously created question banks (one formative, one summative) covering the relevant Workbook topic. Evaluation exercises have shown that students like this testing regime. They particularly like its flexibility and the immediate feedback on the topics. They dislike not getting marks for partially correct answers when numeric single input questions are used, but this issue can be addressed by allowing some tolerance in responses to numeric input questions. A staff concern is allowing students to undertake the formal tests unsupervised, but this is an issue with many types of coursework.

6

6. Trialling and evaluation Our early trialling experiences have been described in [6]. By September 2005, over sixty academics from over 40 UK higher and further education institutions had been involved in the development and evaluation of the resources; universities in Australia, Denmark, Germany, Malaysia, Netherlands, Phillipines and the USA have also expressed interest. This level of interest is clear evidence of the need for the type of learning resources that the HELM project has successfully developed. The resources were put to varying modes of usage. Around 20% of lecturers used the Workbooks as core notes with half of these implementing the HELM CAA regime for both formative and summative testing. Interest in CAA was significant but take-up was limited by the availability or otherwise of QMP linked to an appropriate virtual learning environment (VLE). Of those students who used CAA, the overwhelming majority liked its formative nature, particularly the facilty to do practice tests whenever and wherever they want, provided they have web access, and the immediate feedback. Around 50% of lecturers used the resources as supplementary materials linked to their own lecture materials. And the remaining 30% used them as support materials, lodged on their VLE or in a Support Centre, for students having difficulties coping with their mathematics. It is worth noting that a number of mature students, and also some special needs students, found the HELM resources particularly useful for independent learning. During 2005-06, the HELM team worked with six further UK universities, Leicester, Newcastle, Nottingham, Oxford Brookes, Portsmouth and Salford, to encourage further the effective transfer of practice across institutions. Our principal aims were to:

• convert some HEIs who have been involved with trialling and some newly identified potential HEI users into long-term users of HELM materials.

• monitor and support the HEIs using the HELM project deliverables in different pedagogic ways.

• evaluate the difficulties, successes and failures in transferring HELM to these institutions and to produce a report that will enhance the existing Tutor’s Guide to further aid transferability.

Full details of this work will be published in due course. 7. Summary and conclusions There is a clear need for more flexible mathematics learning resources due, in particular, to the increasing diversity of university intake standards. HELM provides a solution through: Workbooks available in web-based or paper-based forms; web-based CAL segments; a CAA assessment regime which enables regular testing to drive the student’s learning. The large question bank, which can be used for formative and summative assessments, is available either web-delivered or on CD. The HELM Workbooks encourage student engagement during lectures; the HELM Interactive Lessons complement the workbooks and aid understanding; the HELM CAA, with flexible access via web delivery, facilitates regular testing of large numbers of students, provides instant feedback and incorporates both formative and summative testing which helps drive student learning.

7

The learning resources can be used in different pedagogic ways and are popular with staff and students alike. 8. Acknowledgement The Higher Education Funding Council for England (HEFCE) for support through the Fund for the Development of Teaching and Learning phase 4 (FDTL4).

References [1] Hawkes, T., & Savage, M.D, (2000) Measuring the Mathematics Problem. London, The Engineering Council [2] Mathematics Education Centre, http://mec.lboro.ac.uk/ (accessed 4 July 2006) [3] Centre for Excellence in Mathematics & Statistics Support, http://www.sigma-cetl.ac.uk/ (accessed 4 July 2006) [4] HELM Project: http://helm.lboro.ac.uk/ (accessed 4 July 2006) [5] HELM conference: http://www.engsc.ac.uk/nef/events/helmconf.asp (accessed 4 July 2006) [6] Davis, L. E., Harrison, M.C., Palipana, A.S. & Ward, J. P., 'Computer-Based Mathematics Assessment of Engineering Students', Procedings of TIME2004, ISBN 3-901769-59-5, presented at Conference on Technology and its Integration in Mathematics Education, July 15-18, 2004, Montreal, Canada. [7] Green, D.R., Harrison, M.C., Palipana, A.S., Pidcock, D.L., and Ward, J.P. , ''Supporting mathematics learning of engineering students: the HELM project'' , Delta '05, Fifth Southern Symposium on Undergraduate Mathematics and Statistics Teaching and Learning , Queensland, Australia, November 2005, pp 191-203, ISBN: 1-86499-840-7. [8] Macromedia Authorware, http://www.adobe.com/products/authorware/ (accessed 7 July 2006) [9] QuestionMark, http://www.qmark.com/ (accessed 4 July 2006) [10] Cornish, R., Goldie, C. & Robinson, C., 2006, CAA: How many questions is enough? HEA Maths CAA Series: Feb 2006, http://mathstore.ac.uk/articles/maths-caa-series/feb2006/, (accessed 6 July 2006).

8

Appendix 1

The HELM Workbooks

No. Title No. Title 1 Basic Algebra 26 Functions of a Complex

Variable 2 Basic Functions 27 Multiple Integration 3 Equations, Inequalities and Partial Fractions 28 Differential Vector Calculus 4 Trigonometry 29 Integral Vector Calculus 5 Functions and Modelling 30 Introduction to Numerical

Methods 6 Exponential and Logarithmic Functions 31 Numerical Methods of

Approximation 7 Matrices 32 Numerical Initial Value

Problems 8 Matrix Solution of Equations 33 Numerical Boundary Value

Problems 9 Vectors 34 Modelling Motion 10 Complex Numbers 35 Sets and Probability 11 Differentiation 36 Descriptive Statistics 12 Applications of Differentiation 37 Discrete Probability

Distributions 13 Integration 38 Continuous Probability

Distributions 14 Applications of Integration 1 39 The Normal Distribution 15 Applications of Integration 2 40 Sampling Distributions and

Estimation 16 Sequences and Series 41 Hypothesis Testing 17 Conics and Polar Coordinates 42 Goodness of Fit and

Contingency Tables 18 Functions of Several Variables 43 Regression and Correlation 19 Differential Equations 44 Analysis of Variance 20 Laplace Transforms 45 Nonparametric Methods 21 Z Transforms 46 Reliability and Quality Control 22 Eigenvalues and Eigenvectors 47 Mathematics and Physics

Miscellany 23 Fourier Series 48 Engineering Case Studies 24 Fourier Transforms 49 Student’s Guide 25 Partial Differential Equations 50 Tutor's Guide

9

Appendix 2

Example of a HELM Multi-stage Question Objective: To apply a hypothesis test assuming an underlying Normal distribution, so as to test a null hypothesis against an alternative hypothesis. Stage 0: A preamble gives the student specific information on answering multi-stage

questions and then the whole question is presented.

This is a multi-stage question. Credit will be given for each correctly completed stage. If you begin the question you must go on to completion. You may not return to a stage after submitting the answer. You may not return to the question at a later time.

Click on the NEXT button to see the question.

A manufacturer of central heating boilers claims that no more than 30% of boilers of a certain model produced by the firm will require replacement parts in the first five years of operation. To investigate this claim, service records of 150 boilers installed 5 years ago are studied, and it is found that 51 have required replacement parts. Assuming that boilers are independent and that each has a probability p of requiring replacement parts in the first 5 years, test the Null Hypothesis that p = 0.3 against the Alternative Hypothesis that p > 0.3 at the 5% level of significance.

Click on the NEXT button to begin Stage 1.

10

Stage 1: The first part of the question is presented. The correct response is 1.069 and after submitting the answer the student moves to Stage 2. Stage 2: The correct solution for Stage 1 is revealed to the student who now has the task of

finding the appropriate critical value of the standard Normal distribution. The correct response is 1.645 and after submitting the answer the student moves to Stage 3.

STAGE 1 Find the value of the test statistic which has approximately a standard Normal distribution under the Null Hypothesis. Enter your answer, correct to 3 d.p., in the box provided.

Then click SUBMIT.

This stage is worth 2 mark(s)

STAGE 2 The correct answer to Stage 1 was 1.069 Now determine the appropriate critical value of the standard Normal distribution. Enter your answer, correct to 3 d.p., in the box provided.

Then click SUBMIT.

This stage is worth 1 mark(s)

11

Stage 3: The correct solution for Stage 2 is revealed to the student who now has to come to an appropriate conclusion.

This being the last Stage, the question is now completed and the student moves on to the next question.

STAGE 3 The correct answer to Stage 2 was 1.645

On the basis of your results make a statement comparing the Null Hypothesis that p = 0.3 against the Alternative Hypothesis that p > 0.3 at the 5% level of significance.

Consider the options below. A The result is significant at the 5% level. The Null Hypothesis is rejected. The evidence suggests that p > 0.3 B The result is significant at the 5% level. The Null hypothesis is not rejected. The evidence suggests that p < 0.3 C The result is not significant at the 5% level. The Null Hypothesis is rejected. The evidence suggests that p > 0.3 D The result is not significant at the 5% level. The Null Hypothesis is rejected. The evidence suggests that p < 0.3

E The result is not significant at the 5% level. The Null Hypothesis is not rejected. There is insufficient evidence to conclude that p > 0.3 Make your selection below. Then click SUBMIT. Option A is correct.

Option B is correct. Option C is correct. Option D is correct. Option E is correct. This stage is worth 2 mark(s)