HEA 2017 Conference:

Exploring Innovations in

Assessment with Statistical

and Data Analytical

Software Packages

Kathy Maitland

Peter Samuels

6th

July 2017

Background

Increasing importance of data analysis in the workplace (Cheng, 2014; Manyinka et al., 2011)

Increasing use of powerful tools – statistical analysis, data analytics and data mining

Students are not being properly prepared for the workplace:

Using analytical tools is still seen as cheating

Lack of conceptual understanding when using tools

Students often thrown in the deep end with their dissertations and projects

No benchmarking of skill development

Aim

To give participants the opportunity to create

teaching and assessment plans including

benchmarking and online assessment for

different subject areas which include

developing analytical skills with popular

software packages.

Objectives

1. To gain practical experience in combining and

embedding software skills into academic

modules and associated assessments

2. To benchmark skills assessments (both subject-

based and software-based)

3. To explore the feasibility of online assessment

4. To share best practice in incorporating key

software competency skills into taught

academic programmes

Statistical analysis, data

analytics and data mining

Subject Statistical

analysis Data

analytics Data mining

Initial research

question Yes -

specific

Yes -

general No

Systematic pattern

recognition No No Yes

Descriptives Yes Yes Yes

Hypothesis testing Yes Yes Yes

Prediction Possibly Yes Yes

Example software Excel and

SPSS

SAS and

JMP

SAS Enterprise

Miner

Online testing for data analysis?

Online testing of data analysis capability is in its infancy (Czaplewski, 2014)

One promising system is DEWIS from UWE (http://dewisprod.uwe.ac.uk/)

This has the capability of generating randomised data sets using R, numerical input and feedback. Example: http://www.cems.uwe.ac.uk/dewis/welcome/showcase/r_question/stage_two.pdf

Another system is Numbas from Newcastle University (https://www.numbas.org.uk/)

Example questions are available from http://www.statstutor.ac.uk/search/?q=numbas

Both systems are free and allow formative and summative assessment

Sample Dataset:

CensusAtSchool

Questionnaire

Completed by 673 school children

No specific research question but one could be imagined

Hence suitable for both statistical analysis and data mining

Example generic activities

Descriptive statistics using Excel:

Tables, bar charts, pie charts, percentage frequency bar charts, histograms, summary statistics and scatter plots

Inferential statistics using Excel:

Correlation and linear regression

Descriptive statistics using SPSS:

Cross-tabulation, box and whisker plots, error bar charts, histograms with fitted normal curves

Inferential statistics using SPSS:

Assumption checking, independent t-test, Mann-Whitney U test

Data analytics SAS JMP example

The Task

Using descriptive statistics and graphical displays, explore claim payment amounts, and identify factors that appear to influence the amount of the payment.

The Data (MedicalMalpractice.jmp)

The data set contains information about the last 118 claim payments made, covering a six month period. With eight variables:

Amount: Amount of the claim payment in dollars

Severity: The severity rating of damage to the patient, from 1 (emotional trauma) to 9 (death)

Age: Age of the claimant in years

Private Attorney: Whether the claimant was represented by a private attorney

Marital Status: Marital status of the claimant

Specialty: Speciality of the physician involved in the lawsuit

Insurance: Type of medical insurance carried by the patient

Gender: Patient gender

Data analytics SAS JMP example

Analysis: Begin by looking at the key variable of

interest, the amount of claim payment: histogram

and summary statistics for Amount.

(Analyze > Distribution; Select Amount as Y, Columns, and click OK. For a

horizontal layout select Stack under the top red)

Benchmarking

Subject benchmarking – a commonly practice

in course development – possibly driven by

professional bodies

Data analysis software benchmarking – less

common – often at a lower level than subject

benchmarking

Benchmarking assessments using data analysis

software requires both

Benchmarking example –

Computer Science

Sources:

BCS core requirements for accreditation of honours programmes

Additional requirements for CITP

BCS Computing-related practical abilities

Royal Statistical Society Core Knowledge and Skills

SAS Certified Base Programmer for SAS®9

Quality Assurance Agency (QAA) levels (2014)

Examples of blended

assessments: Practical Exam

Questions on academic theory

Multiple choice of similar style to vendor

certification that represents academic

theory in practice

Examples contd.: Case Study

Academic theory in practice through the use of

a case study

Students provide an industrial strength report

using the case study as the vehicle for

assessment

Competency of tool demonstrated in the

production of statistics and charts

Reflection of real world tasks at the appropriate

QAA level

Example: Exploratory

Analysis

Students are required to explore a particular area of interest through data analytics

Locate or obtain appropriate data set through experiment or open source data

Students are required to demonstrate a variety of competencies through the use of data analytics tools

Results are presented through a poster or report

Activity

30 minutes:

Break into small groups of your choosing

Choose a subject area

Choose appropriate software: e.g. data analytics using

SAS / JMP, or statistical modelling using Excel / SPSS,

or data mining using Enterprise Miner / Watson Analytics

Based on the generic examples provided, develop an

approach to teaching and assessment using software for

your subject area for Levels 4 to 6

Explore the use of benchmarking standards and online

assessment

Report back

Conclusions

Future developments of effective teaching and

assessment techniques with data analysis

software packages in different STEM subjects

Thank you for your participation .

If you would like to contact us, especially if you are interested in working on a joint paper on this

subject, please email us at:

kathleen.maitland@bcu.ac.uk

peter.samuels@bcu.ac.uk

References

Chen, J. (2014) Big data for development in China. New York: UNDP.

http://www.cn.undp.org/content/dam/china/docs/Publications/UND

P%20Working%20Paper_Big%20Data%20for%20Development%2

0in%20China_Nov%202014.pdf.

Czaplewski, J. R. (2014) An evaluation of blended instruction in terms of

knowledge acquisition and attitude in an introductory mathematics

course. Doctoral dissertation. University of Minnesota.

Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C.

and Byers, A. (2011) Big data: The next frontier for innovation,

competition, and productivity. McKinsey & Company.

http://www.mckinsey.com/Insights/MGI/Research/Technology_and

_Innovation/Big_data_The_next_frontier_for_innovation.

The Quality Assurance Agency for Higher Education (QAA) (2014) The

UK Quality Code for Higher Education.

http://www.qaa.ac.uk/assuring-standards-and-quality/the-quality-

code/subject-benchmark-statements.