Data and disadvantaged students - using learning analytics for inclusion

Niall Sclater, ConsultantData and disadvantaged students - using learning analytics for inclusion 27/02/2017

While you wait…

http://tiny.cc/data-form »If you haven’t had the

chance to do so, please take some time to look at the data and disadvantaged students pre-session googleform.

02/05/2023

Data and disabled students 2

02/05/2023

Data and disabled students 3

The net of meanings - 1

02/05/2023

Data and disabled students 4

The net of meanings - 2

In this session we will…

Explore : »Ethical issues »Disability definitions and consequences»Potential scenarios »Your priorities»A real life case study

02/05/2023

Data and disabled students 5

“learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs”SoLAR – Society for Learning Analytics Research

02/05/2023

Data and disabled students 6

Learning Analytics Service Toolkit Community

Jisc Learning Analytics Project

Jisc Learning Analytics 2017

86 issues in 9 groups

Group Name QuestionMain type

Importance Responsibility

2 Consent Adverse impact of opting out on individual

If a student is allowed to opt out of data collection and analysis could this have a negative impact on their academic progress?

Ethical 1 Analytics Committee

7 Action Conflict with study goals

What should a student do if the suggestions are in conflict with their study goals?

Ethical 3 Student

8 Adverse impact

Oversimplification How can institutions avoid overly simplistic metrics and decision making which ignore personal circumstances?

Ethical 1 Educational researcher

Group Name QuestionMain type

Importance Responsibility

2 Consent Adverse impact of opting out on individual

If a student is allowed to opt out of data collection and analysis could this have a negative impact on their academic progress?

Ethical 1 Analytics Committee

7 Action Conflict with study goals

What should a student do if the suggestions are in conflict with their study goals?

Ethical 3 Student

8 Adverse impact

Oversimplification How can institutions avoid overly simplistic metrics and decision making which ignore personal circumstances?

Ethical 1 Educational researcher

jisc.ac.uk/guides/code-of-practice-for-learning-analytics

Jisc Learning Analytics 2017

Jisc Learning Analytics 2017

Accessibility Considerations for Learning Analytics

1. Remember that learning analytics is not assessment2. Avoid the labelling of individuals and reinforcing of prejudice and stereotypes3. Maintain disabled students’ confidentiality4. Handle the inference of disabilities from the analytics appropriately5. Ensure that the analytics do not unfairly single out disabled students6. Use analytics to identify modules where there appear to be accessibility issues7. Ensure that student-facing analytics are accessible8. Ensure that interventions are worded appropriately» https://analytics.jiscinvolve.org/wp/2016/12/14/accessibility-considerations-for-lear

ning-analytics/

Jisc Learning Analytics Accessibility Webinar

Contacts

Paul Bailey paul.bailey@jisc.ac.uk Niall Sclater niall.sclater@jisc.ac.uk

Further Information: http://www.analytics.jiscinvolve.org

Join: analytics@jiscmail.ac.uk

Jisc Learning Analytics 2017

Straw poll

02/05/2023

Data and disabled students 14

The data discussion

1. How do you encourage disabled students to disclose?

2. Which of your institutional data sources might be relevant to supporting disabled students?  

3. Which disabled students are visible in your data?

4. How mature is the technology?

Discussion:

02/05/2023

Data and disabled students 15

DSA EHCP

Self disclosedDiagnose

dPublic / private

Brainstorm Scenarios»Design of learner data 

› What is collected (ethics, level of detail e.g. disability and other co-morbid factors, process and outcome),

› Design of interface (usability and accessibility),»Support learner progress

› Documenting/disclosing barriers provides an additional method of early identification for support by tracking progress,

› Informing course design and learner attainment › Improving learning and teaching practice› Comparing progress of disabled versus non disabled learners

»Evaluate institutional/support services› Usage of institution wide assistive technology (e.g. text to speech)› Library uptake of productivity software, ebook usage,

02/05/2023

Data and disabled students 16

Design: Stakeholder engagement:Senior manager»“We've invited a range of

stakeholders to be involved in our learning analytics steering group – including support staff and people with accessibility needs”  

02/05/2023

Data and disabled students 17

Roll of roles…Senior manager

Data and disabled students

Design: Learner engagement Student Union President»“Students are actively involved

in deciding what information they see on their own personal dashboard.”

02/05/2023

18

Roll of roles…Senior manager

Student union president

Data and disabled students

Design: Exam arrangementsExamination Officer»“Some people need access

arrangements for exams. It’s always involved lots of consultation and meetings. Now its just a touch of a button”

02/05/2023

19

Roll of roles…Senior manager

Student union presidentExamination officer

Data and disabled students

Support: ConsistencyStudy skills tutor»Now there’s less room for

students to slip through the net because all the support services have the same information at the same time so  can work together.

02/05/2023

20

Roll of roles…Senior manager

Student union presidentExamination officer

Study skills tutor

Support: ResponsivenessHead of disability service»Student data helps me track the

progress of students who have disclosed a barrier to learning so we can respond more swiftly.

02/05/2023

Data and disabled students 21

Roll of roles…Senior manager

Student union presidentExamination officer

Study skills tutorHead of disability

service

Support: Prioritise Dyslexia specialist»I used to spend ages chasing

students who missed their appointments. Now I can instantly check their other progress and leave them alone if they’re succeeding.

02/05/2023

Data and disabled students 22

Roll of roles…Senior manager

Student union presidentExamination officer

Study skills tutorHead of disability

serviceDyslexia specialist

Data and disabled students

Evaluate: Support strategies Assistive technologist»As well as asking about a

learners disability we've tried to capture more specific detail about the technology strategies recommended through the assessment.

02/05/2023

23

Roll of roles…Senior manager

Student union presidentExamination officer

Study skills tutorHead of disability

serviceDyslexia specialist

Assistive technologist

Data and disabled students

Evaluate: Teaching approachesLecturer»I can see how changes to my

resources and activities have impacted on everyone's engagement, and particularly benefited my disabled students.

02/05/2023

24

Roll of roles…Senior manager

Student union presidentExamination officer

Study skills tutorHead of disability

serviceDyslexia specialist

Assistive technologistLecturer

Evaluate: Teaching approachesLearning technologist»“I can see who is using the

learning platform and how often . This makes it easy to see where content might be difficult to access.”

02/05/2023

Data and disabled students 25

Roll of roles…Senior manager

Student union presidentExamination officer

Study skills tutorHead of disability

serviceDyslexia specialist

Assistive technologistLecturer

Learning technologist

Data and disabled students

Evaluate: Library technology support  Library manager»We can now monitor the

usage/uptake of enabling technology software in our library.  This helps us to adopt a more targeted strategy for promotion of productivity tools to enhance the support we offer.

02/05/2023

26

Roll of roles…Senior manager

Student union presidentExamination officer

Study skills tutorHead of disability

serviceDyslexia specialist

Assistive technologistLecturer

Learning technologistLibrary manager

Data and disabled students

Evaluate: E-books and journals:

Collections manager»“I can see who is using the

resources such as e-books and e-journals. If there are any anomalies I can ask why. By exploring use by different categories of student we can plan more effective intervention and support” 

02/05/2023

27

Roll of roles…Senior manager

Student union presidentExamination officer

Study skills tutorHead of disability

serviceDyslexia specialist

Assistive technologistLecturer

Learning technologistLibrary manager

Collections manager

Evaluate: blended learningE-learning manager»“I can begin to correlate

outcomes for disabled student with online provision in different subject areas. Now I have proof that CPD in blended learning pays dividends for disabled students."

02/05/2023

Data and disabled students 28

Roll of roles…Senior manager

Student union presidentExamination officer

Study skills tutorHead of disability

serviceDyslexia specialist

Assistive technologistLecturer

Learning technologistLibrary manager

Collections managerE-learning manager

Evaluate: Data planningData analyst»“I can begin to plan for the

future in ways that can extend what is currently possible to do. With my colleagues I can begin to shape our data to really meet the needs of a wider group of learners.”

02/05/2023

Data and disabled students 29

Roll of roles…Senior manager

Student union presidentExamination officer

Study skills tutorHead of disability

serviceDyslexia specialist

Assistive technologistLecturer

Learning technologistLibrary manager

Collections managerE-learning manager

Data analyst

Data and disabled students

Where this fits in your institution

http://bit.ly/2lFgpwv

02/05/2023

30

What Can Analytics Contribute to Accessibility in e-Learning Systems and to Disabled Student’s Learning

Martyn Cooper, Rebecca Ferguson and Annika Wolff

Context to the research

• OU – distance educator• Larger than average no. of disabled students • Greater challenges in responding to individual needs of

disabled learners at a distance• Students can declare a disability. But don’t necessarily know

what type and no two disabled students are the same, anyway

The question

• Can learning analytics be used to identify modules with accessibility deficits?

First pass

• Look at average completion rates. – 1338 modules analysed– Can show 50% completion rate if 1 of 2 students with declared

disability drops out.– Low numbers can skew results– Solution: analyse only modules with >25 disabled students = 668

modules

Refined approach using odds ratios

• Odds ratios can determine for 2 groups whether one group is more or less likely to achieve an outcome than another group.

• It is a relative measure of the odds of one outcome occurring, given a particular criteria compared to odds of it happening in absence of the criteria

• In this case:– Outcome is success of students on the course

Using odds ratios to find accessibility issues

• A bigger odds ratio = bigger disparity between groups• But - need to find threshold above which you can say there is a problem

Threshold of > 3 looks sufficient to identify where accessibility is most likely factor to explain difference

Summary

• Low numbers make applying statistical measures very difficult• Not suitable for a large number of modules• Identifies where there might be a problem – but not how to fix

it

Possible Future work

• Use research to find ‘critical learning paths’ to identify accessibility issues on individual modules.

What next

»Follow up email with › feedback form › PDF of slides/notes› Links to Google form, Tricider votes, Niall and Julia

blog posts »Link to recording (if we remembered to press Record!)

»New blog post summarising issues and questions arising from session.

02/05/2023

Data and disabled students 39

jisc.ac.uk

One Castlepark Tower Hill Bristol BS2 0JA

customerservices@jisc.ac.uk

T 020 3697 5800

Data and disabled students

Thank you for listeningSubject specialists Accessibility & Inclusion

Julia.Taylor@jisc.ac.uk Alistair.McNaught@ jisc.ac.uk

02/05/2023

40

Margaret.McKay@ jisc.ac.uk