Analytics for Improved Course Delivery & Design

Richard Burrows - Analytics Specialistrichard.burrows@blackboard.com

Retention vs. Improved Course Delivery & Design

2

33

Information to enable instructors to adapt & personalize their teaching

Informing Course Management

98%

Last Logins Last Logins- in the context of your course

Online Engagement- that understands your peers’ study patterns

Academic History Information - using data from the SIS

Academic Performance- using graded online activity

4Custom Report

5

Provides easy to interpret quantitative data to the course review process in key areas:

Informing Course Review

Student engagement and academic performance

Level, consistency, type of engagement

Allows Course profiling - Supplemental, Complimentary, Social, Evaluative or Holistic

Evaluation of course content & tools being used

Analytics for Learn brings unique insight into both the quality and the delivery of the course

Technology Enhanced Learning Dashboards

• TEL Dashboardo High usage (hot-spots!), can we identify and model good

practice?o Low usage (target staff development resource)o Top 10 (investigate top 10 across all Colleges to identify

good practice)

• Modules with high student numbers (High stakes/Quick wins), raise awareness of:o Adaptive releaseo PeerMarko Group toolo Opportunities to develop peer support mechanisms (cohort

identity)

7

TEL Dashboard

Demonstration data

Technology Enhanced Learning Dashboards

• Single Course Query/Health Check (prior to Staff Development/one-to-one Consultations):

o Usage (student numbers, student access)

o Tool usage, what is and isn’t being used?

o Folder depth? High level of organisation and structure?

o Compare with previous iterations? What is the history of the module?

• College Overview of Tool Usage, are we getting a return on investment?

TEL Dashboard

Demonstration data

How are students spending their time?

10

11

Blackboard Predict for Learn

1212

Sample Model Output – Features & Weights

.00 .05 .10 .15

Current GPA

Course add weeks prior

Transfer credits

EFC

Percent of classes passed

Ethnicity

Class load

Academic year

Course modality

Week Zero Model.00 .10 .20 .30 .40

Earned over attempted

Number of page views

Earned over possible

Percent of classes passed

Days since last page view

Posts to faculty

Weekly Model

13

Instructor’s Dashboard

14

Instructor’s detail report

15

Advisor’s Dashboard

Accuracy of Predictions

Overall Accuracy (F1 Score ) = 84 to 85%

Precision: Of the students we predicted would attend class that week, what percent

actually attended? = 80% to 84%

Recall: Of the students that did attend class that week, what percent did we accurately

predict? = 89% & 84%

Blackboard Analytics Product Portfolio

Blackboard Analytics for Learn• Data warehouse offering for business intelligence/analysis

• Target Teaching Excellence & Course Design

• Data Sources: Blackboard Learn, Student Information System

Past view Current view Future view

Blackboard Predict• Predictive analytics to target interventions for Student Success

• Provides data for faculty and advisors about at-risk students

• Data Sources: Blackboard Learn, Student Information System

Past view Current view Future view

X-Ray for Moodle & Moodlerooms• Predictive analytics to target interventions for Student Success

• Data for faculty about at-risk students, engagement & course design

• Available for Moodle and Moodlerooms

Past view Current view Future view