Janine KiersProduct Manager DelftX MOOCsBased on conference submission with Danika Marquis and Leonie Meijerink

Evaluating Teaching and Learning

What is a successful MOOC?

• Different perspectives• Branding• Innovation • Research• Educational

CC 0 Public domain Pixabay

Quality Assurance cycle

Design

Develop

Run

Evaluate

Evaluating Teaching & Learning

• Quality AssuranceIntention:

• ToolsImplementation:

• Share & interpret resultsExperience:

• SuggestionsImprove:

Tools: how to evaluate Teaching and Learning in a MOOC?

• Passing rate• Student satisfaction• Design

CC 0 Public domain Pixabay

Implementation of evaluationTool: Online Learning Experience

Online Learning Experience

• Flexible (in time, space & content) • Diverse (activities, resources & assessment)• Inclusive (accessible, cultural & gender)• Supportive (guidance & feedback)• Interactive (learner – learner/teacher/content)• Active (learning by doing)• Contextual (real world situations & problems)• Innovative (new tools, strategies & insights)

Implementation of evaluationTools to gather & process data

Sources of data Used in

Student surveys Course report

Forum analysis Course report

Stakeholder interviews Discussion

Platform data Course report

• Collaborative Analysis - eLD & Course Team• Discussion of results• Implement in rerun

Implementation of evaluationTeaching and Learning report

CC BY 2.0 IvanWalsh.com

Evaluation: platform data

• Data analysed (completers, respondents)• Participants’ feedback

• in Likert scale• written feedback

Student expectations

Workload: expected and actual

Describe the course in one word

Evaluation: Student activity

Weekly numbers of active students (blue line), students who watched a video (green line) and students who tried an assignment (orange line).

Evaluation: Replay video

Evaluation: Student performance

• Review questions e.g. < 50%

ExperienceChallenges in interpretation

• No student-teacher personal contact• Survey results are biased

- low response rates & bias for completers

• Heterogeneity of learners• Eliminate content specific results

Learning points from experience

• Standardization of implementation• Standardization of interpretation of OLE • Dedicate time for evaluation

• by Course Team• by support team / eLD

Effect of course evaluation

• Improve evaluated course• Create knowledge base (generalized)• Reflect on support process• Professional development

Thank you

Questions?