Increasing MOOC completion rates through social interactions

A recommendation system

Hugues Labarthe, Rémi Bachelet, François Bouchet & Kalina Yacef

Download this slideshow : http://goo.gl/lkba7HFebruary 22nd, 2016, Graz, Austria

MOOC completion rates are notoriously low

• What we know:

– MOOC students interacting with each other have better odds to get certified (Yang, Wen & Rose, 2014)

• Problem:

– How to improve user-centered experience and facilitate user’s involvement with peers?

2

Questions & data

Research questions:

1. Impact of social interactions between students?

2. Best way to foster peer interaction?

3 data sets:

– Session 2 to 5: Impact of social interactions? Longitudinal Study

– Session5: Students groups. Experiment.

– Session6: Chat & contact recommendation module. Experiment

3

Context: MOOC GdP

4

GdP2 GdP3 GdP4 GdP5 GdP6

Date Fall 2013 Spring 2014 Fall 2014 Spring 2015 Fall 2015

Duration 5 weeks

+ 1 week for

final exam

5 weeks including

1 elective course

+ 1 week for final

exam

6 weeks including

2 elective course

+ 1 week of exams

6 weeks including

2 elective course

+ 1 week of exams

6 weeks including

2 elective course

+ 1 week of exams

Attendance:

- Enrolled

- > 1 quiz in basic

track

- > 1 deliverable in

adv. track

10848

5711

1011

11827

5899

705

19171

8120

1197

17579

4842

725

22300

7305

970

Track success:

- Basic

- Advanced61%

78%

38%

67%

41%

73%

47%

69%

53%

71%

• French MOOC on project management

• Platform: Canvas Engineering: Unow

• 2 sessions/year: enhanced content

• 3 tracks: basic, advanced, team

• Barcamp"richer learning activities for MOOCs : 14+ ideas " EMOOCs2015

5

“Funnel of participation” (Clow, 2013)Attrition rate increasing, “September effect” (in-cursus students = less drop out in these sessions)

Study #1 (Longitudinal): attrition & interactions in 5 sessions (1/3)

6

“Participation funnel” (Clow, 2013)Attrition rate increasing, “September effect” (in-cursus students = less drop out in these sessions)

Study #1 (Longitudinal): attrition & interactions in 5 sessions (1/3)

7

> 50,000 posts & PM - Social networks not included+8k contributors (13.5%), only 3.5% used both posts & PM 3 functions: socializing, learning reinforcement, experience sharing

Longitudinal study : attrition & interactions (2/3)

Longitudinal study : attrition & interactions (3/3)

8

Longitudinal study : attrition & interactions (3/3)

9

The drop in exam pass rate for 5+post comes mostly from the high forum activity of our teaching assistant staff. In our newly published papers, this bias is controlled, as well as involvement in the technical help thread

Longitudinal study : Findings

Interacting is a strong predictor of success

+35% exam pass on average

10

Increase social interactions: Recommendations

– « Introduce yourself » thread• Trigger contributions (“must read” in week zero)

• Boost visibility of this thread: pre-MOOC announcement

– Geolocalization • « where are you ? » map

– Facebook, Google+, Twitter…• Capitalize on existing social networks

• Foster them : news, answers…

– “Question of the week”• starting point for the weekly live Q&A

– Strategies• Reward active posters: Animator badge

• TA team : Do not answer questions too fast, leave this to the other students

11

Study #2: students group to increase interactions –methodology

• Session 5 : Canvas groups features (private forums, wiki, list of group members with link to profile)

• Experiment : 34 groups of variable size:– 4 groups of 50 students– 10 groups of 10 students– 20 groups of 4 students

• 5 grouping by similarity criteria (data from initial research questionnaire)– Geographical proximity (Country)– Study level– Age– Family status– Previous experience with MOOCs

12

Study #2: findings & recommendations

Not many interactions, yet much to be learned!

In groups of 4 & 10, no more than 1 post/person = no real interaction, not many visits after first attempt

13

Criteria Observation Recommendation

Teammates visibility Group page quicklyignored

Make the list of other group membersmore visible

Contact visibility No visit to checkreplies to one’s post

Improve notification system

Sample size Many students do not interact

Larger sample size next time

# of recommended contacts

Low activity even in large groups

More contact recommendations increase the chances of finding other students willing to interact

Individualizedinteraction

Similar interface prevented students from seeing what more it brought

Allow students to create small groups of their own

Study #3: contact recommendation & chat modules

14

A contact recommendation widget in every page of the platform below the navigation column

Advanced ID card with: - profile info,- ways to contact (PM and instant message)- adding to favorites/ignoredAvailable with a simple mouse-over

A permanent chat widget

Notification (+ sound) when a message arrives!

Contact manager, to initiate conversation with favorite(s) contact(s)

Chat manager, to start or pick up a conversation

GdP6: preliminary findings

15

Findings:- Strong curiosity for students’ profiles

- Asynchronous chat interactions- Impact on attrition and performance: currently under review

GdP6:- 22,300 enrolled- 16,127 connected once- 8,053 active

Experiment: 8,674 students- 1792 in control group (no

recommendation, no chat)- 2025 accepted the terms of use- 271 interacted with others

through chat

Conclusion

• Interactions are correlated w/ success, and are key to reducing attrition. Need to find:

• The right combination of factors to recommend students to their peers

• The right interface for them to interact with one another

Can only be achieved through experiments (work in progress)

• Perspectives:– Technology course –interaction design

– Identify students who want to interact = maximize chances of response

16

Thanks for listening!

• Twitter : @R_Bachelet, Googleplus : +Rémi Bachelet

Read the paper online : https://hal.archives-ouvertes.fr/view/index/docid/1277664

Does peer grading work? How to implement and improve it?

Comparing instructor and peer assessment in MOOC GdP

Rémi Bachelet, Drissa Zongo, Aline Bourelle

Download this slideshow : http://goo.gl/GiFvXb