Embed Size (px)
Citation preview
YouEDU
Staging Intelligent Interventionsin MOOC Discussion ForumsAkshay AgrawalJagadish VenkatramanShane LeonardAndreas Paepcke
Agrawal 2
Agrawal 3
Agrawal 4
An Intelligent Forum
Agrawal 5
An Intelligent Forum
• Phase 1: Detect Confusion
Agrawal 6
An Intelligent Forum
• Phase 1: Detect Confusion• Phase 2: Intervene Automatically
Agrawal 7
Challenges
Agrawal 8
Challenges
• Scale:
Agrawal 9
Challenges
• Scale:
• Vocabulary of confusion is domain-dependent
Agrawal 10
Challenges
• Scale:
• Vocabulary of confusion is domain-dependent
• TA overloaded
Agrawal 11
Challenges
• Scale:
• Vocabulary of confusion is domain-dependent
• TA overloaded– lack of forum structure
Agrawal 12
• No tagged forum datasets• Availability of large-scale forum data
Agrawal 13
Why are forums still dumb?
YouEDU
• Pre-work: Dataset– Stanford MOOCPosts
• Phase 1: Confusion Detection– Classify individual posts w.r.t. confusion
• Phase 2: Intervention–Map confused posts to video snippets
Agrawal 14
MOOCPosts Dataset
• 30,000 posts,11 courses– Humanities/Sciences, Medicine,
Education
• Human-coded, 6 dimensions– confusion, sentiment, urgency [1-7]– opinion, question, answer [0/1]
Agrawal 15
http://datastage.stanford.edu/StanfordMoocPosts/
Agrawal 16
Phase 1: Detecting Confusion
Agrawal 17
Agrawal 18
Agrawal 19
Agrawal 20
Classifier Performance
Agrawal 21
Recap: Phase 1
Phase 2: Staging Interventions
Agrawal 22
Agrawal 23
Agrawal 24
Agrawal 25
Agrawal 26
Agrawal 27
Agrawal 28
Agrawal 29
Agrawal 30
Interventions: Evaluation
• 20 confused posts, course in statistics
• Six recommendations for each– randomize order
• 3 human raters– Label each snippet as relevant or
irrelevant
• k-precision– precision of retrieved documents,
limited to the first k recommendationsAgrawal 31
Agrawal 32
YouEDU Architecture
Agrawal 33
Classification Suitehttp://github.com/akshayka/edxclassify
MOOCPosts Datasethttp://datastage.stanford.edu/StanfordMoocPosts/
Agrawal 34
Agrawal 35
Agrawal 36
Agrawal 37
Agrawal 38
