Upload
martin-ebner
View
628
Download
1
Embed Size (px)
Citation preview
W I S S E N T E C H N I K L E I D E N S C H A F T
http://elearning.tugraz.at
Bayesian Modelling of Student Misconceptionsin the one-digit Multiplicationwith Probabilistic ProgrammingBehnam Taraghi, Anna Saranti, Robert Legenstein, Martin Ebner,Educational Technology & Institute of Theoretical Computer ScienceGraz University of Technology
Edinburgh, 29.04.2016
http://elearning.tugraz.at
Presentation Outline
1. Main Idea
2. Previous Work
3. Model Structure
4. Learning the Model’s Parameters
5. Future Work
Behnam Taraghi, Anna Saranti, Robert Legenstein, Martin Ebner, Graz University of TechnologyEdinburgh, 29.04.20162
http://elearning.tugraz.at
Main Idea
Model simple learning misconceptions
Probabilistic graphical models (bayesian networks)
Prediction of future student behaviour
How can this influence learning decision processes?
Behnam Taraghi, Anna Saranti, Robert Legenstein, Martin Ebner, Graz University of TechnologyEdinburgh, 29.04.20163
http://elearning.tugraz.at
Previous Work (1/2)
Derive most prevalent error types in one-digitmultiplication problems
Explore the misconceptions and most difficult vs.easy problems
Cluster one-digit multiplication problems according todifficulty probabilities
Problems: No model, no adaptation, snapshot,inflexible
Solution: Probabilistic Graphical Models
Behnam Taraghi, Anna Saranti, Robert Legenstein, Martin Ebner, Graz University of TechnologyEdinburgh, 29.04.20164
http://elearning.tugraz.at
Previous Work (2/2)
1x1 Trainer: http://schule.learninglab.tugraz.at/einmaleins/ (last access 29 April 2016)
Developed by Graz University of Technology
Applied in different schools in Austria, Germanyand Switzerland
Limited information provided only by the answers,no demographic values
Train the model (Model “learns” its parameters)
Behnam Taraghi, Anna Saranti, Robert Legenstein, Martin Ebner, Graz University of TechnologyEdinburgh, 29.04.20165
http://elearning.tugraz.at
1x1 Trainer Application
Behnam Taraghi, Anna Saranti, Robert Legenstein, Martin Ebner, Graz University of TechnologyEdinburgh, 29.04.20166
http://elearning.tugraz.at
Difficult vs. Easy Questions
Behnam Taraghi, Anna Saranti, Robert Legenstein, Martin Ebner, Graz University of TechnologyEdinburgh, 29.04.20167
http://elearning.tugraz.at
Average Time Consumption
Behnam Taraghi, Anna Saranti, Robert Legenstein, Martin Ebner, Graz University of TechnologyEdinburgh, 29.04.20168
http://elearning.tugraz.at
Clustering One-Digit Multiplication Problems
Behnam Taraghi, Anna Saranti, Robert Legenstein, Martin Ebner, Graz University of TechnologyEdinburgh, 29.04.20169
http://elearning.tugraz.at
Derived Error Types
Error type Description e.g. 56 = 7 ∗ 8operand errors a neighbouring operand is takensplit 1 the neighbouring distance is 1 48 = 6 ∗ 8split 2 the neighbouring distance is 2 for an
operand or is 1 for both operands40 = 5 ∗ 8
operand intrusions a digit of the result matches an operandfirst operand decade digit matches first operand 74←↩ 7 ∗ 8second operand unit digit matches second operand 68←↩ 7 ∗ 8consistency errorsunit consistency only unit digit is correct 76←↩ 56decade consistency only decade digit is correct 51←↩ 56off-by- errorsoff-by-±1 the result differs by x = 1 55 or 57off-by-±2 the result is off by −2 ≤ x ≤ 2 54, 55, 57, 58pattern errors swapped digits in the result 65confusion errors confusion with addition, subtraction and
division operations15 or 1
Behnam Taraghi, Anna Saranti, Robert Legenstein, Martin Ebner, Graz University of TechnologyEdinburgh, 29.04.201610
http://elearning.tugraz.at
Model Structure (1/4)
Perturbation model
Bug library with 6 error types : operand, intrusion,consistency, off-by-±1 and off-by-±2, pattern,confusion with addition, subtraction, and divisionoperation
Unclassified errors
Correct (absence of misconception / error)
Clean up the data samples
Behnam Taraghi, Anna Saranti, Robert Legenstein, Martin Ebner, Graz University of TechnologyEdinburgh, 29.04.201611
http://elearning.tugraz.at
Model Structure (2/4)
Minimal structural assumptions: Errors don’t dependon each other directly
But there are answers to questions that could belongto many error types, f.e. a student may answer thequestion 8 × 5 with a 41, which could be both aconsistency and off-by-+1 error.
Behnam Taraghi, Anna Saranti, Robert Legenstein, Martin Ebner, Graz University of TechnologyEdinburgh, 29.04.201612
http://elearning.tugraz.at
Model Structure (3/4)
Behnam Taraghi, Anna Saranti, Robert Legenstein, Martin Ebner, Graz University of TechnologyEdinburgh, 29.04.201613
http://elearning.tugraz.at
Model Structure (4/4)
The learner’s state of knowledge changes over timeeven during application useThe model represents our estimations about thestudent’s misconceptionsThe more data, the more certain we will be whicherror type is more dominant, how much, etc.“A particular person, when dealing with one-digitmultiplication problems, makes 20% operand errors,10% consistency errors” and so onUncover the similarities and the differences of thelearners
Behnam Taraghi, Anna Saranti, Robert Legenstein, Martin Ebner, Graz University of TechnologyEdinburgh, 29.04.201614
http://elearning.tugraz.at
Learning the Model’s Parameters (1/3)
Behnam Taraghi, Anna Saranti, Robert Legenstein, Martin Ebner, Graz University of TechnologyEdinburgh, 29.04.201615
http://elearning.tugraz.at
Learning the Model’s Parameters (2/3)
The dependencies define the structure of the modelThe quantification of this relationships is expressedby the parameters, which will be learned by the dataInitialisation assumption: non-informative uniformprior
1. Each error type and the unique correct option areequally likely
2. Given a specific error type, is there anypreference or tendency towards a particularanswer?
Behnam Taraghi, Anna Saranti, Robert Legenstein, Martin Ebner, Graz University of TechnologyEdinburgh, 29.04.201616
http://elearning.tugraz.at
Learning the Model’s Parameters (3/3)
Expectation-maximization (EM) - Algorithm, becausethe “LearningState” random variable is hidden
Probabilistic programming library Figaro for Scalahttp://www.scala-lang.orghttps://www.cra.com/work/case-studies/figarohttps://www.cra.com/sites/default/files/pdf/Figaro Tutorial.pdfhttps://www.manning.com/books/practical-probabilistic-programming
Behnam Taraghi, Anna Saranti, Robert Legenstein, Martin Ebner, Graz University of TechnologyEdinburgh, 29.04.201617
http://elearning.tugraz.at
Future Work (1/2)
Are there new types of errors (according to the modeland data)?
Identify groups of learners (stereotypes) with mixtureof Dirichlet distributions (“Learning State” of 20% ofour students has values near to a particular Dirichletdistribution, 30% equals a completely differentDirichlet distribution and so on)
Behnam Taraghi, Anna Saranti, Robert Legenstein, Martin Ebner, Graz University of TechnologyEdinburgh, 29.04.201618
http://elearning.tugraz.at
Future Work (2/2)
Predict the answer and use this to decide what thenext learning issue will be
Probabilistic and maximum a posteriori (MAP)queries can be used to influence all kinds of factors ina learning application such as hints, helping notes,rearrangement of questions’ sequence
Behnam Taraghi, Anna Saranti, Robert Legenstein, Martin Ebner, Graz University of TechnologyEdinburgh, 29.04.201619
http://elearning.tugraz.at
Thanks for your attention!Comments?
Behnam Taraghi, Anna Saranti, Robert Legenstein, Martin Ebner, Graz University of TechnologyEdinburgh, 29.04.201620