What is a “Cognitive Model”?
A simulation of human thinking & resulting behaviorUsually used to explain or predict data on human behavior Like error rates or solution time
Usually implemented as a computer program that can behave like humans Often using AI knowledge representations
like semantic nets, frames, schema, production rules
What are Cognitive Models used for?
Output of basic research Explain results of psychology experiments
Guide design of software systems Have cognitive model “use” the system
Model predicts people’s time & errors(VanLehn) Redesign system to reduce time or errors
Can derive predictions without full implementation (e.g., Ethan)
As a component in an intelligent system Player in a game or training simulation Part of expert system or intelligent tutor
What is an “Intelligent Tutoring System” (ITS)?
A kind of educational softwareUses artificial intelligence techniques to Provide human tutor-like behavior Be more flexible, diagnostic & adaptive Write more general code to get more
capabilities with less effort
Components of an ITS: Interface or problem solving environment,
domain knowledge, student model, pedagogical (tutoring) knowledge
An ITS Success CaseCognitive Tutor Algebra (aka Pump)
Most widely used ITS 1000+ schools across the country Marketed by local spin-off company
Carnegie Learning
“Exemplary Curricula” by US Dept of EdMost cited Journal of AI-ED paper
Koedinger, Anderson, Hadley, & Mark (1997). Intelligent tutoring goes to school in the big city.
. . .
Algebra Cognitive Tutor
Use graphs, graphics calculatorUse graphs, graphics calculator
Analyze real world Analyze real world problem scenariosproblem scenarios
Use table, Use table, spreadsheetspreadsheet
Use equations, Use equations, symbolic symbolic calculatorcalculator
Tracked by Tracked by knowledge tracingknowledge tracing
Model tracing to provide Model tracing to provide context-sensitive context-sensitive InstructionInstruction
Cognitive Tutor AlgebraCourse
Integrated tutor, text, and teacher trainingIn computer lab 2 days/week, classroom 3 days/weekLearn by doing:
Project-based Student-centered Cooperative learning Teacher as facilitator
Replicated Field StudiesControlled, full year classroom experimentsReplicated over 3 years in urban schoolsIn Pittsburgh & Milwaukee
Results:50-100% better on problem solving & representation use.
15-25% better on standardized tests.
Koedinger, Anderson, Hadley, & Mark (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8.
0
10
20
30
40
50
60
Iowa SAT subset ProblemSolving
Represent-ations
Traditional Algebra Course
Cognitive Tutor Algebra
b
CognitiveTutorTechnology
Algebra I GeometryEquationSolver
Algebra II
Cognitive Tutors
ArtificialIntelligence
CognitivePsychology
Research base
Math InstructorsMath EducatorsNCTM Standards
Curriculum Content
Combining Theory & Practice
A Simple Instructional Design Principle
Instruction is most effective when it builds on what students already know
Sequence instruction from easy to hard
Difficulty Factors Assessment:Which Problem Type is Hardest?Story Problem
As a waiter, Ted gets $6 per hour. One night he made $66 in tips and earned a total of $81.90. How many hours did Ted work?
Word Problem
Starting with some number, if I multiply it by 6 and then add 66, I get 81.90. What number did I start with?
Equation
x * 6 + 66 = 81.90
Expert Blindspot:Expertise can impair judgment of student difficulties
0
10
20
30
40
50
60
70
80
90
100
ElementaryTeachers
MiddleSchool
Teachers
High SchoolTeachers
% makingcorrectranking
(equations hardest)
Nathan, M. J. & Koedinger, K. R. (2000). An investigation of teachers' beliefs ofstudents' algebra development. Cognition and Instruction, 18(2), 207-235
Expert BlindspotExperts’ judgments are biased by self-assessing their own performance
Sources of bias in expert judgment: Under-estimate novice’s intuitive,
concrete modes of thinking Over-estimate ease in acquiring formal,
abstract modes of thinking
Result: Inaccurate evaluations, poor design choices
What is the Student Like?
To avoid your expert blindspot, remember:
“The Student Is Not Like Me”
Use Cognitive & HCI methods to find out what students are like
b
CognitiveTutorTechnology
Algebra I GeometryEquationSolver
Algebra II
Cognitive Tutors
ArtificialIntelligence
CognitivePsychology
Research base
Math InstructorsMath EducatorsNCTM Standards
Curriculum Content
Combining Theory & Practice
ACT-R: A Cognitive Theory of Learning and Performance
Big theory … key tenets: Learning by doing, not by listening or watching
Production rules represent performance knowledge:
These units are: Instruction implications: modular context specific
isolate skills, concepts, strategiesaddress "when" as well as "how"
Anderson, J.R., & Lebiere, C. (1998). Atomic Components of Thought. Erlbaum.
Cognitive Model: A system that can solve problems in the various ways students can
Strategy 1: IF the goal is to solve a(bx+c) = d THEN rewrite this as abx + ac = dStrategy 2: IF the goal is to solve a(bx+c) = d
THEN rewrite this as bx + c = d/a
Misconception: IF the goal is to solve a(bx+c) = d THEN rewrite this as abx + c = d
Cognitive Tutor Technology:Use ACT-R theory to individualize instruction
ACT-R production rules are not textbook rules, but “theorems in action” that characterize common thinking patterns
3(2x - 5) = 9
6x - 15 = 9 2x - 5 = 3 6x - 5 = 9
Cognitive Tutor Technology:Use ACT-R theory to individualize instruction
Cognitive Model: A system that can solve problems in the various ways students can
If goal is solve a(bx+c) = dThen rewrite as abx + ac = d
If goal is solve a(bx+c) = dThen rewrite as abx + c = d
If goal is solve a(bx+c) = dThen rewrite as bx+c = d/a
Model Tracing: Follows student through their individual approach to a problem -> context-sensitive instruction
3(2x - 5) = 9
6x - 15 = 9 2x - 5 = 3 6x - 5 = 9
Cognitive Tutor Technology:Use ACT-R theory to individualize instruction
Cognitive Model: A system that can solve problems in the various ways students can
If goal is solve a(bx+c) = dThen rewrite as abx + ac = d
If goal is solve a(bx+c) = dThen rewrite as abx + c = d
Model Tracing: Follows student through their individual approach to a problem -> context-sensitive instruction
Hint message: “Distribute a across the parentheses.”
Bug message: “You need tomultiply c by a also.”
Knowledge Tracing: Assesses student's knowledge growth -> individualized activity selection and pacing
Known? = 85% chance Known? = 45%
The Rules of Mathematics Thinking≠The Rules of Mathematics
Leads to order of operationserror:“x * 3 + 4” is rewritten as “x * 7”
Works for “2x + 3x” but not for “x + 3x”
Overly general productionIF “Num1 + Num2” appears
in an expressionTHEN
replace it with the sum
Overly specific productionIF “ax + bx” appears in an
expression and c = a + bTHEN
replace it with “cx”Production rules are not textbook rules, but “theorems in action” that characterize common thinking patterns