Tradeoffs Between Immediate and Future Learning: Feedback in
a Fraction Addition TutorEliane Stampfer
EARLI SIG 6&7September 13, 2012
Giving Feedback:
When should we tell students something directly, and when should we show them something they
have to interpret for themselves?
Research Question
Correctness Grounded
Faster procedural learning
Better conceptual understanding
Overview
Prior Work: Interpretation is Best
• Algebra expressions drive an animation, showing meaning in terms of story Nathan 1998
• Seeing the consequences of incorrect Excel formulas Mathan & Koedinger 2005
• Invention of formulas for variance prepare students to learn from a lecture Schwartz & Martin 2004
Literature Review
ANIMATE Sample Problem
A helicopter rushes from Central City trying to catch up with a train. The train had left two hours before the helicopter, and the train was going 75 miles per hour. The helicopter flies at 300 miles per hour. The train is 60 miles from a broken bridge – can the helicopter reach it in time?
Literature Review
Nathan 1998
Time: hours
Literature Review
Nathan 1998
Prior Work – Difficulties with Representations
• Relating representations is difficult Ainsworth, Bibby, & Wood 2002
• Students don’t always notice or encode relevant features of the learning environment Siegler 1976, Blair 2009
Literature Review
Tutor Design
Input what students are learning
Feedback
shows nature of error in a form students understand
Works by
Feed Forward
reminding students of what they already know
may point to the next step
Why Grounded Feedback?
Equivalent fractions
would line up
My fraction should be
bigger
Tutor Design
Study Condition 1 - GroundedTutor Design
Study Condition 1 - GroundedTutor Design
Study Condition 1 - GroundedTutor Design
Study Condition 2 - CorrectnessTutor Design
Tutor Similarities
Same Problems
On-Demand Text Hints
Must Solve Each Problem Before Moving On
Tutor Design
Tutor DifferencesGrounded Feedback Correctness Feedback
Shows Tells
Students can re-write any input
Students cannot change correct inputs
Permits student to try incorrect paths
Prevents inefficient paths
Red When Wrong
Eliane Stampfer
Tutor Design
Study Method and Participants
• Pretest • Instruction• Assigned Tutor• Immediate Post Test• Delayed Post-Test 2 weeks later• Participants: all of the 5th graders at a local
school, about 140 (129 completed all parts)
Study Design
InstructionTutor
Process Measures
Grounded Feedback
Correctness Feedback
Problems Attempted (max 20)
9.2 17.7
Hints per problem 7.7 2.5
Significant Differences in problems attempted and hints per problem (p<.01)
Results
Pre-Test, Post-Test, and 2-week Delayed Post Test
3 Test Forms, Matched and Counterbalanced
Pre-Requisite Knowledge, Transfer, Target, and Metacognitive
Test Design
MetacognitiveYour friend solved 2/7 + 1/9. Look at the work your friend did and check the correct statements at the bottom:
Test Design
0123456789
10111213
Pre Post 2-Week Delayed
Pre to Post: Both groups learned (p < .01)Post to Delayed: Significant difference in learning (p = .035)
Correctness Grounded
Results: Full Test
Eliane Stampfer
Target Knowledge
Same Denominator 3/9 + 5/9
One Denominator is a Multiple of the Other 2/12 + 3/4 Unrelated Denominators 1/4 + 3/10 1/3 + 4/11
Test Design
0
1
2
3
4
Pre to Post: Both groups learned (p < .01) Difference in Learning (p = .036)Post to Delayed: Only Grounded improved (p < .01) Difference in Learning (p = .057)
Pre Post 2-Week Delayed
Correctness Grounded
Results: Target Items
Metacognitive Design
Did Metacognitive Skills Improve?Excluding Ceiling at Pretest
Differences in learning from Pre to Delayed-Post are significant (p=.03)
0
0.5
1
1.5
2
2.5
3
3.5
4 Pre Immediate Post 2-Week Delay
Correctness Grounded
Grounded Feedback: Not IntuitiveResults
Condition Pre Immediate Post
2-week Delay
Correctness (47) .66 .48 .58Grounded (90) .47 .57 .74
Condition Pre Immediate Post
2-week Delay
Correctness (16) 0 .38 .40Grounded (48) 0 .54 .73
All StudentsExclude Correct at PretestResults
Differences at 2-week delay: p=.02
Discussion
• Tradeoffs between immediate and future learning
• Grounded Feedback helped even though student didn’t understand it perfectly
• Grounded Feedback may improve conceptual understanding and evaluation skills
Relating Back to the Literature
• Ainsworth et al: relating representations is hard
• Blair and Siegler: students don’t always pay attention to the important parts of the feedback
• Schwartz: struggling at first may prepare students for future learning
Next Steps
• Grounded Feedback may work better when it is more grounded in students’ prior knowledge
• Difficulty Factor Assessment to see why students don’t understand the current fraction bars
• Compare Grounded Feedback to robust worked examples
Acknowledgements
Thanks to my advisor Ken Koedinger, my participants and their teachers, and the Pittsburgh Sciences of Learning Center
This research was supported in part by the Pittsburgh Science of Learning Center through NSF award SBE-0836012, and the Institute of Education Sciences, U.S. Department of Education, through Grant R305B090023 to Carnegie Mellon University.
Tradeoffs Between Immediate and Future Learning: Feedback in
a Fraction Addition TutorEliane Stampfer
EARLI SIG 6&7September 12, 2012
Thank You!
Guided InstructionTutor
Guided vs. Discovery: No Difference
• Same amount of time per problem
• Same number of hints requested per problem
• Further analysis will treat them as one group: Grounded Feedback
Results