Quincy Brown Kallen Tsikalas

• Research Questions & Hypotheses• Theoretical Assumptions: Good, Bad & Ugly• Using CTAT to test hypotheses• The Interface• Beneath the Interface: Models & Behavior Graphs

• Lessons Learned• Extensions to the CTAT Interface Tools• Future work

An Experiment Using CTAT to Explore the Role of

Self-Regulation in the Robust Learning of Middle School Math

Research Questions & Hypotheses

1. Effect of providing a self-regulatory goal. What is the effect of giving students an explicit self-regulatory goal [to be “error detectives”] on their robust learning and the accuracy of their self-efficacy ratings?

2. Effect of providing self-regulatory feedback and practice opportunities. What is the effect of providing students with feedback on and practice with a self-regulatory skill [error detection and correction] on their robust learning and the accuracy of their self-efficacy ratings?

3. Predictive power of accurate self-efficacy ratings. To what extent does the accuracy of students’ self-efficacy ratings effect their learning curve and help-seeking behavior?

Outcome Variables

- Accuracy of self-efficacy ratings

- Learning curves from CTAT data

- Pre-, post-, and delayed post-test scores

How sure are you that you can solve this problem?

Likert scale (1-10)

Theoretical Assumptions

Interventions that target students’ self-regulatory processes can lead to improved cycles of learning and improved academic and non-academic outcomes.

Examples of self-regulatory interventions are training and/or feedback on motivational beliefs, goal-setting, monitoring, self-judgments, etc.

Providing feedback on self-regulatory skills effects students’

Ability to create internal feedback and self-assess Attributions about success or failure Proficiency at help-seeking Willingness to invest effort in dealing with feedback

information

Cognitive load theory may suggest that attending to errors introduces extraneous load which may diminish robust learning.

Using CTAT to Test Hypotheses

2x2 factorial design Control condition = Cognitive Tutor

with no self-regulation enhancements’

Opportunities for assisted practice of cognitive skills

Multiple versions of Cognitive Tutor

Self-Regulatory Goal

+ -

- Control:CogTutor w/ no SR

enhancements

Err

or

IDFe

ed

back

The Interface

Two Versions

Example-Tracing Tutor Executed in Flash Steps on separate screens Dynamic feedback: Students have opportunity to

interact with feedback screens

Full Cognitive Tutor Executive in Flash Interface represents deep mathematical structure

The CTAT Example-Tracing Interface

Executed in Flash Steps on separate screens (Flash frames) Dynamic feedback: Students have opportunity to

interact with error feedback on screens (through Flash movies)

The CTAT Cognitive Tutor Interface Executed in Flash Streamlined format representing deep structure of

mathematics

The CTAT Full Cognitive Tutor

Behavior Graph

Conflict Tree

Working Memory

Cognitive Model

The CTAT Full Cognitive TutorProduction Rules

All production rules functioning

The CTAT Example-Tracing Behavior Graph for the CogTutor Interface

Lessons Learned

How to use the CTAT tools Importance of think-alouds for building

example-tracing and production rules To create correct branching structure To optimize the number of rules – not more

than needed

Potential threats to the efficacy of our intervention: Ken’s talk on design principles

Ideas about new types of learning outcomes (learning curves, help requests that lead to greater learning)

Extensions to CTAT Interface Tools

Multiple screens for one tutor Navigation between screens that

communicates with CTAT Via ActionScript Intratutor communication

Separate functions (e.g., visible and invisible Flash movies) for displaying feedback

Adjustments to Flash Widgets Widgets just to log student actions/ideas

rather than to tutor Debugging of Flash tutorials

Future Work

Extension to mobile devices

Use of student characteristics (e.g., self-efficacy ratings) to guide specific tutoring actions

Use of student characteristics (e.g., accuracy of self-efficacy ratings) to predict learning curves

Special Thanks to…

Everyone who helped us figure out what’s going on!

John and Brett for assistance with Flash widgets and communication between Example-Tracing functions and Flash interface

Jonathan and Vincent for assistance with full cognitive tutor development and production

Noboru for assistance with SimStudent

The PLSC Summer School students and staff for their good humor and great ideas!