Introducing Desirable Difficulties for Educational

Applications in Science

Robert A. Bjork

University of California, Los Angeles

Marcia C. Linn

University of California, Berkeley

www.psych.ucla.edu/iddeas

“Desirable Difficulties”

• Spacing rather than massing study• Interleaving rather than blocking practice

on separate topics or tasks• Varying contextual cues• Reducing feedback to the learner• Testing rather than re-presenting

Learning versus performance• What we can observe is performance, what

we must infer is learning…– and the former is an unreliable index of the

latter

• Instructors are, therefore, susceptible to choosing less-effective conditions of learning over more-effective conditions

• And as learners, we, too, are susceptible to confusing performance with learning

Generation Interleaving Spacing

Goals of the IDDEAS project

• Do such findings extend to to-be-learned materials and retention intervals that are realistic from an educational standpoint?

• And, more broadly, what design principles are fundamental in optimizing educational materials and practices?

WISE (web-based inquiry science environment): http://wise.berkeley.edu

• Advantages as a tool for teachers– Supports authoring and customization– Contains a library of tested projects– Enables collaborative learning, visible thinking,

autonomous investigation– Transportable

• Advantages as an IDDEAS research tool

Current Projects

WISE Platform

• 2 laboratory studies, UCLA

• 2 classroom studies, UCB

Design Principles

• 2 studies, UCLA

Interleaving

• Motor tasks: patterns, force production, bank machine transactions (Lee & Magill, 1983, Simon & Bjork, 1990 Charles et. al, 1990, Jamieson & Robers, 2000)

• Sports: badminton, volleyball, baseball (Bortoli et al, 1992, Goode & Magill, 1986, Hall et al, 1994)

• Abstract learning tasks: mazes, tracking (Carleson et.al, 1989, Jelsma & Van Merrienboer, 1989, Jelsma & Pieters, 1989)

• Logic rules, boolean operators (Schneider et al, 1995, Carleson & Yaure, 1990)

Example of generation effects

Study 1: Overview

• Uses an existing WISE astronomy module:– How scientists determine the habitability of planets

• Design:4 groups– Mass and distance slides either blocked or interleaved;

– Some evidence re-studied via generation or reading

• Forty-eight hour retention interval• Posttest

Posttest questions• Simple sentence-completion: Information presented

and re-studied via generation or reading, or only presented– E. g. “The amount of heat and light emitted by the sun in our solar

system has increased by ____% since the beginning of earth’s history.”

• Integration questions across mass slides or across distance slides– E. g., “Would an object weigh more on the planets in our solar system made

mostly of gas or made mostly of rock? Why?”

• Integration questions across mass and distance slides– E. g., “Imagine a planet that is smaller than Earth and that was located 1.5

AUs from its sun, which is the same strength as the Earth’s sun. How would this planet’s potential for life compare to Earth’s?”

Figure 1. UCLA Study 1 - Interleaving by type of Posttest Integration Question.

Figure 2: UCLA Study 1 -Generation effects on single fact materials

Generation manipulations• Study 1 (UCLA): Sentence Completion

– Generate: ____-type planets are mostly made up of gases.

– Read: Jovian-type planets are mostly made up of gases.

• Study 2, (UCLA; and UCB Classroom Studies 1 and 2):– Sentence level generation

• E.g., “Describe in a sentence how the size of one planet's mass can affect another planet.”

– Knowledge required for successful generation:• Mass or distance Only

• Both mass and distance

Projects in Progress using WISE

• Interactions between Generation & Interleaving (UCLA and Classroom)

• Integration of M + D• Integration of M or D

• Increase Contextual Interference (UCLA and Classroom)• Habitability and Detectability

• Visual Support for Generation (Classroom)• Static versus Animated

Design Principle: Interleaving

• Interleaving as spacing: Practical advantages

• But are effects of interleaving more than the effects of spacing?

• Contextual interference ideas (Battig, 1972, 1979)

– Benefits dues to:• Reloading/ Reconstruction (e.g. Lee & Magill, 1983, 1985)• And/or Development of a higher order representation to

differentiate interleaved materials (Shea & Zimny, 1983, 1988)

Disentangling Interleaving and Spacing

• Initial experiment with second-language materials (courtesy of Hal Pashler):

• Designed to: – Maximize contextual interference– Co-vary (i.e., unconfound) interleaving and

spacing

Learning Materials

English word

Estonian Swahili

“bird”

“dege” “lind”

-8 English words learned in both foreign languages (16 word pairs total)-6 repetitions (anticipation trials) of each word pair

What is tree in Swahili?

?:

The correct word is: mfufumaji

mufuma

Sample anticipation-method learning trial

What is tree in Swahili?

?:

in Estonian?

?:

Sample Test Trial

BLUE same spacing as in blockedRED double spacing vs. blockedPURPLE double spacing vs. blocked

INTERLEAVED BLOCKEDA1 A1B1 A2A2 A3B2 A4A5 A5B5 A6A6 A7B6 A8A1 A1B1 A2A2 A3B2 A4A7 A5B7 A6A8 A7B8 A8A1 A1B1 A2A2 A3B2 A4A5 A5B5 A6A6 A7B6 A8

0

10

20

30

40

50

60

70

80

90

1 2 3 4 5 6 test

Learning Trials

% Correct

Interleaved Double Spacing

Interleaved Same Spacing

Blocked Same Spacing

0

10

20

30

40

50

60

70

80

90

1 2 3 4 5 6 test

Learning Trials

% Correct

Interleaved Double Spacing

Interleaved Same Spacing

Blocked Same Spacing

Implications and necessary next steps

• Interleaving can cause intrusions and errors during learning that then foster long-term retention--and, possibly, transfer

• The effects of interleaving and spacing – May be independent and additive,

– But contextual interference (competition) maybe be necessary to demonstrate benefits of interleaving that go beyond the benefits of spacing

• And, from an educational standpoint, it is essential to see whether the same pattern obtains with more complex and cumulative materials

Our thanks to

The Institute for Education Sciences and the Cognition and Student Learning program

Other members of the IDDEAS research team:– Lindsey E. Richland, Ph.D, (UCLA) – Britte H. Cheng (UC Berkeley) – Jason R. Finley (UCLA) – And a number of undergraduate students, especially Jeff Beyers, Fernando Cervantes,

and Alexandra Hessenius

• Relevant Links– IDDEAS : http://www.psych.ucla.edu/iddeas– WISE : http://wise.berkeley.edu– SCALE: (Synergy Communities: Aggregating Learning about Education) -

http://scale.soe.berkeley.edu– TELS: (Technology Enhanced Learning in Science) -

http://www.telscenter.org