LEARNING AND TESTING

John TaggParker UniversitySeptember 13, 2012

PLEASE HAVE AVAILABLE:

Paper and writing implement Index cards marked A, B, C, and D

DO YOU GIVE EXAMS?

a. Yesb. No

WHAT IS THE MAJOR FUNCTION OF YOUR EXAMS, IF YOU GIVE THEM?

a. To assign gradesb. To reinforce previous learningc. To get students to study before the

examd. To guide students to study after the

exame. Other

WHAT IS THE MOST IMPORTANT GOAL OF YOUR TEACHING?

a. To maximize learning at the time of the exam

b. To maximize learning at the end of the semester

c. To maximize learning a year laterd. To maximize learning five years later

TYPES OF TESTS AND APPROACHES TO LEARNING—NOEL ENTWISTLE

Essays—Encourage a deep approach Multiple-choice—Encourage a surface

approach Short-answer—Can encourage a deep

approach if well designed. Noel Entwistle, Teaching for Understanding

at University: Deep Approaches and Distinctive Ways of Thinking, 2009

FEEDBACK OR EVALUATION?

Testing for Grading = Evaluation Testing for Learning = Feedback

TESTS CAN PROMOTE LEARNING

“A constantly growing body of research demonstrates that tests are themselves learning events. The retrieval processes triggered by tests enhance subsequent recall, sometimes to a much greater degree than do comparable opportunities to restudy the information in question.” Vered Halamish and Robert A. Bjork, “When Does

Testing Enhance Retention? A Distribution-Based Interpretation of Retrieval as Memory Modifier,” Journal of Experimental Psychology, 2011.

Physical Science

Original Final Exam Score

0

2

4

6

8

10

12

14

16

18

20

0 8 16 24 32 40 48 56 64 72 80 88 96

PS100 (BOTH SEMESTERS) N=126

Skewed Left Distribution

Mean 77

Physical Science

Retaken Final Exam Score

0

5

10

15

20

25

0 8 16 24 32 40 48 56 64 72 80 88 96

Approaching Standard Normal

Mean 47

PS100 (BOTH SEMESTERS)

Physical Science

Original Retaken DifferenceMean 77.2 46.7 30.4

PS100: Original vs Retaken ScorePS100: Original vs Retaken Score

0

5

10

15

20

25

0 8 16 24 32 40 48 56 64 72 80 88 96

Score

Fre

qu

en

cy

OriginalRetaken

Physical Science

PS100: DIFFERENCES BETWEEN ORIGINAL AND RETAKEN MEAN SCORESPS100: DIFFERENCES BETWEEN ORIGINAL AND RETAKEN MEAN SCORES

A -34

B -28

C -16

A students forgot most

B students less

C students least

A students forgot most

B students less

C students least

HOW DO STUDENTS STUDY ON THEIR OWN?

“Surveys of college students show that most of them study almost entirely by rereading, with self-testing relatively rarely employed.” Doug Rohrer and Harold Pashler, “Recent

Research on Human Learning Challenges Conventional Instructional Strategies,” Educational Researcher, 2010.

BUT

“Recent studies. . . Have shown that a combination of study and tests is more effective than spending the same amount of time reviewing the material in some other way, such as rereading it.” Rohrer and Pashler, 2010

RETRIEVING IS MORE VALUABLE THAN REVIEWING

“Retrieving an event can be a more potent learning opportunity than restudying it, which flies in the face of educational wisdom that studying creates learning and testing merely measures it.” Henry L. Roediger III and Jeffrey D.

Karpicke, “Intricacies of Spaces Retrieval: A Resolution,” 2011

SELF-THEORIES—OR MINDSETS

Are field-dependent, different in different domains,

Are not reducible to self-confidence or self-esteem,

Are theories about how your intelligence or ability works when applied to challenges in a given domain.

FIXED AND GROWTH MINDSETS

Fixed Mindset

Ability is fixed, by heredity or early experience, and cannot be changed.

Growth Mindset

Ability is malleable, can be increased or decreased by effort and application.

ACADEMIC CHALLENGE

“Entering a challenging scholastic setting with a belief in fixed intelligence seems to set students up for self-doubt, anxiety, and drops in achievement. The [fixed mindset] theory puts a premium on immediate demonstrations of intellectual ability rather than on mastery over time”

--Carol Dweck, Self-Theories: Theoir Role in Motivation, Personality, and Development, 2000

SELF-THEORIES IN PRACTICE

College students were given one of two articles, advocating fixed or growth mindsets,

Then asked to work a set of problems, then another set.

After feedback, given a chance to take a tutorial.

STUDENTS WHO DID WELL ON THE FIRST SET OF PROBLEMS

Those who read the article advocating a growth mindset theory

73.3 % chose the tutorial

Those who read the article advocating a fixed mindset theory

60% chose the tutorial

STUDENTS WHO DID POORLY ON THE FIRST SET OF PROBLEMS

Those who read the article advocating a growth mindset theory

73.3% chose the tutorial

Those who read the article advocating a fixed mindset theory

13.3% chose the tutorial

HOW DOES TESTING AFFECT MINDSET? Testing for pure evaluation reinforces a

fixed mindset. Testing for feedback reinforces a

growth mindset. Embedded testing in interactive

lectures—classroom response system—radically challenges a fixed mindset.

HOW DOES TESTING AFFECT THE TIME HORIZON FOR LEARNING?

WHAT KIND OF TEST DO YOU GIVE MOST FREQUENTLY?

a. Recognitionb. Recallc. Constructed Responsed. Other

WHICH WILL MORE EFFECTIVELY SUPPORT MEMORY?

a. Recognitionb. Recallc. Constructed Response

HOW CAN YOU COMBINE RECOGNITION, RECALL, AND CONSTRUCTIVE RESPONSE?

Classroom Response Systems

TESTS OF RECALL SUPPORT MEMORY BETTER THAN TESTS OF RECOGNITION

“[E]xplicit retrieval, as required by a recall task rather than a recognition task, strengthened knowledge better than a multiple-choice test even when the final test itself involves multiple choice—and thus the effect is not attributable to a simple principle that practicing a given type of test best enhances performance on the same type of test.” Rohrer and Pashler, 2010

TWO KINDS OF TESTS

Blocked (aaabbbccc): problems or questions are grouped with other problems or questions of the same type.

Interleaved (abcbcacab): problems or questions are intermixed with other types of problems or questions.

WHAT KIND OF TEST DO YOU GIVE MOST OFTEN?

a. Blockedb. Interleaved

WHAT KIND OF TEST BEST SUPPORTS RECALL?

a. Blockedb. Interleaved

HOW OFTEN DOES YOUR PROGRAM RETEST INFORMATION FROM PAST SEMESTERS?

a. Frequentlyb. Occasionally c. Never

LONGER DELAY IN REVIEW=LONGER RETENTION

“If a person wishes to retain information for several years, a delayed review of at least several months seems likely to produce a highly favorable return on the time investment—potentially doubling the amount ultimately remembered. . . .” Nicholas J. Cepeda, et al., “Spacing Effects

in Learning: A Temporal Ridgeline of Optimal Retention,” 2008.

HOW CAN TESTS PROMOTE A HOT COGNITIVE ECONOMY?

If students see testing as a process of feedback, then they will use them to reinforce learning.

If students see tests as pure evaluation, then they will be more likely to take a surface approach.

If students see tests as repeatable, they will view them as feedback; if they see them as final, they will view them as evaluation.

THE FINAL EXAM

Final: “Not to be altered or undone; coming at the end: being the last in a series, process, or progress.” Merriam-Webster Collegiate Dictionary

HOW CAN WE FRAME TESTS AS FEEDBACK?

Repeat them Interleave questions from previous

units Revise them— “Buy back” questions or

revise answers Make them preparation for another

assignment or another test Make every exam an “interim exam”—

a step toward future application of tested learning.