Presented at the University of Sussex, April 3, 2008 as part of a seminar on Reading

Comprehension: Intervention and Assessment

When, How, and Why Comprehension Training

Succeeds or Fails

Dr. Scott G. Paris

University of Michigan

sparis@umich.edu

Comprehension Is Dynamic

• No uniform standard for everyone • Texts do not have a single meaning • Comprehension is never static or final• We revise/improve understanding through

repeated readings and discussions about text meaning, structure, and style

• Students mis-understand a great deal but• Partial understandings are building blocks

for better comprehension

Basic reading comprehension is likely when

1. Decoding is fluent

2. Key vocabulary words are familiar

3. Working memory demands are low

4. Strategies are used appropriately

5. Text is interesting and coherent

6. Assessments align with reading purposes

Deeper comprehension is likely when, in addition,

1. Local inferences are automatic2. Global inferences are generated3. Comprehension monitoring occurs4. Selective re-reading and repair strategies

are used5. Motivation is sustained

Analytic comprehension is fostered when

1. Comprehension is critically re-viewed from multiple perspectives

2. Text genre and style are examined

3. Author’s purpose and craft are evaluated

4. Intertextual comparisons are synthesized

5. Social discussion provokes new understanding

6. And all these recursive acts of re-comprehending occur repeatedly as part of meaningful learning

And more, but the main points about comprehension are

1. Fluency increases efficiency

2. Strategies enrich

3. Engagement and discussion amplify

4. Recursive evaluations check, improve, revise, and consolidate

Claim #1

Training on ANY of these aspects of reading can improve comprehension

Hedge: What is gained?

• Training may, at least, provide temporary increases in some comprehension strategies, skills, or metacognition

Why?• Because as Durkin (1979) and later Pressley

et al. (1998) showed, teachers in grades1-5 provide little explicit comprehension instruction compared to time spent on:– Silent reading– Oral reading for fluency– Answering questions and retelling– Filling out worksheets– Writing in response to reading– Other language arts activities

Claim #2

The developmental timing of the specific intervention is important because

a. Highly specific training such as targeted vocabulary words, repeated re-reading, or use of specific strategies, cues, or graphic aids

b. That develop rapidly near the time of training (in the ZPD) and

c. Match assessment outcomes closely

d. Can have rapid and large effects.

In contrast to

a. General reading strategies and processes

b. That develop over long periods of time

c. That only generally align with assessments

d. Reveal modest or nil effects in short interventions and on standardized tests

e. But may have longer term benefits for reading proficiency and motivation

Raises Some General Questions

• What kind of evidence demonstrates successful comprehension training?

• Are significant differences in experimental designs necessary and sufficient as evidence?

• Should pragmatic concerns about time, money, and teacher training/capacity be considered criteria for success?

Some examples of training studies that have improved

comprehension

• With apologies for the

• Brief, incomplete, selective, and American-centric view of training studies

Reciprocal teaching - Palincsar & Brown 1984

• What is taught?

• Four main “comprehension-fostering” strategies taught together:– Question generation– Summarization– Prediction– Clarification

RT - How are strategies taught?

• Peer dialogue via guided instruction with gradual release of responsibility so students take turns as tutor and reader, e.g., “expert scaffolding”

• How, when, and why to use strategies are taught via modeling and dialogue DURING initial instruction in original study

• Later, Palinscar (1987) explicitly taught the four strategies with whole class instruction and worksheet activities BEFORE the RT dialogues

Is RT successful?

• Yes, Rosenshine and Meister (1994) found the median effect sizes on standardized tests to be .32 (but only 2 of 9 studies found sig effects) and on experimenter-designed tests to be .88 (sig in 6 of 7 studies)

Why are the effects larger on experimenter-designed

assessments?• Rosenshine & Meister claim they were easier

because the texts were– Longer and provided more context– Organized with a topic sentence and supporting detail

format– Answering questions required less background

knowledge and less text searching– Texts were similar to passages used during RT

Why is RT successful?

• Explicit strategy teaching

• Expert scaffolding in dialogues

• Peer collaboration/teaching

• Common vocabulary, shared understanding

• Practice on similar texts?

• Dynamic, charismatic teacher?

• Motivation and Hawthorn effects?

Dialogues are critical

• But, in 9 of the 16 studies, the experimenter was the teacher and little evidence is reported about the actual instruction or dialogues

• Critical issues are:– How to scaffold– How to teach teachers to use RT– Use of RT across the curriculum– Grade level to implement

Informed Strategies for Learning - Paris et al. 1984

• 2 classes each of students in Grades 3 and 5 • Four months of 14 lessons in sequenced

instruction that emphasized – Awareness of reading goals, plans, and

strategies– Specific strategies for constructing meaning from

text– Strategies for monitoring and revising

comprehension

Comprehension Lessons• Whole group discussion in 2 lessons/week for

14 weeks taught by Marge Lipson– Lesson A: Teacher explains and models strategies – Lesson B: Discussion reviewed; more independent

practice with strategies– Lesson C: Independent practice and review

• Based on metaphors such as “Be a reading detective” to make cognitive processes more apparent and to use common language to support shared understanding

• Bulletin boards and worksheets for practice

Daily Strategy Instruction

• Ask how, when, and why they used various strategies

• Direct explanations and strategy modeling in whole group activities & guided reading (brief mini-lessons)

• Remind students to use specific strategies and identify/praise appropriate use (teach transfer)

Key Student Insights

• Making thinking public through

• Metacognitive conversations about knowledge construction.

• Effort and agency emphasized for

• Independent strategic reading.

ISL Outcomes

• Gates-MacGinitie Reading Tests to assess achievement - nonsig

• Error detection and cloze tests to assess appropriate strategy use - sig

• Reading Awareness Index and Strategy Ratings - used to assess children’s metacognition about reading - sig

• Like RT, experimenter-designed outcomes showed greater effects than a standardized test, perhaps for the same reasons

Cluster Analyses: Which Children Learned What?

• Cross & Paris (1988, JEdPsych) - clustered students in each grade by reading performance (GM, Cloze, ED) vs reading awareness (RAI,SR)

Main Findings

1. Performance and awareness not strongly related in grade 3

2. Increasing congruence between awareness and performance at grade 5

3. Grade 3: all clusters gained except the lowest group

4. Grade 5: All clusters showed some gains5. Some awareness may be necessary to

benefit from metacognitive instruction

Direct Explanation of Strategies - Duffy et al. 1987

• Teachers in grade 4 modeled strategies such as predicting and summarizing with think alouds in group instruction

• Teachers explained what, how, and why strategies operate to help reading

• Teachers asked students to use strategies and think aloud about what they were doing

• Significant benefits for students compared to traditional instruction

Four Characteristics of Instructional Explanations

1. Information sharing is interactive and responsive

2. Enhanced awareness is an instructional objective

3. The what, how, when, and why features of strategies are explained directly in whole class discussions

4. Teachers consider students’ perspectives as they explain reading strategies gradually, sequentially, and explicitly

Duffy et al. RRQ studies

• Revealed sig effects of whole class instruction on reading comprehension

• But, also revealed– How difficult explanations are and– How effectiveness of training depends on

teachers’ knowledge

Instructional Explanations

• See Wittwer & Renkl (Educational Psychologist, 2008) who argue that IE should focus on:– Learners’ knowledge prerequisites– Concepts and principles– Integrated into ongoing cognitive activities– Supplement, not replace, learners’ knowledge-

construction activities

Instructional Explanations

• Are difficult for teachers (Durkin, 1981; Duffy, 1993; Wittwer & Renkl, 2008) because:

• Mentioning and assigning are more frequent than coherent explanations

• Students respond (perhaps passively) to teacher’s prompts to use strategies rather than being taught the self-regulated use of strategies (Pressley, 1998)

Maybe that is why Pressley’s approach was called

• “Beyond direct explanation: Transactional instruction of reading comprehension strategies” (1992, ESJ)

• TSI lessons include 3 main features:1. Direct explanation/instruction on strategies2. Coordinated and flexible use of strategies3. Cycles of teacher-student transactions in

which the group works to make sense of text

TSI is successful because

• Good explanations of reading strategies• Guided practice with feedback• Metacognitive discourse about texts• Excellent teachers• Practice on tasks like the assessments• Same reasons (and problems) as RT, ISL,

DE approaches

Implementation Issues

• Avoided by early design and training studies because expert researchers provided training

• Apparent when methods given to teachers who have little background or understanding of teaching principles

• Hindered by lack of materials for teaching strategies

Duffy (1993, ESJ)

• Described case studies of teachers trying to implement comprehension strategy training in their classes

• Most had severe challenges understanding and implementing metacognitive explanations of strategic reading

• Professional development requires 2 years of guided practice

Along came the NRP (1999)

• Who reviewed research with rigorous standards and claimed support for only 7 types of instruction that showed increased comprehension

NRP’s Seven Scientifically-Supported Strategies

1. Comprehension monitoring

2. Cooperative learning

3. Graphic and semantic organizers

4. Question answering

5. Question generation

6. Summarization

7. Multiple strategy instruction

What about other strategies and training studies?

• Failed to meet the NRP gold standards including failures to:

• Assign students randomly to conditions• Expose treatment and control subjects equally

to training materials• Provide info about time spent on dependent

variable tasks• Study fidelity of implementation by teachers

and students during instruction• Assess long-term and generalization effects

NRP raises questions about evaluating effects of training and

what counts as evidence for policy-making

• See Slavin (2008) Educational Researcher for rejoinder about “practitioner-friendly syntheses of research”

Post NRP and NCLB research

Two main trends in training studies

1. Peer-assisted learning

2. Strategies embedded in content of curriculum

3. See Liang & Dole, Reading Teacher, May 2006

Peer-based training: Two examples

• Collaborative Strategic Reading • Peer-Assisted Learning Strategies

Collaborative Strategic Reading (Vaughn et al)

• Students learn 4 strategies by using cards as prompts

• Preview card (predict and connect)

• Clunk card (monitor comprehension)

• Get the gist card (main ideas)

• Wrap up card (summarize)

Collaborative Strategic Reading

• Teacher instructs students in use of 4 cards through modeling and think alouds

• Authors provide scripted lessons and cue sheets for teachers so prep time is high initially then decreases

• Students use repetitive routines with cards for months in small groups so they learn how to collaborate and discuss understanding of text

Peer-Assisted Learning Strategies (Fuchs & Fuchs)

• Good and poor readers paired as Coach and Reader

• Coach then Reader reads the same text for 5 minutes.

• Reader retells,

• Coach prompts summarizing, then

• Take turns reading new text and identifying main ideas with cue cards

Peer-Assisted Learning Strategies

Prediction relay with subsequent text1. Predict what will happen next

2. Read half page accurately

3. Check the prediction

4. Summarize important information

Peer-Assisted Learning Strategies

• 30 minutes per day: 3 times/week

• For 15 weeks

• Teacher models and explains to whole class, later monitors effectiveness of pairs

• Teacher preparation high initially then decreases

Content-Based Strategy Instruction

Questioning the Author (Beck & McKeown) Four essential features

1. View the text as fallible information written by fallible authors

2. Interacting with text through questions that enrich understanding

3. Question asking and answering while students read

4. Encourage student collaboration

Goals and Prompts of Questioning the Author

• Initiate the discussion – What is the author trying to say?

• Focus students on the author’s message– That is what the author said, but what does it mean?

• Help students link information– How does that connect with what the author told us?

• Identify author’s presentation style– Does that make sense? Is it said clearly?

• Help students identify text bases for inferences and misinterpretations– Did the author tell us that?

Questioning the Author

• Whole class or small group format

• Discourse-based instruction

• Teacher preparation time is high

• Teacher knowledge of text must be thorough

Focus on Both Content & Strategies: CORI

• Concept-Oriented Reading Instruction (CORI) - Guthrie

• Motivates students to be engaged readers• Teaches strategies and skills• Teaches conceptual knowledge • Uses social collaboration

Four Phases of CORI Instruction

1. Observe and personalize

2. Search and retrieve

3. Comprehend and integrate

4. Communicate to others

CORI Teaching Strategies

• Modeling

• Scaffolding

• Partner practice

• Independent practice

CORI Reading Strategies

• Asking appropriate questions

• Searching for information

• Comprehension

• Organizing and retaining ideas

Is CORI successful?

• Guthrie and Van Meter (1996;1999) showed that CORI students were more engaged and had greater conceptual knowledge compared to traditional instruction

Questions

• How do teachers learn to use CORI?• Does it fit any curriculum? Any grade level?

All students?• How much comprehension instruction

occurs? • Is the gain in text comprehension due to

better reasoning or enhanced engagement?

Responsive Engagement Instruction - Taylor

• High level questioning distinguished effective instruction in high-poverty schools (Taylor et al, 2002)

• High-level student discussions of text increased comprehension and motivation (Taylor et al. 2003)

Responsive Engagement Instruction

• Rich texts with themes & big issues• Ask and answer big “juicy” questions• Make personal connections• Engage in elaborated reasoning• Use response logs to facilitate

discussions• Student-led conversations about texts in

small groups

3 Year Study compared

• Responsive engagement instruction

• Cognitive strategy instruction

• Vocabulary instruction (tier 2 words)

Preliminary Conclusions

• Many positive effects favor REI

• Greater impact of CSI and REI on grades 4/5 than on 2/3

• Professional development revealed teacher difficulties with implementation and teachers asked for 2 years of training

• Combination of CSI starting with REI frame may be best

Content-Rich Comprehension Instruction - Duke

• Provides a reason for comprehending

• Necessary for understanding informational text

• Links with other instructional approaches such as CORI and Questioning the Author and avoids isolated strategy instruction

Content-Rich Comprehension Instruction

• Use content-rich texts

• Use authentic contexts and purposes

• Connects to important world knowledge

An example: IDEAS modelRomance & Vitale (2001)

• Reading/language arts block replaced with a 2 hour daily block devoted to concept-based science lessons

• IDEAS block includes:– Concept mapping– Hands-on activities– Science process and investigations

• Reading science texts• Writing science journals and writing from

concept maps

Review of Approaches

• 1970s - isolated strategy training emulated research on memory strategies

• 1980s - multiple strategies explained with metacognitive & scaffolded instruction

• 1990s to present - strategies were embedded in larger goals of engaged reading and content learning. Peer-assisted learning was used to reinforce strategies

• Future? - building teacher knowledge and using technology to support comprehension

Comprehension Training Fails When

• Explanations are incomplete or unconvincing and teacher knowledge is inadequate to teach self-regulated use of appropriate strategies

• Training is incompatible or not integrated with other learning methods and curricula

• Too brief; too boring; too decontextualized• Superficial discourse, questions,

collaboration, turn-taking, or prompt-following

Comprehension Training Succeeds When

• Teachers understand cognitive processes and strategies of reading deeply and explain them to students

• Teachers stimulate metacognitive discourse about texts, thinking, and meaning-making as part of learning the curriculum

• Recursive and sustained engagements with text are part of daily classroom activities