Web-based training of metacognitive strategies for text comprehension…kpata/haridustehnoloogiaTLU/poor... · 2006-09-19 · comprehension: Focus on poor comprehenders MINA C. JOHNSON-GLENBERG

Web-based training of metacognitive strategies for text

comprehension: Focus on poor comprehenders

MINA C. JOHNSON-GLENBERGWaisman Center University of Wisconsin Madison, WI, USA; The NeuronFarm, LLCMadison, WI, USA

Abstract. Metacognitive reading strategies were trained and practiced using interactive

Web-based tools. Twenty middle school poor reading comprehenders were trained intwo metacognitive strategies using a Web-based application called 3D-Readers. Thetraining texts were science-oriented and merged the narrative and expository genres.

Results from a within-subjects design answered two main experimental questions: (1)Were greater comprehension gains demonstrated after reading experimental texts withembedded verbal (generate questions) and visual (create a model) strategies compared to

control texts? (2) Did the embedded strategies affect elective rereading of the texts? Thedata answered both questions in the affirmative. Comprehension, as assessed withconstructed answers, was significantly higher in the experimental condition, thus dem-

onstrating the efficacy of training verbal and visual strategies in a Web-based envi-ronment. In addition, participants elected to reread more often in the experimentalcondition (as assessed with number of clicks to ‘‘ScrollBack’’ through the text), thusdemonstrating the efficacy of strategy training on text reprocessing. Interestingly, the

poorer comprehenders altered their rereading behavior the most. Implications for Web-based instructional applications are discussed.

Key words: At-risk readers, Comprehension, Computer-assisted learning, Metacognitive

strategies, Reading remediation, Struggling readers, Web-based applications

It is estimated that 95% of schools in the US have Internet access forlearning purposes (http://www.nces.ed.gov). Many classroom teachers areturning to the Web for instructional purposes and there is great potentialfor Web-based training of higher level cognitive activities like readingcomprehension. However, new learning programs must be designed usingresearch-based educational theory to take advantage of the interactivityand synchronicity of the Web. Students can now get immediate feedbackand guidance regarding free text performance, i.e., it is now possible tomove beyond the somewhat rigid assessment formats of multiple choiceand automatically score constructed text on-line. Learning moduledesigners need to be aware of ‘‘potential mismatches between current

Reading and Writing (2005) 18:755–786 � Springer 2005DOI 10.1007/s11145-005-0956-5

technologies and human learning’’ (Reyna, Brainerd, Effken, Bootzin, &Lloyd, 2001, p. 40). This means that design elements should be rigorouslyresearched, analyses should include aptitude by treatment interactions,and elements should be tested on heterogeneous non-mainstream popu-lations. This study focuses on poor reading comprehenders and whetherbetter reading practices can be trained using Web-based tools.

The technology used in this study is based on an experimental softwareprogram called 3D-Readers that harnesses the immediacy and dynam-icism of the Web to teach, assess, and allow practice on two powerfulmetacognitve reading strategies, one verbal and one visual. The verbalstrategy involves the creation of questions while reading the text, and thevisual strategy involves the facilitation of building a mental model bymanipulating images on the screen that construct main ideas. The textsare hybrids of both the expository (science) and narrative (prosodic)genres. Narrative is a natural category and is poorly served by sharpboundary distinctions. Graesser, Golding, and Long’s (1991) descriptionis elegant, ‘‘Narratives are expressions of event-based experiences that (a)are either stored in memory or cognitively constructed, (b) are selected bythe writer to transmit to the reader, (c) are organized in knowledgestructures that can be anticipated by the audience’’ (p. 174). This study’shybrid texts were created to facilitate comprehension in struggling read-ers. The working hypothesis was that using a familiar knowledge struc-ture would decrease the cognitive load associated with processing thenovel scientific concepts. Expository text is more difficult to understandand retain. Narrative resembles more closely orality. The knowledge ofnarrative story structure is acquired before reaching school age (Stein &Glenn, 1979). A further benefit from narrative may be that the genredepicts ‘‘event sequences that people in a culture directly enact or expe-rience’’ (Graesser, Golding, & Long, 1991, p. 172).

To better understand the training program, a quick synopsis of atypical session is presented. Each time readers log on to an experimentaltext on the Website, the following order of events occurs:

(a) Readers first indicate whether they have prior knowledge of the topic.This task activates prior knowledge and facilitates comprehension.(Because the prior knowledge assessment in this first iterationinvolved only a coarse-grained yes/no answer and did not correlatewith any other variables, it will not be discussed further.)

(b) Readers answer seven vocabulary questions, which contain some ofthe more difficult words from the text. These answers are used inconjunction with post-reading vocabulary to assess changes inknowledge. They further activate prior knowledge.

756 MINA C. JOHNSON-GLENBERG

(c) Readers read a section of text and at an appropriate moment areprompted to work on a verbal strategy, i.e., they create a question.They see and hear the instructions ‘‘Pretend you are a teacher. Whatquestion would you ask here?’’ Readers type in the question and it issent to an automatic scoring algorithm.

(d) After the next text section, readers work on a visual strategy designedto facilitate imagery skills and deepen understanding. The visualstrategies are colorful screens split in two. Typically, on the right sideis the toolbox filled with icons depicting important conceptual ele-ments from the text, and on the left is the ‘‘model space’’ where theicons are assembled. That is, readers drag icons into the model spaceto create a visual representation of one of the main ideas.

(e) Readers engage in one more verbal strategy, and one more visualstrategy for a total of four strategies. Note that before each strategythey are prompted to reread if there was anything they did notunderstand. They then exit the text and are not able to re-read.

(f) Readers answer the seven vocabulary questions again as a post-test.(g) Finally, readers answer eight open-ended comprehension questions.

Their constructed answers are scored immediately by the automaticscoring algorithm. Readers are shown their average score after theeight questions so that they can track their progress over time.

The strategies

According to Gough and Tunmer (1986) reading can be portioned intotwo major components: decoding and linguistic comprehension. Thismeans that if readers have adequate decoding skills (and motivation), butstill have problems understanding what they have read, then their primarydeficit lies at the higher level of linguistic comprehension. This would alsoinclude the interfacing between the two skill levels. How can the power ofthe Web be used to remediate higher level reading comprehension deficits?Automatic essay scoring algorithms can be used to score generatedquestions and answers to questions, and advances in streaming technol-ogy now allow for colorful, interactive graphics that might facilitate theconstruction of mental models. Equally important is the aspect ofimmediate, personalized feedback so that readers can repair miscompre-hension while still engaged with the text.

The meta-analysis report of the National Reading Panel in America(NRP, 2000), lists 16 categories of comprehension instruction, includingmental imagery. The NRP results section stipulates that seven of these

757WEB-BASED TRAINING OF METACOGNITIVE STRATEGIES

categories demonstrate firm scientific evidence supporting their efficacy inimproving comprehension in typical readers. This study’s Web-basedinstructional program utilizes five of these seven strategies: Comprehen-sion monitoring, graphic organizers, question answering, question gen-eration, and summarization. (Some summarization is always required inthe final answers). This study’s training package does not use semanticorganizers or cooperative learning, although there are plans to reconfig-ure it for the latter.

The two trained strategies are considered metacognitve. This studyoperates from a definition put forth by Paris, Cross, and Lipson (1984).Metacognition has two fundamental aspects: knowledge about cognitionand self-directed thinking. Self-directed thinking is governed by evalua-tion, planning, and regulation activities. This program aims to modifythese three activities. The hypothesis is that with repeated practice on theverbal and visual strategies, and with repeated immediate feedback onperformance, readers will be aided in their evaluation skills. Evaluationwill, in turn, help activate planning and regulation activities. With prac-tice, this evaluation – or monitoring and self-assessment – should becomemore automatic. Poor comprehenders need fast, flexible, internal, andtangible strategies that they can call on in times of miscomprehension. Amixture of both verbal and visual metacognitive strategies was chosen forthe first iteration of study.

Johnson-Glenberg (2000) describes a metacognitve intervention studywith 45 adequate decoders, but poor comprehenders. In that study 3rdthrough 5th graders (average chronological age 9–11) were randomlyassigned to three groups. Two groups were instructed in two very dif-ferent comprehension strategies. The first was the package of verbalstrategies from Reciprocal Teaching (RT) created by Palincsar andBrown (1984), and the second was a visual package called Visualizing/Verbalizing (V/V) created by Bell (1986, 1991). The third group wascomposed of untreated controls. RT is a comprehension-monitoring andcomprehension-fostering package containing four strategies: summari-zation, prediction, clarification, and question generation. V/V is a pri-marily visual strategy in which readers learn to mentally create, and thendescribe ‘‘movies in their heads’’ as they read. After 10 weeks of training,the two experimental groups demonstrated significant gains on severalmeasures related to reading comprehension when compared to theuntreated control group. The experimental groups constructed betteranswers on both implicit and explicit questions, and demonstrated gainsin word recognition skills. The research supported the hypothesis thatstrategy training aided the readers, and that both verbal and visualstrategies were effective.


Some strands of research, and many classroom teachers, refer to thesestrategies as visual. Although it may be more accurate to call the strate-gies imaginal, we would like to be consistent in our publication termi-nology and will refer to them as visual. This study is multicomponential.In part because research supports that comprehension instruction shouldcontain multiple components (National Reading Panel, 2000; Pressley,2000), and in part because resources did not allow for a mixed 2 · 2crossing the verbal and the visual strategies.

The verbal strategy – creating questions. Of all the specific verbal strategiesavailable, question generation is supported by the strongest scientificevidence (National Reading Panel, 2000). Rosenshine, Meister, andChapman (1996) reviewed intervention studies that taught students howto generate questions and found that training yielded an impressivemedian post-training effect size of .86 on experimenter-designed mea-sures.1 As further support, in a 1996 study, Trabasso and Magliano gavethird grade students three stories to read. After the second story, onegroup of students was asked ‘‘why’’ and ‘‘how’’ questions. This grouprecalled significantly more of the third story. In addition, the ‘‘how’’ and‘‘why’’ question group gave qualitatively more sophisticated explanationsduring the later think-aloud protocol. These results suggest that afterbeing asked ‘‘how’’ and ‘‘why’’ questions subsequent comprehension isenhanced because readers must generate causal links and goal plans. Thecurrent study takes this construct a step further, and allows the readers toask the ‘‘how’’ and ‘‘why’’ questions of themselves. Tovani (2000) assertsthat readers who ask questions improve their comprehension by inter-acting with the text and remaining focused on it. Asking questions fosterscuriosity because readers are forced to continue reading to answer theirquestions, clarify information in the text, and, finally, go beyond literalmeanings and to begin the first step in the processes of deduction orinference.

At appropriate locations in the text, readers are prompted for ques-tions and they type in their questions. These are automatically scored forquality. Quality was assessed with several methods to be described later.The two main criteria for higher scores were that the answer be multi-word and that the question began with ‘‘how’’ or ‘‘why’’. For the firstcriterion, the answer to the question should require a sentence or manywords to answer. Effective questions were ones that required deeperprocessing of the text or even inferencing. Examples were given ofquestions that were: not very effective – ‘‘Did you like the story?’’, moreeffective/good – ‘‘What part of the eye do you see when someone has ‘red-eye’ in a photograph?’’, and most effective/excellent – ‘‘How does


someone get ‘red-eye’ in a photograph?’’. For the second criterion, it wasexplained to the readers before the experimental condition that generallyquestions that started with either ‘‘how’’ or ‘‘why’’ were requestinginformation that required deeper processing.

The visual strategy – building mental models. The ultimate act of reading isthe creation of a mental model (Johnson-Laird, 1983 – also called asituation model, Kintsch, 1988). Although not all mental models are vi-sual, research on the importance of imagery and visualization duringinformation processing and reading has a long and respected history.Based primarily on Paivio’s Dual Code Theory (Paivio, 1986; Sadoski &Paivio, 2001), numerous studies report significant comprehension gainsby experimental groups that were either encouraged, or taught to visu-alize while reading (Bell, 1991; Gambrell & Jaywitz, 1993; Johnson-Glenberg, 2000; Levin, 1973; Mayer & Sims, 1994; Oakhill & Patel, 1991;Pressley, 1977). Creating a visual model on the screen concretely aidspoor readers in building internal visual models. By manipulating andbuilding images, readers discover and/or confirm how the segments ofsequential text fit together in a ‘three dimensional’ gestalt. In this currentstudy an incomplete, stylized version of a visual ‘‘mental model’’ appearson the screen. The reader interacts with the model, and is in turn suppliedwith immediate feedback regarding performance.

The visual strategies are designed to serve two purposes: (1) to train incomprehension by serving as a stepping stone between text and mentalrepresentation, and (2) to assess text comprehension. As an example,readers might first be shown an animation of how a practice sentenceshould be built (the training stage). The practice sentence at the top of thescreen might explain that the color of an object is really the lightwavesthat the object is reflecting. Therefore, a banana is reflecting. . .? Theanimation would show the correct icons being dragged out of the toolboxand placed in order – first a flashlight which would emit white light, andthen a banana, and then rays with a yellow hue that bounce off of thebanana. Readers are then expected to build their own model of a realsentence from the text which would require near transfer (e.g., ‘‘Whatcolor would the iguana be reflecting?’’). Figure 1 illustrates one black andwhite screenshot of this intensely colorful interactive visual strategy.

When readers feel confident about their self-assembled models, theysubmit them for scoring and receive a percent correct score (the assess-ment stage). Readers are allowed up to three submissions. After the finalsubmission, if the reader does not have a 100% score, the system usesanimation to build the correct model.


Rereading (ScrollBacks). Rereading has been operationalized as Scroll-Backs. Readers could click on one of two buttons to scroll back through asection of text to reread it. Only four lines of text were visible at a time.Mature (good) readers use the strategy of rereading (Pressley, 2000).Memory for text has been shown to improve with rereading (Amlund,Kardash, & Kulhavy, 1986). Millis, Simon, and tenBroek (1998) positthat rereading facilitates comprehension because it allows readers tocomplete processes which produced less than ideal outputs from priorreading. Poor readers are often disinclined to reread text. They may notunderstand that the goal of reading is to extract meaning (several poorreaders have reported to the author that the goal of reading is to ‘‘get tothe end’’). Poor comprehenders may be unaware that they are not com-prehending, or they may not know that such a strategy even exists. Weincluded rereading as a variable in this study to ascertain whetherintroduction to and practice on other higher-order, more metacognitivereading strategies might affect the strategy of rereading.

Vocabulary. The pre-reading vocabulary test serves as a brief assessmentof content knowledge and stimulates prior knowledge. Seven of the moredifficult words are culled from the text, and a multiple choice format is

Figure 1. Screenshot of an interactive visual strategy.


used for assessment. Although multiple choice has its limitations, it isquickly administered and allows for a straight-forward subtractionmethod to assess change in knowledge at the time of post-test. SeeAppendix A for a sample of the vocabulary test.

A word on essay scoring algorithms

Most reading assessment relies on answering multiple choice questions.This is a limited knowledge assessment paradigm. In addition to adding alevel of chance (usually 25–33%) to a reader’s score, it relies on a rigid,one-correct-answer template. This format erroneously sends readers insearch of the ‘‘one true meaning’’ of the text (Pearson & Hamm, 2001).The eight final questions in this study are open-ended and require stu-dents to construct more authentic answers. The system uses the High-dimensional Expert Match Algorithm (HEMA) to score a reader’s gen-erated questions and constructed answers.

The inclusion of HEMA also means that there are checks and balancesin the system. One of the attractive elements of employing both visual andverbal strategies is that the system can immediately cross-reference areader’s performance on both types of strategies and alert teachers to anydiscrepancies. This could happen in either the verbal or visual domains. Itis important to note that even though HEMA represents the vanguard ofcomputerized automated scoring, a computer will never be as flexible andcreative as a human scorer. No claims are made that the processing inHEMA is isomorphic to human cognition. HEMA is merely a tooldesigned to mimic, as closely as possible, human scoring of written lan-guage. Language is by definition generative and evolving; decades of AIresearch have taught us that the true correlation between human expertassessment and computer-generated assessment will never be 1.00.HEMA makes this Web-based remediation system unique in that itallows readers to create their own questions (which in the next iterationwill be scored and sent as feedback). HEMA allows readers to answeropen-ended questions and receive immediate feedback. Space constraintsdo not allow for more detail on the algorithm; it is mentioned here pri-marily as a reliability measure for the human scorers.

Predictions

To measure the outcome validity of the strategy training, two differentcomprehension measures were chosen as dependent variables. The first


was constructed answers, the prediction was that in the experimentalcondition (with the embedded strategies) participants would demonstratesignificantly greater gains on the final answers – as scored by both humanexperts and the HEMA algorithm. The second variable was vocabularytest gain from pretest to post-test, the prediction was that in the experi-mental condition the gain would also be greater. Data were gathered onnumber of ScrollBacks (rereads), and the scoring of the readers’ generatedquestions (by both humans and HEMA). The prediction was that in theexperimental condition readers would use the ScrollBack mechanismmore frequently because they would be engaged in deeper text processingand would now be more aware or cognizant of the fact that they neededto reread a section. It was also predicted that readers who generatedhigher level questions would also demonstrate better comprehension onthe final constructed answers.

Method

Participants

The study included 20 participants from one middle school in a lower-middle class neighborhood of a small midwestern city in the US. Sixparticipants were in the 7th grade and 14 were in the 6th grade with anaverage chronological age of 12.5 (range from 11.10 to 13.6). The readingspecialist at the school handled the majority of recruiting and scheduling,utilizing prior contact with the students and/or referrals from teachers.We asked teachers for students who were poor comprehenders. Wespecified that they either be below the mean on the state’s standardizedreading test (TerraNova Test – CTB/McGraw-Hill, 1997) or currentlyreading below the mean for the class in text comprehension. We remindedteachers that these children are often poor listeners as well, so they mightalso be the children who do not follow directions well and ask repeatedly,‘‘What are we supposed to be doing?’’ In addition, we stipulated that theybe able to decode texts written for the middle of 5th grade. Poor com-prehenders were selected for two reasons. First, they are the primarytarget audience for this software, and second, they are the readers whooften show the greatest gains from comprehension instruction programs.The poor comprehenders in Oakhill and Patel’s (1991) study demon-strated a greater benefit from imagery training than the better com-prehenders. Furthermore, teachers were asked to not recommend readerswho were English language learners (ELL) or who were classified ashaving special needs (EEN-Educable Exceptional Needs).


Standardized reading scores from the previous spring’s TerraNovaTest (CTB/McGraw-Hill, 1997) which measures both decoding andcomprehension, revealed an average percentile score of 33.26%(SD=17.30) for the sample. One student was above the 50%ile (at the72%ile), which was surprising since he was only the 8th ranked in finalcomprehension scores in this study. Perhaps his teacher felt that he wasstarting to lag behind his classmates during the fall. Because deleting hisdata did not alter the results, his data have been retained in the analyses.Two participants left the study. One was a new transfer student whoapparently was decoding at around the 3rd grade level, and one self-selected to stop after several sessions so that he could remain in hisclassroom. The ethnicity breakdown was as follows: 45% African-American, 40% White, 10% Asian, and 5% Hispanic. The sample was55% female.

Materials

Room and computers. The main room for the study was the computer lab.It was a glassed-in room in the middle of the library. When all studentswere present in the two largest groups, two computers were used in thelibrary and a trainer stayed in the library with them. There were rarelyother students in the library at this time. The computers were all Internetconnected PCs, of 486 speed or higher; the majority manufactured byCompaq.

Texts. The seven original texts (the practice text was repeated once) werewritten by the author and two middle school teachers whose specialitieswere science and reading instruction. The texts were rewritten on averagesix times by committee until they were balanced on several key variables.Very specific guidelines were created and followed, the texts had to be (a)engaging in an age-appropriate and prosodic manner, (b) synchronizedwith the State of Wisconsin middle school content standards for science(via FOSS – Full Option Science Program, Lawrence Hall of Science;http://www.lhs.berkeley.edu/foss/), (c) visual enough to facilitate thegraphical mental model strategies, and (d) less than 1,800 words so theycould be finished in the half hour time period. The topics, which wereappropriate for the school district’s content standards, were (1) how tomeasure a wave (for the short practice texts), (2) volcanoes, (3) satellites,(4) color as wavelength, (5) how the eye works, (6) biomes, and (7) tele-scopes. The texts contained an average of 1,445 words and the averageFlesch-Kincaid Readability score was 5.22 (beginning of 5th grade). The


texts were written with a mixture of narrative and expository elements.For example, in the text on biomes the twin brother and sister hiked up amountain where they experienced the four major biomes in one setting –as they joked/fretted about being followed by a bear. See Appendix B forthe text on color as wavelength.

ScrollBacks. We operationalized rereading with ScrollBacks. Text wasdouble spaced 14 point font. In a window in the center of the screen fourlines (including blanks) were legible at a time. The text before and after thereading window was masked with case-sensitive Xs and the backgroundwas grey. The reading text background was white. On the right side of thescreen were four buttons. The bottom two buttons scrolled the readerforward, and the top two scrolled the reader back through one section oftext. There were five sections per text. The double upward-pointing arrowscrolled readers up five lines of text. The single upward-pointing arrowscrolled readers up one line of text. This was explained to readers in thefirst session. Figure 2 illustrates a text page and the scrolling interface.

HEMA. Because we were committed to assessing knowledge throughconstructed, or free text, we needed to design an algorithm powerful, yet

Figure 2. Example of the text reading and scrolling interface.


flexible, enough to score the range of readers’ responses. Among severalversions of automatic essay scoring algorithms on the market, one of theleaders is KAT (Knowledge Assessment Technologies) based on LatentSemantic Analysis (Landauer & Dumais, 1997). The algebra in HEMA issignificantly different from other automated systems and sometimesincludes neural networks. Our purpose is different as well, it is to assessthe very short, very specific types of question and answers that childrenages 10 through 17 create for hybrid-style texts. We designed two versionsof HEMA to be able to score both the questions generated and theanswers constructed. For purposes of this article HEMA is only discussedinsofar as it supplements interrater reliability.

Design

This studywas allowed access to only one school and because it was a ‘‘pull-out’’ programwith a specific population, it had access to a small number ofparticipants. To control for variability a within subjects designwas utilized.Because it was necessary to also control for cognitive carry-over effectsassociated with educational interventions, the control condition alwayscame first. In the control condition participants unscrambled four ana-grams embedded in the texts. In the experimental condition the two meta-cognitive strategies were embedded twice in the texts (for a total of four). Tocontrol for experimental story effects, order of text was varied between fourorder conditions in a modified Latin Square. Participants were randomlyassigned to order condition. Participants were seen in three groups a day,the number of participants in each group was eight, five, and seven.

There were eight sessions in total. The anagram-embedded control textswere always read in the first three sessions. In the control sessions studentsstopped reading four times to unscramble anagrams. The control condi-tion’s purpose was to serve as a lexical (not metacognitive) task, and toequate both conditions for time on task. Otherwise, the strategies wouldjust have made the experimental condition last longer. The anagram wordswere comprised of the more difficult words from the section of text that hadjust been read. The writers chose the ‘‘more difficult’’ words by consensus;words had to be longer than four letters and by group consensus consideredlow frequency for that age group. Readers were always warned before ananagramappeared andprompted – via voiceover and text – if theywished togo back and reread anything (similar to the prompting in the experimentalcondition with strategies). There were four anagrams in each control text,except in the short practice texts which contained two. The anagram tasktimed out after 3 minutes and readers were then shown the answers.


The experimental texts contained two verbal and two visual strategieseach. In both conditions no one text followed another more than onetime. The control texts were written at the same time, and in the samestyle as the experimental texts. Texts were randomly selected to becomeeither experimental or control. Prior topic knowledge and pre-readingvocabulary scores were not statistically significantly different between thetwo conditions. Sessions lasted approximately 30 minutes. The entirestudy lasted 2 weeks.

Trainers. There were two full-time trainers, their job was to troubleshootproblems with the computers and navigation – not to answer contentquestions. There were three ‘‘as-needed’’ tutors who came in during theone-on-one ‘‘What is a strategy’’ session at time 4. The trainers includedthe author (who has over 15 years of clinical and research experienceteaching reading) and the school’s reading specialist. The tutors includedtwo graduate students in either the Educational or Cognitive Psychologyprograms, and the computer lab manager who was training to become aPrincipal. All tutors received several sessions of training by the author,and during session 4 everyone followed a protocol booklet (protocolavailable at www.3D-Readers.com under T3). In designing the protocolbooklet to aid with instructor fidelity, much time was spent developingexplanations using age-appropriate language to describe and rationalizethe use of strategies. It is well known that motivation is a substantialcomponent in whether poor readers will actually stop and work meta-cognitive strategies on their own. (We also stressed that these were notjust strategies to be used on the computer, but anywhere and anytime thatstudents were reading they should stop and use their ‘‘new tools’’.)

Procedure

The time line was as follows:

Time 1: Introductions were made and navigation discussed. The readersread the first short practice control text called ‘‘The Wave’’ I withtwo embedded anagrams. This text was shorter than the otherswith only 740 words. (Note: This text appeared again as theexperimental practice text at time 4.) It also contained only fourvocabulary choices and six final questions.

Time 2: Read Control text.Time 3: Read Control text.Time 4: Human tutors sat with the readers for approximately 10 minutes

(tutor to reader ratio 1:1, in three instances 1:2) and worked


through the protocol booklet explaining to the students what thestrategies were and why they were important.2 The readers thenread through the experimental version of ‘‘The Wave’’ II practicetext. This time, instead of anagrams, readers worked through twoof the metacognitive strategies. The text contained one verbalstrategy (create a question) and one visual strategy (build a model).No results from this session were included in the analyses, asrereading would surely favor the experimental condition.

Time 5 through Time 8: Readers worked through the four full experi-mental texts with four strategies in each text.

Results

The results section reports on the following four variables: (1) compre-hension assessed with open-ended answers, (2) comprehension assessedwith vocabulary gains, (3) rereading assessed with ScrollBacks, and (4)question generation. All analyses were conducted using SPSS version10.0. All Alpha levels were .05 and bi-directional.

Comprehension – constructed final answers. Once readers exited the textthey could not go back to reread. This means that the final questions (andpost-reading vocabulary) had to be answered from memory. Studentstyped in constructed answers to the eight final questions for each text (sixin the practice text ‘‘The Wave’’ I). These answers served as the primarymeasure for comprehension. They were scored using two methods: humanexperts and HEMA. The author and a graduate student in EducationalPsychology scored the answers. A significant interrater reliability wasfound, Pearson r=.92, P<.001. All interrater differences greater than 2%were resolved through discussion. Because one scorer was not blind tocondition, another reliability measure was used, the assuredly unbiasedscoring of the HEMA algorithm. The average human score significantlycorrelated with the HEMA scores, r=.79, P<.001. Thus, the functionswere very similar. However, the mean intercepts of the two types of scorersdiffered by a significant average of 13 points (t>4.00). The originalHEMA algorithm was propagated with a large range of answers, many ofwhich were too sophisticated for this 6th and 7th grade sample. HEMAdid an excellent job overall of assessing an answer’s relative value, but thebaseline was obviously set too high. (Future iterations will remedy this.)

In order to address the hypothesis of whether embedding strategiesaided in comprehension, a paired t-test compared final constructedanswers during the control condition with final constructed answers


Table

1.HumanandHEMA

means,SDs,grandmeans,eff

ectsizes(coher’sd)andcorrelationsbysessionandconditionforcomprehensionassessedwithconstructed

an-

swers.

Scoring

type

Control

Grand

mean

(GM)

Experim

ental

Grand

mean

(GM)

Effect

size

(GM)

r (GM)

Human/

HEMA

12

3(1–3)

56

78

(5–8)

Human

46.58

(25.19)

36.84

(20.99)

36.37

(22.07)

40.33

(19.58)

51.17

(21.92)

47.91

(20.73)

44.31

(21.59)

46.61

(21.17)

48.18

(15.69)

.45

Control.81**

HEMA

28.51

(19.73)

24.71

(8.51)

28.77

(10.17)

27.25

(13.59)

35.95

(11.96)

34.83

(12.57)

36.34

(10.29)

35.92

(11.86)

35.76

(11.67)

.67

Exper..76**

Note:**P<.01.


during the experimental condition. Using the human expert scores, theanalysis revealed a significant mean difference of 7.84, t(19)=3.14,P=.005; using the HEMA scores, the analysis revealed in a significantmean difference of 8.52, t(19)=4.60, P<.001. Table 1 shows the relevantstatistics. Results support the hypothesis that participants did signifi-cantly better on the texts that encouraged and allowed them to work onmetacognitive strategies.

Because of the constraints of the within subjects design a furtherimportant question presents itself, does comprehension simply improvewith time via a practice effect? Is there a linear progression, such that asreaders work through more texts their scores simply continue to improve?Figure 3 demonstrates that in each condition there were actually slightlynegative slopes associated with time.

To answer this question inferentially, a within subjects growth curveanalysis was conducted to compare the changes in comprehension acrossthe control texts with the changes in comprehension across the experi-mental texts. We wanted to determine if there was a significant interactionbetween time and condition. The interaction of time and condition on

Figure 3. Lowess function fit through the seven sessions: 1–3 are control; 5–8 are

experimental.


comprehension was quantified for each student by an individual regres-sion coefficient. A paired t-test was then used to determine if the means ofthe interaction coefficients differed from zero. The t values were nonsig-nificant at less than .90. Thus, there is not much evidence for a practiceeffect that would explain the experimental condition’s overall compre-hension superiority.3

Comprehension – vocabulary gains. The three text authors chose 10 of themost irregular, ambiguous, or age-group-infrequent words from a text.Then the seven words with the most consensual overlap were selected asthe vocabulary items. At least five of the seven had to relate specifically tothe scientific topic (see Appendix A). Readers answered the seven multiplechoice vocabulary items before each text (only four in ‘‘The Wave’’shorter texts), and they answered the same questions after reading. Ta-ble 2 presents the descriptive statistics.

A paired t-test revealed that the average vocabulary gain from pre-reading to post-reading in the control condition was significant, meangain=16.43, t(19)=4.57, P<.001. In addition, the average vocabularygain in the experimental conditionwith strategieswas also significant,meangain=10.79, t(19)=4.22, P<.001. The interaction between gain and con-dition, i.e., the difference in post-reading minus pre-reading scores betweenthe two conditions, was not significantly different, t(19)=1.48, NS.

We had predicted the experimental group would demonstrate greatergains in the post-reading condition, but this was not the case. Two rea-sons may explain the results. The first reason may be that the anagramtask encouraged readers to focus on the lexical and orthographic levels ofthe text. Anagrams (and the vocabulary words) were typically the harderwords from the text. Thus, extra attention may have been given to thenew, more difficult, and often irregular words during reading in order tounscramble them later. This attention could have aided post-readingvocabulary definition, as the vocabulary words were also the more diffi-cult, and irregular words from the text. The second reason may be thatthe control condition included the short practice text (‘‘The Wave’’ I)which had only four vocabulary choices and contained somewhat lesssophisticated decoys than the full texts. Many of the greatest gains in

Table 2. Vocabulary: percent correct, means, SD, t-tests, and effect sizes.

Conditions Pre-reading Post-reading t-test (pre to post) Effect size

Control (1–3) 56.39 (16.47) 72.73 (14.93) t(19)=4.57, P<.001 1.04

Experimental (5–8) 60.22 (17.39) 71.01 (17.08) t(19)=4.22, P<.001 .63


vocabulary were seen in this text. The median and mode for post-readingvocabulary were 100% (scored by 13 out of 20 participants) on text 1. Onall the other vocabulary pre-reading and post-reading tests the medianwas never greater than 71%, and no more than four participants everscored 100%. (Note that the second reading of the Wave practice text II(time 4) was not included in the analyses.)

Finally, how does vocabulary correlate with constructed answers?Table 3 illustrates the significant correlations between vocabulary andconstructed final answers as scored by both human experts and HEMA.

Rereading as assessed with ScrollBacks. ScrollBacks were computerizedtallies of elective rereading. The majority of ScrollBacks were 0 and 1 (noscrolling to reread, or scrolling up only one line). Because some readersdid utilize the double arrow (which scrolled up a block of five lines oftext), we had several outlier high scores, ranging from 34 to 64. Thedistribution of the raw data was non-normal; however, trimming thescores seemed too arbitrary. In order to run parametric tests on these datathe numeral 1 was added to each score, and then all scores were naturallog transformed. Thus a normal distribution was approximated and 0remained 0:1 became .69, and 64 became 4.17. Table 4 lists the condi-tional means and correlations between ScrollBacks, question generation,post-reading vocabulary and constructed answers.

We predicted there would be significantly more ScrollBacks in theexperimental condition, and the analyses revealed this to be the case,paired t(19)=2.16, P=.04. What we did not predict was that on averagethe participants who were scrolling back more often would be the rela-tively poorer comprehenders.

Figure 4 illustrates the relationship between the average number ofScrollBacks in the experimental condition minus the number in thecontrol condition and participants’ comprehension as assessed in thecontrol condition (before being altered by strategies). The correlationbetween the difference in conditional ScrollBacks and participants’ earliercomprehension was ).66 (n=20, P=.002).

Table 3. Correlations with between post-reading vocabulary and constructed answers.

Post-reading vocabulary and constructed answers Control Experimental

Human-scored .59** .81**

HEMA-scored .47* .58**

Note: n=20, *P=.037, **P<.009.


Table

4.Means,SDs,andcorrelationsbetweenrelevantreadingvariables.

MeanandSD

Correlations

Cont.

Exp.

Voc.

Cont.

(post)

Voc.

Exp.

(post)

Human

answ

er

–Cont.

Human

answ

er

–Exp.

HEMA

answ

er

–Cont.

HEMA

answ

er

–Exp.

ScrollBack

aLogN

(0–4.17)

0.96

(0.69)

1.35*

(0.95)b

0.01c

)0.69**

0.21

)0.45*

0.28

)0.12

Questiongeneration–Human(0–100%)

74.28

(13.97)

0.54*

0.63**

0.56*

Questiongeneration–HEMA

(0–100%)

64.51

d**

(7.07)

0.19

0.33

0.32

aScrollBack

isnaturallogtransform

ed.Thenumeral1wasadded

toeach

score;withouttransform

,therawcontrolmeanwas(SD)=

4.65(5.75),

raw

experim

entalmeanwas(SD)=

8.07(10.42).

bScrollBack

paired

t-test,t(19)=

2.16,P

=.04.

c Themost

relevantvocabulary

score,whichispost-readingisreported.In

addition,thevocabulary

controlcorrelationwithvocabpost-reading

minusvocabpre-readingdifference

was.32(N

S).Thevocabexperim

entalcorrelationwithvocabdifference

was.48(P=

.03).

dPaired

t-test

betweenthehumanandHEMA

questiongenerationscores,t(19)=

3.91,P=

.001.


Question generation. Readers were prompted (via both written text andvoiceover) to type in questions during their reading of the experimentaltexts. They did this once during the experimental practice text (time 4)and twice during the full experimental texts (times 5–8). The readers’ self-generated questions were saved in a database and scored by HEMA.However, the scoring of questions is contingent on a more ill-definedproblem space than the scoring of constructed answers, and so in thisiteration readers did not receive immediate feedback regarding the qualityof their questions.

The readers’ 165 generated questions were scored three times by (a) theauthor, (b) a graduate student in Educational Psychology, and (c) theHEMA algorithm. During the one-on-one, time 4 training session readerswere instructed that questions that required multi-word answers, and/orbegan with ‘‘how’’ and ‘‘why’’ were worth more points. A rubric wascreated for the human scorers which also included these criteria. How-ever, a question that used ‘‘how’’ as a count word, e.g., ‘‘How manysatellites are in the world?’’, would not receive as high a score as, ‘‘Howdo signals travel around the world?’’ The Pearson r correlation betweenthe two human scorers was .69 (P<.001). This was not as high as was

Figure 4. Relationship between ScrollBacks and control condition comprehension.


seen in the scoring of constructed answers. After meeting to discuss dis-crepancies and come to closer consensus, the scorers’ interrater correla-tion increased significantly to .85 (P<.000). Human scorers could not beblind to condition, as question generation was one of the experimentalstrategies. The Pearson r correlation between the human and the HEMAscores was .61 (n=20, P=.004). Again, humans scored the questions at ahigher baseline than HEMA did overall. Table 4 shows that the humansscored the questions statistically significantly higher (M=74.28) thanHEMA (M=64.51), paired t(19)=3.91, P=.001. The algorithm’s base-line output will have to be adjusted upwards in this instance as well.

Discussion

The training of reading comprehension using interactive, multi-mediaWeb-based tools is a relatively new field and there are many questions leftto be resolved. Hopefully, this study has moved the field forward byaddressing two important questions: (1) will training poor comprehendersin visual and verbal strategies result in significant comprehension gains,and (2) will training poor comprehenders in visual and verbal strategieseffect elective rereading of texts.

Comprehension. This study demonstrates that scores on the most sensitivecomprehension measure – constructed answers to final questions – aresignificantly higher in the experimental condition with strategies, than inthe control condition. This study also presents preliminary evidence thatthis gain is not simply a linear artifact of time and practice. Although thevocabulary gains were not significantly different between the conditions,the unscrambling of anagrams may have activated some low-level lexical/verbal processes that resulted in vocabulary gains which were not con-tingent on metacogntive processing. However, it is the use of higher-levelverbal strategies and the addition of the visual/imaginal processing thatappear to result in an increase in deeper comprehension for the readers, asassessed by the constructed responses. Because the verbal and visualstrategies were not separately administered, the question of which strategyeffects the greatest change cannot be addressed. Many researchers supportthe use of multi-componential reading programs (NRP, 2000; Pressley,2000). It may well be the case that the majority of proficient readers useboth verbal and visual processes during situation or mental model crea-tion. This is a view supported by Paivio’s (1986) Dual Code Theory.

The Dual Code Theory posits three levels of processing or meaning forboth the verbal and visual codes. The first level is the representational


level, this involves the initial activation of one or both code systems. Thestructure at this level can be ‘‘described as the availability in memory ofmodality-specific logogens and imagens (neuronal structures)’’ (Sadoski& Paivio, 2001, p. 71). At the second level are referential connectionswhich operate between visual and verbal systems. At the third level areassociative connections which operate within systems connecting imagensand logogens to one another. If comprehension is conceived of as apattern of neural activation composed of both verbal and visual elements,then a pattern that is both highly activated and veracious represents‘‘good comprehension’’. The computerized training system in this studymay have increased text comprehension for two reasons. First, questiongeneration, by activating the verbal code, may force readers to reviewcurrent knowledge and ascertain where their knowledge structures areincomplete or fuzzy. Question generation, and answering open-endedquestions at the end of the text would certainly activate both represen-tational and associative links in the verbal system. Second, the visualstrategy which entails ‘‘building a mental model on screen’’ may activateall three levels of representational, associative and referential links in bothverbal and visual systems. Aspects of the text are turned into imagery,lines from the text are repeated, and then readers manipulate and verifywhere icons should be placed on screen. Activating all three levels andcommunicating between and within the two verbal and visual systemsrepresents powerful across-the-board cognitive processing. The morepractice in effortful cognitive processing that poor comprehenders receive,the more proficient they should become at activating these processes onother texts, and in other literacy situations. Completing these Web-basedstrategies forces upon the reader self-directed thinking activities.Although this training system supplies a measure of evaluation, thereaders themselves must then plan and regulate their cognition, reading,and repair strategies thereafter. Oakhill and Patel (1991) hypothesizedthat training a visual strategy to their poor comprehenders may havesignificantly enhanced integration skills (using Dual Code terminologythese may be interpreted as both referential and associative connections)or circumvented memory limitations.

ScrollBacks. The analysis of ScrollBacks was edifying. Rereading hasbeen shown to improve comprehension and metacomprehension accuracyin college-age students (Rawson, Dunlosky, & Theide, 2000). On average,the readers in this study used significantly more ScrollBacks in theexperimental condition. However, it was the relatively poorer compreh-enders who were utilizing the technique more often than the relativelybetter comprehenders. In the control condition (with the anagrams),


ScrollBacks were somewhat positively related to all comprehensionvariables – vocabulary, human and HEMA scored final constructedanswers. However, in the experimental condition ScrollBacks were neg-atively correlated with all the comprehension variables – significantly sowith vocabulary and human-scored open-ended answers. Perhaps becausethe poorer comprehenders came to realize, via the system’s feedback, thatthey were struggling and should avail themselves of the strategy. Perhapsby integrating the system’s immediate feedback and their own growingmetacognitive awareness, the poorer comprehenders began to moreconsciously, or at least electively, utilize one of the new tools available tothem. Research supports that different readers use text reprocessingstrategies differently, e.g., amount of prior knowledge significantly affectstext reprocessing (Haenggi & Perfetti, 1992). Interestingly, Haenggi andPerfetti also demonstrate that, on average, college readers show equalbenefits in a comparison between three different types of text reprocessingstrategies: (1) rereading, (2) rewriting notes, or (3) rereading notes. Takingnotes would normally be considered a more effortful strategy and, thus,more beneficial; however, simply rereading the text increased collegestudents’ comprehension scores in equal measure.

Importance of findings. The findings are important for several reasons.First, the results demonstrate that Web-based strategy training tools cansignificantly increase struggling readers’ comprehension scores. Byallowing readers to create questions as they read, and to manipulategraphics on screen to create stylized simulations of their ‘‘mental mod-els’’, the readers’ final comprehension scores increased significantly. Theexperimental scores increased on average 8% beyond the control scores.When assessing the difference between the control and experimentalconditions with human-scored constructed answers, a Cohen’s d (orEffect Size – ES) of .45 was found, and in the HEMA-scored condition(with smaller SDs), the ES was .67. As a comparison, in a meta-analysisby Rosenshine and Meister (1994) on the metacognitive interventionReciprocal Teaching, the median ES with standardized measures was .32.

Second, programming for the visual strategies is expensive andresources dictated the number of media-rich, interactive experimentaltexts that could be built. Thus, these results are especially unusual andheartening given that the study lasted only two weeks. Duffy et al. (1987)and Meloth (1990) demonstrated that it can take up to 16 weeks forsignificant higher level metacognitive differences to emerge in poorreaders using hard copy materials. Third, this study demonstrates that itis possible to move away from the multiple choice format when testingwith computers. There is still refining to be done on the scoring algorithm,


but the strong correlations with human scores would lead one to believethat much (though certainly never all) of constructed text can be accu-rately and reliably scored on-line. This can be said for the larger popu-lation of struggling middle school readers that this sample mayextrapolate to, it remains to be seen whether the HEMA algorithm willscale up to the average high school and college students who will be usingmore sophisticated syntax. Fourth, the new generation of formal andinformal school assessments expects students to write coherent andorthographically correct short essays. Web-based tools that require con-structed text, and that encourage students to practice the extremelyvaluable skill of writing are important additions to the educationaltoolbox. Fifth, reading instruction needs more empirical Web-basedresearch. The following quote is from the meta-analysis Report of theNational Reading Panel (NRP, 2000): ‘‘Particularly striking in its absenceis research on Internet applications as they might be incorporated inreading instruction’’ (p. 6-2).

Future directions. There are two reciprocal directions to focus on in thefuture: new experiments and further system design enhancements. Con-trolled studies are needed to ascertain which components benefited textcomprehension. One question is how the hybrid text is affecting com-prehension. All of the texts address scientific concepts, but these conceptsare introduced in an engaging, narrative structure. This narrative struc-ture is more familiar to middle school children than the expositorystructure. One hypothesis to be tested is whether the hybrid story formatis more felicitous for younger readers who may now be struggling with thenovel expository format, than it is for older readers (who may find itdistracting). Even though the case can be made for multi-componentialtraining packages in the applied domain, it would be of scientific interestto ascertain the ES associated with each individual strategy. In addition,are there interactions between text type (expository, narrative, or hybrid)and metacognitive strategy (verbal, visual, or mixture)?

Future studies will include more participants and texts, and usebetween-subjects designs to facilitate multiple conditional comparisonswithout cognitive carryover effects. In addition, a more pedagogicallyrelevant control condition will be included. Instead of unscramblinganagrams, readers will be asked to locate information in the text. This issimilar to a control condition used by Lovett et al. (1996). Locatinginformation is a valuable scholastic tool that should not create interfer-ence with the question generation or visual strategies. With more par-ticipants there would also be enough statistical power to assess foraptitude by treatment interactions. It is hypothesized that practice


manipulating images on screen to simulate important textual conceptswill especially aid poor visualizers.

The issue of transfer needs to be addressed in a systematic manner.The strategies could be ‘‘faded’’ towards the end of training so thatperformance could be ascertained on computer texts without embeddedstrategies. In addition, a far transfer assessment with hard copy texts maybe even more ecologically valid.

The computer program will be altered to take into account lessonslearned in this study. Some are simple fixes like altering the awkwardarrow–click interface. Some fixes are more complex like increasing thebaseline on the HEMA algorithm. In future iterations, readers will receivefeedback after every submission of a final constructed answer item,instead of after all eight have been submitted. In addition, if the score isless than 90% correct, readers will be automatically taken back to thelocation in the text where the answer was located and given a secondchance. The system will keep track of both answers and a reader whonever shows improvement when given a second chance will be flagged forteacher intervention. Further, it would be of interest to track preciseplacement of ScrollBacks to ascertain what information precedes aScrollBack. Think-aloud protocols would help to isolate whether readersare engaged in verification or elaboration processes (Millis & King, 2001).

The new generation of Web-based tools needs to be constructed on topof theory-driven, scientifically, and experimentally sound structures(Reyna et al. 2001). Such tools can be extremely worthwhile for strugglingreaders who may not be receiving enough of the one-on-one attention andfeedback that is so critical to mastering higher level reading compre-hension strategies. This study demonstrates that positive change incomprehension and rereading performance can be effected using Web-based training tools.

Acknowledgements

This research was supported by a grant from NICHD R43-HD40696-01. Special thanks are given to Renee Duffy, Matthew Zeidenberg, L.Chris Crawley, Amy Waite, Lori Hillyer, and Gary McClelland.

Notes

1. It should be noted that the effect size dropped to .34 on standardized tests. It is

difficult to find comprehension changes with standardized measures (Paris, Cross, &


Lipson, 1984), which is why we did not follow up on this sample’s end-of-yearstandardized measures.

2. It is our goal to eventually create a Web-based system that contains on-line animated

tutors. Resources did not allow for the creation of animated tutors in this iteration,human tutors presented the 10-minute strategy introduction in session 4. That 4% ofthe intervention time was spent with humans does not seriously detract from the

claim that this system is primarily ‘‘Web-based’’.3. Although, it should be noted that the large SD resulted in an effect size of .19, and

power was less than .40 to detect a significant difference.

Appendix A

Vocabularies I and II

‘‘These words are going to be in the story. Click on the word or phrase on the right

that best matches the word on the left.’’

1. Light a. a form of energy*

b. not early

c. pretty

2. Visible spectrum a. looks invisible

b. the colors you can see*

c. is divisible by three

3. Reflect a. small pieces of dirt or other substance

b. part of a bicycle

c. to bounce off of a surface*

4. Astonishment a. a party

b. surprise*

c. a gravel pathway

5. Absorb a. take turns

b. nice

c. to take something in*

6. Absence a. lack of*

b. sore

c. a dream-inducing drink

7. Sample a. not very clever

b. an example of something*

c. a type of light


Appendix B

Shameka Sees the Light

By Julie Fitzpatrick and Mina C. Johnson-Glenberg

Learning Goals: Light travels in waves; absorption and reflection; color is reflected light;color depends on wavelength in source light

‘‘Take your time and read this story. You will stop four times to work strategies. Whenyou are finished you will be asked eight questions. Use the buttons on the right to movethrough the text. You can also go back and reread anything you want to.’’

‘‘Shameka! What’s up?’’ asked Jasmine, as she plopped her books down in the darkhallway. Shameka had been waiting by the door for her big sister because she was so

excited.

‘‘Look Jas! Mom says if you and I can agree on a color, we can paint our room!

Mom and I picked up these cards of sample colors from the paint store this morning.What do you think of this light pink?’’

Jasmine took the sample from Shameka’s hand and stared thoughtfully at the tinysquare of color on the card. ‘‘I don’t know. . .It looks kind of. . .muddy red, to me,’’ shereplied unhappily, as she handed the card back to Shameka. Shameka frowned as she

took the card. She walked over to the window where the sun was just setting. Shewondered aloud, ‘‘That’s funny. When I picked it out, I thought it was a nice, pinkcolor. Now, it just looks all red.’’

‘‘You are about to work a strategy would you like to reread anything?’’

‘‘Pretend you are a teacher, type in a question you would ask the class here to see if

they were understanding.’’

‘‘Well, bring it in the kitchen and I’ll look at it again.’’ Suggested Jasmine, ‘‘Man, I

could eat a horse. Let’s start supper.’’ Shameka called out to her brother loafing in theliving room doing nothing, ‘‘Michael, turn on the lamp in the living room so Mom cantell we’re home when she gets to the corner.’’

Click. A warm glow lit the other room. The two girls looked at one another inastonishment. Wow, he had actually done their bidding the first time he was asked!

Shameka set the sample card on the counter. Michael came in the kitchen, grabbedthe card and ran back into the living room.

‘‘Hey! Come back with that!’’ Shameka exclaimed.

‘‘Why would anyone pick this color?’’ he teased, ‘‘Pink? Gross!’’ The two girlsturned to look at each other with their heads tilted in puzzlement. Now the sample

looked pink again? Michael came back to the doorway. ‘‘What? What’d I say?’’ Heasked.

After supper, Shameka and Jasmine discussed the mysterious paint sample withMom. ‘‘To understand what is happening with the paint, you will need to know moreabout light,’’ said Mom.

‘‘OK,’’ agreed Shameka, ‘‘What is light?’’

‘‘Light is a form of energy,’’ explained Mom, ‘‘Sunlight travels in bursts, called

waves. ‘‘Um, that’s great, Mom, but what does it have to do with the color of the paintsample?’’ Jasmine was getting impatient.


‘‘We’ll get to that,’’ laughed Mom. ‘‘The waves act in different ways when they runinto something.’’

‘‘Hold on, do you mean light travels in waves like the kind at the water park?’’interrupted Shameka.

Mom replied, ‘‘Well, sort of like that. You can still measure the waves from crest tocrest, but they are so very teeny tiny that we don’t feel pushed around by them like we

do by big water waves. The light waves are arranged by how long they are. This is calledthe visible spectrum. The word visible means we can actually see each wave length as acolor. Spectrum tells us they are arranged in some kind of order.’’

The children were quiet for a while.

‘‘So, what color is made up of the longest waves we can see?’’ asked Jasmine.

‘‘Red,’’ said Mom, ‘‘Then the remaining shorter waves are orange, then yellow,green, blue, indigo, and the shortest waves, that we can still see are violet.’’

‘‘Hey, that sounds like a rainbow,’’ said Shameka in surprise.

Mom nodded. ‘‘Yes, a rainbow shows the visible spectrum, or at least part of it.’’

‘‘How do you remember all those colors in order, Mom?’’ wondered Jasmine. Shewas impressed.

‘‘I just think of the initial letters of the colors as the name Roy G. Biv,’’ said Mom.

‘‘Hey, Mom! That’s a great trick! I’m going to use it to remember other importantthings, like She Has A Man Eating Killer Appetite – Shameka!’’ joked Michael as he

threw himself back on the couch.

She gave him one of her famous soon-you’ll-be running-for-cover looks, but did not

let him distract her from the conversation. ‘‘So, are you saying that colors are light,Mom?’’

‘‘Yes,’’ her mother replied, ‘‘Color is light.’’

‘‘You are about to work a strategy would you like to reread anything?’’Part I – drag color name to appropriate wavelength – assess learning of ROY G BIV

Part II – drag all wavelengths together in box to create white light.

‘‘Now can we get back to how the paint sample changed color?’’ Jasmine suggested.

‘‘OK, do you remember when I said that waves act in different ways when they run intosomething?’’ reminded Mom.

‘‘Umm. . .yeah,’’ answered her three listeners doubtfully.

Mom launched into a careful explanation. ‘‘When a light wave hits an object, like apaint sample, or a chair, or anything at all, it can be absorbed by that object, or it can be

reflected by that object. What does it mean to be absorbed by something?’’

Shameka answered, ‘‘You mean like a sponge absorbs water.’’ Mom went on,

‘‘That’s very close, like a sponge the object can absorb some colors, and the colors thataren’t absorbed are bounced off. We say that the light that bounces off the object isreflected. We see the reflected light as the color of the object.’’ Mom smiled at her

listeners expectantly.


‘‘Pretend you are a teacher, type in a question you would ask the class here to see ifthey were understanding.’’


They thought about what she had said. Shameka scratched her ear, finally, she askedslowly, ‘‘What if an object absorbs all the light that hits it? What color would it be? No

color? Would it be invisible?’’

Her mom smiled. ‘‘Remember color is light. What would the absence or lack of light

look like?’’

‘‘Nighttime!’’ shouted Jasmine and Michael at exactly the same time. They laughed

at each other.

‘‘True,’’ said mom, ‘‘But, we call the night color black. If an object absorbs all lightit looks black to us.’’ ‘‘Whoa.’’ said, Shameka.

‘‘Well, what if the object reflected all the light that hit it? Would it be white?’’ askedMichael. All three children looked hopefully at Mom.

‘‘Yes!’’ she smiled, ‘‘If the light hitting the object had all the visible spectrumwavelengths in it – if it was white light.’’

‘‘OK, OK, suppose you had an object – let’s say like that stinky iguana that Minahas. And, he is green. You’re saying the iguana absorbs all the wavelengths except

green,’’ said Jasmine, excitedly. ‘‘That would mean the iguana’s only reflecting the greenwavelengths, right?’’


‘‘Yes,’’ said Mom, ‘‘that’s right. The iguana is only reflecting green and he’sabsorbing everything else.’’

‘‘OK, so if white light hits the iguana he would look green, right?’’ continued Jas-mine, ‘‘But suppose the light that hit him only contained red wavelengths. Pretend it’s

nighttime and the traffic light outside the window is stuck on red. NOW what colorwould the iguana be if only red wavelengths were hitting him?’’ Jasmine asked the groupwhile arching her eyebrows.

‘‘Black!’’ piped up Michael. ‘‘He would be black because no light would be reflected.No light, no color.’’ He looked expectantly at Mom, who nodded.

‘‘Yes! I knew it! ‘‘ Michael rejoiced as he did a little victory dance.

Now it was Shameka’s turn. ‘‘What if a paint sample absorbed all the wavelengths

except mostly pink? Would it be pink in white light and then look more red aroundsunset when the light seems more red?’’ asked Shameka with a confident grin.

‘‘Yes,’’ saidMomwith satisfaction, ‘‘If there was only a little light coming through thewindows at the end of the day, with hardly any of the blue wavelengths, the light wouldseem redder. The paint sample would look almost red. It would reflect very little pink.’’

‘‘Now I get it,’’ added Jasmine. ‘‘When Shameka picked the paint sample earlier inthe day, the white sunlight had lots of blue, indigo and violet wavelengths in it for the

paint sample to reflect. When I looked at it in the setting sun, there were fewer of theshorter blue wavelengths so it only looked totally red.’’

‘‘Then I took the sample in here by the light from the lamp, and it looked pinkagain,’’ finished Michael. ‘‘I still say pink is the pits,’’ he smirked.

Shameka and Jasmine took another look at the paint sample with Mom. ‘‘Well?

What do you think?’’ asked Mom.

The girls looked at each other and grinned. ‘‘I think tomorrow we’ll be painting!’’

shouted Shameka, as she threw the paint sample at Michael, ‘‘and you can help!’’

‘‘Click here to exit the text. You will not be able to go back and reread.’’


Final Open-ended Questions

1. Name the seven main colors that are in the visible spectrum.2. How does light travel?

3. Are they having horse for dinner? ___Why did Jasmine say that?4. What does it mean ‘‘to loaf’’?5. Why were the girls puzzled when Michael said that pink was gross?

6. Why did the paint sample look different in the evening?

Transfer Questions

7. If a dog reflects ALL the light that hits it in the middle of a sunny day, what color isthe dog?

8. Light that is absorbed creates heat in the object that absorbs it. Which would keep

you cooler in the summer a white shirt or a black shirt? ____ Why?

References

Amlund, J. T., Kardash, C. A. M., & Kulhavy, R. W. (1986). Repetitive reading andrecall of expository text. Reading Research Quarterly, 21, 49–58.

Bell, N. (1986). Visualizing and verbalizing for language comprehension and thinking PasoRobles, CA: Academy of Reading Publications.

Bell, N. (1991). Gestalt memory: A critical factor in language comprehension. Annals ofDyslexia, 41, 246–260.

Duffy, G. G., Roehler, L. R., Sivan, E., Rackliffe, G., Book, C., Meloth, M. S., Vavrus,

L. G., Wesselman, R., Putnam, J., & Bassiri, D. (1987). Effects of explaining thereasoning associated with using reading strategies. Reading Research Quarterly, 22,347–368.

FOSS – Full option science program, Lawrence Hall of Science, retrieved October5:2003, from http://www.lhs.berkeley.edu/foss/.

Gambrell, L. B., & Jaywitz, P. B. (1993). Mental imagery, text illustrations and

children’s story comprehension and recall. Reading Research Quarterly, 28, 265–273.

Gough, P., & Tunmer, W. E. (1986). Decoding, reading, and reading disability.Remedial and Special Education, 7, 6–10.

Graesser, A., Golding, J. M., & Long, D. L. (1991). Narrative representation and

comprehension. In R. Barr, M. L. Kamil, P. Mosenthal, & P. D. Pearson (Eds.),Handbook of reading research (Vol. II, pp. 171–205). White Plains, NY: LongmanPublishing Group.

Haenggi, D., & Perfetti, C. A. (1992). Individual differences in reprocessing of text.Journal of Educational Psychology, 84(2), 182–192.

Johnson-Glenberg, M. C. (2000). Training reading comprehension in adequate

decoders/poor comprehenders: Verbal vs visual strategies. Journal of EducationalPsychology, 92, 772–782.

Johnson-Laird, P. N. (1983). Mental models: Towards a cognitive science of language,inference, and consciousness Cambridge, MA: Harvard University Press.

Kintsch, W. (1988). The role of knowledge in discourse comprehension: A constructionintegration model. Psychological Review, 95, 163–182.


Landauer, T. K., & Dumais, S. T. (1997). A solution to Plato’s problem: The latentsemantic analysis theory of acquisition, induction, and representation of knowledge.

Psychological Review, 104, 211–240.

Levin, J. R. (1973). Inducing comprehension in poor readers: A test of a recent model.Journal of Educational Psychology, 65, 19–24.

Lovett, M. W., Borden, S. L., Warren-Chaplin, P. M., Lacerenza, L., DeLuca, T., &Giovinazzo, R. (1996). Text comprehension training for disabled readers: Anevaluation of reciprocal teaching and text analysis training programs. Brain andLanguage, 54, 447–480.

Mayer, R. E., & Sims, V. K. (1994). For whom is a picture worth a thousand words?Extensions of a dual-coding theory of multimedia learning. Journal of EducationalPsychology, 86, 389–401.

Meloth, M. S. (1990). Changes in poor readers’ knowledge of cognition and theassociation of knowledge of cognition with regulation of cognition and readingcomprehension. Journal of Educational Psychology, 82, 792–798.

Millis, K., & King, A. (2001). Rereading strategically: The influences of comprehensionability and a prior reading on the memory for expository text. Reading Psychology,22, 41–65.

Millis, K., Simon, S., & tenBroek, J. (1998). Resource allocation during the rereading ofscientific texts. Memory & Cognition, 26, 232–246.

National Center for Education Statistics (NCES). Retrieved August 10, 2002, fromwww.NCES.ed.gov.

National Reading Panel (Report of the), Teaching Children to Read (2000). Reports ofthe Subgroups. Retrieved June 5, 2001, from www.nichd.nih.gov/publications/nrp/report.htm.

Oakhill, J., & Patel, S. (1991). Can imagery training help children who havecomprehension problems?. Journal of Research in Reading, 12, 106–115.

Paivio, A. (1986). Mental representation: A dual coding approach New York: Oxford

University Press.

Palinscar, A. S., & Brown, A. L. (1984). Reciprocal teaching of comprehension-fosteringand comprehension-monitoring activities. Cognition and Instruction, 1, 117–175.

Paris, S. G., Cross, D. R., & Lipson, M. Y. (1984). Informed strategies for learning: Aprogram to improve children’s reading awareness and comprehension. Journal ofEducational Psychology, 76, 1239–1252.

Pearson, P. D., & Hamm D. N. (2001). The assessment of reading comprehension: A

review of practices: past, present and future. Technical Report for RANDCorporation.

Pressley, M. (1977). Imagery and children’s learning: Putting the picture in develop-

mental perspective. Review of Educational Research, 47, 585–622.

Pressley, M. (2000). What should comprehension instruction be the instruction of? InM. L. Kamil, P. B. Mosenthal, et al. (Eds.),Handbook of Reading Research (Vol. III,

pp. 545–561). Mahwah, NJ Lawrence Erlbaum Associates.

Rawson, K. A., Dunlosky, J., & Theide, K. W. (2000). The rereading effect:Metacomprehension accuracy improves across reading trials. Memory and Cogni-tion, 28, 1004–1010.

Reyna, V. F., Brainerd, C. J., Effken, J., Bootzin, R., & Lloyd, F. J. (2001). Thepsychology of human–computer mismatches. In C. R. Wolfe (Ed.), Learning andteaching on the world wide web (pp. 23–41). San Diego, CA: Academic Press.


Rosenshine, B., Meister, C., & Chapman, S. (1996). Teaching students to generatequestions: A review of the intervention studies. Review of Educational Research, 66,

181–221.

Rosenshine, B., & Meister, C. (1994). Reciprocal teaching: A review of the research.Review of Educational Research, 64, 479–530.

Sadoski, M., & Paivio, A. (2001). Imagery and text: A dual coding theory of reading andwriting Mahwah, NJ: Lawrence Erlbaum Associates.

Stein, N. L., & Glenn, C. G. (1979). An analysis of story comprehension in elementaryschool children. In R. Freedle (Ed.), New directions in discourse processing (pp. 53–

120). Norwood, NJ: Ablex.

TerraNova Reading and Language Arts Test (1997), Monterey, CA: CTB/McGraw-Hill.

Tovani, C. (2000). I read it but I don’t get it: Comprehension strategies for adolescentreaders Portland, ME: Stenhouse Publishers.

Trabasso, T., & Magliano, J. P. (1996). How do children understand what they read and

what can we do to help them?. In M. Graves, P. Broekvan den & B. Taylor (Eds.),The first R: A right of all children (pp. 255–288). New York: Teachers College Press.

Address for correspondence: Mine C. Johnson-Glenberg, Waisman Center, 1500 High-land Avenue, Madison, WI 53705, USA

E-mail: [email protected], [email protected]


Documents

Web-based training of metacognitive strategies for text comprehension…kpata/haridustehnoloogiaTLU/poor... · 2006-09-19 · comprehension: Focus on poor comprehenders MINA C. JOHNSON-GLENBERG