Examining Hypermedia as a Examining Hypermedia as a Means to Improve Access to the Means to Improve Access to the

General Education Curriculum for General Education Curriculum for Students with Reading DifficultiesStudents with Reading Difficulties

Dr. Matthew T. MarinoDr. Matthew T. Marino

Washington State UniversityWashington State University

20072007

OverviewOverview

Context for this investigation Context for this investigation

Students with LD in reading and poor readersStudents with LD in reading and poor readers

Adolescents and expository texts Adolescents and expository texts

Barriers to the accommodations processBarriers to the accommodations process

Improving reading comprehensionImproving reading comprehension

Technology in education Technology in education

Can hypermedia improve access to the general education Can hypermedia improve access to the general education curriculum?curriculum?

Current ResearchCurrent Research

Students with Learning Disabilities (LD)Students with Learning Disabilities (LD)

Students with LD comprise the largest subgroup of students Students with LD comprise the largest subgroup of students served under IDEA served under IDEA (U.S. Department of Education, 2002)(U.S. Department of Education, 2002)

The number of students age 12 to 17 identified with LD has The number of students age 12 to 17 identified with LD has increased 44% in the past 10 years increased 44% in the past 10 years (Lyon et al, 2001)(Lyon et al, 2001)

Only 62% of students with LD graduate with a diploma Only 62% of students with LD graduate with a diploma (U.S. (U.S. Department of Education, 2002)Department of Education, 2002)

Educational costs for students with LD are 1.6 times that of Educational costs for students with LD are 1.6 times that of regular education students regular education students (Special Education Expenditure Project [SEEP], (Special Education Expenditure Project [SEEP], 2003)2003)

Students served under IDEA must be provided with access Students served under IDEA must be provided with access to the general education curriculum to the greatest extent to the general education curriculum to the greatest extent possiblepossible (IDEA, 1997)(IDEA, 1997)

Review of Literature

LD Identification Process & ImplicationsLD Identification Process & Implications

IQ-achievement discrepancy identification is IQ-achievement discrepancy identification is changingchanging

LD in reading defined LD in reading defined IQ does not strongly correlate with reading IQ does not strongly correlate with reading

achievement or rate of reading developmentachievement or rate of reading development (Share, McGee, & Silva, 1989; Vellutino, Scanlon, & Lyon, 2000)(Share, McGee, & Silva, 1989; Vellutino, Scanlon, & Lyon, 2000)

IQ does not predict a student’s ability to read or IQ does not predict a student’s ability to read or profit from remediationprofit from remediation (Siegel, 1988; Vellutino et al., 2000) (Siegel, 1988; Vellutino et al., 2000)

Review of Literature

LD in Reading and Poor ReadersLD in Reading and Poor Readers

Poor readers defined Poor readers defined (Vellutino et al., 2000)(Vellutino et al., 2000)

Poor readers show many of the same characteristics Poor readers show many of the same characteristics as students with LD in reading as students with LD in reading ((Fletcher et al., 1994; Stanovich and Fletcher et al., 1994; Stanovich and Siegel, 1994)Siegel, 1994)

Students with LD in reading and poor readers Students with LD in reading and poor readers possess virtually indistinguishable reading growth possess virtually indistinguishable reading growth curves in grades 1 through 12 curves in grades 1 through 12 (Vellutino et al., 2000)(Vellutino et al., 2000)

For the purposes of this study, students with LD in For the purposes of this study, students with LD in reading will be combined with poor readers and reading will be combined with poor readers and referred to as students with reading difficulties (RD) referred to as students with reading difficulties (RD)

Students with RD in this study scored below the 25% Students with RD in this study scored below the 25% on standardized measures of reading achievement on standardized measures of reading achievement during the academic year prior to this study.during the academic year prior to this study.

Review of Literature

Statement of the ProblemStatement of the Problem

More than 12.4 million students experience significant More than 12.4 million students experience significant difficulties learning to readdifficulties learning to read (National Center on Educational Statistics, (National Center on Educational Statistics, 2003).2003).

Students with RD are failing to make adequate yearly Students with RD are failing to make adequate yearly progress in the general education curriculumprogress in the general education curriculum (Lyon et al., 2001; (Lyon et al., 2001; Mastropieri, Scruggs, & Graetz, 2003; U.S. Department of Education, 2003).Mastropieri, Scruggs, & Graetz, 2003; U.S. Department of Education, 2003).

Traditional methods of instruction (e.g., lectures based on Traditional methods of instruction (e.g., lectures based on readings in expository texts) are not effective for students readings in expository texts) are not effective for students with RDwith RD (Lapp, Flood, & Ranck-Buhr, 1995; Maccinin, Gagnon, & Hughes, 2002).(Lapp, Flood, & Ranck-Buhr, 1995; Maccinin, Gagnon, & Hughes, 2002).

The number of students with RD in inclusive classrooms is The number of students with RD in inclusive classrooms is increasingincreasing (Lyon et al. 2001).(Lyon et al. 2001).

Review of Literature

Adolescents and Expository TextsAdolescents and Expository Texts

Expository text definedExpository text defined

Students with RD…Students with RD… Often lack prior knowledge Often lack prior knowledge (Gersten, Fuchs, Williams, & Baker, 2001)(Gersten, Fuchs, Williams, & Baker, 2001)

Are unaware of the text structures they are readingAre unaware of the text structures they are reading (Meyer, (Meyer, Brandt, & Bluth, 1980)Brandt, & Bluth, 1980)

Retrieve information randomly Retrieve information randomly (Wilson & Rupley, 1997)(Wilson & Rupley, 1997)

Have difficulty determining essential information Have difficulty determining essential information (Engert & (Engert & Thomas, 1987)Thomas, 1987)

Do not utilize text cues Do not utilize text cues (Gersten et al, 2001)(Gersten et al, 2001)

Fail to recognize when they are not comprehending new Fail to recognize when they are not comprehending new information information (Gersten et al., 2001)(Gersten et al., 2001)

Review of Literature

This Leads To…This Leads To…

Low levels of reading comprehensionLow levels of reading comprehension (Gersten et at., 2001)(Gersten et at., 2001)

An inability to formulate questions and hypothesesAn inability to formulate questions and hypotheses (Wilson (Wilson

& Rupley, 1997)& Rupley, 1997)

Failure to make abstract connectionsFailure to make abstract connections (Engert & Thomas, 1987)(Engert & Thomas, 1987)

Frustration, lower motivation, expected failureFrustration, lower motivation, expected failure (McKinney, (McKinney,

Osborne, & Schulte, 1993) Osborne, & Schulte, 1993)

What do we do for these students?

Review of Literature

Barriers to the Accommodations ProcessBarriers to the Accommodations Process

Teachers at the secondary level feel pressured Teachers at the secondary level feel pressured to progress through the curriculum to progress through the curriculum (Mastropieri, Scruggs, & (Mastropieri, Scruggs, & Graetz, 2003)Graetz, 2003)

Teachers’ ability to provide meaningful Teachers’ ability to provide meaningful accommodations is hampered by large class accommodations is hampered by large class sizes and a lack of resources sizes and a lack of resources (Lancaster, Schumaker, & (Lancaster, Schumaker, & Deshler, 2002)Deshler, 2002)

Many general education teachers do not have Many general education teachers do not have the time or expertise to provide meaningful the time or expertise to provide meaningful accommodations accommodations (Mastropieri et al., 2003)(Mastropieri et al., 2003)

How can we improve this process?Review of Literature

Improving Reading ComprehensionImproving Reading Comprehension

Modify instructional materials to match the Modify instructional materials to match the student’s reading abilitystudent’s reading ability (Mastropieri & Scruggs, 2004)(Mastropieri & Scruggs, 2004)

Present information using multiple modalitiesPresent information using multiple modalities (MacArthor, Ferretti, Okolo, & Cavalier, 2001)(MacArthor, Ferretti, Okolo, & Cavalier, 2001)

Offer repeated reading and practice opportunitiesOffer repeated reading and practice opportunities (Jenkins, Stein, & Wysocki, 1984; Bryant, 2003)(Jenkins, Stein, & Wysocki, 1984; Bryant, 2003)

Provide students with opportunities to gain Provide students with opportunities to gain additional background knowledge quickly additional background knowledge quickly (MacArthor et (MacArthor et al., 2001)al., 2001)

How can we do this?

Review of Literature

Consider TechnologyConsider Technology Current policy calls for the increased use of Current policy calls for the increased use of

technology to support student learningtechnology to support student learning (U.S. Department of (U.S. Department of Education, 2003) Education, 2003)

Technology use in regular education classrooms is Technology use in regular education classrooms is rapidly increasingrapidly increasing (U.S. Department of Education, 2003; Vannatta & O’Bannon, (U.S. Department of Education, 2003; Vannatta & O’Bannon, 2002 ) 2002 )

Technology may improve access to the general Technology may improve access to the general education curriculumeducation curriculum (Pucket, 2004; Behrmann & Jerome, 2002; Edyburn, (Pucket, 2004; Behrmann & Jerome, 2002; Edyburn, 2002; Fisher & Frey, 2001)2002; Fisher & Frey, 2001)

Technology should be used in concert with other Technology should be used in concert with other instructional methods (e.g., classroom discourse)instructional methods (e.g., classroom discourse) (Yerrick, 2000; De La Paz & MacArthor, 2003; Fuchs, Fuchs, & Kazdan, 1999)(Yerrick, 2000; De La Paz & MacArthor, 2003; Fuchs, Fuchs, & Kazdan, 1999)

Review of Literature

Can Technology Improve Access to Can Technology Improve Access to the General Education Curriculum?the General Education Curriculum?

Hypermedia definedHypermedia defined Hypermedia may eliminate the overuse of expository textsHypermedia may eliminate the overuse of expository texts

(Lancaster et al., 2002)(Lancaster et al., 2002)

Hypermedia provides:Hypermedia provides: Information on demandInformation on demand (McKenna, Reinking, Labbo, & Keiffer, 1999)(McKenna, Reinking, Labbo, & Keiffer, 1999)

Tools that support cognitive processesTools that support cognitive processes (Lajoie, 1993)(Lajoie, 1993)

Built-in accommodations by allowing teachers to modify Built-in accommodations by allowing teachers to modify task difficulty and select appropriate readability levelstask difficulty and select appropriate readability levels (Edyburn, 2000; Behrmann & Jerome, 2002; Pucket, 2004)(Edyburn, 2000; Behrmann & Jerome, 2002; Pucket, 2004)

Review of Literature

Hypermedia ResearchHypermedia Research

Hypermedia allows teachers to monitor student Hypermedia allows teachers to monitor student progress and resource use progress and resource use (McKenna et al., 1999)(McKenna et al., 1999)

Students and teachers report positive outcomes from Students and teachers report positive outcomes from using hypermedia using hypermedia (Lancaster et al.,, 2002; Garthwait, 2004; Lewis, 2000)(Lancaster et al.,, 2002; Garthwait, 2004; Lewis, 2000)

There are a limited number of studies examining the There are a limited number of studies examining the efficacy of hypermedia as a means of improving efficacy of hypermedia as a means of improving comprehension for students with RD comprehension for students with RD (Maccini, Gagnon, & (Maccini, Gagnon, & Hughes, 2002)Hughes, 2002)

Research with students in regular education Research with students in regular education classrooms is promising, but inconclusive classrooms is promising, but inconclusive (Oliver, 1999; Land, (Oliver, 1999; Land, 2000; Liu, 2004)2000; Liu, 2004)

Review of Literature

Need for Current ResearchNeed for Current Research

Preliminary research examining the effects of Preliminary research examining the effects of technology-based textual modifications (e.g., technology-based textual modifications (e.g., readability levels within hypermedia programs) is readability levels within hypermedia programs) is inconclusive inconclusive (MacArthur et al., 2001).(MacArthur et al., 2001).

Research is needed to determine the types of Research is needed to determine the types of comprehension instruction that are most beneficial for comprehension instruction that are most beneficial for low ability readers in secondary content area courses low ability readers in secondary content area courses (Report of the National Reading Panel, 2000).(Report of the National Reading Panel, 2000).

Additional research is needed to determine which Additional research is needed to determine which cognitive tools are most beneficial to students with cognitive tools are most beneficial to students with RD in middle school science classes RD in middle school science classes (Liu, 2004).(Liu, 2004).

Review of Literature

Theoretical frameworkTheoretical framework

Knowledge Construction Through Conceptual Change Knowledge Construction Through Conceptual Change (KCTCC) expands on the theory of Schema Change(KCTCC) expands on the theory of Schema Change (Winn, (Winn, 2004).2004).

Perpetual cycle of learning where prior knowledge Perpetual cycle of learning where prior knowledge directs how individuals seek, identify, and interpret directs how individuals seek, identify, and interpret informationinformation (Neisser, 1976(Neisser, 1976).).

The way in which individuals construct knowledge can The way in which individuals construct knowledge can not be predicted or supported using teacher directed not be predicted or supported using teacher directed instructional designsinstructional designs (Duffy & Jonassen, 1992).(Duffy & Jonassen, 1992).

KCTCC is most effective in problem-based learning KCTCC is most effective in problem-based learning environments that incorporate technologyenvironments that incorporate technology (Winn, 2004).(Winn, 2004).

Review of Literature

Universal Design for Learning (UDL)Universal Design for Learning (UDL)

UDL supports KCTCC by:UDL supports KCTCC by: Assuming students enter learning experiences with Assuming students enter learning experiences with

varying degrees of prior knowledge varying degrees of prior knowledge (Hitchcock, Meyer, Rose, & (Hitchcock, Meyer, Rose, & Jackson, 2002).Jackson, 2002).

Using technology to incorporate video clips, graphic Using technology to incorporate video clips, graphic organizers, illustrations, and text modifications into the organizers, illustrations, and text modifications into the curriculum curriculum (Mastropieri, Scruggs, Bakken, & Whedon, 1996).(Mastropieri, Scruggs, Bakken, & Whedon, 1996).

Allowing learners to access information quickly using Allowing learners to access information quickly using multiple modalities multiple modalities (MacArthur, Ferretti, Okolo, & Cavalier, 2001).(MacArthur, Ferretti, Okolo, & Cavalier, 2001).

Review of Literature

Why Middle School Science?Why Middle School Science?

Science is one of the most difficult subjects for students Science is one of the most difficult subjects for students with RD to learn due to its complex vocabulary and with RD to learn due to its complex vocabulary and theoretical nature theoretical nature (Mastropieri et al., 2003).(Mastropieri et al., 2003).

Students with RD in middle school science classes Students with RD in middle school science classes typically read at the 4th or 5th grade leveltypically read at the 4th or 5th grade level (Mastropieri et al., 2003). (Mastropieri et al., 2003).

If textual modifications and cognitive tools can improve If textual modifications and cognitive tools can improve access to the middle school science curriculum, their access to the middle school science curriculum, their application in other content areas may yield similar application in other content areas may yield similar results.results.

Review of Literature

Research PurposeResearch PurposeStatement of the research problem:Statement of the research problem: There is a need to determine whether hypermedia can There is a need to determine whether hypermedia can

improve access to the general education curriculum for improve access to the general education curriculum for students with RD and low ability readers students with RD and low ability readers (Lyon and Moats, 1997; (Lyon and Moats, 1997; MacArthur et al., 2001; National Reading Panel, 2000).MacArthur et al., 2001; National Reading Panel, 2000).

For the purposes of this study:For the purposes of this study: Low ability readers are defined as students scoring Low ability readers are defined as students scoring << 50% 50%

on the Degrees of Reading Power (DRP) subtest of the on the Degrees of Reading Power (DRP) subtest of the Connecticut Mastery Test. The DRP is a norm-referenced Connecticut Mastery Test. The DRP is a norm-referenced measure of reading achievementmeasure of reading achievement (Touchstone Applied Science (Touchstone Applied Science

Associates, 2004)Associates, 2004).. Students with reading difficulties (RD) are defined as Students with reading difficulties (RD) are defined as

scoring scoring << 25% on the DRP. 25% on the DRP. Research Questions

*

Research QuestionsResearch Questions1)1) Are there posttest differences on the science posttest and Are there posttest differences on the science posttest and

solutions forms measure between students with RD (solutions forms measure between students with RD (<<25th 25th percentile) and those who are not RD (25th - 50th percentile)?percentile) and those who are not RD (25th - 50th percentile)?

2)2) Are there posttest differences on the science posttest and solution Are there posttest differences on the science posttest and solution forms measure between students who participated in the 4th forms measure between students who participated in the 4th grade readability condition and those who participated in the 8th grade readability condition and those who participated in the 8th grade readability condition?grade readability condition?

3)3) Is there an interaction between reading ability (i.e., students Is there an interaction between reading ability (i.e., students <<25th percentile and students in the 26th - 50th percentile) and 25th percentile and students in the 26th - 50th percentile) and treatment condition (4th grade readability and 8th grade treatment condition (4th grade readability and 8th grade readability)? If so, what is the nature of the interaction?readability)? If so, what is the nature of the interaction?

4)4) Is there a relationship between students’ reading ability, use of Is there a relationship between students’ reading ability, use of cognitive tools, and their comprehension of scientific concepts and cognitive tools, and their comprehension of scientific concepts and processes as measured on the posttest and solutions form processes as measured on the posttest and solutions form measure? If so, what is the nature of the relationship?measure? If so, what is the nature of the relationship?

Research Questions

*

Research DesignResearch questions 1 - 3 utilize the following factorial design with random assignment.

Condition DRPScore(factor)

ScienceKnowledgePosttest

SolutionsFormScores

DRP Group 1

Treatment (4th grade text)

Control (8th grade text)

< 25%

DRP Group 2

Treatment (4th grade text)

Control (8th grade text)

26 – 50%

Note. DRP = Degrees of Reading Power.

Research question 4 consisted of an analysis of students’ use of Alien Rescue’s 11cognitive tools over the course of the three-week intervention

Methods Design

*

Threats to ValidityThreats to ValidityThreat Control Measures / Instrumentation

Student differences DRP scores (factor)Block designRandom assignmentPretest (establish differences in student groups at the outset)Sample size based on power analysisDemographic data surveyMultiple outcome measures (posttest and 6 solutions forms)Post-intervention student survey

Teacher Effects Pre-intervention teacher surveyTeacher classroom data formResearcher observation formPost-intervention teacher survey

Treatment Compliance Teachers manualTeacher training (1 day with 2 follow-up trainings)Teacher classroom data form Researcher observation form

Treatment Fidelity Teacher classroom data form (completed after each class)Researcher observation form (6 unscheduled formal observations per teacher)

Methods Design

*

SettingSetting Four middle schools in New England volunteered to participate in the studyFour middle schools in New England volunteered to participate in the study Schools were selected based on: 1) administrative consent, 2) 100% Schools were selected based on: 1) administrative consent, 2) 100%

teacher agreement to participate, and 3) available technology resourcesteacher agreement to participate, and 3) available technology resources

School Grade Students Teachers Classes ERG

A 8 279 3 15 D

B 8 187 2 8 D

C 7 444 5 25 B

D 6 334 6 14 B

Setting & Participants

*

Student Demographic DataStudent Demographic Data

50% male, 50% female50% male, 50% female 87% White, 5% African American, 4% Asian, 4% 87% White, 5% African American, 4% Asian, 4%

HispanicHispanic 89% have a computer at home they can use89% have a computer at home they can use 54% have a personal computer54% have a personal computer

Response to the statement: Response to the statement: ““I am good at using computers”I am good at using computers”32% strongly agree32% strongly agree62% agree62% agree5% disagree5% disagree1% strongly disagree1% strongly disagree

Setting & Participants

*

ParticipantsParticipants Students (N = 1153) were grouped based on 2004 DRP scoresStudents (N = 1153) were grouped based on 2004 DRP scores

Group 1 - Students with RD ( Group 1 - Students with RD ( << 25% on DRP) 25% on DRP) Group 2 - Poor readers (26 - 50% on DRP)Group 2 - Poor readers (26 - 50% on DRP) Group 3 - Proficient readers ( > 50% on DRP)Group 3 - Proficient readers ( > 50% on DRP)

Students from groups 1 (n = 113) and 2 (n = 189) were Students from groups 1 (n = 113) and 2 (n = 189) were randomly randomly assigned at the student level within each classassigned at the student level within each class to either the 4th to either the 4th grade readability or 8th grade readability condition.grade readability or 8th grade readability condition.

Students in group 3 (n = 851) received text from the program at the Students in group 3 (n = 851) received text from the program at the 8th grade level8th grade level

Setting & Participants

*

The Intervention: Alien Rescue

Instrumentation

*

Performance MeasuresPerformance Measures

Pre/PosttestPre/Posttest 25 item paper & pencil multiple 25 item paper & pencil multiple

choice testchoice test Reliability of .85 established in Reliability of .85 established in

previous study (Pedersen & previous study (Pedersen & Williams, 2004)Williams, 2004)

19 items assess 19 items assess knowledge and knowledge and comprehensioncomprehension (e.g., A world (e.g., A world would have a magnetic field would have a magnetic field if________ )if________ )

6 items assess students’ ability to 6 items assess students’ ability to apply knowledgeapply knowledge (e.g., Scientist (e.g., Scientist want to measure Mars’ want to measure Mars’ atmosphere. What instrument atmosphere. What instrument would they use ______ ?)would they use ______ ?)

Six solution formsSix solution forms One paper and pencil solution One paper and pencil solution

form for each alien speciesform for each alien species Two column form that requires Two column form that requires

students to students to analyze, synthesize, analyze, synthesize, and evaluate dataand evaluate data from the Alien from the Alien Rescue programRescue program

Scaffolds the learning process Scaffolds the learning process through prompts (e.g., magnetic through prompts (e.g., magnetic field)field)

Contains open-ended and Contains open-ended and narrative response itemsnarrative response items

Piloted with 300 students (Marino, Piloted with 300 students (Marino, 2005)2005)

Established inter-rater reliability Established inter-rater reliability of .90of .90

Instrumentation

*

ProceduresProceduresPre-Pre-implementationimplementation

Teacher trainingTeacher training

Group students based on DRP scoresGroup students based on DRP scores

Random assignment at student levelRandom assignment at student level

Pretest (first day of intervention)Pretest (first day of intervention)

ImplementationImplementation

by classroom by classroom teachersteachers

Students learn to use program during 3 day explorationStudents learn to use program during 3 day exploration

Remainder of 3 weeks spent solving the problem and Remainder of 3 weeks spent solving the problem and completing the solutions formscompleting the solutions forms

Posttest during last day of interventionPosttest during last day of intervention

Solutions forms and post-intervention surveys collectedSolutions forms and post-intervention surveys collected

Post-Post-interventionintervention

Solutions forms scored by teacher and researcherSolutions forms scored by teacher and researcher

Pretest analysisPretest analysis

Factor and reliability analysis of posttest measuresFactor and reliability analysis of posttest measures

Analysis of research questionsAnalysis of research questions

Methods

*

Preliminary AnalysesPreliminary Analyses Students who were absent more than 3 days Students who were absent more than 3 days

(20%) of the intervention were excluded from (20%) of the intervention were excluded from analyses.analyses.

Independent samples t-test results, t (302) = Independent samples t-test results, t (302) = 1.22, p =.22, d = .01 indicated that there were no 1.22, p =.22, d = .01 indicated that there were no significant differences (DRP group 1 vs. 2) on significant differences (DRP group 1 vs. 2) on the pretest measure.the pretest measure.

Principal component factor analysis of posttest Principal component factor analysis of posttest measures (posttest & solutions forms) indicates measures (posttest & solutions forms) indicates 68% of variability was explained by posttest.68% of variability was explained by posttest.

Solutions forms grouped into one subscale (total Solutions forms grouped into one subscale (total score) with one factor solution reliability of .94. score) with one factor solution reliability of .94.

Analysis

*

Analysis: Research questions 1 - 3Analysis: Research questions 1 - 3

Compute descriptive statisticsCompute descriptive statistics Separate two-way ANOVA’s Separate two-way ANOVA’s DRP group (DRP group 1 vs. DRP group 2) DRP group (DRP group 1 vs. DRP group 2) and treatment condition (4th grade text vs. and treatment condition (4th grade text vs.

8th grade text) as between-subjects 8th grade text) as between-subjects independent variables. Posttest and independent variables. Posttest and combined solutions forms scores (total score) combined solutions forms scores (total score) as dependent variables.as dependent variables.

Analysis & Results

*

Results: Research Questions 1 - 3Results: Research Questions 1 - 3

RQ1: Differences by DRP groupRQ1: Differences by DRP group Results of the ANOVA for the posttest were not significant

F (1, 298) = 3.552, p = .060, d = 0.01 Results of the ANOVA for total score were significant

F (1, 281) = 3.974, p = .047, d = 0.01

RQ2: Differences by treatment (4th vs. 8th grade text)RQ2: Differences by treatment (4th vs. 8th grade text) Results of the ANOVA for the posttest were not significant

F (1, 298) = 0.369, p = .554, d =.001. Results of the ANOVA for total score were not significant

F (1, 281) = 0.03, p = .872, d < .001.

RQ3: Interaction between treatment and DRP groupRQ3: Interaction between treatment and DRP group Results of the ANOVA for posttest were not significant

F (1, 298) = 2.56, p = .111, d = .008 Results of the ANOVA for total score were not significant

F (1, 281) = 1.28, p = .259, d = .004

Results

*

Student LearningStudent Learning

Paired samples t-test (pre/posttest) with DRP Paired samples t-test (pre/posttest) with DRP groups collapsed was significant (t = 25.719, groups collapsed was significant (t = 25.719,

p < .001, d = 1.5)p < .001, d = 1.5) Posttest mean was 14.22 (56%) on a 25-point Posttest mean was 14.22 (56%) on a 25-point

scalescale

On Solutions Forms (216-point scale)On Solutions Forms (216-point scale) DRP 1 mean score 142.05 (66%)DRP 1 mean score 142.05 (66%) DRP 2 mean score 151.77 (70%)DRP 2 mean score 151.77 (70%)

Did they use the built-in cognitive tools? Results

*

Analysis Research Question 4Analysis Research Question 4 Group cognitive tools into four categories Group cognitive tools into four categories (Lajoie, 1993; Liu, (Lajoie, 1993; Liu,

2004; Liu & Bera, 2005)2004; Liu & Bera, 2005) Tools that share cognitive overload (e.g., databases)Tools that share cognitive overload (e.g., databases) Tools that support cognitive process (e.g., field journal)Tools that support cognitive process (e.g., field journal) Tools that support out-of-reach activities (e.g., astroengineering)Tools that support out-of-reach activities (e.g., astroengineering) Tools that support hypothesis testing (e.g., telemetry)Tools that support hypothesis testing (e.g., telemetry)

Obtain correlations between students’ use of tools and Obtain correlations between students’ use of tools and performance on posttest and total scoreperformance on posttest and total score

Separate one-way ANOVAs with tool use category as the Separate one-way ANOVAs with tool use category as the dependent variable and DRP group as independent dependent variable and DRP group as independent variable.variable.

Simultaneous multiple linear regression analysis to Simultaneous multiple linear regression analysis to determine how tool use (by category) predicts determine how tool use (by category) predicts performance on posttest and total score measuresperformance on posttest and total score measures

Analysis & Results

*

Results Research Question 4Results Research Question 4 The strongest correlation was between tools that share

cognitive overload and the posttest (r = .224). ANOVA (tool use DV, DRP IV) was significant for each of

the four tool categories. Cog. overload F (2, 954) = 12.60, p < .001, d = .03 Cog. process F (2, 954) = 4.86, p = .008, d = .01 Out-of-reach F (2, 954) = 3.07, p = .047, d = .006 Hypothesis F (2, 954) = 5.57, p = .004, d = .01

Scheffe’s post hoc analysis indicates significant differences between students by DRP groups

Analysis & Results

*

Results of Multiple Regression - RQ4Results of Multiple Regression - RQ4

Multiple regression indicates main effects for tools that share cognitive overload and tools that support out-of-reach activities on posttest scores.

A significant interaction between DRP group and tools that support cognitive overload was present on the posttest regression analysis.

A significant interaction between DRP group and hypothesis testing was present on the solution form regression analysis.

Analysis & Results

What does this tell us?

*

Interpreting the Multiple Regression AnalysisInterpreting the Multiple Regression Analysis

On the Posttest Regression Analysis:On the Posttest Regression Analysis:For every unit increase in the SQRT of cognitive overload For every unit increase in the SQRT of cognitive overload

We predict a .792 unit increase in posttest score for students in We predict a .792 unit increase in posttest score for students in DRP groups 1 & 2.DRP groups 1 & 2.

We predict a .33 unit increase for students in DRP group 3.We predict a .33 unit increase for students in DRP group 3.

For every unit increase in the SQRT of tools that support For every unit increase in the SQRT of tools that support out-of-reach activities students’ scores on the posttest out-of-reach activities students’ scores on the posttest decrease by .674 units.decrease by .674 units.

On the Solutions Form Regression Analysis: On the Solutions Form Regression Analysis: Students in DRP 3 gained 7.886 units for each unit gain using Students in DRP 3 gained 7.886 units for each unit gain using

tools that support hypothesis testing.tools that support hypothesis testing. Students in DRP groups 1 & 2 gained 1.264 units for each unit Students in DRP groups 1 & 2 gained 1.264 units for each unit

gain using tools that support hypothesis testing.gain using tools that support hypothesis testing.

Analysis & Results

*

DiscussionDiscussion

Students in DRP groups 1 and 2 (<50%) Students in DRP groups 1 and 2 (<50%) performed in similar ways during the performed in similar ways during the intervention. intervention.

There was a lack of treatment effect due to There was a lack of treatment effect due to readability level of text.readability level of text.

Students used cognitive tools differently Students used cognitive tools differently depending on their reading ability level.depending on their reading ability level.

*

Discussion

ImplicationsImplications Students need systematic, explicit, scaffolded instruction to Students need systematic, explicit, scaffolded instruction to

utilize cognitive toolsutilize cognitive tools (Gersten & Baker, 1998)(Gersten & Baker, 1998)

Opportunities for students to reflect on and have immediate Opportunities for students to reflect on and have immediate feedback regarding tool usefeedback regarding tool use (Swanson & Hoskyn, 1998)(Swanson & Hoskyn, 1998)

Inclusion of tools that support cognitive overload and out-Inclusion of tools that support cognitive overload and out-of-reach activitiesof-reach activities (Land, 2000; Liu, 2004; Williams & Peterson, 2004)(Land, 2000; Liu, 2004; Williams & Peterson, 2004)

Technology and content area training for special education Technology and content area training for special education teachersteachers (Sharpe & Hawes, 2003)(Sharpe & Hawes, 2003)

Increased collaboration between special education and Increased collaboration between special education and regular education teachersregular education teachers (Moore & Keefe, 2001)(Moore & Keefe, 2001)

Improved instructional strategies for general education Improved instructional strategies for general education teachersteachers (Washburn-Moses, 2005)(Washburn-Moses, 2005)

*

Implications

LimitationsLimitations Paper and pencil assessmentsPaper and pencil assessments Levels and types of classroom discourseLevels and types of classroom discourse Tool use data highly skewedTool use data highly skewed Student differences with problem-based learningStudent differences with problem-based learning Sample (SES, representation of minority Sample (SES, representation of minority

populations)populations) Teacher effectsTeacher effects

*

Limitations

Questions?Questions?*

LinksLinks Descriptives by DRP groupDescriptives by DRP group Descriptives by Treatment ConditionDescriptives by Treatment Condition Two-Way ANOVA PosttestTwo-Way ANOVA Posttest Two-Way ANOVA Total ScoreTwo-Way ANOVA Total Score Tool Category CorrelationsTool Category Correlations Posttest Multiple RegressionPosttest Multiple Regression Total Score Multiple RegressionTotal Score Multiple Regression Scheffe’s Post Hoc for Tool UseScheffe’s Post Hoc for Tool Use Threats to ValidityThreats to Validity Cell sizes for low ability readersCell sizes for low ability readers Teacher EffectsTeacher Effects Research QuestionsResearch Questions

Descriptive Statistics by Reading Ability

Pretest Posttest Total Score

Mean SD Mean SD Mean SD

DRP Group 1 7.81 3.12 13.95 4.48 128.3 39.06

DRP Group 2 8.26 3.24 14.85 3.69 136.04 14.86

Results

Maximum Posttest Score = 25Maximum Total Score = 216

RQ 1 -3

*

Descriptive Statistics by Treatment Condition

Pretest Posttest Total ScoreMean SD Mean SD Mean SD

4th gradereading level

8.24 3.13 14.25 4.07 132.02 26.65

8th gradereading level

8.09 3.30 14.55 4.1 132.38 27.26

Maximum Posttest Score = 25Maximum Total Score = 216

RQ 1 - 3

*

Results

Posttest Two-Way ANOVASource df SS MS F

DRP group 1 56.69 56.69 0.060

Treatment (4th vs. 8th reading level) 1 5.89 5.89 0.544

DRP group x Treatment 1 40.82 40.82 0.111

Within cells 298 4755.49 15.958

Total 301 4869.42

*p < .05

RQ 1 -3

*

Results

Total Score Two-Way ANOVASource df SS MS F

DRP group 1 5217.51 5217.51 .047

Treatment (4th vs. 8th reading level) 1 34.35 34.35 .872

DRP group x Treatment 1 1676.80 1676.80 .259

Within cells 278 364946.8

Total 281 372721.31

*p < .05

RQ 1 - 3

*

Results

Tool Category CorrelationsTool Category Posttest Solutions

Forms

Tools that share cognitive overload .224** -.005

Tools that support the cognitive

process

.085** -.039

Tools that support out-of reach

activities

.108** .051

Tools that support hypothesis testing .174** .115**

**. Correlation is significant at the 0.01 level (2-tailed).

*

RQ 4

Results

Scheffe’s Post Hoc Analysis for Tool Use

DRP Group 1 DRP Group 2 DRP Group 3

Cognitive Tool Category Mean SD Mean SD Mean SD

Share cognitive overload 29.41a 16.71 30.48b 16.21 36.17a,b 17.78

Support cognitive process 16.93 a 15.33 17.99 16.74 21.41a 18.3

Out-of-reach activities 11.20 7.41 11.88 7.97 13.57 8.57

Hypothesis testing 4.65 a 4.62 5.21 4.60 6.12 a 5.38

Note. Means in a row sharing subscripts are significantly different. For all measures,higher scores indicate a higher level of tool use.

*

RQ 4

Results

Multiple Regression PosttestUnstandardized

Coefficients

Standardized

Coefficients

Predictor Variable B Std.

Error

Beta t Sig.

SQRT share cognitive overload .792 .194 .258 4.088 < .001**

SQRT support the cognitive process -7.235E-02 .148 -.031 -.488 .626

SQRT support out-of reach activities -.676 .300 -.172 -2.257 .024*

SQRT support hypothesis testing .331 .277 .082 1.197 .232

DRP Dichotomous ( > 50% / < 50%) 4.744 1.116 .484 4.252 < .001**

Int. DRP X share cognitive overload -.462 .232 -.301 -1.990 .047*

Int. DRP X support the cog process -4.039E-02 .174 -.023 -.232 .816

Int. DRP X out-of-reach activities .534 .357 .219 1.498 .134

Int. DRP X hypothesis testing .202 .339 .062 .595 .552

*. p < .05. **. p < .01.

*

MR 4 Results

Multiple Regression Total ScoreUnstandardized

Coefficients

Standardized

Coefficients

Predictor Variable B Std.

Error

Beta t Sig.

SQRT share cognitive overload -.726 1.736 -.030 -.418 .676

SQRT support the cognitive process -1.194 1.315 -.065 -.908 .364

SQRT support out-of reach activities 1.642 2.610 .053 .629 .529

SQRT support hypothesis testing 1.264 2.429 .040 .521 .603

DRP Dichotomous ( > 50% / < 50%) 39.804 9.751 .504 4.082 < .001**

Int. DRP X share cognitive overload -2.568 2.070 -.208 -1.241 .215

Int. DRP X support the cog process -.787 1.537 -.055 -.512 .609

Int. DRP X out-of-reach activities -3.813 3.099 -.194 -1.231 .219

Int. DRP X hypothesis testing 6.622 2.917 .252 2.229 .026*

*. p < .05. **. p < .01.

*

MR 4 Results

Teacher EffectsDifferences in Student Posttest Measures by Teacher.

Posttest Total Score

Teacher Students Mean SD Mean SD

11 87 18.71a 4.05 158.94a 32.73

12 93 19.75b 3.59 143.03b 28.56

13 91 17.05 4.55 125.10a,c 22.64

41 84 16.86 3.60 166.42b,I,j,m,o 24.65

42 90 13.96b,c,d,e,g,h.i,j,l 4.7 124.13a,b,d,e,f,g,h,I,k,l,m,n,o 24.84

21 94 17.84d 4.23 181.88a,b,c,d 16.28

22 88 18.03e 4.31 182.56a,b,e 12.84

23 87 18.37f 4.28 173.93b,c,f 18.75

24 63 20.19g 3.65 155.76c,d,e,g 16.58

25 103 17.68h 4.60 174.21c,h 13.63

31 70 18.43i 3.61 146.48c,d,e,f,h,I,j 21.77

32 70 17.96j 3.99 130.15a,d,e,f,g,h 32.76

33 19 18.58k 3.47 166.16c,j,k 30.13

34 69 17.61l 3.92 84.33a,b,c,d,e,f,g,h,I,j,k,l 21.35

35 24 13.29a,b,f,g,i 3.99 67.01a,b,c,d,e,f,g,h,I,j,k,m 14.63

36 74 13.43a,b,c,d,e,f,g,h.i,j,k,l 3.95 151.66c,d,e,f,h,j,l,m,n 28.06

Note. Items in same column with similar subscripts are significant (p < .05).

Results of one-way ANOVA with posttest and solution form scores as dependent variables and Teacher as the independent variable were significant.

Posttest F (15, 1190) = 16.131, p < .001, d = 0.17

Solutions Forms F (15, 1153) = 112.334, p < .001, d = .59

*

Results

Cell sizes for low ability readers

66 93

47 96

Group 1 Group 2

Text presented at 4th grade level

Text presented at 8th grade level

Methods

*

Power analysis indicates optimum group sizes of 68

(f = .20, d = .57) to achieve a power of .80.