Designing and using assessment Designing and using assessment systems to prevent reading systems to prevent reading difficulties in young childrendifficulties in young children
Dr. Joseph TorgesenFlorida State University and
Florida Center for Reading Research
Western North Carolina LD/ADD Symposium, November, 2005
1. Increase the quality, consistency, and 1. Increase the quality, consistency, and reachreach of instruction in every K-3 classroomof instruction in every K-3 classroom
2. Conduct timely and valid assessments of 2. Conduct timely and valid assessments of reading growth to identify struggling readersreading growth to identify struggling readers
3. Provide more intensive interventions to 3. Provide more intensive interventions to “catch up” the struggling readers“catch up” the struggling readers
The prevention of reading difficulties is a The prevention of reading difficulties is a school-level challengeschool-level challenge
A model for preventing reading failure in A model for preventing reading failure in grades K-3: The big Ideasgrades K-3: The big Ideas
A model for preventing reading failure in A model for preventing reading failure in grades K-3: The big Ideasgrades K-3: The big Ideas
1. Increase the quality, consistency, and 1. Increase the quality, consistency, and reachreach of instruction in every K-3 classroomof instruction in every K-3 classroom
2. Conduct timely and valid assessments of 2. Conduct timely and valid assessments of reading growth to identify struggling readersreading growth to identify struggling readers
3. Provide more intensive interventions to 3. Provide more intensive interventions to “catch up” the struggling readers“catch up” the struggling readers
Systematic assessments of reading growth: Big Systematic assessments of reading growth: Big IdeasIdeas
Progress monitoringProgress monitoring in growth of critical reading in growth of critical reading skills for all children during the year to help plan skills for all children during the year to help plan instructioninstruction
End of year outcome assessmentsEnd of year outcome assessments in the critical in the critical elements of reading growth –is the child on track to elements of reading growth –is the child on track to read at grade level by third grade-how effective is read at grade level by third grade-how effective is our program?our program?A data management system that supports the use A data management system that supports the use of assessment information in planning instructionof assessment information in planning instruction
Screening assessmentsScreening assessments that identify children who that identify children who are lagging behind in growth of critical skillsare lagging behind in growth of critical skills
Diagnostic assessments Diagnostic assessments to help provide additional to help provide additional information useful for focusing and planning information useful for focusing and planning instructioninstruction
Systematic assessments of reading growth: Big Systematic assessments of reading growth: Big IdeasIdeas
Screening assessmentsScreening assessments that identify children who that identify children who are lagging behind in growth of critical skillsare lagging behind in growth of critical skills
What should we screen for?What should we screen for?
Kindergarten – letter knowledge, phonemic awareness, Kindergarten – letter knowledge, phonemic awareness, vocabularyvocabulary
11stst Grade–, phonemic awareness, phonics, vocabulary Grade–, phonemic awareness, phonics, vocabulary
22ndnd Grade– phonics, reading fluency, vocabulary Grade– phonics, reading fluency, vocabulary
33rdrd Grade– reading fluency, vocabulary Grade– reading fluency, vocabulary
More or less comprehensive screeners – DIBELS, TPRI, More or less comprehensive screeners – DIBELS, TPRI, PALS, PPVT, have information about reliability and validityPALS, PPVT, have information about reliability and validity
Systematic assessments of reading growth: Big Systematic assessments of reading growth: Big IdeasIdeas
Progress monitoringProgress monitoring in growth of critical reading in growth of critical reading skills for all children during the year to help plan skills for all children during the year to help plan instructioninstructionWhat can we progress monitor at presentWhat can we progress monitor at present
Kindergarten – letter knowledge, phonemic awareness, Kindergarten – letter knowledge, phonemic awareness, phonicsphonics
11stst Grade–, phonemic awareness, phonics, fluency Grade–, phonemic awareness, phonics, fluency22ndnd Grade– phonics, reading fluency Grade– phonics, reading fluency33rdrd Grade– reading fluency Grade– reading fluency
Different frequencies for different groupsDifferent frequencies for different groupsWhole class – three, four times a yearWhole class – three, four times a yearStudents receiving interventions – more frequentlyStudents receiving interventions – more frequently
What can teachers learn from PM assessments?What can teachers learn from PM assessments?
Information from progress monitoringInformation from progress monitoring
Are the children actually learning what I am Are the children actually learning what I am teaching?teaching?
Is my intervention strong enough to place the Is my intervention strong enough to place the children on a growth trajectory that ends in grade children on a growth trajectory that ends in grade level performance by the end of the year?level performance by the end of the year?
Are the children ready to move forward in the Are the children ready to move forward in the curriculum?curriculum?
Sept Dec Feb May
2nd Grade Growth in Oral Reading Fluency
16
32
64
48
80
96
Co
rre
ct w
ord
s p
er m
inu
te
What can teachers learn from these What can teachers learn from these assessments?assessments?Information from progress monitoringInformation from progress monitoring
Are the children actually learning what I am Are the children actually learning what I am teaching?teaching?
Is my intervention strong enough to place the Is my intervention strong enough to place the children on a growth trajectory that ends in grade children on a growth trajectory that ends in grade level performance by the end of the year?level performance by the end of the year?
Decisions to be madeDecisions to be made
Should I reteach the last unit to some of my children?Should I reteach the last unit to some of my children?
Should I move the child to a smaller group, or program Should I move the child to a smaller group, or program more instructional time?more instructional time?
Are the children ready to move forward in the Are the children ready to move forward in the curriculum?curriculum?
Should I seek help to implement a more powerful Should I seek help to implement a more powerful instructional strategy?instructional strategy?
In order to monitor progress adequately, we need In order to monitor progress adequately, we need two different kinds of information about progresstwo different kinds of information about progress
Information from curriculum embedded tests or teacher Information from curriculum embedded tests or teacher obs.obs.
Are the children actually learning what I am Are the children actually learning what I am teaching?teaching?
Information from “index” tests like the DIBELS or Information from “index” tests like the DIBELS or TPRITPRI
Are the children ready to move forward in the Are the children ready to move forward in the curriculum?curriculum?
Is my instruction powerful enough to place the child Is my instruction powerful enough to place the child on a trajectory for grade level achievement by the end on a trajectory for grade level achievement by the end of the year?of the year?
Systematic assessments of reading growth: Big Systematic assessments of reading growth: Big IdeasIdeas
End of year outcome assessmentsEnd of year outcome assessments in the critical in the critical elements of reading growth –is the child on track to elements of reading growth –is the child on track to read at grade level by third grade-how effective is read at grade level by third grade-how effective is our program?our program?
Kindergarten – phonemic awareness, letter knowledge, Kindergarten – phonemic awareness, letter knowledge, phonics, vocabularyphonics, vocabulary
11stst Grade– phonics, text fluency, reading comprehension, Grade– phonics, text fluency, reading comprehension, vocabularyvocabulary
22ndnd Grade– text fluency, reading comprehension, vocabulary Grade– text fluency, reading comprehension, vocabulary
33rdrd Grade– text fluency, reading comprehension, vocabulary Grade– text fluency, reading comprehension, vocabulary
Diagnostic TestingDiagnostic Testing
Purpose of diagnostic assessmentPurpose of diagnostic assessment-- -- to provide to provide information that is useful in planning more information that is useful in planning more effective inseffective instructiontruction
Diagnostic tests should be given when there is Diagnostic tests should be given when there is a a clear expectationclear expectation that they will provide new, that they will provide new, or more reliable, information about a child’s or more reliable, information about a child’s reading difficulties that can be used reading difficulties that can be used to provide to provide more powerful instructionmore powerful instruction. .
Diagnostic testing within the rich Diagnostic testing within the rich assessment context of screening, PM, assessment context of screening, PM, and outcome testsand outcome testsIf schools are implementing If schools are implementing screeningscreening, , progress progress monitoringmonitoring, and , and outcome assessmentsoutcome assessments in a reliable in a reliable and valid way, this should substantially reduce the and valid way, this should substantially reduce the need for additional testing using formal diagnostic need for additional testing using formal diagnostic instruments. instruments.
Will the diagnostic assessment actually provide Will the diagnostic assessment actually provide moremore information, or more reliable information, about the child’s information, or more reliable information, about the child’s strengths and weaknesses in the components of reading strengths and weaknesses in the components of reading than they already possess. If it will provide additional than they already possess. If it will provide additional information, will this new information be useful in planning information, will this new information be useful in planning additional instruction for the child?additional instruction for the child?
Diagnostic testing within the rich Diagnostic testing within the rich assessment context of the Reading First assessment context of the Reading First assessment planassessment planShould students identified as “at risk” at the Should students identified as “at risk” at the beginning of the year automatically be beginning of the year automatically be administered a diagnostic test to provide additional administered a diagnostic test to provide additional information to help plan instruction?information to help plan instruction?If you know a child is low in phonemic awareness, If you know a child is low in phonemic awareness, letter knowledge, and vocabulary at the beginning letter knowledge, and vocabulary at the beginning of K, what more can we reliably measure of K, what more can we reliably measure to guide to guide instructioninstruction??If you know a child is low in phonemic awareness, If you know a child is low in phonemic awareness, and knowledge of the alphabetic principal, and and knowledge of the alphabetic principal, and vocabulary at the beginning of 1vocabulary at the beginning of 1stst grade, what grade, what more can we reliably measure more can we reliably measure to guide instructionto guide instruction??
Diagnostic testing within the rich Diagnostic testing within the rich assessment context of the Reading First assessment context of the Reading First assessment planassessment planOn the basis of the On the basis of the diagnostic informationdiagnostic information available available from the screening measures, interventions from the screening measures, interventions focused on critical components of reading could focused on critical components of reading could begin immediatelybegin immediately, rather than waiting for , rather than waiting for additional diagnostic information generated by additional diagnostic information generated by diagnostic tests. The major argument for not doing diagnostic tests. The major argument for not doing additional diagnostic testing in this case is that it additional diagnostic testing in this case is that it would be would be unlikely to add any additional information unlikely to add any additional information critical to planning effective interventionscritical to planning effective interventions, and it , and it might delay the process of beginning the needed might delay the process of beginning the needed interventions for these “at risk” students.interventions for these “at risk” students.
Systematic assessments of reading growth: Big Systematic assessments of reading growth: Big IdeasIdeas
A data management system that supports the use A data management system that supports the use of assessment information in planning instructionof assessment information in planning instruction
The Florida Progress Monitoring and Reporting Network The Florida Progress Monitoring and Reporting Network (PMRN). Web-based data management and reporting (PMRN). Web-based data management and reporting networknetwork
Immediate reports for teachers, principals, district Immediate reports for teachers, principals, district staffstaff
School level entry of screening, progress School level entry of screening, progress monitoring, and some outcome scoresmonitoring, and some outcome scores
Tied to the Florida Student Information System, so Tied to the Florida Student Information System, so that student progress is tracked longitudinally, that student progress is tracked longitudinally, and from school to schooland from school to school
Class Status Report for teacher or principal
This is an example of a first grade classroom report for December.
The teacher can immediately see which of her students are at most risk for not being at grade level at the end of the year, and can use this report to group students for targeted instruction and support
Student Progress Report for teacher or principal
This report shows the progress of a third grade student in becoming a fluent and accurate reader in third grade. At the beginning of the year, the student was classified as high risk on this measure, but the student received excellent intervention, and by the end of the year, he was performing in the low risk range.
Scores of 60% of the class
median
5th percentile for class
Class Tracking Tool
Kindergarten
LNF
Full Year Students only
Sorted by Overall
School Distribution Report
Interval 4
KG-LNF
District Status Report
This report allows district level staff to quickly examine the performance of each of the schools in the district on critical measures of early reading skill.
Using Screening and Using Screening and outcome data to outcome data to
evaluate the evaluate the strength of strength of
Instruction at state, Instruction at state, classroom, and classroom, and
school levelschool level
0
10
20
30
40
50
60
70
80
K 1 2 3
2004-2005
% of students at “grade level” level at Assessment 1 % of students at “grade level” level at Assessment 1 using combined index from DIBELS measuresusing combined index from DIBELS measures
3939
7575
6565
3838
0
10
20
30
40
50
60
70
80
K 1 2 3
2004-2005
% of students at “grade level” level at Assessment 4% of students at “grade level” level at Assessment 4
6767
52524848
3939
Notice the drop in % of students Notice the drop in % of students at grade level from beginning to at grade level from beginning to end of yearend of year
0
10
20
30
40
50
60
70
80
K 1 2 3
2004-2005
Percent of students at High Risk level at Assess 1Percent of students at High Risk level at Assess 1
2323
991414
3333
0
10
20
30
40
50
60
70
80
K 1 2 3
2004-2005
Percent of students at High Risk level at Assess 4Percent of students at High Risk level at Assess 4
1818 2020
2828
2222
Notice the increase in % of Notice the increase in % of students at high risk from students at high risk from beginning to end of yearbeginning to end of year
What are the areas in most need of immediate What are the areas in most need of immediate improvement?improvement?
Stronger support for the growth of text Stronger support for the growth of text reading fluency in second gradereading fluency in second grade
37th percentile
53rd percentile
:
Instructional Emphasis for Second Grade
3.0%
5.1%
49.0% 21.2%
5.5%
3.2%
12.9%
Other
Writing/language art
Comprehension Text reading
Fluency
Spelling
Word study/phonics
Comprehension42%
Non-instructional6%
Writing or lang. arts 3%
Fluency4%
Spelling2%
Word study/phonics21%
Text reading22%
2004 2005
About half our second graders began second grade not having met the February 1st grade benchmark in NWF
Slightly more than 20% still hadn’t met the 1st grade benchmark at the end of second grade
One problem that arises from so many students One problem that arises from so many students coming into 2coming into 2ndnd grade still weak in effective, grade still weak in effective, accurate word reading strategiesaccurate word reading strategies
Growth in fluency requires Growth in fluency requires accurateaccurate practice practice
A A majormajor factor underlying growth in fluency factor underlying growth in fluency for struggling readers is how fast the number for struggling readers is how fast the number of words they can recognize “by sight” of words they can recognize “by sight” increasesincreasesChildren must read unfamiliar words with Children must read unfamiliar words with perfect perfect accuracy on multiple occasions before accuracy on multiple occasions before they can become sight wordsthey can become sight words
Sight vocabulary must grow Sight vocabulary must grow very rapidlyvery rapidly in in second grade to keep pace with normative second grade to keep pace with normative developmentdevelopment
Our third graders are “holding their Our third graders are “holding their own” in fluency development, but own” in fluency development, but they come into third grade too far they come into third grade too far behind in the development of fluent behind in the development of fluent reading skillsreading skills
31rd percentile
32nd percentile
What are the areas in most need of immediate What are the areas in most need of immediate improvement?improvement?
Stronger support for the growth of text Stronger support for the growth of text reading fluency in second gradereading fluency in second grade
More powerful instruction toward mastery of More powerful instruction toward mastery of the alphabetic principle early in first gradethe alphabetic principle early in first grade
47th percentile
62nd percentile
Over ½ of our students did not make the benchmark on time
What are the areas in most need of immediate What are the areas in most need of immediate improvement?improvement?
Stronger support for the growth of text Stronger support for the growth of text reading fluency in second gradereading fluency in second grade
More powerful instruction toward mastery of More powerful instruction toward mastery of the alphabetic principle early in first gradethe alphabetic principle early in first grade
Creative work to develop and support stronger Creative work to develop and support stronger interventions for struggling students in grades interventions for struggling students in grades 1,2, and 3 – school level must become 1,2, and 3 – school level must become involvedinvolvedContinued efforts to expand vocabulary Continued efforts to expand vocabulary instruction-linked to comprehension- instruction-linked to comprehension- accessing words in multiple contexts accessing words in multiple contexts
Another way to think about improvements and Another way to think about improvements and performance – two indices of instructional performance – two indices of instructional effectivenesseffectiveness
Effectiveness of Core Instruction (ECI)Effectiveness of Core Instruction (ECI) – what – what percentage of students who began the year at percentage of students who began the year at grade level on the screening measures finish the grade level on the screening measures finish the year at grade level?year at grade level?
Effectiveness of Interventions(EI)Effectiveness of Interventions(EI) – what percentage – what percentage of students who began the year at some level of of students who began the year at some level of risk on the screening measures finish the year at risk on the screening measures finish the year at grade level?grade level?
The ECI and EIThe ECI and EI
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 19 20 21
screening
One academic year
9 of 13 students who were at grade level continue at grade level
ECI = .69ECI = .69
2 of 7 students who began the year “at risk” finished the year at grade level
EI = .29EI = .29
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 19 20 21
outcome
Core and Intervention effectiveness in year 1 Core and Intervention effectiveness in year 1 and 2and 2
Core ProgramCore Program InterventionIntervention
K year 1K year 1 year 2year 2
82%82%87%87%
45%45%55%55%
11stst year 1 year 1 year 2year 2
57%57%65%65%
14%14%16%16%
22ndnd year 1 year 1 year 2year 2
70%70%71%71%
6%6%5%5%
33rdrd year 1 year 1 year 2year 2
84%84%81%81%
15%15%14%14%
District EI %tile %Min %FRL ECI % Grade
Orange 39 99 84 89 82 A
Collier 36 97 98 99 79 D
Orange 34 95 83 80 95 A
Collier 33 93 94 93 93 B
Palm Bea. 33 91 78 75 84 C
Sarasota 32 89 80 85 89 B
Orange 32 89 93 93 67 B
Lee 32 89 67 73 84 B
Palm Bea. 31 89 95 87 66 C
Broward 31 89 99 66 67 B
Schools with High Average Effectiveness of Iterventions
District EI %tile %Min %FRL ECI % Grade
Hillsboro. 8 2 88 93 4 C
Polk 8 3 55 74 10 C
St. Lucie 10 4 58 71 14 C
Polk 10 5 66 89 3 C
Pinellas 10 5 60 74 23 B
Volusia 11 6 68 89 1 C
Brevard 12 7 79 97 32 D
Marion 12 7 47 81 52 A
Hillsboro. 12 7 84 84 45 B
Hillsboro. 12 8 63 58 66 A
Schools with Low Average Effectiveness of Interventions
A final concluding thought….A final concluding thought….
There is no question but that “leaving no There is no question but that “leaving no child behind in reading” is going to be a child behind in reading” is going to be a significant challenge…significant challenge…
It will involve professional development for It will involve professional development for teachers, school reorganization, teachers, school reorganization, careful careful assessmentsassessments, and a relentless focus on the , and a relentless focus on the individual needs of every child…individual needs of every child…
But, its not the most difficult thing we could But, its not the most difficult thing we could be faced with…be faced with…
Consider this task for example…Consider this task for example…
Questions/Questions/
DiscussionDiscussion