Chapter 23

Using Technology to Help Make Assessment Decisions

Computers and Instructional Decision Making

• Computers and software allow teachers to individualize curriculum and instruction for students

• What a student knows– What skills has the student mastered?– Should instruction be individualized, or does the

student fit with a group in the classroom?– Are they behind in the curriculum?

• Rate of learning– Combine information about what a student knows

with what they need to know

Rate of student progress

Class and Individual Student Diagnostic Reports

Class and Individual Student Diagnostic Reports

Class and Individual Student Diagnostic Reports

Class and Individual Student Diagnostic Reports

Technology Enhanced Progress Monitoring Measures

• Periodic – Completed by hand or by computer on a periodic basis

(e.g. every 3-5 weeks)• Continuous – Used daily to monitor overall progress

Measures Reviewed* Periodic Continuous AIMSweb Accelerated Math

STAR ReadingSTAR Math

* For a more complete list see table 23.1 and 23.2 in the textbook

Periodic Progress Monitoring Measures: AIMSweb

• General– Collects and manages progress monitoring data– Enter data from:• Curriculum based measurement (CBM) • Dynamic Indicators of Basic Early Literacy Skills (DIBELS)

– Web-based– Comprehensive:• Allows measures across a broad content domain• Most recently, screening students at-risk for behavior

problems

Periodic Progress Monitoring Measures: AIMSweb

• Reliability and Validity– Adequate evidence for reliability and content

validity – National norms

• Potential disadvantages– Recent work highlighting problems with

comparability across CBM measures (i.e. high amounts of error)

– Ease of use (hand scoring and entering)

Periodic Progress Monitoring Measures: STAR Reading

• General– Computer-adaptive test designed to provide

estimates of students’ reading and comprehension levels relative to national norms

– Short comprehension and extended comprehension items

• Scores– Grade equivalents, percentile ranks, standard

scores, normal-curve equivalents, and scaled scores

Periodic Progress Monitoring Measures: STAR Reading

• Norms– 27,807 students in 247 schools – Sample characteristics approximate the U.S.

population

• Reliability– High test-retest reliability

• Validity – Adequate criterion-related validity evidence

Periodic Progress Monitoring Measures: STAR Math

• General– Computer adaptive test designed to provide

estimates of students’ math achievement levels relative to national norms

– Grades 1-12– 3 minutes to solve each multiple choice item

(items are spread across the major strands of math content)

Periodic Progress Monitoring Measures: STAR Math

• Scores– Grade equivalents, percentile ranks, standard scores,

normal-curve equivalents, and scaled scores• Norms– 29,185 students in 312 schools– Sample is similar to U.S. population

• Reliability– Adequate test-retest reliability for screening and

progress monitoring• Validity– Moderately high criterion-related validity evidence

Continuous Progress Monitoring Measures: Accelerated Math

• General– Technology-enhanced system designed to measure

student progress and manage student practice• Process:

1.Students are placed at an instructional level using STAR Math

2.Complete exercises (generated by the software) at instructional level

3.Score work using the software 4.Monitor accuracy and task completion at the student,

class, school, or district level

Computer Scoring Systems• Available for a variety of tests – including those

used for diagnostic and eligibility decisions• Advantages– Reduce time– Reduce human-error– Assist with more sophisticated analyses

• Note that:– Human-error is not removed, it is simply minimized– Great care must be paid to the development of

software used to generate scores that will be used for important decisions

– Storing electronic data introduces a new challenge for education professionals

Computer Scoring Systems

• Beyond simple calculation– Software used to evaluate constructed responses– Educational reports (i.e. interpretation)• Users should be actively involved in the report writing

process and should refrain from accepting automated reports “as is”