The Many Meanings of “Multiple Measures”

Susan Brookhart

Volume 2009, Volume 67:3

ASCD, November 2009, pp. 6-12

Would you choose a house using one measure alone?

Why use multiple measures for decisions in education?

• Construct validity– The degree to which a

score can convey meaningful information about an attribute it measures

• Decision validity– The degree to which

several relevant types of information can inform decision-making

Multiple Measures

• Measures of different constructs

• Different measures of the same construct

• Multiple opportunities to pass the same test

Using Multiple Measures for Educational Decisions

Conjunctive Approach

(All measures count)

Compensatory Approach

(High performance on one measure can

compensate for lower performance on

another measure)

Complementary Approach

(High performance on any measure counts)

Measures of different constructs

School accreditation ratings based upon

student achievement meeting identified

targets in Reading, Math, Science, and

Social Studies

An outside agency identifies the “best

schools identified by computing an index

of weighted scores

AYP “Safe Harbor” by having a percentage of students

who scored below proficiency decreasing by ten

percentage points from the previous year

Different measurers of the same construct

Students have to pass a reading

comprehension test on two stories at the same reading level before the

student is allowed to read stories at the next

higher reading level

Teachers determine standards-based grades in a course using scores on multiple assessments

measuring the same GLCE or HSCE

Teachers allow student choice on assessment tasks

to demonstrate their understanding of the learning

targets for a unit

Multiple opportunities to pass the same test

Students meeting all requirements will

graduate after passing an exit exam, no matter

how many opportunities

Teachers allow students to retake a unit test to

demonstrate mastery of the unit’s outcomes

Students must pass one mathematics test in order to

graduate; students can choose the state test or an

end-of-course exam in either Algebra I or Geometry

Other Examples

• NCLB accountability is conjunctive (i.e., aggregate and subgroups must reach threshold to make AYP)

• Most classroom grading policies are compensatory (i.e., average, percentage)

• Getting a driver’s license is complementary (i.e., passing one of the requirements when you want)

Guiding Principle for Multiple Measures

• Know your purpose!

–What do you need to know?

–Why do you need to know it?

Suggestions for UsingMultiple Measures

for Decision Making

• Classroom assessments linked to the same construct to determine mastery

• Granting credit for graduation requirements

• Evaluating school programs