DCAS “Standard Setting”

Appoquinimink School DistrictBoard of Education

December 14, 2010Odessa, DE

NOT the world we live in today!

The Bottom Line• The level of performance required for

each Performance Level has gone up.– The level of performance that earned many

students a PL3 last year will earn those students a PL2 this year.

– Many students are likely to drop a Performance Level this year, even if their level of performance increases.

• This is not a function of the standards or of the test – it is solely a function of the raising of requirements for each Performance Level.

The Bottom Line• Two factors will present serious

challenges to schools’ efforts to make AYP1. The targets are going up:

• Math targets will rise from 67% to 75%• Reading targets will rise from 79% to 84%

2. The level of performance needed for students to “meet standard” has gone up.

The Bottom Line• Schools that are on Academic Watch or

are Under Improvement will find it much harder to raise their ratings.

• Many schools are likely to go on Academic Watch for the first time.– Some schools that have always been Superior

will drop straight to Academic Watch.– This will be particularly shocking to elementary

schools that have never had anything but a Superior rating.

The Bottom Line

Under DSTP Under DCAS

The Process

• “Standard Setting” was really the setting of the cut points between Performance Levels.

• Ordered Item Book: 66 sample test items, in order of difficulty as determined by students’ performance on the spring field test.

• Descriptions of the state standards, by grade level.

• “Think of the lowest performing student who just barely meets the standard.”– “What item will he get right 2/3 of the time?”– This defines the bottom level of performance for

PL3.

The Charge to the Panels

• Raise the standards.• Rationale: This will…

– bring Delaware’s state assessment more in line with NAEP and PISA

– “drive school improvement”

FROM DOE’s PRESENTATIONS

Benchmarks for Meets Standard

11

Percent of Students in DSTP and DCAS Meets Standard in Reading

0

10

20

30

40

50

60

70

G2R G3R G4R G5R G6R G7R G8R G9R G10R

DSTP DCAS

12

DSTP and Projected DCAS, Performance Level 3, Meets Standard, Reading

Spring 2011 AYP Target: 84%

13

Percent of Students in DSTP and DCAS Meets Standard in Mathematics

0

10

20

30

40

50

60

70

G2M G3M G4M G5M G6M G7M G8M G9M G10M

DSTP DCAS

DSTP and Projected DCAS, Performance Level 3, Meets Standard, Mathematics

Spring 2011 AYP Target: 75%

14

In general, the interim DCAS standards are higher than the previous DSTP performance standards.

This will result in fewer students performing at and above the interim DCAS performance standards.

This is consistent with current federal and state education reform initiatives across the country.

What was the Main Outcome of Standards Setting?

The interim standards will also be used as part of Delaware’s growth model reporting and indices of teacher and school effectiveness.

The Department is organizing a DCAS communication plan with regional forums, media articles, website resources, and other briefings as needed.

15

What are the Next Steps?

Sample Reading ItemOn reading tests, students can see the passage and all

the associated items

Sample Math Item The test includes some items where the students

respond graphically

Scale Score and Performance Level for Student

Teachers have immediate access to a printable score report for each student

Scale Score and Performance Level for Student

SO NOW WHAT?

The Implications

• For Teachers

– The experienced teacher’s mental image of what level of student performance meets standard is no longer valid.

– It probably will take several administrations of the test for teachers to “recalibrate” their understanding of the levels of performance that are associated with the different Performance Levels.

Table from DSBA Report

DSTP DCAS NAEP PISA

PL 5 – Well Above Standard

PL 4 – Advanced

Proficient

Level 4 – Meets International Standard

PL 4 – Above Standard

Level 3 – Approaching International Standard

PL 4 Cut Score Raised

PL 3 – Meets the Standard

PL 3 – Meets Standard

Below StandardPL 3 Cut Score Raised

PL 2 – Below Standard

Below StandardPL 2 – Below Standard

PL 2 Cut Score Raised

PL 1 – Well Below Standard

PL 1 – Well Below Standard

!

The Implications

• For Principals

– Instructional Leadership• What do these changes mean for teachers? What

support will they need?• What do these changes mean for students, both

collectively and as individuals? What support will they need?

• Prepare before the tests; Analyze & use the data after the tests

– School Ratings• Staff & Student Morale• Front-line Public Relations

The Implications

• For District CIA Staff

– Doubly important that the district’s curriculum be:• Aligned with the standards• Aligned vertically within the district (no instructional

gaps)• Rigorous• Competitive on a national, even global, playing field

The Implications

• For Superintendents

– Manage Change at all levels

– Communication• Constituents & School Boards: prep for what’s to

come – before the headlines hit• DOE• Partners in Education – DSBA & others

The Implications

• For School Boards

– Need to be aware of the possibilities.

– Need to understand that a drop in ratings may come even if there’s an improvement in student performance.

– Need to have answers for the parking lot conversations &the evening phone calls

– Need to be able to support administration, not (necessarily) blame them

The Future – Short Term

• Monitor the testing process & learn from it– Technical & logistical issues– “Implementation dip”

• Make good use of the immediate feedback– Recalibrate our understanding of cut scores– Adapt “on the fly”– Cycle between short term (instructional

strategies, individual student help) and long term (curriculum alignment & revisions)

• Keep working for the best interests of our students.

The Future – Long Term

Paper & Pencil → Computer Administered

Constructed Response→ Multiple Choice

State Standards → National Standards

State Assessment → National Assessment ???

DSTP → DCAS → NCAT*

*National Computer-Administered Test (not a real test…yet)

Reading

• DOE Press Release, 9/17/10• Jim Wilson article

– Although Jim is a State Board of Education Member, his concern is what he sees as shortcomings of the national initiative.  He is very pleased with the way Dr. Lowry and her staff have addressed the issue of raising our state standards and cut scores to move Delaware closer to the proficiency standards established by the NAEP.

Questions?