New York State School Boards Association August … · New York State School Boards Association August 14, 2015 Data Use for Accountability and ... statement about Elvis from “Naked

New York State School

Boards Association

August 14, 2015

Data Use for Accountability and

Student Achievement

From School Performance to

Teacher Growth Scores

Fred Cohen, Nassau BOCES

Data Warehouse Specialist

[email protected] 608-6640

mailto:[email protected]

Robert E. Lupinskie Center for Curriculum, Instruction and Technology

One Merrick Ave · Westbury, New York 11590

(516) 608-6640 · Fax: (516) 608-6616

Data for the 2014-2015

assessments (is, are) now

available!

First, a quick question.

Which is correct?

Now that I know that “data” is

usually considered plural, what

do I do with “these” data?



(516) 608-6640 · Fax: (516) 608-6616

Here’s an idea suggested by

Robert Burn’s 1786 poem,

“The Louse”

“And if some Power this small gift give us

To see ourselves as others see us!

It would from many a blunder free us….”

(modern translation)



(516) 608-6640 · Fax: (516) 608-6616

How ‘m I doin’? Mayor Ed Koch

Or, in the immortal words of New York

City’s three-time mayor, Ed Koch

But before we can decide what

data to use in our self-reflection,

some guidelines developed over

the past 49 years



(516) 608-6640 · Fax: (516) 608-6616

There are four parts to any

self-reflection or data analysis

• Trends—compare yourself to yourself.

• The competition—compare yourself to everyone

else (especially those most similar to you).

• Benchmarks—compare yourself to state or

national benchmarks, targets, or expectations.

• Disaggregate the data—break down the data

into subgroups and other meaningful divisions.



(516) 608-6640 · Fax: (516) 608-6616



(516) 608-6640 · Fax: (516) 608-6616

How is each separate school and

each subgroup population doing?

• Overall, our students are doing quite well.

• But what if district scores soar while one

school lags far behind?

• And what if special education students,

students of limited English proficiency, low

income students, or other subgroups are

failing?



(516) 608-6640 · Fax: (516) 608-6616

Data savvy people are aware of this

additional obligation to

“Disaggregate the data!”

• It is not sufficient to achieve overall

success. (“Our school/district is

performing well.”)

• Every distinct population must also

perform well.



(516) 608-6640 · Fax: (516) 608-6616

That makes four questions which

must be answered to begin a true

data analysis.



(516) 608-6640 · Fax: (516) 608-6616

Four Key Issues to Consider 1. Trends—how did my district do compared to last year?

(and the year before)

2. The overall competition—how are other districts, especially similar districts, doing? (in the county, state)

3. State targets—what benchmarks or expectations have been set by the State and Federal Government? (Adequate Yearly Progress or AYP)

4. When we disaggregate our data, how are each of our subgroups doing? (both genders, all ethnicities, special ed, limited English proficient, and low income students). Also disaggregate by school and teacher.

What follows are samples of

Nassau BOCES Instructional

Data Warehouse Reports that

Offer these Insightful

Reflections



(516) 608-6640 · Fax: (516) 608-6616

The Snapshot Report—A Simple Performance

Level Comparative Analysis



(516) 608-6640 · Fax: (516) 608-6616

Disaggregate

by school

Compare to last year

Compare to everyone else

Compare to state targets

Disaggregate the data

2014/2015



(516) 608-6640 · Fax: (516) 608-6616

The Trends Analysis—A Longitudinal Perspective





69.0

The Subgroup Analysis Report—Disaggregating Results

by Subgroup



(516) 608-6640 · Fax: (516) 608-6616

89.2%

69.1%

84.5%




The purpose of disaggregation 2014-2015

The Gap Report

A Question by Question

Comparison to the Nassau County

Region



(516) 608-6640 · Fax: (516) 608-6616



(516) 608-6640 · Fax: (516) 608-6616

Grade 3-8

Assessments

and

Regents

Assessments







(516) 608-6640 · Fax: (516) 608-6616



(516) 608-6640 · Fax: (516) 608-6616

The Gap Report for Released Questions—A Question by

Question Comparison to the Nassau County Region with

links to actual questions released by SED



(516) 608-6640 · Fax: (516) 608-6616



(516) 608-6640 · Fax: (516) 608-6616



(516) 608-6640 · Fax: (516) 608-6616

The Wrong Answer Summary Analysis (Released Questions)—

A Summary of the Wrong Answers Chosen by Students



(516) 608-6640 · Fax: (516) 608-6616



(516) 608-6640 · Fax: (516) 608-6616



(516) 608-6640 · Fax: (516) 608-6616

Teacher by Teacher Analysis

(Disaggregate the data)

Teacher # Students % Correct B % Distractor D

A 82 59% 33%

B 87 52% 40%

C 101 66% 27%

D 76 49% 40%

E 97 44% 42%

F 71 73% 15%



(516) 608-6640 · Fax: (516) 608-6616

Teachers must have access to their released question data, as soon

after their September return to school, as possible.

Even the Most Ardent Critics of Current NY

Test Policy Support This Test Analysis

• From Carol Burris’ Hechinger Report

blog: “I am not discounting the importance

of using student achievement in assessing

teacher quality and in helping teachers

improve. Prior to the mandated use of

scores, I used achievement results to help

guide my observations, review teacher

lesson plans and design professional

development.”

Time for a musical interlude!



(516) 608-6640 · Fax: (516) 608-6616

I adapted the following

statement about Elvis from

“Naked Scientists Podcasts” as

reported by Free Movement

Magazine, November 13, 2007



(516) 608-6640 · Fax: (516) 608-6616



(516) 608-6640 · Fax: (516) 608-6616

• The number of Elvies Presley

impersonators has reached an all-time

record high – there are now at least

85,000 Elvis’s around the world, compared

to only 170 in 1967 when Elvis died. At

this rate of growth, experts predict that, by

2019, Elvis impersonators will make up a

third of the world population.



(516) 608-6640 · Fax: (516) 608-6616

Mistakes Made on the this Slide

• Elvies’ name is incorrectly spelled.

• Elvis died in 1977 not 1967.

• The data analysis predicting the trend is

meant to be comically absurd.

• Therefore, each of us should look at all

data conclusions with skepticism, and

each of us should do our own analyses

of all data conclusions.



(516) 608-6640 · Fax: (516) 608-6616

Overall Best Source for School Data

• 2013-2014 School and District Report

Cards

• https://reportcards.nysed.gov/



(516) 608-6640 · Fax: (516) 608-6616

Other Source of School Data

http://www.p12.nysed.gov/irs/

http://www.p12.nysed.gov/irs/



(516) 608-6640 · Fax: (516) 608-6616



(516) 608-6640 · Fax: (516) 608-6616 Grand Island



(516) 608-6640 · Fax: (516) 608-6616

What data are available on the

Fiscal Supplement?



(516) 608-6640 · Fax: (516) 608-6616



(516) 608-6640 · Fax: (516) 608-6616

Average

Need/Resource Capacity



(516) 608-6640 · Fax: (516) 608-6616

Need

________________________________

Resource Capacity

% Low Income Students

____________________

Average Price of Homes



(516) 608-6640 · Fax: (516) 608-6616



(516) 608-6640 · Fax: (516) 608-6616

School Report Cards Have Four Parts



(516) 608-6640 · Fax: (516) 608-6616



(516) 608-6640 · Fax: (516) 608-6616

School Report Card Accountability

“Big Ideas” • Districts and schools must meet NYSED defined

performance levels or targets.

• Each district and school must also meet similar performance levels or targets for each subgroup (disaggregated data).

• Target levels are raised each year.

• Under No Child Left Behind (NCLB), all students, and, therefore, all subgroups had to be proficient by 2013-14!

• District and school proficiency is defined by something called a Performance Index.

ESEA Waiver—Growth over Achievement



(516) 608-6640 · Fax: (516) 608-6616

Performance Index

PI – Performance Index: is a value from 0 to 200 that

is assigned to an accountability group, indicating how

that group performed on a required State test (or

approved alternative) in English language arts,

mathematics, or science.

Levels of Performance

1 Basic Below Standard

2 Basic Proficiency Meets Basic Standard

3 Proficient Meets Proficiency Standard

4 Advanced Proficiency Exceeds Proficiency Standard



(516) 608-6640 · Fax: (516) 608-6616



(516) 608-6640 · Fax: (516) 608-6616

For 2013

assessments



(516) 608-6640 · Fax: (516) 608-6616

For 2013

assessments



(516) 608-6640 · Fax: (516) 608-6616

Calculating PI at the

Elementary/Middle Level

Performance Index

100 × [(Count of Continuously Enrolled Tested

Students Performing at Levels 2, 3, and 4 + the

Count at Levels 3 and 4) ÷ Count of All

Continuously Enrolled Tested Students]

or

Double the percentage of students scoring at

Level 3 and Level 4, and add the percentage

at Level 2.



(516) 608-6640 · Fax: (516) 608-6616

District A—25% at each

Performance Level

• 50% (Level 3 and 4) X 2 = 100

• Plus 25% (Level 2) = 125

• Performance Index = PI = 125

The school or district Performance

Index must meet or exceed SED’s

Annual Measurable Objective in

order to make Adequate Yearly

Progress.

If your PI is at or above the required

AMO, you make AYP.

Essence of Accountability

• Your performance (as measured by a

Performance Index) must meet the state

standard (Annual Measurable Objective).

• If it is, you make Adequate Yearly

Progress.

• Said another way, if your PI meets the

state AMO, you make AYP!



(516) 608-6640 · Fax: (516) 608-6616



(516) 608-6640 · Fax: (516) 608-6616

AMO is the height of the wall

PI is how high I’ve jumped

If I get over the wall, I’ve made AYP

NYS Requirements

Every district has numerous

hurdles to vault.

• Elementary-middle school and high school

• ELA and Math

• Every school and total district

• Every special population—multiple ethnic

groups, ELL, special education, and

poverty classifications

• And every combination of the above



(516) 608-6640 · Fax: (516) 608-6616

What are the differences

between growth and proficiency? • Proficiency measures the height of the wall you

hurdled.

• Growth measures the difference between the

height of the wall you hurdled this year compared

to the height of the wall you hurdled last year.

• Proficiency measured only school and district

performance.

• Growth will measure teacher and principal

performance too.



(516) 608-6640 · Fax: (516) 608-6616

How is teacher performance

measured by student

assessment scores?



(516) 608-6640 · Fax: (516) 608-6616

To understand the new teacher

growth evaluation system, you

must meet my friend HEDI. • HEDI stands for the four teacher ratings: Highly

Effective, Effective, Developing, and Ineffective.

• Each of the four classifications is further

subdivided, creating a zero to twenty point scale.

• Highly Effective………..18-20 points

• Effective………………...9-17 points

• Developing……………..3-8 points

• Ineffective………………0-2 points



(516) 608-6640 · Fax: (516) 608-6616

Each teacher’s growth score is based

on the mean growth percentile of the

students in her or his class.

• Did students increase or decrease their

overall ranking or position in New York

State compared to similar students?

• Sarah was in the 10th percentile statewide

compared to other ELL students last year,

and now she’s in the 35th percentile.

Sarah will improve my teacher rating. Robert E. Lupinskie Center for Curriculum, Instruction and Technology


(516) 608-6640 · Fax: (516) 608-6616

Will All Teachers Get Growth

Scores from SED?

• Only teachers of grade 4-8 ELA and Math

will get Growth or Mean Growth Percentile

(MGP’s) HEDI Scores from SED.

• 80% of teachers will get Student Learning

Objective Growth or HEDI Scores from the

district.



(516) 608-6640 · Fax: (516) 608-6616



(516) 608-6640 · Fax: (516) 608-6616

•K, 1st, 2nd, 3rd grade teachers

•Music, Art, Phys Ed teachers

•Teachers of high school courses including

Regents exam courses

Which teachers will get SLO’s

instead of MGP growth

scores?

What is a

Student Learning Objective? • Student Learning Objectives (SLO’s)–

a long-term academic goal for a teacher’s

students, which represents the most

important learning for the year/course.



(516) 608-6640 · Fax: (516) 608-6616

Measuring growth by SLOs and

the other components of a

teacher evaluation system

continue to be a challenge for all!



(516) 608-6640 · Fax: (516) 608-6616

Summary and Final Exam

• What are four issues to consider when looking at

school related data?

• How does a school, district or subgroup make

AYP?

• Describe the two different methods used to

evaluate teachers under SED’s new

accountability system.



(516) 608-6640 · Fax: (516) 608-6616

If I’ve done my job well, this is your responsibility, not mine!

Review of Key Slides

Final Summary

There are four parts to any

self-reflection or data analysis

• Trends—compare yourself to yourself.

• The competition—compare yourself to everyone

else (especially those most similar to you).

• Benchmarks—compare yourself to state or

national benchmarks, targets, or expectations.

• Disaggregate the data—break down the data

into subgroups and other meaningful divisions.



(516) 608-6640 · Fax: (516) 608-6616

The school or district Performance

Index must meet or exceed SED’s

Annual Measurable Objective in

order to make Adequate Yearly

Progress.

If your PI is at or above the required

AMO, you make AYP.

Will All Teachers Get Growth

Scores from SED?

• Only teachers of grade 4-8 ELA and Math

will get Growth or Mean Growth Percentile

(MGP’s) HEDI Scores from SED.

• 80% of teachers will get Student Learning

Objective Growth or HEDI Scores from the

district.



(516) 608-6640 · Fax: (516) 608-6616

A final “Do!” and a final “Do not”

• Do seek out the data you need to hear

rather that the data you want to hear!

• Do not use data1 the way a drunk uses a

lamppost—for support rather than

illumination. Andrew Lang

1 Statistics



(516) 608-6640 · Fax: (516) 608-6616

New York State School

Boards Association

August 14, 2015

Data Use for Accountability and

Student Achievement

From School Performance to

Teacher Growth Scores

Fred Cohen, Nassau BOCES

Data Warehouse Specialist

[email protected] 608-6640

mailto:[email protected]

Documents

New York State School Boards Association August … · New York State School Boards Association August 14, 2015 Data Use for Accountability and ... statement about Elvis from “Naked