Student Achievement Dashboard Training. Objectives ●PWBAT understand the purpose for using the dashboard to analyze student achievement ●PWBAT understand

Student Achievement Dashboard Training

Objectives

● PWBAT understand the purpose for using the dashboard to analyze student achievement

● PWBAT understand the concepts and metrics used in the dashboard

● PWBAT know how to use the dashboard

Purpose & Functions

Purpose - To inform school and district leadership about the performance and growth of students

Questions We Sought to Answer● Proficiency

o How did each school/grade perform?o Which schools/grades have the highest/lowest proficiency?

● Growtho What was the growth for each school/grade?o Which schools/grades have the highest/lowest growth?

...this is a new way of thinking about our data!

Metrics

● Proficiency● Median Percentile Rank● Student Growth Percentile● Quartiles

Methodology

● We grouped all students in the district that have the same test for the previous year (2013) and the current year (2014).

● Only students with BOTH scores AND who are promoted are included in each dataset.

● For each group, we converted the FCAT/EOC scale score to a percentile rank.

Quick Tutorial on Percentiles

A percentile is a measure that indicates the percent of a distribution that is equal to or below it.

For example, let’s say Lara scored at the 12th percentile on a test. This means that she performed better than 12% of all other students who took the test. And let’s say Marcus scored at the 79th percentile. This means that he scored better than 79% of the other students who took the test.

Groups & CountsReading

2013 2014 # of Students

Grade 3 25,313

Grade 3 Grade 4 23,298







Math

2013 2014 # of Students

Grade 3 25,301





Grade 6 Grade 7 Alg 3,357



Grade 7 Alg Grade 8 Geo 2,349




Proficiency

First, and foremost, we are most concerned with proficiency. We want more of our students to be able to read and do math proficiently.

Proficiency

Proficiency Example: Let’s say there are 85 students that have a 2014 FCAT Grade 4 score (current year) AND a 2013 FCAT Grade 3 score (previous year).

In 2013, 43% of these students passed the 3rd grade test. But in 2014, 48% (of these same 85 kids) passed the 4th grade test. These proficiency rates exclude students that do not have BOTH scores.

We are NOT comparing “this year’s 4th grade students” to “last year’s 4th grade students.” We are comparing “this year’s 4th grade students” to “how they did in 3rd grade (the previous year).”

Grade 3 does NOT have a prior year test, and as a result, this group is NOT included in the “All” totals for 2013 and 2014.

ProficiencyExample: Let’s say there are 41 8th grade students that have a 2014 Algebra 1 EOC score (current year) AND a 2013 FCAT Grade 7 score (previous year).

In 2013, all of these students passed the FCAT Math Grade 7 test. But in 2014, only 92% (of these same 41 kids) passed the Algebra 1 EOC. These proficiency rates exclude students that do not have BOTH scores.

Growth

While important, proficiency rates don’t tell the whole story about what is happening at a school. Many of our students start well below grade level, so in order to catch them up, we need for them to make growth. Growth metrics account for where a group of students started the year (proficiency rates alone don’t do this).

Three growth metrics:● Change in median percentile rank● Student Growth Percentiles● Quartiles

Growth Example

Students

Aaron

Alesiah

Anna

Carlos

Christopher

Darryl

Dontavius

Eric

Jacqueline

Kevin

Leah

Tamara

Let’s take a look at a group of 12 students.

In this example, these students took the FCAT Reading Grade 6 test in 2013 and the FCAT Reading Grade 7 test in 2014.

Percentile Rank (Prior Yr.)2013 FCAT Grade 6

Student FCAT Scale Score

FCAT Percentile

Aaron 211 24

Alesiah 232 60

Anna 211 24

Carlos 177 3

Christopher 204 16

Darryl 217 33

Dontavius 245 82

Eric 226 48

Jacqueline 196 10

Kevin 167 0

Leah 220 38

Tamara 191 8

Anna’s score of 211 puts her in the 24th percentile (she performed better than 24% of all students who took this test).

Dontavius’ score of 245 puts him in the 82nd percentile (he did better than 82% of all students who took this test).

Each student’s FCAT/EOC scale score is converted into a percentile. In this example, the scale scores are compared against all 24,212 students in this group.

Median Percentile Rank (Prior Yr.)2013 FCAT Grade 6


FCAT Percentile

Dontavius 245 82

Alesiah 232 60

Eric 226 48

Leah 220 38

Darryl 217 33

Aaron 211 24

Anna 211 24

Christopher 204 16

Jacqueline 196 10

Tamara 191 8

Carlos 177 3

Kevin 167 0

Median 211 24

We sorted the list of 2013 Grade 6 scores and found the median. The median FCAT scale score is 211, which corresponds to a median percentile of 24.

This means that half of the students in this group scored above the 24th percentile and half scored below.

Percentile Rank (Current Year)2014 FCAT Grade 7


FCAT Percentile

Aaron 224 34

Alesiah 245 71

Anna 215 21

Carlos 197 7

Christopher 201 9

Darryl 221 29

Dontavius 258 88

Eric 235 55

Jacqueline 204 11

Kevin 186 3

Leah 229 43

Tamara 223 31

Here are the scores for these same 12 students on the 2014 FCAT Grade 7 test (the next year).

Again, each scale score is converted into a percentile based on how the student did compared to the 24,212 MDCPS students in their reference group.

Median Percentile Rank (Current Yr.)2014 FCAT Grade 7


FCAT Percentile

Dontavius 258 88

Alesiah 245 71

Eric 235 55

Leah 229 43

Aaron 224 34

Tamara 223 31

Darryl 221 29

Anna 215 21

Jacqueline 204 11

Christopher 201 9

Carlos 197 7

Kevin 186 3

Median 222 30

With the 2014 Grade 7 scores, we sorted the list and found the median. The median FCAT scale score is 222, which corresponds to a median percentile of 30.

This means that half of the students in this group scored above the 30th percentile and half scored below on this test.

Change in Median Percentiles2013 FCAT Grade 6


FCAT Percentile

Aaron 211 24

Alesiah 232 60

Anna 211 24

Carlos 177 3

Christopher 204 16

Darryl 217 33

Dontavius 245 82

Eric 226 48

Jacqueline 196 10

Kevin 167 0

Leah 220 38

Tamara 191 8

Median 22 24

2014 FCAT Grade 7

FCAT Scale Score

FCAT Percentile

224 34

245 71

215 21

197 7

201 9

221 29

258 88

235 55

204 11

186 3

229 43

223 31

30

Now we can compare the median percentile from the previous year (grade 6) to the current year (grade 7).

In this example, the median percentile increased from 24 to 30.

Bubble Chart

(24, 30)

We can plot the point on a graph, where the horizontal axis is the median percentile rank of the previous year and the vertical axis is the median percentile rank of the current year.

Bubble Chart

* Each bubble represents a school or grade.

* Bubble size is proportional to the number of students.

The diagonal line represents performance in which the median score for the current year was the same as the median score for the previous year.

Bubbles centered above the line represent groups of students whose median percentile rank INCREASED from the previous year.

Bubbles centered below the line represent groups of students whose median percentile rank DECREASED from the previous year.

Student Growth Percentiles

We discussed the performance of our middle student, but how did each student do when compared to other students with similar scores?

To answer this question, we calculate a Student Growth Percentile for each student.

Let’s use Aaron as an example...

Student Growth Percentiles (SGP)2013 FCAT Grade 6


FCAT Percentile

Aaron 211 24

2014 FCAT Grade 7

FCAT Scale Score

FCAT Percentile

224 34

Aaron received a scale score of 211 on the 2013 FCAT Grade 6 test. The scale score is at the 24th percentile, which means that Aaron scored better than 24% of all students who took the test (and worse than 76%).

In 2014, Aaron took the FCAT Grade 7 test and received a scale score of 224. This scale score is at the 34th percentile, which means that he did better than 34% of all students who took this test.

His percentile rank in 2014 improved, but it is still on the lower end of achievement. So how did Aaron do compared to where he started? To calculate this, we look at his “academic peers,” which consist of all other students in the district with a similar score the prior year.


200

283

167

175

225

275

250

24,212 total students

211

Out of the 24,212 students that took the 2013 FCAT Reading Grade 6 test, 400 students scored a 211 and Aaron is one of them. These students constitute Aaron’s “academic peers.”

2013 FCAT Reading Grade 6

200

289

171175

225

275

250

400 total students in Aaron’s “Academic Peer” group


On the 2014 FCAT Reading Grade 7 test, Aaron scored a 224. Other students have different scores - 171, 193, 209, 238, 260, etc. Compared to all students in his academic peer group, Aaron’s score of 224 was better than 64% of them.

So Aaron’s Student Growth Percentile is 64.

Aaron

224


We calculate a Student Growth Percentile for EACH student.

Here are the SGPs for Dontavius and Kevin...


200

283

167

175

225

275

250


245

Out of the 24,212 students that took the 2013 FCAT Reading Grade 6 test, 325 students scored a 245 and Dontavius is one of them. These students constitute Dontavius’ “academic peers.”


200

289

171175

225

275

250

325 total students in Dontavius’ “Academic Peer” group


On the 2014 FCAT Reading Grade 7 test, Dontavius scored a 258. Other students have different scores - 201, 216, 244, 280, etc. Compared to all students in his academic peer group, Dontavius’ score of 258 was better than 73% of them.

So Dontavius’ Student Growth Percentile is 73.

258

Dontavius


200

283

167

175

225

275

250


245

Out of the 24,212 students that took the 2013 FCAT Reading Grade 6 test, 480 students scored a 167 and Kevin is one of them. These students constitute Kevin’s “academic peers.”


200

289

171175

225

275

250

480 total students in Kevin’s “Academic Peer” group


On the 2014 FCAT Reading Grade 7 test, Kevin scored a 186. Other students have different scores - 171, 191, 199, 205, 226, etc. Compared to all students in his academic peer group, Kevin’s score of 186 was better than 38% of them.

So Kevin’s Student Growth Percentile is 38.

186

Kevin

Student Growth Percentiles (SGP)

Student

Aaron

Alesiah

Anna

Carlos

Christopher

Darryl

Dontavius

Eric

Jacqueline

Kevin

Leah

Tamara

Student Growth Percentile

64

74

33

60

16

34

73

56

56

38

58

85

Each student has a Student Growth Percentile that is based on how they performed in relation to their own academic peer group.

We just calculated the SGPs for Aaron, Dontavius, and Kevin.

Median SGP

Student Growth Percentile

85

74

73

64

60

58

56

56

38

34

33

16

Student

Tamara

Alesiah

Dontavius

Aaron

Carlos

Leah

Eric

Jacqueline

Kevin

Darryl

Anna

Christopher

For each group (e.g., school or grade), we calculate a “Median Student Growth Percentile” by taking the median of the SGPs.

In this example, the Median Student Growth Percentile is 57. This means that half of the students have a SGP above 57 and half have a SGP below.

Larger median SGPs indicate higher rates of growth across the distribution of scores. Smaller median SGPs indicate lower rates of growth across the distribution.

The district median SGP is 48.

Sample School

All

5

4

Median Percentile Rank

Gr Size 2013 2014

4 61 38 46

5 46 36 27

All 107 38 38

For this elementary school, the median percentile rank for the 61 4th grade students increased 8 percentile points, which is why the bubble is positioned above the line. The median percentile rank for the 5th grade students decreased, which is why the 5th grade bubble is below the line.

Combining both grades, the median percentile rank for all 107 students was unchanged.

Clearly, 5th grade is the relative priority.

Sample School

Median Percentile Rank

Gr Size 2013 2014 Median SGP

4 61 38 46 53

5 46 36 27 30

All 107 38 38 41

Keeping the bubbles positioned as they are, we can add the “Median Student Growth Percentile” metric to the graph as the bubble color.

Remember, this is a DIFFERENT metric than the median percentile rank metric. For 4th grade, the Median SGP of 53 indicates that half of these 61 students have a SGP below 53 and half have an SGP above 53. For 5th grade, the Median SGP of 30 indicates that half of these 46 students have a SGP below 30 and half have an SGP above 30.

5th Grade (All Schools)

Reading Math

Quartiles2013 FCAT Grade 6


FCAT Percentile

Dontavius 245 82

Alesiah 232 60

Eric 226 48

Leah 220 38

Darryl 217 33

Aaron 211 24

Anna 211 24

Christopher 204 16

Jacqueline 196 10

Tamara 191 8

Carlos 177 3

Kevin 167 0

2014 FCAT Grade 7


FCAT Percentile

Dontavius 258 88

Alesiah 245 71

Eric 235 55

Leah 229 43

Aaron 224 34

Tamara 223 31

Darryl 221 29

Anna 215 21

Jacqueline 204 11

Christopher 201 9

Carlos 197 7

Kevin 186 3

Q3 = 43

Q2 = 24

Q1 = 9

Q3 = 49

Q2 = 30

Q1 = 10

Quartiles

Quartiles

On the 2013 FCAT Grade 6 test... On the 2014 FCAT Grade 7 test...

...25% of students had a percentile rank between 43 and 100.



25% of students had a percentile rank between 0 and 9.





Review of Metrics

Proficiency● Criterion-referenced metric

Growth● Norm-referenced metrics● Students in a particular group are compared against all

other students in that group (e.g., all other 24,000 MDCPS students that have a 5th grade score AND a 4th grade score for the previous year)

District-Wide (for each grouping)

MDCPS

Because the median percentile rank is a norm-referenced metric, for each group (e.g. 4th grade to 5th grade students), the median percentile rank for all MDCPS students in that group is approximately 50 for the prior year and 50 for the current year. So if we plotted a bubble for the district, it would be centered around (50, 50). And the bubble would be huge because it represents 24,000 students.

Similarly, the median Student Growth Percentile for the district is 48. This means that about half of the students in a group have a SGP above 48 and half have a SGP below 48. For this reason, the district’s bubble is white (not a shade of green or red).

Again, norm-referenced metrics compare a large group of students to each other.

District-Wide (for each grouping)On the 2013 test...

And on the 2014 test...

...25% of students had a percentile rank between 75 and 100. This is the 4th quartile.

...25% of students had a percentile rank between 50 and 75. This is the 3rd quartile.

...25% of students had a percentile rank between 25 and 50. This is the 2nd quartile.

25% of students had a percentile rank between 0 and 25. This is the 1st quartile.

Because the district is the reference group and we calculated the percentile ranks compared to our MDCPS students, it makes sense that approx. 25% of the students in the group are in each quartile for both years.

So if there are 24,000 students in a grouping (remember this grouping only contains students with BOTH scores), approximately 6,000 students would be in each quartile:

* 6,000 students would have a percentile rank between 0 and 25




Example Dashboard

Interesting Case

Questions

??

?

??

Extra

● Helpful video from Georgia about SGPs

https://www.youtube.com/watch?v=dyArv7184ZY

Bubble Chart w/ Median SGPs

The median Student Growth Percentiles (SGPs) are displayed as the bubble color. Entities with lower median SGPs are red and entities with higher median SGPs are green.

Quartiles

200

225

260

250



150

175

200

225

269

250

154

175

140

155

175

180

255

140

Documents

Student Achievement Dashboard Training. Objectives ●PWBAT understand the purpose for using the dashboard to analyze student achievement ●PWBAT understand