Upload
nancy-underwood
View
216
Download
1
Embed Size (px)
Citation preview
2
Bottom Line Up Front
All districts received secure access to their 2013-14 teacher and principal/AP/VP Median Student Growth Percentile (mSGP) data on January 8, 2015.
• Teachers and school leaders have the highest impact on student achievement of any in-school factor.
• NJDOE has worked with NJ educators in taking a long and thoughtful approach to implementing both evaluations and mSGP.
• mSGP data is an important part, but only one part of an educator’s evaluation. These scores will be used to finalize 2013-14 evaluations and to inform educators’ ongoing professional development.
By statute, mSGPs (like all aspects of an individual’s evaluation) are confidential and should not be shared publicly.
3
Context for SGPs in New Jersey
• Student Growth Percentile (SGP) methodology is not new; SGP was adopted in the 2009-2010 school year and first used for school accountability in the 2011-2012 annual performance reports.
• TEACHNJ Act requires use of at least one measure of student growth in evaluations (18A:6-123: “Standardized assessments shall be used as a measure of student progress…”).– AchieveNJ incorporates 2 measures of growth for teachers:
mSGP and SGOs and 3 measures for principals: mSGP, SGO Average, Administrator Goals.
• NJDOE has been building capacity to share mSGP for over 4 years, including distribution of 2011-12 teacher data to pilot districts and 2012-13 teacher data to all districts.
• mSGP data is derived from roster and staff submission data provided by districts through a submission process in place for many years.
4
Timeline of SGP Development in New Jersey
A thoughtful, multi-year approach to ensure data is accurate and usable
2014 20152011 2012 20132010
Federal Mandate for Stimulus Funds: States Must Calculate “Student Growth”; Link Teachers to Students
Student SGPs Provided to All Districts in NJ SMART
NJ Adopts SGP Methodology for Calculating Student Growth
SGP Training Begins for Districts; SGP Video Released
TEACHNJ Act Passed; Growth
Measures Required for Evaluation
District SGP Profile Reports Deployed via NJ SMART
School SGPs Used in School Performance Reports per NJ’s Federal ESEA Waiver
2012-13 Teacher mSGP Reports Provided to All Districts for Learning Purposes and Data Preview
2011-12 Teacher Median SGP (mSGP) Reports Provided to Pilot Districts for Learning Purposes
2013-14 Teacher & Principal mSGP Reports Provided to All Districts for Use in Evaluations; Score Verification Process Announced
Evaluation Pilot Advisory Committee
Provides Feedback on Usefulness of
SGP Data2013-14 mSGP Score Verification & Certification Process Completed by Districts
5
2013-14 mSGP Data
• The 2013-14 mSGP data counts for 30% of qualifying teachers’ and 20 or 30% of qualifying principals’ 2013-14 evaluations.
• Evaluation data of a particular employee shall be confidential in accordance with the TEACHNJ Act and N.J.S.A. 18A:6-120.d and 121.d.
– Educator mSGP data should be handled in the secure manner one would treat, handle, and store any part of a confidential personnel record and should not be released to the public.
• While a dry run for teacher mSGP data was conducted last year to improve roster verification processes, if educators identify a problem with the 2013-14 mSGP score, the Department is offering options for addressing the issue.
6
Student Growth PercentileStudent Growth Objectives
mSGP Qualification and Weighting: Teachers
1. Teach a 4th–8th grade Math or Language Arts Literacy (LAL) class, and
2. Be the teacher of record for at least 60% of the course prior to the NJ ASK assessment, and
3. Have at least 20 students with valid SGP scores who are enrolled in the class for at least 70% of the school year before they take the NJ ASK.
To receive an mSGP score, teachers must:
Teacher Practice
55%15%
30%
2013–14
7
mSGP Qualification and Weighting: Principals
1. Be assigned to schools attended by more than 20 separate students who took the 4th- to 8th-grade Mathematics or Language Arts NJ ASK; and
2. Be assigned to qualifying students reported as having attended the school for one full year prior to the administration of the NJ ASK, according to the data submitted by the district in its State Submission.
To receive an mSGP score, principals/APs/VPs must:
Principal PracticeEvaluation Leadership
SGO AverageAdministrator GoalsSchool SGP
30%
20%10%
20%
20%
Single-Grade SGP Principals/APs/VPs(1 SGP grade in school)
30%
20%
10%
10%
30%
Multi-Grade SGP Principals/APs/VPs
(2 or more SGP grades in school)
2013-14
8
Individual mSGP Report
Individual mSGP reports provide:
• Background information about SGPs and mSGPs
• The educator’s mSGP score for 2013-14; Chart depicting an
example of a teacher’s score is displayed below
• Suggestions for how to interpret and utilize the data, as
well as access to additional resources
TEACHER MSGP
# STUDENTS ASSIGNED
MSGP SCORE(1.0-4.0)
Language Arts Literacy 46 22
Mathematics 68 22Overall 52 22 3.0
9
Addressing Data Issues
• The accuracy of mSGP Data is dependent on course roster and staff submission data by the school district.
• Districts may expect to receive mSGP scores for educators who ended up not qualifying – and vice versa.
• Incorrect district data submissions may have led to errors in mSGP scores for individual educators.
• Guided by feedback from last year’s practice release, the Department has created mSGP score verification and correction procedures. • Depending on the issue and the district’s ability to document evidence
for a given error, the mSGP score may be recalculated or removed from an educator’s summative rating.
• Detailed guidance has been posted here.
10
Potential Data Issues
• Educators expecting but not receiving the mSGP score– Option A: For a teacher with 2 SGOs for 2013-14 OR for a
principal/AP/VP, recalculate summative rating using other component scores
– Option B: For a teacher with 1 SGO, eliminate summative rating
• Educators seeking to correct the mSGP score– Option A: Provide documentation of errors, correct mSGP
score, and recalculate summative rating– Option B: Remove mSGP score for a teacher with 2 SGOs OR
for a principal/AP/VP, and recalculate summative rating– Option C: Remove mSGP score for a teacher with 1 SGO,
eliminate summative rating.Note: State law requires multiple measures of student achievement; thus, if district data errors result in the need to negate the mSGP score for a teacher – and that teacher did not set 2 SGOs – then the teacher cannot receive a final summative rating for 2013-14. This scenario does not apply to principals, whose evaluations already include both Administrator Goals and the SGO Average in addition to potential mSGP scores.
11
Evaluation Score Certification Tool
• Districts will have an opportunity to certify that all 2013-14 evaluation data is correct or to make changes where necessary.
– The Department will release the 2013-14 Evaluation Score Certification Tool, a new electronic application for districts to use in certifying final 2013-14 summative ratings for all educators, in late January.
– This interface will allow districts to review data, correct any errors that occurred in the original NJ SMART submission, and certify the accuracy of each staff member’s final score.
– Districts will have approximately one month to complete this process after release of the tool.
12
Next Steps
• Districts: download secure mSGP reports and provide individual educator reports confidentially, ideally through one-on-one conference– Department resources:
http://www.nj.gov/education/AchieveNJ/teacher/percentile.shtml
• Educators: review data, share questions/feedback with supervisor
• Districts Identifying Data Issues: consult Department guidance: http://www.nj.gov/education/AchieveNJ/resources/1314mSGPdataprocedures.pdf
• Department: release 2013-14 Evaluation Score Certification Tool in coming weeks
• Districts: certify all 2013-14 evaluation scores within 4 weeks of Tool access
13
Implementation Timeline: Common Core, State Assessments, and Student Growth Data
CCSS curriculu
m alignment begins
(K-2 math)
All curriculum aligned to
CCSS
NJ ASK aligned to
CCSS in ELA (3-8) and Math (3-5)
2011-12 median Student Growth
Percentiles (mSGPs)
released to pilot
districts
NJ ASK completely aligned to
CCSS
PARCC piloted in
classrooms across 1,276
schools
2012-13 mSGPs
released to all districts as practice
exercise
Full PARCC Implemen
tation
2013-14 mSGP data
released
‘10-’11
‘12-’13
‘13-’14
‘14-’15
‘11-’12
Setting the Context
14
SGP and the PARCC Transition
Multi-Year Preparation•CCSS adopted 4 years ago •NJ ASK incrementally aligned to CCSS in content and rigor•2014 NJ ASK scores show little change compared to prior years despite increased rigor•81% of schools who will use PARCC engaged in pilot testing technology
Growth, Not Proficiency
•To calculate SGP, student growth is compared with growth of academic peers taking the same assessments statewide•SGP does not depend on tests having a consistent scale and is not a criterion-reference metric
Multiple Measures,
Lower Stakes•mSGP is one of multiple measures for the educators in tested grades and subjects; others include educator practice, SGOs, and additional goals and evaluation leadership for principals•mSGP weight reduced to 10% for all educators for 2014-15 to recognize adjustment to new assessment
15
Appendix: Calculating SGP and mSGP
• The following slides depict and explain how SGP and mSGP scores are calculated.
• For a video overview of SGP calculation, please access http://survey.pcgus.com/njgrowth/player.html.
16
Calculating Student Growth Percentiles
• Student Growth Percentiles (SGPs) measure how much a student has learned from one year to the next compared to students with a similar NJ ASK performance history from across the state (“academic peers”).
• SGP scores range from 1 to 99 All students can show growth.
17
Why Student Growth?
A student’s NJ ASK score does not tell the whole story.
Partially Proficient
Gr. 3 Gr. 4 Gr. 5
100
200
250
150160 165
NJ ASK Scale Score by Grade
NJ A
SK
Sca
le
Proficient
Advanced Proficient
230205
220
Albert
Maria
Under our current system, schools and parents might only notice that Maria is “Proficient” and that Albert is “Partially Proficient.”
18
SGP Considers Growth, Not Proficiency
Albert’s Prior
Scores
Academic Peers’ Prior
Scores
Albert has taken the 5th-grade NJ ASK. How does his score compare to those of
his academic peers?
3rd Gr. 150
4th Gr. 160
5th Gr. 165
3rd Gr. ≈150
4th Gr. ≈160
5th Gr. ???
19
Understanding Academic Peers
• Multiple consecutive years of data are required to calculate SGP.– Since 3rd grade is the first year of the NJ ASK, 4th grade is the
first year for which we can calculate SGP.
• The more years of data available, the more precise an academic peer group is.– By 8th grade, we have up to 6 years of scores for the calculation
(as long as students have taken the same assessments).– If a student does not have an NJ ASK score for a given year, an
SGP cannot be calculated until he/she has 2 consecutive years of scores.
• The peer group includes all students in a grade cohort who have taken the exact same assessments, i.e., all 7th graders who took the 3rd, 4th, 5th and 6th-grade Math NJ ASK and the 7th-grade Math NJ ASK
• Academic peer groups are recalculated every year, considering all assessment data for a given cohort of students.
20
Determining an SGPN
J A
SK
Sco
re
Partially Proficient100
200
150160 165
Proficient
Advanced Proficient
250
Gr.3 Gr.5 Gr.4
Partially Proficient100
200
150160
200Proficient
Advanced Proficient
110
250
Gr.3 Gr.5 Gr.4
Albert’s 5th-Grade NJ ASK ScoreAlbert’s Academic Peers’ NJ ASK Scores
Albert scored 165. His academic peers scored between 110 and 200. How did Albert do in
comparison to them?
29%
70%
165
21
Determining an SGP
A comparison to his academic peers allows us to see that Albert actually outperformed 70% of students who, up
until this year, performed in a similar manner to Albert. Albert’s SGP score would be 70.
1%
99%
70%
Albert’s Score
Academic Peers’ Scores
5th Gr. 165
5th Gr.
110 - 200
SGP
22
Determining a Teacher’s mSGP Rating
22
Albert’s teacher would receive a
median SGP score of 51.
Albert’s SGP is
arranged along with the SGPs of
all his teacher’s students
from low to high.
Student SGP ScoreHugh 12Eve 16Clarence 22Clayton 24Earnestine 25Helen 31Clinton 35Tim 39Jennifer 44Jaquelyn 46Lance 51Roxie 53Laura 57Julio 61Selena 65Ashlee 66Albert 70Mathew 72Marcus 85Charles 89Milton 97
23
mSGP Technical Rules
In future years, teachers should know that:
• If two or three years of data are available, the Department will use the best available score — either the teacher’s median score of their current roster or the median of all student scores over the years available.
• The mSGP score, along with the observation and SGO scores, will be placed on a 1.0 - 4.0 scale and weighted appropriately to determine a summative evaluation rating.
Student SGP Score
Hugh 12Eve 16Clarence 22Clayton 24Earnestine 25Helen 31Clinton 35Tim 39Jennifer 44Jaquelyn 46Lance 51Roxie 53Laura 57Julio 61Selena 65Ashlee 66Albert 70Mathew 72Marcus 85Charles 89Milton 97
Mel 56Laura 57Regina 58Marissa 60Julio 61Faye 63Selena 65Ashlee 66Jackie 67Courtney 68Albert 70Matthew 72Laura 77Jack 78Jared 80Rick 84Marcus 85Charles 89Michelle 92Molly 95Milton 97
Student SGP Score
Hugh 12Eve 16John 16Charles 20Annie 20Clarence 22Clayton 24Earnestine 25Jake 27Helen 31Rachel 33Clinton 35Tim 39George 41Amber 42Jennifer 44Jaquelyn 46Bobby 50Lance 51Roxie 53Mike 55
Median over 2 years = 56Median this Year = 51
24
mSGP Conversion from 1 - 99 to 1.0 - 4.0
mSGP Score
Evaluation Rating
1 – 20 121 1.122 1.223 1.324 1.425 1.526 1.627 1.728 1.829 1.930 231 2.132 2.233 2.334 2.4
mSGP Score
Evaluation Rating
65 3.566 3.567 3.568 3.669 3.670 3.671 3.772 3.773 3.774 3.875 3.876 3.877 3.978 3.979 3.9
80 - 99 4
mSGP Score
Evaluation Rating
35 2.536 2.537 2.638 2.639 2.740 2.741 2.842 2.843 2.944 2.945 346 347 348 349 3
mSGP Score
Evaluation Rating
50 351 352 353 354 355 356 3.157 3.158 3.259 3.260 3.361 3.362 3.463 3.464 3.4
Based on the mSGP score, Albert’s teacher receives a rating of 3 for this component, which is then combined with other
evaluation components to get a summative rating.
25
SGP Conversion Chart Explained
mSGP ScoreEvaluation
Rating35 2.536 2.537 2.638 2.639 2.740 2.741 2.842 2.843 2.944 2.945 3.046 3.047 3.048 3.049 3.050 3.051 3.052 3.053 3.054 3.055 3.056 3.157 3.158 3.259 3.260 3.361 3.362 3.463 3.464 3.4
Why are all the values between 45 and 55 set to the same score (3.0)? • The Department believes that
educators in the middle of the mSGP distribution are driving significant academic growth in their students.
• Educators whose students achieve scores in this range should be recognized by receiving a rating on par with their impact.
26
SGP Conversion Chart Explained
mSGP Score
Evaluation Rating
1 – 20 1.0
21 1.122 1.223 1.324 1.425 1.526 1.627 1.728 1.829 1.930 2.031 2.132 2.233 2.334 2.4
Why are the values at the extreme ends of the distribution, 1-20 = 1 in this case (and 80-99 = 4), set to the same score? • When more than half of a
teacher's students are in the top 20 percentile points on the SGP scale it is an indication of very high growth.
• When more than half of a teacher's students are in the bottom percentile points this is an indicator of low growth to be considered with other evidence.
mSGP Score
Evaluation Rating
65 3.566 3.567 3.568 3.669 3.670 3.671 3.772 3.773 3.774 3.875 3.876 3.877 3.978 3.979 3.9
80 - 99 4.0
27
SGP Conversion Chart Explained
mSGP Score
Evaluation Rating
65 3.566 3.567 3.568 3.669 3.670 3.671 3.772 3.773 3.774 3.875 3.876 3.877 3.978 3.979 3.9
80 - 99 4.0
Why Decimals? Why Tenths? • The use of decimals instead of whole
numbers enables the scale to increase/decrease gradually, improving the statistical efficiency of the conversion.
• This prevents large rating differences that may not accurately reflect significant differences in student learning.