Measuring Educator Effectiveness

Tom Corbett, Governor ▪ Carolyn C. Dumaresq, Acting Secretary of Education www.education.state.pa.us

Measuring Educator Effectiveness


September 11, 2013 1



Project Goal

• To develop educator effectiveness models that will reform the way we evaluate school professionals as well as the critical components of training and professional growth.

The term “educator” includes teachers, education specialists, and principals.

2



Educator Effectiveness

3



Background• A multi-phase project that began with $800,000 Gates Foundation

grant to facilitate the development of statewide policy, tools and processes to evaluate teachers and principals in which student achievement is a significant factor affecting performance ratings

• PDE is closely following the work of the Pittsburgh Public Schools, recipients of a $40 million Gates Foundation grant that is more comprehensive in scope but similar in redesigning evaluation policy, tools and processes

• A University of Pittsburgh researcher is conducting a qualitative analysis regarding the effectiveness of principal training and the comprehensiveness, validity, transparency, practicality, and quality of the teacher evaluation system

• A third party researcher is conducting a quantitative analysis evaluating the relationships between professional practices as measured by classroom observation scores and teacher contributions to student achievement 4



2010-2011 Phase I Sites

• Allentown School District• Cornell School District• Mohawk School District• IU 5 – Northwest Tri-

County

5



2011-2012 Phase II Sites

• 119 Local Education Agencies, including School Districts, CTCs, Charter Schools and IUs

• 363 Buildings• 658 Supervisors• 2,348 Teachers in PSSA tested subjects• 2,586 Teachers in non-PSSA tested

subjects6



2012-2013 Phase III Sites • 293 Local Education Agencies, including

School Districts, CTCs, Charter Schools and IUs

• 1,255 Schools• 1,972 Supervisors• 20,360 Teachers in PSSA tested subjects• 3,771 Teachers in non-PSSA tested

subjects

7



The Danielson Framework for Teaching



Danielson Framework for Teaching

• Starting with the 2013-2014 school year, all professional educators holding an Instructional Certificate will be evaluated using the Danielson Framework for Teaching

• Professional development linked to Danielson components is available on SAS

9



Observation/Evidence from the

Danielson Framework• Classroom observations by Principal/supervisor,

including evidence that demonstrates behaviors associated with improving student achievement:– Planning and preparation, including selecting standards-based

lesson goals and designing effective instruction and assessment;

– Classroom environment, including establishing a culture for learning and appropriate classroom management techniques that maximize instructional time;

– Instruction, including the use of research-based strategies which engage students in meaningful learning and utilize assessment results to make decisions abut student needs; and

– Professional responsibilities, including using systems for managing student data and communicating with student families 10



Danielson Framework, Domain 3: Instruction Component 1. Failing 2. Needs

Improvement 3. Proficient 4. Distinguished

3a: Communicating with students

Expectations for learning, directions and procedures, and explanations of content are unclear or confusing to students. Teacher’s use of language contains errors or is inappropriate to students’ cultures or levels of development

Expectations for learning, directions and procedures, and explanations of content are clarified after initial confusion; teacher’s use of language is correct but may not be completely appropriate to students’ cultures or levels of development.

Expectations for learning, directions and procedures, and explanations of content are clear to students. Communications are appropriate to students’ cultures and levels of development.

Expectations for learning, directions and procedures, and explanations of content are clear to students. Teacher’s oral and written communication is clear and expressive, appropriate to students’ cultures and levels of development, and anticipates possible student misconceptions.

3b: Using questioning and discussion techniques

Teacher’s questions are low-level or inappropriate, eliciting limited student participation, and elicit recitation rather than discussion.

Some of the teacher’s questions elicit a thoughtful response, but most are low-level, posed in rapid succession. Teacher’s attempts to engage all students in the discussions are only partially successful.

Most of the teacher’s questions elicit a thoughtful response, and the teacher allows sufficient time for students to answer. The students are engaged and participate in the discussion, with the teacher stepping aside when appropriate.

Questions reflect high expectations and are culturally and developmentally appropriate. Students formulate many of the high-level questions and ensure that all voices are heard.

11



Multiple Measures of Educator Effectiveness



Building Level Data, 15%

Teacher Specific Data, 15%

Elective Data, 20%

Observation/ Practice, 50%

Teacher Observation & PracticeEffective 2013-2014 SYDanielson Framework DomainsPlanning and PreparationClassroom EnvironmentInstructionProfessional Responsibilities

Building Level Data/School Performance Pro-fileEffective 2013-2014 SYIndicators of Academic AchievementIndicators of Closing the Achievement Gap, All Stu-

dentsIndicators of Closing the Achievement Gap, Histori-

cally Underperforming StudentsAcademic Growth PVAASOther Academic IndicatorsCredit for Advanced Achievement

Teacher Specific DataPVAAS / Growth 3 Year Rolling Average2013-2014 SY2014-2015 SY2015-2016 SYOther data as provided in Act 82

Elective Data/SLOsOptional 2013-2014 SYEffective 2014-2015 SYDistrict Designed Measures and Examina-tionsNationally Recognized Standardized TestsIndustry Certification ExaminationsStudent Projects Pursuant to Local Re-quirementsStudent Portfolios Pursuant to Local Re-quirements

Teacher Effectiveness System in Act 82 of 2012

13



Building Level Data, 15%

Observation/ Practice, 50%

Teacher Observation & Practice Effective 2013-2014Danielson Framework DomainsPlanning and PreparationClassroom EnvironmentInstructionProfessional Responsibilities

Building Level Data/School Performance ProfileEffective 2013-2014 SYIndicators of Academic AchievementIndicators of Closing the Achievement Gap, All StudentsIndicators of Closing the Achievement Gap, His-

torically Underperforming StudentsAcademic Growth PVAASOther Academic IndicatorsCredit for Advanced Achievement

Elective Data/SLOsOptional 2013-2014 SYEffective 2014-2015 SYDistrict Designed Measures and Examina-tionsNationally Recognized Standardized TestsIndustry Certification ExaminationsStudent Projects Pursuant to Local Re-quirementsStudent Portfolios Pursuant to Local Re-quirements

Elective Data, 35%

14



15

CTCsPart-time CTC's FFT Building Score Teacher Specific-

PVAAS Elective-SLOs2013-2014 100% 2014-2015 65% 35%2015-2016 65% 35%2015-2016 65% 15%* 20%

Full-time CTCs FFT Building Score Teacher Specific-

PVAAS Elective-SLOs2013-2014 85% 15% 2014-2015 50% 15% 35%2015-2016 50% 15% 35%2015-2016 50% 15% 15%* 20%

Career and Technology Centers



The Elective Portion of Teacher Effectiveness

Measuring Student Performance for Teachers in Tested and Non‐Tested Grades and Subjects

16



Student Learning Objectives (as demonstrated through:)

17

• District Designed Measures and Examinations

• Nationally Recognized Standardized Tests• Industry Certification Examinations• Individual Teacher Developed Examinations• Student Projects Pursuant to Local

Requirements• Student Portfolios Pursuant to Local

Requirements



• Focuses on Student Learning Objectives (SLOs) that are aligned to PA Standards

• Measures student performance through student growth or mastery

• Guides the data collection process • Includes a teacher effectiveness measure

18

PA’s SLO Template



Non-Teaching Professionals Educational Specialist



Who are Educational Specialists?

• Educational Specialists are defined in Pennsylvania School Code with the scope of their certificates and assignments described in Certification and Staffing Polices and Guidelines (CSPGs).

• Currently CSPG 75 through 81 list the following specialist certifications: – Dental Hygienist – Elementary School Counselor – Home and School Visitor– Instructional Technology Specialist– Secondary School Counselor– School Nurse– School Psychologist

• PDE is in the process of working with stakeholder groups from across the Commonwealth to revise the Danielson Framework for Teaching to reflect the specific roles and functions of the identified specialist groups 20



Specialists-What about Other Licensed Staff Hired under Teacher Contracts?

• Given that many LEAs hire licensed professionals under teacher contracts who are not certificated as specialists under Pennsylvania School Code, PDE has made a decision to develop revised Danielson Framework for Teaching rubrics for the following roles:– Occupational Therapist– Physical Therapist– Social Workers– Behavior Specialists

• PDE is in the process of working with stakeholder groups from across the Commonwealth to revise the Danielson Framework for Teaching to reflect the specific roles and functions of the identified licensed professionals.

21



Observa-tion/ Prac-tice 80%

Student Perfor-mance 20%

Observation and PracticeDanielson Framework Domains1. Planning and Preparation2. Educational Environment3. Delivery of Service 4. Professional Development

Student Performance/Multiple Measures

Non Teaching Professional Employee Effectiveness System in Act 82 of 2012

Effective 2014-2015 SY

22



Non-Teaching Professionals (ICNDI)

Instructional certifications who do not provide direct instruction to students

To determine whether you are a teaching professional, you must be able to answer yes to the following two questions:

• Are you working under your instructional certification?• Do you provide direct instruction* to students in a particular subject or grade level? *Direct instruction is defined as planning and providing the instruction, and assessing the effectiveness of the instruction.

Under Act 82, if you are working under your instructional certification but do not provide direct instruction to students you are considered a non-teaching professional. Act 82 applies to non-teaching professionals in 2014-2015. PDE will publish a rating tool for Non-Teaching Professionals in the Pennsylvania Bulletin by June 30, 2014.

23



Non-Teaching Professionals- Supervisors

Educational Supervisors are defined in Pennsylvania School Code with the scope of their certificates and assignments described in Certification and Staffing Polices and Guidelines

– CSPG 88 - Supervisor of Curriculum and Instruction – CSPG 89 - Supervisor of Pupil Services – CSPG 90 - Supervisor of Single Area – CSPG 91 - Supervisor of Special Education – CSPG 92 - Supervisor of Vocational Education

24

http://www.portal.state.pa.us/portal/http;/www.portal.state.pa.us;80/portal/server.pt/gateway/PTARGS_0_123236_1090624_0_0_18/CSPG%2088%20Supervisor%20Curriculum%20&%20Instruction%206-2005.pdf

http://www.portal.state.pa.us/portal/server.pt?open=18&objID=1246820&mode=2

http://www.portal.state.pa.us/portal/http;/www.portal.state.pa.us;80/portal/server.pt/gateway/PTARGS_0_123236_1090625_0_0_18/CSPG%2090%20Supervisor%20Single%20Area%206-2005.pdf





Principal Effectiveness



Principal EffectivenessDomain 1 - Strategic/Cultural Leadership : The school leader will systematically and collaboratively develop a positive culture to promote continuous student growth and staff development. The leader articulates and models a clear vision of the school’s culture that involves students, families, and staff.

Domain 2- Managerial Leadership: The school leader will ensure that the school has processes and systems in place for budgeting, staffing, problem solving, communicating expectations and scheduling that result in organizing the work routines in the building. The school leader must efficiently, effectively, and safely manage the building to foster staff accountability and student achievement.

26



Principal EffectivenessDomain 3 – Leadership for Learning: The school leader assures school-wide conversations occur regarding standards for curriculum, instruction, assessment, and data on student learning based on research and best practices, and ensures that the ideas developed are integrated into the school’s curriculum and instructional approaches.

Domain 4 – School and Community Leadership: The school leader will promote the success of all students, the positive interactions among building stakeholders, and the professional growth of staff by acting with integrity, fairness and in an ethical manner.

27



Draft –

For Refe

rence

Only

28



Building Level Data,

15%Correlation

Data Based on Teacher-Level

Measures15%

Elective Data20%

Observa-tion/ Ev-idence

50%

Observation/ EvidenceFramework for Leadership Domains1. Strategic/Cultural Leadership2. Systems Leadership3. Leadership for Learning4. Professional and Community

Leadership

Building Level Data/School Performance ProfileIndicators of Academic AchievementIndicators of Closing the Achievement Gap, All StudentsIndicators of Closing the Achievement Gap,

Historically Underperforming StudentsAcademic Growth PVAASOther Academic IndicatorsCredit for Advanced Achievement

Relationship based on Teacher Level Measures

Elective Data/SLOsDistrict Designed Measures and ExaminationsNationally Recognized Standardized TestsIndustry Certification ExaminationsStudent Projects Pursuant to Local RequirementsStudent Portfolios Pursuant to Local Requirements

Principal Effectiveness System in Act 82 of 2012

Effective 2014-2015 SY

47



Additional Items



Alternative Rating System

• Must be approved by PDE• Must meet or exceed the measures of

effectiveness in the PA Educator Effectiveness System

• Must identify the employee as Distinguished, Proficient, Needs Improvement or Failing

31



Differentiated Supervision• Participation in Differentiated Supervision is

available to any tenured professional who has received at least a Proficient rating on the Danielson Framework of Teaching in the previous two years.

• Each tenured professional will be assigned to the Formal Observation Mode for one (1) year during the supervision cycle.

• Tenured professionals newly hired by a district will be eligible to participate in Differentiated Supervision after successfully completing his/her first year in the Formal Observation Mode.

32



Differentiated SupervisionAdditional Guidelines will include: • Principals maintain the right to remove a teacher from the Differentiated

Supervision program and place them into the Formal Observation or Intensive Supervision modes of supervision.

• Informal observations may occur at anytime regardless of the supervision mode a teacher has selected or has been assigned for that school year.

• Differentiated Supervision Modes must align to the Danielson Framework for Teaching and is related to a district or school initiative designed to improve instructional practices.

• Each mode must be approved by the principal.• The professional is required to complete a mid year and an end of the

year self-reflection report regarding their goal setting, planning, procedures, results, and impact on student achievement.

• All professionals enrolled in Differentiated Supervision will also be rated on Domain 4 of the Danielson model: Professional Responsibilities

33



Professional Development Opportunities



Inter-rater ReliabilityTeachscape’s Focus for Principals

• Professional development for those conducting observations• 20 hours of training content

o Self pacedo Total training time can be faster or longer

• 11 online learning modules for observerso Orientation to observer trainingo Minimizing biaso 9 modules on Framework for Teaching

o Overviewo Framework for Teaching Components in Domain 2 and

Domain 3detailed rater training on each component and performance level

• Over 100+ master scored videos 35



Professional Development on SASTeachscape’s Framework for Teachers Effectiveness Series

(FFTES)

• For anyone• Self-Paced and Video-Rich Online Learning Resources• The Framework for Teaching Effectiveness Series includes 2

sets of learning modules:1. Laying the Foundation

Orientation to course materialsUnderstanding the Framework for TeachingApplying the Framework for Teaching2. Framework Components8 individual learning modules which explore the eight observable components in Domains 2 and 3 of Danielson’s Framework for Teaching

36



Additional Professional Development on SAS

• Professional development courses are aligned to the Danielson, 2011 Framework for Teaching

• Stand alone: 20 courses are currently available on SAS

• The following course models are in development:• Prerequisite Model: Courses designed to build on

each other• Deepening Knowledge Model: Pre-requisites not

required; educators select courses based on their knowledge/experience level

37



Questions?



39

Documents

Measuring Educator Effectiveness