Lisa Foehr Rhode Island Department of Education Thanos Patelis Center for Assessment

The Design and Implementation of

Educator Evaluation Systems, Variability of

Systems and the Role of a Theory of Action

Rhode IslandLisa FoehrRhode Island Department of Education

Thanos PatelisCenter for Assessment

Presentation at the National Conference on Student AssessmentNew Orleans, June 25, 2014

2Foehr & Patelis – NCSA - 06/25/2014

Overview Overview of the educator evaluation

system Some results Variability that exists Theory of Action Recap


Overview of the System In 2012-2013, full implementation of educator

evaluations. Multiple measures are utilized to develop

final effectiveness ratings for each educator.


Overview of the System The evaluation system is embedded within a year-long process

representing goal setting, observation, data collection, and reflection.


Overview of the System

Professional Practice Rating involves observation using a rubric on eight components adapted from Domains 2 and 3 of the 2013 version of Danielson’s Framework for Teaching.


Practice

Adapted from Domains 2 and 3 of the 2013 version of Charlotte Danielson’s Framework for Teaching


Foundations


Overview of the System

+


Results from 2012-2013 - Teachers

Highly Effective = 47.2% Effective = 48.4% Developing = 3.7% Ineffective = 0.5%


Highly Effective Effective Developing Ineffective0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

Rhode Island Model (n=6,200)

Innovation Model (n=3,576)

Results - Teachers


Results – Building Administrators


Variability Rhode Island has four approved systems for teachers and two

approved systems for building administrators: Teacher Evaluation Systems

1. Coventry Teacher Evaluation System Used by one LEA

2. Learning Community Teacher Evaluation System1. Used by a charter chool

3. RI Innovation Consortium Teacher Evaluation and Support System Used by 7 LEA’s

4. RI Model Teacher Evaluation and Support System Used by 46 LEAs

Building Administrator Evaluation Systems 1. Coventry Building Administrator Evaluation System

Used by one LEA2. RI Model Building Administrator Evaluation System

Used by 53 LEAs A new evaluation system for support professionals is being

developed and gradually implemented in the 2013-14 school year in preparation for full implementation during the 2014-15 school year.


Variability (cont’d) LEAs have the opportunity to establish local policies and

structures that will support the effective implementation of their educator evaluations. Local variability can be found on the professional practice and

foundations components. There are state-wide guidelines for the development that

includes training. But there is local variability with the number (above the minimum of two), the type of assessments used, how targets are set, etc.

Variability permissible if meet RI educator standards. To assist LEAs with this essential work, RIDE is producing

several guidance documents to help LEAs align local policies and long-term plans both with their own visions for great teaching and school leadership, and with Rhode Island’s Educator Evaluation System Standards.


Variability (cont’d) In the 2013-14 school year a Differentiated

Evaluation Process for Teachers was introduced.

The Differentiated Evaluation Process for Teachers provides schools and districts with greater flexibility for classroom observations, evaluation conferences and Professional Growth Plans. The Differentiated Evaluation Process for Teachers

establishes new minimum requirements for classroom observations and evaluation conferences, but schools and districts may choose to conduct more than the minimum of either.


Role of Theory of Action Summarizes the goals and

expectations of the system. Assists in the evolution and

development of the systems. Offers a means to evaluate local

policies and structures. Permits the development of a

validation agenda.


Overarching Goals of RI’s Theory of Action

Obtain measurable improvements in student learning and achievement.

Obtain measurable improvement in overall educator effectiveness.

Establish a fair, reliable, legally defensible educator effectiveness measure to support human capital decisions.

Incrementally increase the number of new educators who reflect the range of knowledge, skills and competencies necessary to be effective.

Improve perceptions of educator quality and accountability across various stakeholder groups.

Increase support provided to educators to improve effectiveness.

Source: Hall, 2013.


Evaluation Criteria & Claims Three criteria:

Professional Practice – a measure of effective instruction and classroom environment

Professional Foundations – a measure of instructional planning and the contributions teacher make as members of their learning community

Student Learning – a measure of a teachers’ impact on student learning.

For each of the criteria produced statements for the following: Description Data source Scoring methodology Performance categories Rational for inclusion in the system Associated claims

Source: Hall, 2013.


Evaluation Criteria & Claims

Source: Hall, 2013.

Sample for one criterion…

Criterion:• Professional Practice (PP)

Claim:• Teachers who score high on the PP display

the instructional skills and classroom environment expected from teachers.

• Teachers who score low on the PP can benefit fro actionable feedback and targeted professional development.


Recap Educator evaluations systems in Rhode Island (now two years old)

represent an effort to capture multiple measures involving professional practice, professional foundations and student learning.

Variability within Rhode Island is possible, as long as systems conform to the RI Professional Teaching Standards, RI Educational Leadership Standards and the RI Code of Professional Responsibility and as expressed in the Rhode Island Theory of Action.

Ongoing feedback is captured and incorporated into revisions. Results show comparability. Efforts and processes represent major undertaking for the

department. Theory of Action assists in decision making, particularly around

understanding and accommodating variability. Theory of Action permits the development of a validation agenda to

substantiate claims.


ReferencesBell, C. A., Gitomer, D. H., McCaffrey, D.,

Hamre, B., Pianta, R., & Qi, Y. (2012). An argument approach to observation protocol validity. Educational Assessment, 17, 1-26.

Danielson, C. (2013). The Framework for Teaching Evaluation Instrument. Princeton, NJ: The Danielson Group.

Hall, E. (2013). Validation Framework for the Rhode Island Model Evaluation and Support System. Unpublished report produced for the Rhode Island Department of Education.

21

Questions and Comments?

Center for Assessmentwww.nciea.org

Thanos [email protected]

Foehr & Patelis – NCSA - 06/25/2014

Documents

Lisa Foehr Rhode Island Department of Education Thanos Patelis Center for Assessment