Second Annual Race to the Top Professional Development ......Nov 02, 2011 · RttT PD Evaluation: Year 2, Part II—Local Outcomes Baseline March 2013 Consortium for Educational Research

Second Annual Race to the Top

Professional Development Evaluation

Report Part II—Local Outcomes Baseline Study

Authors:

Jenifer Corn, Avril Smart, Lauren Bryant, Jennifer

Maxfield, Meredith Walton, Ruchi Patel, Sara Pilzer Weiss,

Brandy Parker, Shaun Kellogg Friday Institute for

Educational Innovation

Contributors:

Nate Barrett, Julie Marks, Tina Patterson, Carolina Institute

for Public Policy

March 2013

Consortium for Educational Research and Evaluation–North Carolina

Acknowledgements

We wish to thank Lynne Johnson, Maria Petrie-Martin, Neill Kimrey, Yvette Stewart, Eliz

Colbert, Robert Sox, and Stephen Jackson of the North Carolina Department of Public

Instruction for their time and assistance with data collection. We would also like to thank NCDPI

Division staff who led the professional development sessions we observed, as well as the

teachers and staff who participated in our site visits and focus group. We are grateful to Trip

Stallings and Glenn Kleiman from North Carolina State University’s Friday Institute for

Educational Innovation for valuable feedback on drafts of this report, and to Larissa Leroux for

editorial assistance.

RttT PD Evaluation: Year 2, Part II—Local Outcomes Baseline

March 2013

Consortium for Educational Research and Evaluation–North Carolina

Table of Contents

Executive Summary ........................................................................................................................ 3

Introduction ..................................................................................................................................... 7

Purpose of the RttT Evaluation and of this Report ...................................................................... 8

Overview of RttT Professional Development Activities and Supports Addressed in this Report .... 9

2011-12 NCDPI-RESA Sessions ........................................................................................... 10

2012 Summer Institutes .......................................................................................................... 10

Method .......................................................................................................................................... 12

Statewide Data Sources ............................................................................................................. 13

LEA Professional Development Coordinator Survey ............................................................ 13

RttT Omnibus Survey ................................................................................................................ 13

Longitudinal Sample.................................................................................................................. 14

School Sample Selection Process ........................................................................................... 14

Description of the Sample ...................................................................................................... 15

Longitudinal Data Sources and Procedure ................................................................................ 15

CLASS Observation Protocol ................................................................................................ 16

Teacher Professional Development Survey ........................................................................... 17

LEA Professional Development Coordinator Interviews....................................................... 19

Principal Interviews................................................................................................................ 20

Teacher Focus Groups ............................................................................................................ 20

Limitations ................................................................................................................................. 20

Results ........................................................................................................................................... 22

Evaluation Question 3.A.: To what extent did leader participants improve practices for support

of organizational change, including capacity to provide high-quality local professional

development aligned to RttT priorities? .................................................................................... 22

Planning, Accessing, and Implementing Local Professional Development........................... 23

Alignment with RttT Priorities ............................................................................................... 27

Quality of Local Professional Development .......................................................................... 29

Summary of Findings for Evaluation Question 3A ................................................................ 31

Evaluation Question 3.B.: How did school culture/organizational conditions change to support

RttT priorities? ........................................................................................................................... 32

Changes in Conditions related to Transition to the New State Standards .............................. 32

Changes in Conditions related to Data-based Decision Making and Revised Teacher

Evaluation Process ................................................................................................................. 34


March 2013

Consortium for Educational Research and Evaluation–North Carolina 2

Evaluation Question 3.C.: To what extent did teachers improve classroom practice? ............. 36

Next Steps for the Professional Development Evaluation ............................................................ 42

Continued Evaluation of the RttT Annual Professional Development Cycle ........................... 42

Continued Evaluation of the Distinguished Leadership in Practice (DLP) Principal Institutes ...... 43

Continued Collaboration with NCDPI Staff .............................................................................. 43

References ..................................................................................................................................... 44

Appendix A. RttT Professional Development Teacher Survey Items .......................................... 45

Appendix B. LEA Professional Development Coordinator Interview Protocol ........................... 50

Appendix C: Principal Interviews Interview Protocol .................................................................. 52

Appendix D. Teacher Focus Group Protocol ................................................................................ 54


March 2013


SECOND ANNUAL RACE TO THE TOP PROFESSIONAL DEVELOPMENT

EVALUATION REPORT, PART II—LOCAL OUTCOMES BASELINE STUDY

Executive Summary

The North Carolina Race to the Top (RttT) professional development plan is an expansive and

multi-faceted effort to increase student achievement by updating the knowledge and skills of the

state’s entire K-12 public education workforce. This initiative is driven by a host of recent

changes, including: adoption of new Common Core State Standards and North Carolina Essential

Standards; increased use of data to inform classroom and school decisions; rapid changes in the

technologies and digital resources available for teaching and learning; new teacher and

administrator evaluation processes; and an increased emphasis on formative assessment to

inform instructional decisions.

The human resources challenge of the initiative—to provide the state’s 100,000 teachers and

2,400 principals with professional development that will enable them to extend their knowledge,

improve professional practices, and, ultimately, increase student achievement overall and close

achievement gaps among student groups—is formidable. The timeframe (the four-year period of

the grant), diversity of the State (from large metropolitan local education agencies [LEAs] to

small, rural, and resource-limited LEAs, many of which continue to struggle under the weight of

fiscal constraints), and expectations (to create a statewide professional development

infrastructure that can be sustained after RttT funding ends) only increase that challenge. The

RttT professional development evaluation is being conducted in full recognition of these

circumstances, as well as of the deep commitment of the members of the RttT Professional

Development Implementation Team. The intent of the evaluation is to provide data-driven

information that can support reflection about and improvement of this effort.

Four general questions guide the evaluation:

1. State Strategies: To what extent did the state implement and support proposed RttT

professional development efforts?

2. Short-Term Outcomes: What were direct outcomes of State-level RttT professional

development efforts?

3. Intermediate Outcomes: To what extent did RttT professional development efforts

successfully update the NC education workforce?

4. Impacts on Student Performance: To what extent are gains in student performance

outcomes associated with RttT professional development?

The Evaluation Team is providing this second annual assessment of progress in three separate

but related reports. This report—the Local Outcomes Baseline Study— provides a baseline (first-

year) assessment of the State’s progress toward updating the education workforce in North

Carolina (Evaluation Question 3). For this report, the Evaluation Team (a) collected and

analyzed relevant data from all 115 LEAs, and (b) identified a purposeful sample of 27 schools

to examine more deeply the extent to which LEA and school staff increased capacity to provide

high-quality professional development. The other two reports address (1) the state’s progress in


March 2013


delivering face-to-face professional development statewide, and (2) implementation and impact

of the State’s Online Professional Development.

Summary of Major Baseline Findings

To address Evaluation Question 3, the Team examined the extent to which participation in the

state’s professional development efforts impacted (a) local capacity to provide high-quality

professional development, (b) shifts in local organizational conditions to support RttT priorities,

and (c) changes to instructional practice. Results from these baseline analyses will be used in

future reports to gauge progress toward updating the education workforce statewide.


of organizational change, including capacity to provide high-quality local PD aligned to RttT

priorities?

Planning and Accessing Professional Development.

For most schools, focus group and interview data reveal that school-level professional

development was informed by what was being offered at the state level and typically was

designed around resources available at the LEA level. In line with the state-endorsed “train the

trainer” model, LEA-level Professional Development Coordinators indicated that they were

primarily responsible for identifying and disseminating the most relevant professional

development resources and making them available to principals and teachers in their LEAs. Most

LEA Coordinators mentioned using tools like wikis, website postings, and staff development

newsletters and calendars to condense professional development resources and make them more

easily accessible to teachers and principals. However, despite the multiple methods of

dissemination described by LEA leaders, data indicate that some principals and teachers still

remained uninformed about NCDPI-support professional development opportunities.

Implementing Professional Development.

Most Professional Development Coordinators agreed that their LEA leadership had the capacity

to implement and plan professional development, but fewer believed that their LEA leaders used

data to inform their professional development decisions. Some LEAs called on teams of school-

level leaders to develop implementation strategies. In most schools, professional development

training was implemented through Professional Learning Communities (PLCs), and in some

LEAs, training opportunities were extended beyond the school year.

Alignment with RttT Priorities.

LEA Professional Development Coordinators statewide and teachers in the representative school

sample held different perceptions about whether professional development helped teachers build

their knowledge and skills related to some RttT priorities, such as understanding revised state

standards (91% of Coordinators but only 64% of teachers agreed that it helped) and deepening

their content knowledge (81% and 58%, respectively). For other priority areas, the perceptions of

both groups were more closely aligned, but were typically low (e.g., only 51% and 52%,

respectively, agreed that teachers built knowledge and skills related to revised state assessments).


March 2013


In addition, while 92% of Coordinators reported that LEA-provided RttT professional

development was aligned with and built upon existing professional development initiatives, only

64% of the coordinators agreed that their LEAs utilized data on staff technology proficiency

when planning RttT professional development. Furthermore, only 60% agreed that their LEAs

have plans for how to use the NCDPI Professional Development Leads in each region for

support in their delivery of professional development. Overall, these differing perceptions

suggest a lack of consistency among LEA PD coordinators in their efforts to align professional

development with RttT priorities at the local level.

Quality of Local Professional Development

As they reflected on the academic year, about 70% of teachers surveyed agreed or strongly

agreed that their local professional development experiences were both sustained and coherent,

though fewer believed that they had time to reflect on and experiment with what they learned—a

sentiment that was echoed in focus group responses. Teachers generally gave lower ratings for

their experiences with LEA-provided professional development than did LEA Professional

Development Coordinators, but interview and focus group data highlighted principals’ and

teachers’ positive experiences with local coaches and specialists, as well as their positive

impressions of the value of PLCs.

Evaluation Question 3.B.: How did school culture/organizational conditions change to

support RttT priorities?

Changes in Conditions related to Transition to the New Standards

Most LEA Professional Development Coordinators agreed that their LEAs have strategies in

place for communicating about availability of, collaborating on, and integrating state resources

into professional development related to revised state standards, but preparation for integrating

other aspects of the curriculum with those standards varies. Results from the RttT Omnibus

Survey showed that nearly all teachers felt strongly about their own preparation for the transition

to Common Core but were less sure about whether school leaders or other teachers in their

school really understood how to implement the new standards effectively.

Changes in Conditions related to Data-based Decision Making and the Revised Teacher

Evaluation Process

Between 2010-11 and 2011-12, there were slight decreases in teacher perceptions of their schools’

use of data for decision-making, as well as in their perceptions of the new teacher evaluation

process, but both changes may be the result of a sizeable increase in response rates. The Evaluation

Team will monitor annual survey results to determine whether these patterns persist.

Evaluation Question 3.C.: To what extent did teachers improve classroom practice?

Data gathered for this section provide baseline information about current learning activities for

comparison to similar data in subsequent years of RttT. Overall, Math teachers most frequently

reported daily instructional time on instructional strategies related to new standards, followed by

English Language Arts (ELA), Science, and Social Studies teachers. Across each content area:


March 2013


ELA: The highest percentage of teachers estimated that students spent time almost daily on listening skills (76%) and on general vocabulary (66%).

Math: The highest percentage of teachers reported that on almost a daily basis, students made sense of problems (80%), persevered in solving problems (72%), and used appropriate tools

strategically (67%).

Science: Teachers reported that their students practiced participating in hands-on activities (32%), completed activities with a real-world context (24%), and used tools to gather data

(19%).

Social Studies: Students spent time almost daily on recognizing and appreciating contributions of diverse cultural groups (30%), demonstrating chronological thinking (29%),

and analyzing cause-and-effect relationships (28%).

On average, observed classrooms in the sample schools received midrange ratings across the five

dimensions of the Evaluation Team’s standard classroom observation protocol1 that are most

directly aligned with implementation of the new Common Core and Essential Standards (Regard

for Student Perspectives, Quality of Feedback, Instruction Dialogue, Analysis and Problem

Solving, and Content Understanding). Across content areas, teachers sometimes scaffolded

discussion in the classroom and made attempts to integrate student feedback. ELA classrooms

were scored slightly higher on average than were other core courses in the frequency and quality

of examples of three of the dimensions of effective teaching (Regard for Student Perspectives,

Quality of Feedback, and Instruction Dialogue). Social Studies classrooms received the lowest

average ratings for observed frequency and quality of the other two dimensions (Analysis and

Problem Solving and Content Understanding). Finally, teacher surveys indicated that while some

formative assessment practices were common across classrooms, others were employed

infrequently. These observation data mirror results from teachers’ self-report surveys about

frequency of effective, standards-based teaching practices.

Summary of Results

Overall, the baseline results presented in this study indicate that schools and LEAs have taken

some initial steps toward developing a process for professional development that supports RttT

priorities, including transition to the new state standards, implementation of formative and

summative assessments, use of data to support instruction, and effective utilization of the new

North Carolina Educator Evaluation System. These aggregated results provide a cursory baseline

assessment of Year 1 progress toward achieving and sustaining local capacity to provide high-

quality professional development, supporting shifts in local organizational conditions to support

RttT priorities, and encouraging changes in instructional practice.

1 Classroom Assessment Scoring System, or CLASS (http://www.teachstone.org/about-the-class/). In all, there are

12 CLASS domains.

http://www.teachstone.org/about-the-class/


March 2013


Introduction

Providing high-quality, accessible professional development to all teachers and principals is a

key component of the professional development plan funded by North Carolina’s federal Race to

the Top (RttT) grant. The United States Department of Education’s grant application process

recognized the important role that professional development must play in the successful

implementation of the RttT education reforms by requiring states to develop comprehensive

strategies for both the expansion of their professional development offerings and the evaluation

of the effectiveness of that professional development. North Carolina’s RttT-funded professional

development plans are ambitious, with a top-level goal of updating the entire education

workforce to ensure that each of the state’s 100,000 teachers and 2,400 principals has the

knowledge and skills necessary to foster student achievement.

The challenges addressed by the North Carolina Department of Public Instruction’s (NCDPI)

RttT Professional Development Implementation Team are formidable. The Implementation

Team must develop strategies that, in coordination with Local Education Agencies (LEAs),

Regional Education Service Agencies (RESAs), and other organizations, will engage all of the

state’s teachers and administrators in professional development that will extend their knowledge

and guide them in making significant changes in their professional practices in order to increase

overall student achievement and close achievement gaps among student groups. The plan

requires differentiation of professional development to meet the needs of educators with a wide

range of backgrounds, preparation, experience, and expertise, ranging from the over 10,000

experienced National Board Certified teachers in North Carolina to the over 20,000 teachers who

are in their first three years of teaching and who in many cases entered teaching through an

alternative licensure pathway. The plan requires reaching teachers from kindergarten through

high school, across all disciplinary areas. The plan also requires professional development for

principals, assistant principals, curriculum specialists, and all of the other administrators

involved in guiding and supporting teachers through transitions to new standards, student

assessments, data systems, technologies, teacher evaluations, and overall expectations for both

themselves and their students. All of this is to be accomplished within the four-year period of the

grant, across a large and diverse state with many small, rural, and resource-limited LEAs. In

addition, the plan is expected to result in a statewide professional development infrastructure that

can be sustained after RttT funding ends. While the grant provides significant funding, the

professional development initiative must proceed during a time of severe economic constraints

that limit additional state and local resources that can be brought to bear on the challenges; this

limitation is far greater than was anticipated when the RttT proposal was developed.

The RttT professional development evaluation is being conducted with full recognition of the

enormous challenges being addressed by the Professional Development Implementation Team

and the deep commitment of the members of the team to do so. The Evaluation Team further

recognizes that the RttT professional development plan is breaking new ground: There are no

established, proven models for creating a professional development system of this scale to

support so many educators through so many changes in so short a period of time. The Evaluation

Team’s intent for this report is to provide data-driven information that can support reflection

about and improvement of the RttT professional development effort.


March 2013


Purpose of the RttT Evaluation and of this Report

The roles of the RttT Evaluation Team are to (1) document the activities of the RttT initiatives;

(2) provide timely, formative data, analyses, and recommendations to help the initiative teams

improve their ongoing work; and (3) provide summative evaluation results in Year 4 of the

project to determine whether the RttT initiatives met their goals and to inform future policy and

program decisions to sustain, modify, or discontinue initiatives after the grant-funded period.

The RttT Evaluation Team is documenting the professional development initiative activities and

collecting data about participation in, satisfaction with, and the impact of locally- and state-

supported professional development activities. In addition, the team is collecting information

through LEA and school field visits and teacher and administrator surveys to inform summative

evaluation analyses that examine the extent to which participation in RttT professional

development activities has increased the capacity of the education workforce to deliver effective

instruction. Ultimately, the evaluation will use state assessment data to gauge changes in student

performance. The full plan is described in greater detail in Appendix A.

Four general questions guide the overall evaluation effort, with a number of more specific

questions embedded in each one. These overall questions are organized to reflect the general

sequence in which they can be addressed over the four years of RttT:

1. State Strategies: To what extent did the state implement and support proposed RttT

professional development efforts?

2. Short-Term Outcomes: What were direct outcomes of state-level RttT professional

development efforts?

3. Intermediate Outcomes: To what extent did RttT professional development efforts

successfully update the North Carolina education workforce?

4. Impacts on Student Performance: To what extent are gains in student performance

outcomes associated with RttT professional development?

The Year 2 professional development evaluation serves three primary purposes, each of which is

addressed in a separate report, as described below. Each of the three reports is organized around

specific evaluation questions; this document is Report II of the Year 2 RttT evaluation.

Report I: RttT Statewide Face-to-Face Professional Development Formative Evaluation—Report I documents the current status and direct outcomes of the state’s RttT

face-to-face professional development efforts in the NCDPI Annual Professional

Development Cycle described below, addressing questions under Overall Evaluation

Question 1 (State Strategies) and questions under Overall Evaluation Question 2 (Short-Term

Outcomes). It is intended to contribute to the RttT Professional Development Implementation

Team’s ongoing efforts to provide the most supportive and useful professional development

possible for teachers and administrators.

Report II: RttT Professional Development Local Outcomes Baseline Study—Report II (this report) provides detailed information concerning near- and longer-term impacts of


March 2013


statewide face-to-face RttT professional development efforts at the local level. The

Evaluation Team collected and analyzed relevant data from all 115 LEAs and identified a

purposeful sample of 27 schools to examine more deeply the extent to which LEA and

school staff increased capacity to provide high-quality professional development and focused

on shifts in local organizational conditions to support RttT priorities, and changes to

instructional practice. That is, this section provides baseline data for Evaluation Question 3

(Intermediate Outcomes) and related sub-questions:

A. To what extent did leader participants improve practices for support of organizational change, including capacity to provide high-quality local PD aligned to RttT priorities?

B. How did school culture/organizational conditions change to support RttT priorities?

C. To what extent did teachers improve classroom practice?

Report III: RttT Online Professional Development Evaluation Report—Report III also addresses Evaluation Questions 1, 2, and 3 (State Strategies, Short-Term Outcomes, and

Intermediate Outcomes), but with a focus on all RttT-supported online professional

development. It provides detailed information about the extent to which the online

professional development components of the RttT proposal have been carried out, and the

immediate impact of RttT online professional development efforts on educators.

The next major assessment, scheduled for delivery in Fall 2013, will consider short-term

outcomes in more detail and will begin to address intermediate outcomes, while reports in

following years (after the local professional development programs are in operation) will address

the impact on teachers and, to the extent possible, students. Since the schedule of the Evaluation

Team’s deliverables calls only for annual reports, the team will actively coordinate with NCDPI

to share preliminary survey data reports with appropriate staff, in order to provide timely

information as the Professional Development Implementation Team continues to work with

LEAs and charter schools and develops additional resources.

Overview of RttT Professional Development Activities and Supports Addressed in this Report

As shown in the diagram of the Annual Professional Development Cycle provided by NCDPI

(Figure 1, following page), the overall professional development plan is built around annual

cycles comprised of Summer Institutes, formative support for LEA and charter school

Professional Development Leadership Teams, and additional face-to-face support sessions

provided by the NCDPI RttT Professional Development Implementation Team in collaboration

with RESAs. Through implementation of this Annual Professional Development Cycle, NCDPI

aims to both guide and support the development of local professional development through a

high-quality, systemic, blended approach to effective professional development, defined as job-

embedded, research-driven, data-informed, professional, community-based, and aligned to the

RttT initiatives. A blended approach to professional development is designed to address the

needs of LEAs, charter schools, and individual educators through face‐to‐face sessions and online resources, such as learning modules or webinars.


March 2013


Figure 1. NCDPI Annual Professional Development Cycle

2011-12 NCDPI-RESA Sessions

NCDPI offered over 100 face-to-face NCDPI-RESA partnership sessions across the eight North

Carolina State Board of Education regions from August 2011 to June 2012. Sessions lasted

anywhere from three hours to two days and focused on Content Support (Common Core and

Essential Standards), Fidelity Support, Teacher Effectiveness/New Accountability Model, North

Carolina Professional Training Standards for Principals and Assistant Principals, Detailed

Scope of Work/Technical Assistance, Principal Training for Common Core and Essential

Standards, and Principal and Assistant Principal Training (ITES Standards). These sessions were

developed and facilitated by the NCDPI RttT Professional Development Implementation Team in

partnership with RESA staff.2

2012 Summer Institutes

Six two-day Summer Institutes were held across the state to provide opportunities for local

professional development leaders to expand their learning about the Common Core State

Standards and the North Carolina Essential Standards. LEA and charter school teams, comprised

of up to 18 members, including key leaders from each LEA and charter school, attended these

summer trainings. These sessions were conducted in collaboration with LEAs and facilitated by

2 For descriptions of these sessions, please see Table 10 in Report I (Report I: RttT Statewide Face-to-Face

Professional Development Formative Evaluation).


March 2013


the NCDPI RttT Professional Development Implementation Team. This year’s Summer Institutes

focused on how to facilitate learning for all students and included sessions organized by topic:

Content Strands, Leaders with Leaders, Facilitative Team Time, and Assessment and

Accountability.


March 2013


Method

In order to examine more deeply the extent to which LEA and school staff increased their

capacity to provide high-quality professional development at the local level, the Evaluation

Team collected and analyzed relevant data from all 115 LEAs (state strategies), and identified a

purposeful sample of 27 schools (short-term outcomes). Eight data sources were used over the

course of the 2011-12 academic year and the summer of 2012 (Table 1). The Evaluation Team

used a mixed methods approach to address the broad evaluation questions discussed above.

Quantitative methods included surveys and structured observations. Qualitative approaches

included interviews and focus groups, as well as open-ended survey questions. This section

discusses each of the data sources used and the corresponding methodology. Percentages

presented in the following tables have been rounded to the nearest whole number; therefore, not

all percentages add up to exactly 100.

Table 1. RttT Evaluation Data Sources, 2011-12

*RttT priorities include 1) transition to new state standards, 2) implementation of new formative/summative

assessments, 3) use of data literacy for instruction, 4) use of technology, and 5) effective use of the revised North

Carolina Educator Evaluation System.

Data source Purpose

Statewide

RttT LEA Professional

Development Coordinator

Survey

Gather information about local capacity for providing high-

quality professional development

RttT Omnibus Teacher

Survey

Assess change across a wide range of constructs that may be

influenced by the collective set of NC RttT activities

Longitudinal

CLASS Observation Protocol Gather information about classroom practices

RttT Professional

Development Teacher

Survey

Gather information about teachers’ attitudes, experiences,

knowledge, etc., about professional development and RttT

priorities*

RttT Professional

Development LEA

Coordinator Interviews

Discuss local capacity for providing high-quality professional

development

RttT Professional

Development Principal

Interviews

Discuss principals’ experience and recommendations about

RttT professional development related to RttT priorities*

RttT Professional

Development Teacher Focus

Groups

Discuss teachers’ experience and recommendations about RttT

professional development related to RttT priorities*


March 2013


Statewide Data Sources

LEA Professional Development Coordinator Survey

The Evaluation Team created and distributed a survey to LEA Professional Development

Coordinators to gain information on how the LEAs were developing and implementing

professional development programs that addressed the RttT initiatives, as well as whether they

were accessing and using NCDPI’s RttT resources and events. The survey items asked the

participants to use a 7-point Likert scale to rate their LEA in terms of how well their local

professional development efforts helped their teachers move toward RttT initiative goals (e.g.,

“Professional development experiences in my district this year have helped teachers implement

the state Standard Course of Study”). Participants also were asked to indicate the progress of

their LEA in building the capacity to continue serving their educators after the completion of the

RttT grant (e.g., “In my district, leaders have the skills to plan and design high-quality

professional development”).

The survey was distributed via email in February 2012 to LEA leadership with a request that it

be forwarded to the Professional Development Coordinator. Due to this deployment method, the

data were cleaned to remove any non-Professional Development Coordinators who may have

taken the survey. The Team received a total of 274 responses to the survey. After restricting the

data to those who specifically identified themselves as serving in a professional development

coordinator capacity, the final sample size used in the analysis was 159. Survey responses were

received from all eight regions, from 92 of the 115 LEAs (Table 2).

Table 2. LEA Professional Development Coordinator Survey Response Rate by Region, 2012

Region

Number of LEAs that

Completed the Survey

Number of LEAs in

Region Response Rate

1 13 15 87%

2 13 14 93%

3 12 14 86%

4 10 12 83%

5 10 15 67%

6 7 9 78%

7 16 19 84%

8 11 17 65%

Total 92 115 80%

RttT Omnibus Survey

The overall research plan for the four years of the NC RttT evaluation includes annual

administration of an Omnibus Teacher and Principal Survey to a statewide sample in February of

each school year. The survey was designed to assess change across a wide range of constructs

that may be influenced by the collective set of NC RttT activities, with items in several

constructs corresponding to specific initiatives, including professional development.


March 2013


The Omnibus Survey is comprised of 170 items across 23 dimensions. Examples of key

dimensions include teacher-leadership respect, teacher-teacher trust, teacher knowledge sharing,

and teacher-student relationships. Each respondent received a random sub-sample of the

questions to decrease respondent burden. The survey was administered online via Qualtrics, and

the sample of questions for each participant was estimated to take between 15 and 30 minutes to

complete.

For this report, five dimensions on the survey were identified to help understand the evolution of

the professional development component of RttT. The tables in this report were generated using

data from individual teacher responses to items in the following dimensions:

1. Quality of Professional Development

2. Alignment of Professional Development

3. Data-Driven Instruction

4. Evaluation of Teachers

5. Formative Assessment

For the first four of these dimensions, items were measured on a 7-point Likert scale: (1)

Strongly disagree, (2) Disagree, (3) Slightly disagree, (4) Neither agree nor disagree, (5) Slightly

agree, (6) Agree, and (7) Strongly agree. Items in the fifth dimension, Formative Assessment,

asked how often different formative assessment strategies were used; frequency was measured

on a 5-point scale: (1) Never, (2) A few times a year, (3) Once or twice a month, (4) Once or

twice a week, and (5) Almost daily. Since the nature of the questions and scale of the data were

different, the results for the Formative Assessment dimension are reported separately, after the

results for the first four professional development dimensions.

Longitudinal Sample

School Sample Selection Process

In order to gain a more accurate perspective on the impacts of the RttT professional development

initiatives on educators across the state, a purposeful sample of schools was selected to

participate in the longitudinal descriptive study. School type, professional development ratings,

and student achievement were the three primary variables that influenced the selection of the

schools. The Evaluation Team categorized all schools in the state by type (elementary, middle, or

high school), tercile of recent professional development quality (high, midrange, or low) based

on the results from the 2010 TWC survey, and tercile of school performance composite scores

(high, midrange, or low) based on the North Carolina ABCs of Public Education school

accountability model. School type, urbanicity, achievement status, teacher characteristics,

student characteristics, and other variables also were considered in the selection of schools for

the sample. Finally, considerations were made to ensure that the schools were geographically

representative. This selection process yielded a stratified sample of 27 schools.


March 2013


Description of the Sample

This section describes the sample schools in general terms; specific information that could be

used to identify individual schools was removed to prevent violation of confidentiality.

Schools. The longitudinal sample included nine traditional elementary schools (grades K-5),

seven middle schools (grades 6-8), nine high schools (grades 9-12), and two schools that blend

either middle and elementary grades or middle and high school grades. Across all 27 schools in

our sample, 83% of their Adequate Yearly Progress goals in the 2010-11 academic year were

met, and collectively, 75% of their students were at or above grade level. Approximately 19% of

schools were located in cities, 11% were suburban, 26% were in a town, and 44% were rural.

Schools were distributed across all eight State Board of Education regions.

Figure 2. Geographic Distribution of Longitudinal Professional Development Study Schools

Teachers and students. Across the sample schools, about 26% of teachers had advanced degrees,

18% were National Board Certified, and 96% were fully licensed. Eighteen percent of teachers in

the sample had three or fewer years teaching experience.

The ethnicities of students in the sample were comparable to those across the state (North

Carolina Department of Public Instruction, 2012). On average, about 55% were White/

Caucasian, 26% were Black/African American, 14% identified as Hispanic/Latino, 4% were

multiracial, 1% were Asian, and less than 1% were American Indian/Native American or Pacific

Islander. Approximately 55% of the students in the sample were eligible to receive free or

reduced-price lunch.

Longitudinal Data Sources and Procedure

During the 2011-12 academic year, the Evaluation Team gathered quantitative and qualitative

data on professional development activities in the 27 schools. Quantitative data were gathered

through structured classroom observations and teacher surveys. Qualitative data included semi-

structured interviews and focus groups with LEA Professional Development Coordinators,

principals, and teachers in the sample schools. All interviews and focus groups were facilitated

by Evaluation Team members. The following section describes each data source and the

procedures used to analyze the data collected.


March 2013


CLASS Observation Protocol

The Classroom Assessment Scoring System (CLASS) was used to gather data regarding

teachers’ progress in the transition to new state content standards (Common Core and Essential

Standards), which was a primary goal of the RttT-funded professional development. Research

has shown CLASS to be both valid and reliable (Center for Advanced Study of Teaching and

Learning, 2011), and it can be used in a wide range of classroom situations. In addition, the

tool’s depth offers several advantages over similar tools. CLASS offers different versions for

multiple grade levels, and its 7-point rating scale offers more rating flexibility and refinement

than do scales found in other tools considered by the Evaluation Team. All RttT evaluators who

conducted the classroom observations completed a certification process that consisted of two

days of training and successful completion of a CLASS Reliability Test.

CLASS was developed to assess classroom quality based on observed instructional interactions

between teacher and students, as well as productivity evident in classroom settings (Teachstone,

2012). The system has a total of 12 dimensions, but for this evaluation, the Team chose to focus

on the 5 CLASS dimensions that most closely align with instructional strategies outlined in the

Common Core and Essential Standards (Regard for Student Perspectives, Content

Understanding, Analysis and Problem Solving, Quality Feedback, and Instructional Dialogue;

Table 3). Each dimension was scored using the 7-point scale, with 1 being the lowest.

During the 2011-12 academic year, the Evaluation Team observed over 120 hours of teaching in

core content classrooms (English-Language Arts [ELA], Math, Social Studies, and Science)

within our 27 sample schools. Observers spent between one and two hours in each classroom in

both the fall and spring semesters.

Table 3. CLASS Observation Tool Dimensions Chosen for Use in the RttT Evaluation

CLASS Dimension Description

Regard for

Student

Perspectives

(RSP)

This dimension focuses on the extent to which the teacher is able to meet and capitalize on

the social and developmental needs and goals of adolescents by providing opportunities for

student autonomy and leadership. Also considered are the extent to which student ideas and

opinions are valued and content is made useful and relevant to adolescents.

Content

Understanding

(CU)

Content understanding refers to both the depth of lesson content and the approaches used to

help students comprehend the framework, key ideas, and procedures in an academic

discipline. At a high level, this refers to interactions among the teacher and students that lead

to an integrated understanding of facts, skills, concepts, and principles.

Analysis and

Problem Solving

(APS)

This dimension assesses the degree to which the teacher facilitates students’ use of higher

level thinking skills, such as analysis, problem solving, reasoning, and creation through the

application of knowledge and skills. Opportunities for demonstrating metacognition (i.e.,

thinking about thinking) are also included.

Quality of

Feedback (QF)

Quality of feedback assesses the degree to which feedback expands and extends learning and

understanding and encourages student participation. In secondary classrooms, significant

feedback may also be provided by peers. Regardless of the source, the focus here should be

on the nature of the feedback provided and the extent to which it “pushes” learning.

Instructional

Dialogue (ID)

Instructional dialogue captures the purposeful use of dialogue—structured, cumulative

questioning and discussion that guide and prompt students—to facilitate students’

understanding of content. The extent to which these dialogues are distributed across all

students in the class and across the class period is important to this rating.


March 2013


CLASS data were compiled for all classroom observation sessions during the 2011-12 academic

year, for a total of 208 observed classrooms (grades 1 through 12) and 702 observations in all.

The data were grouped by school performance level and by professional development rating (as

described above). Summary statistics for the various groupings were then calculated for the

classroom characteristics and the CLASS variables of interest listed above.

On average, classrooms contained one adult and 20 students. ELA represented over 30% of the

content observed in classrooms, followed by Math and Science, each of which represented 25%

of observed content. Twenty-one percent of classrooms taught Social Studies, and the remaining

focused on non-core content. Generally, classrooms observed for this sample used “whole class”

instructional formats for some or all of the instruction observed (69% of classrooms), followed

by “small group” and “individual instruction” formats (approximately 40% each). It is important

to note that because the study included elementary schools, Evaluation Team members had the

option to indicate observation of multiple content areas for each classroom characteristic per

observation. For example, in an elementary classroom, an observer may see both ELA and Math

taught, with the teacher using multiple instructional formats for each, all within the same

observation period.

Teacher Professional Development Survey

The Teacher Professional Development Survey (Appendix A) was designed to gauge teachers’

perceptions of the quality and utility of state- and locally-provided professional development

training. More specifically, items on the survey asked teachers to rate LEA leadership capacity

for providing professional development, the potential impacts of professional development on

teachers’ skills and knowledge in the classroom, and the integration of Common Core and

Essential Standards into classroom instructional practices. The items about instructional practices

aligned to the new standards were developed in close collaboration with NCDPI leaders in the

Curriculum and Instruction division for the core content areas.

Principals in each sample school disseminated the survey to teachers in their schools between

October 2011 and March 2012. Survey items included both Likert agreement scales and

frequency scales. Of the 1,211 participants who started the survey, 22 participants indicated that

they either did not teach a class (e.g., some participants indicated that they were counselors or

principals), were not certified to teach (e.g., teacher aides), or left the “certified to teach”

question blank; these participants were removed from the initial sample, reducing the sample to

1,189.

In general, teachers who completed the Teacher Professional Development Survey were

representative of teachers across the state. Similar to state-reported ratios in the Public Schools

of North Carolina statistical profile for the 2011-12 school year3 teachers in this sample were

75% female (n = 891), and over 80% were Caucasian/White (n = 890; Table 4, following page).

3 North Carolina Public Schools Statistical Profile, 2011–12

http://apps.schools.nc.gov/pls/apex/f?p=1:21:0::NO:::


March 2013


Table 4. Participant Representation by Ethnicity, Teacher Professional Development Survey

Ethnicity

Percentage of

Respondents

American Indian/Alaska Native 1%

Asian*


March 2013


Table 6. Participants’ Education Experience, Teacher Professional Development Survey

Education Experience

Percentage of

Respondents

Bachelor’s degree 34%

Bachelor’s degree plus credits 23%

Certificate of Advanced Study 1%

Master’s degree 25%

Master’s degree plus credits 12%

Doctorate* 0%

I am a lateral entry teacher 4%

*Representation for those holding a doctorate is 0.5%

Response rate. Based on self-reported data from principals in our sample regarding the total

number of teachers in their schools, the overall survey response rate on the Teacher Professional

Development Survey was 77% (Table 7).

Table 7. Survey Response Rates, Teacher Professional Development Survey

School Level

Mean Response

Rate

Elementary School 83%

Middle School 56%

High School 87%

Overall 77%

Response rates for each question on the survey varied for a few reasons. First, participants were

able to skip questions. Second, many items offered the response option of “Not Applicable”; this

response option was treated as missing when calculating proportions of responses. Finally, the

response options for the Quality of Professional Development and the District Capacity items

initially contained an error; the “Neither disagree nor agree” option offered only “Agree.”

Because it is not possible to know if the participants who saw this item were influenced by this

erroneous response option, all participants who viewed the incorrect scale were removed from

that section of the analysis.

Analysis of the participant survey data focused on frequency of responses. Data presented from

this survey in the following sections either show proportions of respondents who agreed or

strongly agreed with each item, or they show the frequency of all participant responses,

depending on the scale used.

LEA Professional Development Coordinator Interviews

Principals at each sample school were asked to identify the person in their LEA who coordinated

professional development training and assistance for their school. The Professional Development


March 2013


Coordinators were asked to participate in a 30-minute interview to discuss their LEAs’ capacity

to provide high-quality professional development. The interview protocol (Appendix B)

contained questions specific to strategies used in selecting appropriate professional development

for teachers and staff, integration of RttT priority areas into professional development training,

and the use of state-provided resources. Interviews were conducted between October 2011 and

March 2012.

Principal Interviews

Principals from each sample school participated in two 30-minute semi-structured interviews in

the fall and spring semesters, during which they discussed their experience with professional

development planning and implementation. The interview protocol (Appendix C) contained

questions specific to strategies used in selecting appropriate professional development for

teachers and staff, integration of RttT priority areas into professional development training, and

the use of state-provided resources.

Teacher Focus Groups

Six to eight core content teachers (in English, Math, Science, and Social Studies) from each

school were selected to participate in a 45- to 60-minute focus group. Teachers from each school

were assembled onsite and engaged in a discussion facilitated by an Evaluation Team member.

The focus group protocol (Appendix D) was designed to gather teachers’ opinions of the

availability and use of professional development resources at their school, as well as of their

overall impressions of the transition to the new Common Core and Essential Standards.

The Evaluation Team collected over 45 hours of interview and focus group data. After each

audio recording was transcribed, Atlas.ti software was used to facilitate qualitative analysis of

the data. An a priori coding scheme was developed based on the evaluation questions outlined in

the introduction of this report. The coding scheme was refined and expanded as the team

interacted with the data. Results from the analysis contributed to the baseline outcomes in this

report.

Limitations

Due to changing schedules and available resources on the RttT Professional Development

Evaluation Team, a few data collection strategies did not reach their full potential.

CLASS Observation Protocol: Evaluators were able to observe only a small proportion of the teachers in the schools in our longitudinal study, and only for a short time—four teachers in a

school for two hours at a time, twice a year. The intent is not to rate teachers but to use this

valid and reliable classroom observation tool to systematically record observed instructional

interactions between teacher and students. For the purposes of this study, these data will be

used to describe teachers’ transition to the new state content standards (Common Core and

Essential Standards), as evidenced by effective instructional practices required for the

teaching to the new standards.

Teacher Professional Development Survey: When transferring the survey to the online survey portal, a clerical error resulted in two “agree” response options for the items about Quality of


March 2013


Professional Development and LEA Capacity. Because it is not possible to know if

participants were influenced by this erroneous response option, all participants who viewed

the incorrect scale for those constructs were removed from analysis.

Specific strategies have been identified to address these limitations for the remainder of the

evaluation and are outlined in the final section of this report, Next Steps for the Professional

Development Evaluation.


March 2013


Results

This report provides a baseline (first-year) assessment of the State’s progress toward updating

the education workforce in North Carolina, as measured by the status of intermediate outcomes

related to that goal (Figure 3). These outcomes are expected to result from educator participation

in statewide face-to-face RttT professional development events designed to impact (a) local

capacity to provide high-quality professional development, (b) shifts in local organizational

conditions to support RttT priorities, and (c) changes to instructional practice. That is, this report

provides baseline data for addressing Evaluation Question 3— To what extent did RttT

professional development efforts successfully update the North Carolina education workforce?—

and related sub-questions.

Figure 3. Logic Model of Local-Level Outcomes for Statewide RttT Professional Development

*RttT priorities for North Carolina include: successfully transitioning to the new standards, implementation of

formative and summative assessments, use of data to support instruction, and effective utilization of the new North

Carolina Educator Evaluation System (NCEES).


of organizational change, including capacity to provide high-quality local professional

development aligned to RttT priorities?

In order to assess progress toward meeting intermediate outcomes related to updating the North

Carolina education workforce at this point in the RttT grant, qualitative and quantitative data

were collected to: (1) describe whether local capacity has been built to plan, access, and

implement high-quality professional development (Figure 4, following page); (2) provide

baseline information about the extent to which there have been organizational shifts at the school

and LEA level related to the RttT priorities; and (3) provide baseline information about the

quality of the professional development being provided.


March 2013


Figure 4. Logic Model of Outcomes for Statewide RttT Professional Development—Local

Capacity Focus

*RttT priorities for North Carolina include: successfully transitioning to the new standards, implementation of

formative and summative assessments, use of data to support instruction, and effective utilization of the new North

Carolina Educator Evaluation System (NCEES).

Planning, Accessing, and Implementing Local Professional Development

Planning and Accessing Local Professional Development. Focus group and interview data

revealed that, for most schools, school-level professional development was informed by what

was being offered at the state level and typically was designed around resources available at the

LEA level. When asked about professional development training around specific RttT priority

areas in particular, some coordinators and principals noted a strong reliance on state and LEA

guidance (e.g.: “We go with whatever is provided by our LEA;” “I think our professional

development is through what is offered by the state”). In the few cases where local-level

leadership were more directly involved in the planning process, Professional Development

Coordinators and principals highlighted cross-LEA sessions where professional development

strategies were exchanged among LEAs in their region. One Coordinator’s comments

summarized the appreciation expressed by local leadership for the chance to engage in the

planning process:

When you’re working within a district, you’re isolated within that district and you don’t

know what else is going on around you. It was good to hear what smaller districts could

do, and what some of the larger districts comparable to [our] county could be doing.

Since we are all going toward the new assessments and new curriculum, it’s good to hear


March 2013


what others are doing to prepare their teachers. Then we can decide within our county

cohort what things we think might work, if we need to go back in and redesign our plan,

or move forward with what we think our best practice is.

Some schools also relied on in-house experts or outside vendors (e.g., Reading 3D or PD 360) to

plan supplemental professional development training for their teachers.

Qualitative data from the sample schools detailed the ways in which information about

professional development availability typically was disseminated by LEA leadership to teachers

and principals at the school level. In line with the state-endorsed “train the trainer” model, LEA-

level Professional Development Coordinators indicated that they were primarily responsible for

identifying and disseminating the most relevant professional development resources and making

them available to principals and teachers in their LEAs. Most LEA Coordinators mentioned

using tools like wikis, website postings, and staff development newsletters and calendars to

condense professional development resources and make them more easily accessible to teachers

and principals. Other methods of increasing awareness of professional development

opportunities included teachers and administrators sharing information at faculty meetings and in

subject-area or grade-level Professional Learning Communities (PLCs).

Despite the multiple methods of information dissemination described by LEA leaders, data

revealed that some principals and teachers still remained uninformed about NCDPI-supported

professional development opportunities. For example, when focus group moderators asked

teachers whether they were aware of any online professional development opportunities offered

by NCDPI, teachers from four schools were unaware of any of the online modules being offered,

and teachers from four other schools indicated that they were aware of or had accessed the NC

FALCON modules developed prior to RttT but were not aware of the new online modules

released in the summer and fall of 2011. This lack of awareness was likely due to either a LEA

Professional Development Coordinator’s decision to postpone the rollout of the modules,

unresolved decisions about what modules or module components to use, or, as in the case of two

LEAs, a decision not to make the modules mandatory because of duplication of prior local

efforts or concerns over their quality.

While most schools were able to locate and access some professional development resources,

more evidence is needed to fully assess changes in local professional development training

patterns. While there are indications that, at the state level, NCDPI leadership has recognized the

need to create a better structure to describe how to connect state-implemented professional

development to professional development training at the LEA and school levels, a need for

specific communication strategies for professional development at the local level persists, as

evidenced by teachers’ and principals’ lack of awareness of professional development resources,

despite LEA Professional Development Coordinators’ express role in disseminating state-

provided professional development resources. Differences in professional development needs

across schools and the availability of resources impact North Carolina educators’ ability to

translate state-level professional development resources into local-level training. This baseline

data will be used as a point of comparison for future evaluations of LEA and school ability to

locate and access professional development resources.


March 2013


Implementing Local Professional Development. The Evaluation Team’s approach to examining

LEA and school preparedness to implement professional development was two-fold. First, data

from the statewide LEA RttT Professional Development Coordinator Survey was used to

examine the extent to which LEA leaders believed they had the capacity to implement high-

quality professional development. Second, qualitative data from principals and teachers in our

sample schools provided detailed information about the process by which LEA and school

leadership selected, planned, and implemented professional development at the local level.

Table 8 presents results from the RttT LEA Professional Development Coordinator Survey on

perceptions of LEA leadership capacity to develop professional development programs that suit

the needs of their schools and LEAs.

Table 8. Perceptions of LEA PD Coordinators about LEA Capacity to Implement Professional

Development

In my district, leaders…

Percentage of

Respondents

who Strongly

Agree/Agree

(n = 144-150)

Have knowledge about high-quality professional development defined by research

and national and state standards 83%

Have the skills to plan and design high-quality professional development 83%

Have the skills to implement high-quality professional development 87%

Have the skills to select high-quality professional development 92%

Have positive attitudes about high-quality professional development 94%

Have a district-wide commitment to high-quality professional development 91%

Support Communities of Practice around high-quality professional development 81%

Provide opportunities for networking and support (both online and offline) in high-

quality professional development 77%

Have a district-level strategic plan for professional development in place 86%

Align school-level professional development plans to district plans 80%

Distribute responsibilities for leadership for professional development among

multiple school and district leaders 86%

Use data from teacher performance evaluations to create individual professional

development plans for teachers 68%

Aggregate data from teacher performance evaluations and use data to identify

school/district professional development needs (Teacher Professional Development

Survey: Use data from teacher evaluations to identify school/district professional

development needs)

64%

Use survey data to select, plan, and design professional development 78%

Use summative student data to select, plan, and design professional development 79%

Use formative student data to select, plan, and design professional development 67%

Source: RttT LEA Professional Development Coordinator Survey


March 2013


Survey results revealed that statewide, a large majority of Professional Development

Coordinators agreed or strongly agreed that their LEA leadership had the capacity to implement

and plan various aspects of professional development. For instance, 91% of Professional

Development Coordinators agreed or strongly agreed that their LEA leaders had a commitment

to implementing high-quality professional development, and 86% of Professional Development

Coordinators believed that their LEAs had a strategic plan in place for professional development.

Results were slightly less favorable regarding LEA leaders’ use of data to inform professional

development decisions, with 79% of coordinators agreeing that LEA leaders used summative

student data to select, plan, and design professional development, 67% agreeing that leaders used

formative student data for the same purpose, and only 64% agreeing that LEA leaders used data

from teacher performance evaluation results to identify professional development needs.

The design for professional development training implementation sometimes was a group effort

at the LEA level (for example, collaboration among principals from across an LEA), as

explained by this focus group participant:

This particular year, how we are rolling out the Common Core, we brought in a group of

principals, elementary level, middle school level, and high school level, to look at how

they thought we should roll this out, and who should be trained, and who should be the

ones taking the information back to the schools. We are very large. And so there’s no way

that we’re going to have the face-to-face contact with each teacher within the district. So

we had to develop a plan to use the trainer model, so to speak, in order to at least get the

information out, so that they can begin the teaching of the Common Core.

At the school level, professional development training was designed primarily for delivery

within PLCs or staff meetings. Some schools provided supplemental training resources through

online tools (e.g., Live Binder or Moodle) as additional support for teachers in their transition to

the new state standards. Generally, teachers felt that PLCs were designed to provide an

opportunity to gather with other teachers in their content areas or grade levels to review

professional development resources and walk through any changes in curriculum related to the

new state standards that would affect their instructional practice. For many teachers, PLCs or

staff meetings were their first entrée into professional development training focused on RttT. The

following comments reflect teacher and principal feedback regarding the usefulness of designing

PLCs around professional development goals:

I think obviously the professional learning communities, it’s really helped us to grow

those and it has also enabled us to really utilize that blended approach where we can use

some online, some face-to-face, and we’re using Moodle.

We’re asking teachers to use these [additional] tools to learn the standards themselves.

We’re using the online environment to actually use these modules that DPI set up and

then take them a step further and use the Moodle environment to hold the PLC

conversations. So once they view the module, then we have them go and do online

discussions within their PLC, and so each PLC is doing something a little differently.

Implementation was not limited to 2011-12 school year professional development. Many LEAs

provided summer pre-training opportunities designed to give teachers early chances to explore


March 2013


new Common Core materials and find ways to apply them to their specific grade levels. Most

teachers in the focus groups noted that their experience in summer training was helpful in

preparing them for the use of state-implemented strategies related to Common Core. For a small

proportion of LEAs and schools, implementation of professional development around the new

state standards began one or two years before the 2011-12 school year. One Professional

Development Coordinator commended her LEA for extending professional development

strategies around the Common Core and Essential Standards from the previous year:

Our county is trying to be proactive and they made sure at the end of the last school year,

the 2011-11 school year, that we had the actual Common Core curriculum in our hands,

and then over the summer and through this year, they’re making sure we’re unpacking

the standards; looking at them, understanding for each grade level: Do you know what is

being transitioned out of your grade? Do you know what’s being transitioned into your

grade? I know that there are a lot of counties that are surrounding us that are not doing

those things to prepare. We’re already beginning to teach those things and starting to

close the gap this year so that it won’t be as hard of a transition for the teachers next

year.

Alignment with RttT Priorities

Items from the LEA Professional Development Coordinator Survey and Professional

Development Teacher Survey specifically asked about teachers’ participation in professional

development aligned to RttT priorities (Table 9). Only about half of LEA Professional

Development Coordinators from across the state and teachers in the sample schools agreed or

strongly agreed that teachers built their knowledge and skills for revised state assessments (51%

and 52%, respectively) and teaching diverse student populations (53% and 59%, respectively).

Perceptions of LEA Professional Development Coordinators statewide and teachers in our

representative sample differed in other areas, most particularly about whether teachers built their

knowledge and skills for revised state standards (91% and 64%, respectively) and whether they

deepened their content knowledge (81% and 58%, respectively).

Table 9. Perceptions of Participation in Professional Development on RttT Priorities

To what extent do you feel that teacher knowledge and

skills have been enhanced in each of the following areas as a

result of participation in professional development?

Percentage of Respondents who

Strongly Agree/Agree

PD Coordinators

(n = 146-148)

Teachers

(n = 864-890)

Revised state standards 91% 64%

Instructional materials 73% 58%

Approaches to formative assessment 79% 64%

Use of technology 67% 70%

Strategies for teaching diverse student populations 53% 59%

Deepening content knowledge 81% 58%

Leadership development 71% 53%

Revised state assessments 51% 52%

Revised NC Teacher Evaluation Process 81% 63%

Sources: RttT LEA Professional Development Coordinator Survey, RttT Professional Development Teacher Survey


March 2013


When asked to rate their level of agreement with statements about the alignment of their LEA’s

professional development training and RttT priorities, most coordinators (>85%) agreed that their

LEAs had plans, strategies, resources, and systems in place to support RttT Professional

Development (Table 10). Fully 92% of coordinators reported that LEA-provided RttT professional

development was aligned with and built upon existing professional development initiatives.

Table 10. Professional Development Coordinators’ Perceptions of LEA Capacity to Support RttT

Professional Development

To what extent do you agree/disagree with the following?

Percentage of

Respondents who

Strongly Agree/Agree

(n = 139-144)

Our Race to the Top professional development is aligned with and builds on existing

professional development initiatives in the district. 92%

Our district has . . .

a strategic plan for the deployment of Race to the Top professional development. 92%

an implementation timeline in place for Race to the Top professional development. 90%

a process in place for how we will communicate Race to the Top professional development

initiatives and opportunities to all district staff. 92%

ensured that Race to the Top professional development activities are supported with up-to-

date board-approved policies and procedures. 85%

designated key personnel who are responsible for various components of our Race to the Top

professional development plan. 93%

a system for monitoring and evaluating our Race to the Top professional development plan. 79% communicated with our local testing coordinator to ensure all staff has NC Education login

capabilities. 88%

a comprehensive plan to coordinate the integration of face-to-face and online professional

development. 78%

utilized data on staff’s technology proficiency when planning Race to the Top professional

development. 64%

ensured all staff has equitable access to technology for accessing Race to the Top

professional development resources. 85%

determined that all online resources related to Race to the Top are accessible in the LEA. 89%

specific strategies for collaboration as we roll out Race to the Top professional development. 91%

provided educators an online space (e.g., wiki, website, Moodle, etc.) for sharing resources,

experiences, and/or information to related to RttT. 77%

collaborated with district technology staff to determine the extent of online communication

and collaboration tools available for PD. 72%

designated key personnel to be responsible for monitoring and recording the progress of staff

in completing online and face-to-face Race to the Top PD. 81%

developed a plan for implementing PLCs (online and/or face-to-face) related to Race to the Top. 73% designated key personnel responsible for scheduling and monitoring PLCs related to Race to

the Top. 70%

a plan for how to use the DPI PD Leads in each region for support in our Race to the Top

professional development. 60%

a long-range plan in place for Race to the Top professional development for the next three years. 71% Our district’s Race to the Top professional development . . .

goals are aligned with state initiatives. 94%

team can articulate their role in our Race to the Top professional development plan. 85%

differentiates to meet participants’ needs with additional resources and training. 75%

Source: RttT LEA Professional Development Coordinator Survey


March 2013


However, coordinators highlighted two areas in which they thought LEAs were less effective in

implementing RttT professional development. Only 64% of coordinators agreed that their LEAs

utilized data about staff’s technology proficiency when planning RttT professional development.

In addition, only 60% agreed that their LEAs have plans for how to use the NCDPI Professional

Development Leads in each region for support in their RttT professional development. Similar to

some of the results reported above regarding perceptions of LEA capacity to provide appropriate

professional development, these results also suggest a lack of capacity to use data to guide the

selection, design, and implementation of high-quality professional development.

Quality of Local Professional Development

Data from the RttT Omnibus Teacher Survey provides insight into North Carolina teachers’

general perceptions of the overall quality of professional development they experienced in 2011

and 2012 (Table 11). About 70% of teachers agreed or strongly agreed that professional

development experiences were both sustained and coherent in the 2010-11 and 2011-12 school

years. However, fewer (63%) agreed or strongly agreed that they had enough time to think about,

try, and evaluate new ideas in 2010-11, and that number fell further (to only 56%) in 2011-12.

Table 11. Teacher Perceptions of Overall Quality of Professional Development

Overall, my professional development experiences this year have...

2011

Strongly

Agree/Agree

(n = 1,709)

2012

Strongly

Agree/Agree

(n = 10,011)

Been sustained, rather than short-term. 70% 70%

Been coherently focused, rather than unrelated. 70% 69%

Included enough time to think carefully about, try, and evaluate new ideas. 63% 56%

Source: RttT Omnibus Teacher Survey

Both the RttT Omnibus Teacher Survey and LEA Professional Development Survey results

demonstrate that, although a majority of respondents agreed that their experiences were positive,

teachers generally gave lower ratings for their experiences with LEA-provided professional

development than did LEA Professional Development Coordinators (Tables 11 and 12).

In their assessment of the quality of LEA-provided professional development, 93% of the

Professional Development Coordinators agreed that teachers were provided opportunities to

work productively with colleagues in their school, compared to only 70% of teachers. Similarly,

66% of Coordinators but only 46% of teachers agreed that teachers were provided the

opportunity to work with teachers from other schools, and 81% of Coordinators but only 63% of

teaches thought most of what teachers learned in professional development addressed the needs

of the students in the classroom (Table 12, following page).


March 2013


Table 12. Professional Development Coordinators’ Perceptions of LEA-Level Professional

Development Alignment

To what degree do you agree with the following statements about

the PD experiences provided in your district this year?

Strongly

Agree/Agree

(n = 152-154)a

Strongly

Agree/Agree

(n = 10,011)b

Teachers are provided opportunities to work productively with

colleagues in their school 93% 70%

Teachers are provided opportunities to work productively with

teachers from other schools 66% 46%

Most of what teachers learn in professional development addresses the

needs of the students in their classroom 81% 63%

Sources: a RttT LEA Professional Development Coordinator Survey;

b RttT Omnibus Teacher Survey

Data gathered in interviews and focus groups that was related to the quality of local professional

development was somewhat more encouraging, highlighting principals

Documents

Second Annual Race to the Top Professional Development ......Nov 02, 2011 · RttT PD Evaluation: Year 2, Part II—Local Outcomes Baseline March 2013 Consortium for Educational Research