Paul Cobb and the MIST Team Vanderbilt University University of Washington

A Provisional Agenda for Research on Improving the Quality of Mathematics Teaching on a Large

Scale

Paul Cobb and the MIST TeamVanderbilt University

University of WashingtonMichigan State University

McGill University

Purpose

• Outline a set of issues that need to be addressed if research is to provide adequate guidance for large-scale instructional improvement efforts in mathematics – Across a large urban district

Overview

• Preamble: what counts as high-quality instruction

• Background: ongoing work as a setting for appreciating the limitations of current research

• Proposal for issues that need to be addressed

What Counts as Instructional Quality?• Has to be justified in terms of students’

learning of mathematics that is worth knowing– Conceptual understanding as well as procedural

fluency– Justifying solutions, evaluating the reasonableness

of solutions, generalizing from solutions, making connections among multiple representations of mathematical ideas

Research on Students’ Mathematical Learning

• Rigorous mathematical tasks• Individual or small group work• Whole class discussion– Teacher presses students to: • Explain and justify their reasoning • Make connections between different solutions

Goals for Teachers Learning• High-leverage instructional practices – Planning and conducting productive whole class

discussions– Setting up rigorous mathematical tasks

• Specific types of knowledge implicated in the enactment of these practices– Mathematical knowledge for teaching– Vision of high-quality mathematics instruction– View of students’ mathematical capabilities

Challenge for Districts

• How to organize, support, and press for teacher learning across the entire system– What guidance can research provide?

Background: MIST Project

• Four-year collaboration with four large urban districts – 360,000 students – 2007-2011

• Continued collaboration with two districts – 180,000 students – 2011-2015

• Investigate (and support) the districts’ instructional improvement efforts in middle-grades mathematics

Background: MIST Project

• High proportion of students from traditionally underserved groups– Limited financial resources– High teacher turn over– High proportion of novice teachers

• Atypical in one respect:– Aiming at ambitious goals for student learning and

thus for teachers’ instructional practices

District Participants• 30 middle-grades mathematics teachers in 6-

10 schools in each district• Mathematics coaches• School leaders– Principals, assistant principals

• District leaders– Across central office units that had a stake in

mathematics teaching and learning

Collaboration with Districts

October

• Interview district leaders to document current strategies for improving middle-school mathematics

January-March

• Audio-recorded interviews with the 200 participants to document how the districts’ strategies are actually playing out in schools and classrooms


February-May

• Analyze transcripts of the 200 interviews• Identify and explain gaps between each district’s

intended and implemented improvement strategies• Develop a detailed report for leaders in each district

• Shared findings and made actionable recommendations

May

• Meet with district leaders to discuss our findings and recommendations


• District leaders attempt to act on our recommendations to a significant extent

• Become co-designers of district improvement strategies – Participants in as well as observers of the districts’

instructional improvement efforts

Collaboration with Districts• Formulating recommendations: Have to

address concrete organizational design problems

• Occasion to appreciate – The types of problems that district leaders have to

address – Extent to which current research can provide

guidance – hence this talk

Research Goal

• Develop an empirically grounded theory of action for instructional improvement at scale– Can inform other districts’ instructional

improvement efforts

Ongoing Analyses

• Initial conjectures about supports and accountability relations – Drew on then available literature

• Conjectures informed initial recommendations to districts

• District leaders acted on recommendations – opportunity to test and revise conjectures

Retrospective Analyses• On-line surveys for teachers, coaches, and school leaders• Video-recordings of two consecutive lessons in the 120

participating teachers’ classrooms– Coded using the Instructional Quality Assessment (IQA)

• Assessments of teachers’ and coaches’ Mathematical Knowledge for Teaching (MKT)

• Video-recordings of select district teacher professional development

• Audio-recordings of teacher collaborative planning time• Student achievement data

Theory of Action for Instructional Improvement at Scale

• A coherent instructional system:– Instructional materials + professional

development + assessments to inform instruction + additional supports for struggling students

• Mathematics coaches’ practices in providing job-embedded support for teachers’ learning

Theory of Action for Instructional Improvement at Scale

• School leaders’ practices as instructional leaders in mathematics

• District leaders’ practices in supporting the development of school-level capacity for instructional improvement

Research Team

Paul CobbTom Smith

Kara JacksonErin Henrick

Ilana HornKen Frank

Research TeamJessica Rigby Mollie AppelgateJonee Wilson Adrian Larbi-Cherif Brooks RosenquistCharlotte MunozBritnie Kane Jason Brasel Brette Garner Seth HunterEmily Kern Megan WebsterMahtab Nazemi I-Chien Chen

Explicit goals for students’ mathematics learning

Vision of high-quality instruction Instructional materials

Teacher professional development

Additional supports for struggling students

Assessments to inform ongoing instructional improvement

Component 1: Coherent

Instructional System

(Bryk et al., 2010; Newmann et al., 2001)

Vision of high-quality instruction:• Small set of high-leverage

practices that are potentially learnable in the context of high-quality professional development

Instructional materials:• Grounded in student learning

trajectories that aim at significant mathematical ideas



Explicit goals for students’ learning

Vision of high-quality instruction Instructional materials

Teacher professional development

• Pull-out Professional Development (PD): • Specific PD designs – promising

findings• Grounded in classroom practice –

pedagogies of investigation and enactment

• Most work in pre-service – have extrapolate to in-service

• Teacher Collaborative Time (TCT) • Most researcher-led – potentially

productive types of activities• Naturally occurring –

characteristics of productive teacher groups



• Formative assessment systems– Aligned with ambitious

goals for students’ learning

– Grounded in trajectories of students’ learning

Assessments to inform instruction



• Goal: support struggling students to participate effectively in mainstream instruction

Additional supports for struggling students



Coherent Instructional System• Collaborating districts: fragments of a coherent

instructional system– Strengths: explicit goals for students’ learning, vision

of high-quality instruction, instructional materials– Challenge: teacher professional development –

district capacity– Challenge: TCT – expertise + leadership of meetings –

district capacity– Weakness: additional supports for struggling students

– not aligned with mainstream classroom instruction

Needed Research: Developing District Capacity

• Researchers typically assume full responsibility for “building” particular elements

• The problem of scale involves supporting districts’ development of the capacity to create, coordinate, and sustain the elements of such a system

Developing District Capacity: Sacrificial Offering

• Example: co-designing and co-leading PD for coaches with district mathematics specialists – Support the development coaches’ capacity to

design and lead high-quality teacher PD – Gradual hand over of responsibility to district

mathematics specialists • Overall goal: Investigate how to support districts’

development of capacity to develop and sustain a cadre of mathematics coaches

Needed Research: Interrelations Between Elements of the System

• Current research typically focuses on the individual elements of a coherent instructional system

• Also need to investigate interrelations between various the elements, and between elements and other components of ToA– Which are preconditions for the development of

other elements/components?

Developing District Capacity: Sacrificial Offering

• Example: co-designing and co-leading PD for school leaders– School leaders press for instructional

improvement– Coaches support teachers in meeting those

expectations• Investigate development of aligned support and

press for teachers’ improvement of their instructional practices

Mathematics Coaching• Finding: Teachers’ improvement of their

instructional practices depends crucially on their access to colleagues who have already developed accomplished practices

• Three of our four districts: Small proportion of accomplished teachers

• Critical role of coaches as more accomplished colleagues

Mathematics Coaching

• Design and lead pull-out PD • Work with groups of teachers during TCT – Current research + our findings indicate

importance of leadership/expertise in TCT• Support teachers one-on-one in their

classrooms– Build on pull-out PD and work in TCT

Current Research on Coaching

• Provides little guidance on:– Types of activities in which coaches might engage

teachers – Coaches practices as they enact these activities – Supporting the development of a cadre of

accomplished mathematics coaches • One of the collaborating districts: four years of

sustained professional development for coaches• Only slightly ahead of the teachers they were expected

to support

Needed Research: Delineating Goals for Coaches’ Learning

• Develop testable conjectures about potentially productive coaching practices

• Working with groups of teachers: • Draw on studies of researcher-led and naturally

occurring PLCs – Kazemi and Franke: Potentially productive activities– Horn: Press teachers of key issues


• Working with teachers in their classrooms:– Draw on research on teacher learning and

professional development• Points to importance of modeling and especially co-

teaching • Observation/feedback at specific points in teachers’

development – e.g., Tuning their enactment specific practices


• Important to explicate the forms of knowledgeability implicated in the enactment of proposed practices– Researchers do not typically report what they

needed to know

Needed Research: Specifying Forms of Knowledgeability

• Mathematical knowledge for teaching (MKT) • Additional candidate: an envisioned trajectory

for teachers’ learning– Classroom management– Student engagement– Teacher questioning

• Informs decisions about which aspects of practice to work on with teachers

Needed Research: Designs for Supporting Coaches’ Learning

• Both settings: – Draw on very limited literature on coach PD– Extrapolate from work on teacher learning and

teacher PD

Needed Research: Test and Revise Designs for Coaches’ Learning

• Design experiments to:– Test and refine conjectures about supports for

coaches’ development of target practices– Assess in terms of improvements in:• Coaches’ practices• Quality of teachers’ classroom instruction • Student learning

Coach PD: A Sacrificial Offering

Coach PD Session

Coaches lead PD with pilot group

MIST & District Math Leaders collaboratively

plan for upcoming session

MIST views video-recording of pilot PD in

light of goals for coaches’ learning

School Instructional Leadership• Standards-based reform: Principals and

assistant principals increasingly expected to act as instructional leaders in specific content areas– Manage instruction rather than manage around

instruction

Current Research School Instructional Leadership

• No consensus on what school leaders need to know and be able to do in order to be effective instructional leaders in mathematics– General content-independent characteristics of

high-quality instruction• Observe instruction and provide feedback

– MKT, student mathematical learning, high-quality mathematics instruction, teacher learning• Coach mathematics teachers

Findings• Interviews – vision of high-quality

mathematics instruction (VHQMI)– Form rather than function views– Consistent with teachers’ accounts of the

feedback they receive from school leaders• Extensive professional development– Focused on general, content-independent

characteristics of high-quality instruction

Initial Findings• General characteristics of high-quality inquiry-

oriented instruction– Too abstract – not able to connect to concrete

instructional practices • MKT, student mathematical learning, high-

quality mathematics instruction, teacher learning– Beyond the capacity of most districts

Needed Research: Delineating Goals for School Leaders’ Learning

• Develop testable conjectures about potentially productive school leadership practices

• Justify in terms of: – Direct support/press for teachers’ learning– Indirect support – developing conditions for

teacher learning

Current Bets• Identify and capitalize on instructional

expertise in the school• Observe mathematics instruction and provide

feedback• Participate in teacher collaborative time (TCT)• Support coach to support teachers’ learning

School Leadership RoutineAttend TeacherCollaborative

Time

ObserveClassroom Instruction

Meet withMathematics

Coach

School LeaderAnd Coach:

Quality of IndividualTeachers’ Instruction

+How to

Support Instructional Improvement

+Jointly Plan for

TeacherCollaborative

Time

Needed Research: Delineating Goals for School Leaders’ Learning

• Important to explicate the forms of knowledgeability implicated in the enactment of the proposed practices – Vision of high-quality mathematics instruction• Observing instruction and giving feedback• Identifying and leveraging instructional expertise

Needed Research: Designs for Supporting School Leaders’ Learning

• Extrapolate from work on teacher learning and teacher PD – Vision of high-quality mathematics instruction• Distinguish between high- and low-rigor tasks• Distinguish between strong and weak enactments of

specific high-leverage instructional practices

Needed Research: Test and Revise Designs for School Leaders’ Learning

• Design experiments to:– Test and refine conjectures about supports for

school leaders’ development of target practices– Assess in terms of improvements in:• Instructional leaders’ practices• Direct and indirect supports for teachers’ learning • Quality of classroom instruction • Student learning

Coaching and School Instructional Leadership

• Important to take account of relations between members of different role groups– Coaches’ effectiveness in supporting teachers’

learning depends on relationship with school leaders

– Coach is a potential support for school leaders’ learning

District Leadership• Also important to take account of coaches’

and school leaders’ relations with district leaders – Supports for their learning – What they are held accountable for• Potential tension between raising student achievement

in the short term and improving the quality of classroom instruction in the long term

Why Does Current Research Provide Only Limited Guidance?

• Math education, teacher education, and the learning sciences:– Student learning, instructional activities and tools – Teaching, teacher professional development,

teacher collaborative time • Typically bracket out the school and district

contexts in which teachers’ learning occurs

Why Does Current Research Provide Only Limited Guidance?

• Educational policy and leadership – Typically instructionally agnostic • Content + vision of high quality instruction matter

– Research in policy and leadership can be relevant• Have to read through lens of what counts as high-

quality mathematics instruction

Supporting Instructional Improvement at Scale

• A problem of both teacher learning and organizational learning – Need to specify goals for organizational as well

and teachers’ learning • Differentiate between organizational change and

organizational learning

Organizational Learning• Conjectured practices, social relations, tools,

and routines are provisional goals for organizational learning – Provisional goal for the organizational learning of a

district: The creation and ongoing refinement of a coherent instructional system

– Provisional goal for the organizational learning of a school: Establishment of the school leadership routine

Papers and instruments downloadable at:

http://vanderbi.lt/mist

Documents

Paul Cobb and the MIST Team Vanderbilt University University of Washington