Root Cause Analysis

Data Summit, March 1, 2018Green Bay, WI

Facilitated by:Judy Sargent, WISExplore Team, CESA 7

Mary Ann Hudziak, WISExplore Team, CESA 6

1

Access to Presentation Slides:http://bit.ly/2orhrMc

Intended Outcomes

Participants will ...• Build common understanding of a critical student data

problem and a theory of action• Collaboratively generate potential root causes as

hypotheses of practices • Categorize potential root causes in practices• Prioritize to determine the greatest “influencer” practices

for leverage • Describe the practice of highest priority• Determine and understand next steps

2

Prepare to Use Technology

• Use laptops with power until 11:30 with access to the internet.

• Use your copy of the slides to record your notes.

• You will be provided a Root Cause Analysis Worksheet to document your work today.

• Please open and save a copy of this worksheet. Provide access to all team members.

• Be sure that all team members can view the same screens to foster collaboration and conversation. 3

Community Agreements

● Take care of yourself● Be fully engaged and take risk● Be mindful of and assist other learners ( novices to

experts)● Be fully present

4

Setting the Stage for Today’s Work

We will engage in root cause analysis based on your own school data.• Be sure you have online access to data from a local source

of from WISEdash for Districts or WISEdash Public.

5

Data Inquiry Process

6

Interconnected Data Inquiry Process

7

● Student data inquiry provides the pathway to inquiry and analysis of root causes in practices.

● The combination of student data inquiry and subsequent practices data inquiry provides a solid informed basis for implementing improvement actions.

Today we will practice and hone in on the

linkages between the student data inquiry and

practices inquiry.8

Critical Problem for Students

Determining a Critical Problem

9

A critical data problem is …1. An issue determined from a comprehensive student inquiry,

that should includea. All students data inquiryb. Equity gaps data inquiryc. Content data inquiry

2. Analysis of student performance data, ideally supported from multiple data sources

3. Agreed upon by a collaborative team

Determining Criticality

10

To determine criticality in data findings, consider…● Persistence in the data pattern● Multiple years showing a trend (at least 3 years)● Size of differences in the numbers (larger differences

more likely to be critical)● Impact on students, including numbers of students

impacted

Activity Steps

1. Share perceptions

2. Verify perceptions with data

3. Identify the critical problem (problem statement).

11

Perceptions

Without looking at data, what do you “think” is the most critical student performance problem in your school?

● On a post-it note, each person on the team lists his/her perception of the single most critical student performance problem in their school.

12

● Each person shares perception with the team

● Use a group process to come to consensus to agree upon the most critical student performance problem and record this in your worksheet.

● Be ready to support your selection.

Verify Perceptions with DataIs there data to support your perception?• Use WISEdash Public, secure or local data sources to find a data

picture(s) that clearly supports the performance problem selected.• Insert the data picture(s) that represents your critical student data

problem in the worksheet.• Add a caption or data finding statement to the data picture that

includes basic information about the data.• Write a rationale for selecting this data picture in your worksheet.

13

If the data does not support the collective perception, regroup to reach new consensus.

Problem StatementA problem statement should:

• Be concise and focus on only one problem. • Answer the 5 W’s (Who, what, where, when, why)• Be one or two sentences long.• Include the size or magnitude of the problem• Include the negative points of the current situation and its

importance• Not suggest a solution.

14Source: https://www.sheffield.ac.uk/polopoly_fs/1.440722!/file/HowtoWriteaProblemStatement.pdf

Problem Statement-Sample 1

15

.Problem Statement

The percentage of ELA proficient & advanced EcD

elementary students in grades 3-5 is persistently low over two years in all schools, while non-EcD students is persistently

higher, revealing an unchanging significant gap.Rationale:

Across all elementary schools, the difference between EcD and Non-EcD students in ELA achievement is very large, negatively impacting about 50% of our total student population.

Problem Statement-Sample 2

16

Rationale:Less than 50% of all students in grades 3-8 were proficient & advanced in the Forward exam; furthermore the achievement percentages shift to fewer students at proficient or above grade by grade, until the lowest achievement is in grade 8.

Problem StatementMath proficiency is very

low--less than 50%-- for all students in grades 3-8 on the Wisconsin Forward Exam in 2016-17, indicating that our

students will not be college & career ready.

Problem Statement

Given your agreed-upon critical data picture and rationale, write a problem statement for your school.

17 18

Situation Appraisal

Writing a HypothesisHYPOTHESES OF

PRACTICES Are clearly written and actionable

Focus on alterable educator practices

Are likely root causes for the problem statement

Are actionable in immediate and long-term

Are interconnected

Are locally generated and owned by the team/staff

POOR HYPOTHESES Are vaguely written

Are phrases or topics

Blame others/point fingers

Focus on unalterable factors

Contain random acts

Focus on a “product/program or tool” instead of system or implementation

Not like this ... …more like this.

19

Hypotheses about Root CausesGuidelines

20

1. Keep critical student data in mind.

2. Pose hypotheses about alterable factors, rather than unalterable factors, such as student characteristics.

Posing Hypotheses of Practices Based on the critical data problem, follow this process to pose hypotheses of practice.

Hypotheses must be:● About alterable professional practices● Likely contributors to the critical data finding● Not focused on characteristics of children, families or

unalterable factors● Captured from all team members● Recorded

21

Situational AppraisalSituation Appraisal. As a team reflect about the priority critical

problem indicated above and write hypothesis of practice. Directions:1. Study the priority data finding and the underlying data for that

finding.2. Reflect silently on your own to think about why this is occurring.

Focus on issues within your control -- adult professional practices. These practices are your hypotheses of practice (potential root causes for the data findings).

3. Jot down your ideas about hypotheses of practices until the time is up--one idea per post-it note.

4. Three minutes will be allowed for this activity.

22

Situational Appraisal, continued5. Now, in round robin fashion, each team member states one

hypothesis until all ideas have been shared. There is no discussion at this point.

6. Next, as a team, view and discuss the list. At this time, items may be collapsed together to clarify.

7. Write each thought as a Hypothesis of Practice (HOP) using the “We believe …” sentence stem as detailed in the table on the worksheet. HOPs should be written so that all stakeholders fully understand what you mean by the hypothesis as a potential root cause.

23

Break

24

25

Categorizing Hypotheses of PracticeFishbone

Categorizing Practices

• Categories for practices help us visualize the areas of work for major focus.

• There are many frameworks that can help us organize our potential root causes.

• Today, we are providing six categories of educator practices that will help you frame your HOPs.

• We will use a “fishbone”process to organize ourHOPs - potential root causes.

26

LINK for blank Fishbone

Fishbone CategorizationDirections: 1. Find the poster for your fishbone.2. Write your problem statement at the “head” of the fish.3. Discuss each HOP, and collaboratively,

determine which of the six categories isappropriate for that practice. (if you havecategories that you prefer,please modify your fishbone)

4. Place the HOP post-it in that category.5. Repeat these steps until all post-its have been categorized

collaboratively.6. Reflect on the HOPs in the fishbone to ensure that they connect to

critical data picture and problem statement as potential root causes.7. Team members may add HOPs as desired to reach team consensus.8. Transfer the problem statement and these HOPs statements into the

worksheet table to document your fishbone categorization. 27 28

29

Prioritizing Hypotheses of Practice

Winnowing the List

30

The list of HOPs on your fishbone is a collection of many ideas. The team needs to discuss and provide some judgement around this categorized list of ideas, choosing which ideas are most likely potential root causes for the student data problem.Directions:● Individual voting: Each person gets 5 votes to select the root causes

on your fishbone that they think have the highest likelihood of producing the desired student outcome (use a marker to mark their dots on the fishbone post-its). They may place as many of their 5 votes on any one root cause.

● Collaboratively, select the HOPS with the most number of votes received. You can use a maximum of 10 HOPS to move forward.

In your worksheet, BOLD or HIGHLIGHT these (10) HOPS.

Everyone does their own thing

Math proficiency is very low--less than 50%-- for all students in grades 3-8 on the Wisconsin Forward Exam in 2016-17, indicating that our

students will not be college & career ready.

We don’t know how to handle behavior problems

Our resources are not aligned to standards

CurriculumStandards

Organizational Structures

Climate & Culture

Family & Community

InstructionalStrategies

Strategic Assessment

We don’t know how to engage parents to follow up with students homework

We changed our schedule and have shorter math periods.

Staff turnover We’ve had a lot of change in staff

We only use STAR data 3 times a year and don’t do anything with it

We’ve had these results for so many years, we feel overwhelmed

We don’t really teach to the standards

During staff meetings, we focus on irrelevant stuff, and not teaching

We waste our time during PLC time with district requirements

Lack of differentiation in the classroom

When we teach the lower kids, it slows the whole class down and we can’t meet expectations

We adopted new materials without good training

We don’t know what the students are being taught in RtI (is it matched to our curriculum?)

We always have to teach to “catch” students up from the year before because they’re lacking skills

Winnowing the List

31

Some teachers are using their own materials in addition to the adopted texts, and away from the text.

Pull out programming isn’t the same as universal

Circle of Hypotheses

32

• Place up to 10 hypothesis statements with dots in a circle on chart paper, as shown.

• Assign random numbers to these hypotheses. Write the number directly on the post-it.

Everyone does their own thing

3

We don’t really teach to the standards

7

Some teachers are using their own materials in addition to the adopted texts, and away from the text.

6

We always have to teach to “catch” students up from the year before because they’re lacking skills

1

When we teach the lower kids, it slows the whole class down and we can’t meet expectations

8

We adopted new materials without good training

2

Our resources are not aligned to standards

4

We changed our schedule and have shorter math periods.

9

Lack of differentiation in the classroom

5

Finding Relationships

• Study the relationship between each pair systematically and ask:

1) Does a relationship exist?

2) Which potential root cause has the greatest influence over the other?

• Draw a line between any two that have a relationship.

33

Everyone does their own thing

3

We don’t really teach to the standards

7

Some teachers are using their own materials in addition to the adopted texts, and away from the text.

6

We always have to teach to “catch” students up from the year before because they’re lacking skills

1

When we teach the lower kids, it slows the whole class down and we can’t meet expectations

8

We adopted new materials without good training

2

Our resources are not aligned to standards

4

We changed our schedule and have shorter math periods.

9

Lack of differentiation in the classroom

5

Direction of Influence

34

● Put an arrowhead on the end of the line which points to the influenced item.

● Cautions: There can never be a two-headed arrow. i.e. the arrow can only move in one direction.

● Count the number of arrows going into each item (ins) and number of arrows stemming from each item (outs).

● Place these numbers in parenthesis by each item.

Everyone does their own thing

3

Counting “Ins and Outs”

35

Everyone does their own thing

3

We don’t really teach to the standards

7

Some teachers are using their own materials in addition to the adopted texts, and away from the text.

6

We always have to teach to “catch” students up from the year before because they’re lacking skills

1

When we teach the lower kids, it slows the whole class down and we can’t meet expectations

8

We adopted new materials without good training

2

Our resources are not aligned to standards

4

We changed our schedule and have shorter math periods.

9In:Out1:2

In:Out2:0

In:Out2:0

In:Out1:1

In:Out1:0

In:Out1:1

In:Out0:4

In:Out1:1

In:Out1:1

Lack of differentiation in the classroom

5

Take a Photo & Record Hypotheses

36

Snap a photo of your Influencer Circle and import it into your worksheet!

Recording Results

Documenting the results of the influencer activity will help with communicating this process with stakeholders.Directions:1. Record each of the potential root causes from your circle in Column A of the

table in your worksheet. 2. Beginning with root cause you numbered “1”, find all the post-its (other root

causes) that this root cause is pointing to. 3. Record the numbers of those root causes (that the arrows are pointing to) in

column B.

37

Describing the Practice

38

Describing the Practice

Directions:1. Write the hypothesis statement you have identified with the most influence

(from page 2) in the table below.2. Name the practice from the prioritized hypotheses.3. Develop a Theory of Action about this practice. Shift your hypothesis

statement to an action-oriented statement. “We believe we can improve this (current problem) if we (do what)” Add details that help inform the rationale for the HOP(S).

4. Describe the instructional or leadership practice in general terms. What would teachers be doing if this practice? What would leaders be doing?

39

Next Steps

The next step will be to complete a practice inquiry for the hypothesis of practice you have determined to be a potential root cause for the student data problem identified.To get started, it is recommended that you would:1. Investigate research to provide a rationale for this practice

indicating its contribution to solving the critical problem.2. Collaboratively develop a description

of this practice at its expected level for teachers and for leaders.

3. Discuss how these steps can inform an investigation into the prioritized practice in your school.

40

Additional Resources

• The tools and resources we are sharing with you today were developed by the WISExplore Team.

• These will eventually be available to everyone in WISElearn

• Each CESA has trained WISEcoaches to help you continue the work you’ve started today, or to lead you through the complete data inquiry process from start to finish.

41

Wrap Up & Feedback Form

42

Please take a few minutes to complete this feedback form to help us get better.