C ontinuous Q uality I mprovement

Continuous Quality Improvement

Michelle P. [email protected]

1Departments of Education and Public Welfare | www.education.state.pa.us | www.dpw.state.pa.us

MIECHV Grantee Meeting

6/5/2013

Session Agenda1. Introduction to Quality Improvement2. Pennsylvania’s MIECHV CQI Plan3. CQI Team4. Culture of Quality5. CQI Plan6. Model for Improvement

2

1. Introduction to Quality Improvement

2. Pennsylvania’s MIECHV CQI Plan3. CQI Team4. Culture of Quality5. CQI Plan6. Model for Improvement

3

Brainstorm:What is CQI?

4

• Quality improvement comes from a desire to make things better

• At first glance, quality improvement might seem like more work piled on to an already impossible work load

• But if quality improvement can ultimately lessen that workload, and allow you to provide more efficient/better home visits for your families, leading to better outcomes, isn’t it worth it?

5

The BIG picture

6

Quality Assurance Quality Improvement

Motivation Measuring compliance with standards

Continuously improving processes to meet

standards

Means Inspection PreventionAttitude Required

DefensiveChosen

ProactiveFocus Outliers: "bad apples"

IndividualsProcessesSystems

Scope Agency Overall Client CareResponsibility

Few All

Quality Improvement Quality Assurance

Quality assurance (QA) measures compliance against certain necessary standards. QA can be reactive, retrospective, policing, and in many ways punitive. It often involves determining who is at fault after something goes wrong.

However, standards and measures developed for quality assurance can inform the quality improvement process. 7

Shifting Away fromQuality Assurance

o a process of continuous improvemento involves both prospective and retrospective reviewso improvement -- measuring where you are, and figuring out ways to make things bettero create systems to prevent errors from happeningo improving how things worko trying to find where the “defect” in the system iso figuring out new ways to do thingso “think outside the box”

8

So, what is quality improvement?

1. Introduction to Quality Improvement2. Pennsylvania’s MIECHV CQI Plan3. CQI Team4. Culture of Quality5. CQI Plan6. Model for Improvement

9

An OpportunityComprehensive, multi-level CQI is a federal grant requirement and a commitment Pennsylvania has made in its MIECHV CQI Plan

. . . but more importantly, it is an opportunity:• To refine and share best practices;• To enhance decision-making;• To individualize services for participating

families across sites and programs10

A Shared Responsibility• CQI leverages the expertise and

perspective of project participants across roles, levels, and agencies

• To work well, CQI needs to be a safe space for constructive input from all

• It can’t just be something the state, the programs, or the “data people” are doing alone

• Community input is vital• Communication is key – challenges and

successes need to be freely shared 11

Pennsylvania’s MIECHV CQI Plan

• 3 teams

• Bottom-up approach

12

MIECHV State CQI Team

MIECHV Program CQI Team

MIECHV Local CQI Teams

Survey Results30 different agencies implementing MIECHV programs25 agencies responded to the survey21 have a CQI plan, CQI lead and/or a CQI team3 have a CQI lead and/or a CQI team, but no CQI plan1 does not have a CQI lead/team or a plan

13

Discussion:Share current CQI processes

14


15

Initial Activities (of all Teams)

• learn about CQI• define the role and function of a CQI team• determine the frequency of CQI team meetings

and activities• identify the people/positions responsible for

elements of the CQI plan• provide oversight of the CQI Plan• define quality for the MIECHV program• identify goals and objectives

16

Team Membersshould include the

following:• Lead – oversees the team; facilitates

the meetings• Scribe – captures all documentation• Member – provides information and

helps make good decisions• Technical Expert – subject matter

experts17

MIECHV State CQI Team• Provides an opportunity to address

statewide issues and consolidate information and issues from all other teams.

• Meetings will be held quarterly in-person.

18

19

1 Members are optional attendees at meetings.2 Denotes subcontractors.


Possible issues to be addressed are:• Monitor data collected across programs• Discuss unresolved issues from Program and Local Teams• Ongoing comprehensive program evaluation• Monitors comprehensive program model fidelity• Communicate shared successes and resolved issues• Report on aggregated data• Assist in the development of the PA MIECHV State CQI plan

Through tracking the output and process indicators related to the constructs, the MIECHV State CQI Team will be able to identify specific agencies that require additional support. CQI activities will focus on ensuring that state goals are met. Data will be analyzed for progression with enrollment, provider training, frequency of home visits, and data entry compliance, in addition to monitoring whether one site or model is doing better than others on certain benchmarks and why, including identification of best practices and lessons learned.

20


MIECHV Program CQI Team• Provides an opportunity to address

issues that impact the MIECHV programs as a whole and any unresolved issues presented by the MIECHV Local CQI Teams.

• Meetings will be held every other month via webinar/phone (in-person when possible) .

21

22

MIECHV Program CQI Team

MIECHV Program CQI TeamPossible issues to be addressed by the MIECHV Program CQI Team

are:• Discussion of common issues (and shared successes) across

agencies and models• Discussion of unresolved issues brought up from MIECHV Local CQI

Teams• Discussion of data• Discussion of model fidelity• Review and provide feedback to State on the use of assessment

tools, program questionnaires, and implementation of the MIECHV program

The overall goal of the MIECHV Program CQI Team is to continuously improve the CQI process so that the MIECHV Local CQI Teams have fewer challenges and greater success in optimally serving participants. Members of the MIECHV Program CQI Team will work directly with the MIECHV Local CQI Teams. 23

MIECHV Local CQI Teams• Provide an opportunity to address

issues that impact individual MIECHV agencies.

• Meetings will be held monthly in-person.

24

Discussion:Local CQI Team Members

25

MIECHV Local CQI Teams

26

Program Director Home Visiting Supervisor

Data Entry Staff Home Visitors

All Support Staff

MIECHV Local CQI TeamsThe MIECHV Local CQI Teams will be required to:• Have a CQI plan, which will be revised, at a minimum, annually• Focus on agency level data• Monitor model fidelity at the agency level• Determine areas of improvement• Address quality improvement challenges (i.e. staff retention,

participant attrition)• Identify issues with program evaluation(s), communication,

information systems, hiring, safety, etc. Issues that are not resolved can be brought to the MIECHV Program CQI Team. The EDUCATIONAL RESEARCH ASSOCIATE 2 and STATISTICAL ANALYST 1 will be tasked with regular dialogue with local sites around CQI.

27


28

Components of a Culture of Quality

• Generate buy-in at all levels: movement from an ‘us’ to a ‘we’

• Understanding the system: process steps that lead to the outcome

• Data collected that are relevant and meaningful

• Utilization of data to monitor progress towards established target

• Small-scale tests of interventions29

Understanding of Current Culture

• Mission, vision and values support culture

• Determination of where we are today vs. where we want to be

• Consistent and frequent methods of communication

• Lines of accountability• Expectations for performance• Processes and procedures

30

31

Culture of

Quality

Attitude

Data

Transparency

Outcomes

Commitment

Agencies strive to reach pre-determined targets. Every situation can be a

learning experience. Data are essential to

day-to-day work.

Data collection infrastructure exists.

Reporting metrics are meaningful. Data

elements are relevant, accurate, and

important. Reports are timely and recent.

Reports show performance relative to

targets. Reports show changes over time.

Reports break out small units.

All members are committed. Key members are trained in QI methods and support strategic initiatives. Frontline practitioners are supported by leadership. Improvement is guided by team approach and accountability. Data are readily accessible.

Practices and methods are shared. Results and outcomes

are shared. Individual contributions are recognized.

Open communication and critical scrutiny are welcomed. Shared learning decreases the

need to ‘reinvent the wheel.’

Outcomes are clearly articulated and understood. Outcomes are measured against an established target. All process measures are tied to outcomes. There is a deep understanding of systems, processes, and procedures. Performance is reported frequently.

Organizational Culture of Quality Checklist

TransparentCurious and eager to learnOpen to failure as a learning

opportunityCommitted to improving processes

and outcomesData-driven with sufficient MIS

capacityTrained from top to bottom in CQI

methods32

MIECHV CQI Process Map 1: Data Collection Process

MIECHV CQI Process Map 1 provides an overview of the data collection process, from client intake to data extracts from PA MIECHV and NFP ETO data systems.

Client enrolledIntakeData collected

by home visitors

Data brought back to local

MIECHV agency

Data entered into PA MIECHV

& NFP ETO data systems

Data extracts from PA

MIECHV & NFP ETO data

systems

MIECHV CQI Process Map 2: Data Sharing ProcessOnce all data are received , the STATISTICAL ANALYST 1 will analyze the data and create trend reports. These reports will be shared each month with the MIECHV Local CQI Teams. After reviewing the data at the agency level, the MIECHV Local CQI/Data lead will work with the STATISTICAL ANALYST 1 to improve data quality. The STATISTICAL ANALYST will then upload all data into an Excel spreadsheet to perform data clean- up while observing for outliers and continuing to follow-up as needed with sites for missing / questionable data. The INFANT TODDLER SPECIALISTS and Model TAs will also follow-up with sites on any program issues/TA needs. The STATISTICAL ANALYST 1 will conduct data analysis and create reports for the MIECHV Local CQI Teams; this will enable the local agencies to see how data quality can affect results, as well as to see their final results.

Data extracts from PA MIECHV (pulled on 10th of each month)

and NFP ETO data systems (pulled on 17th of each month)

Preliminary data analysis by STATISTICAL ANALYST 1

Share data with MIECHV Local CQI Teams (last Thursday of the

month)

Follow up with MIECHV Local CQI Teams re: data issues

(STATISTICAL ANALYST 1)

Final data review, clean up, and analysis by STATISTICAL ANALYST

1

MIECHV CQI Process Map 2: Data Sharing Process

Reports will also be shared with the MIECHV Program CQI team (every other month), MIECHV State CQI team (quarterly), Home Visitation Stakeholder Committee (quarterly), and the Early Learning Council (twice a year). While reports for the MIECHV Program and State CQI teams will include data that is identified at the agency level, the other two teams will only see data that is de-identified. The goal of sharing data with the Home Visitation Stakeholder Committee and the Early Learning Council is to provide them information about the children and families being served through MIECHV as well as Pennsylvania’s progress towards meeting the benchmarks.

Share data with MIECHV Program CQI Team (every other month)

Data review and discussion with MIECHV State CQI Team (quarterly)


36

Discussion:CQI plan

37


38

Model for Improvement2 parts:(1) THREE fundamental questions

– What are we trying to accomplish? Set an aim.

– How will we know that a change is an improvement? Establish measures.

– What changes can we make that will result in improvement? Take action.

– We know that improvements require change, however not all changes are an improvement. We have to test whether a change is an improvement, and that’s where the PDSA cycle comes in. 39

Model for Improvement2 parts:(2) Plan-Do-Study-Act (PDSA) cycle

– Plan a change. Clarify the objective, make a prediction, figure out the who, what, where, when, and how.

– Do (Test) the change. Carry out the plan, document any unexpected problems or challenges, begin analyzing data.

– Study (Review) the tests. Complete data analysis, compare results to theory and prediction, summarize and present data.

– Take Action based on what has been learned.

– If the change did not work, go through the cycle again with a different change. If the change was successful, use what has been learned to begin planning new improvements.

40

How Do Tests Lead to Improvements?

41

Implementation of change

Wide-scale tests of change

Follow-up tests

Very small scale test

Changes that result in improvement

The cycles build on each other…

Things to Remember• Keep it small – design the first test

for one home visitor, 5 clients• Scale down the time frame (weeks,

days, hours)• Keep the first tests simple• See what happens, act on that

knowledge, and then scale-up the test

• And remember You’ll never make a change if you don’t… actually make a change.

42

43

Pizza Delivery Example.pdf

44

Time to Practice

45

Survey ResultsTop eight ranked areas of improvement (in order from the highest to the lowest ranked):

1. retention of families2. improvements in family

self-sufficiency/stability3. families are connected to needed services and

social supports4. recruitment of families5. increase in screenings for domestic violence6. improved parenting skills7. community partnerships8. prevention of injuries and maltreatment

46

Next Steps Identify agency-level CQI team

members Share current CQI plan Plan initial agency-level CQI meeting Identify initial measures and areas

of focus Seek technical assistance

47

Documents

C ontinuous Q uality I mprovement