1 Design for Improvement: Aims, Indicators & Evidence Diana Chamrad Senior Improvement Advisor, USA Jemimah Owande Improvement Advisor, Kenya Flora Nyagawa

1

Design for Improvement: Aims, Indicators & Evidence

Diana ChamradSenior Improvement Advisor, USA

Jemimah OwandeImprovement Advisor, Kenya

Flora NyagawaImprovement Advisor, Tanzania

USAID Applying Science to Strengthen and Improve Systems

Outline of the session

• Improvement 101– What is Quality?– What is Quality Improvement?– Principles of Improvement– Systems Thinking– A model for improvement and concepts to help you improve

• Design for Improvement – Group Design– Improvement Aim– Evidence Based Interventions– System Level– Developmental Level of Child– Barriers to Achieving Aims– Change Ideas– Indicators

2


How do you define quality?

3


• How would you define Quality of Care?

• Think about the last time you accessed a service. It could be a service related to health, food, having your car repaired. What made it a quality service?

4


Dimensions of Quality

5

Institute of Medicine Others

Equitable Appropriateness

Safe Accessible

Timely Affordable

Effective Technically sound

Efficient Continuous

Patient Centered Participatory

Sustainable

Compassionate


Definition of Quality

• So simply put,

Quality is how well a product or service matches the expectations and needs of a consumer.

6


• Quality Assurance – purpose is to assess when we have achieved the intended quality of service or product (standards) – yes/no identification.

• Quality Improvement – purpose is to move the system from current state of performance to a new state of performance as defined by our use of Quality

7


Quality Improvement

• The techniques and methods used to take us from where we are, to where want to be

• Identifies where gaps exist between services actually provided and expectations for services

• TESTING AND CHANGE to reduce the gaps

8


Principles of Designing Strong Improvement Work based on Best Evidence

• Improvement work starts with defining clear, measurable improvement aims to achieve an outcome-oriented goal

• Improvement aims are based on evidence–based best practices (content)

• A minimum number of indicators are defined to measure whether improvement is occurring (or not) to achieve the aim

• Change ideas describe specific actions that may help overcome local obstacles to reliably implement the evidence-based best practices to achieve an aim

• Aim is Client-centered: quality is about needs of client, not needs of provider

9


Focus on “Systems Thinking”

10


Complex Dynamic Systems

• Step 1 – Everyone stand up

• Step 2 – Without speaking; pick two people but don’t say who they are or point at them (Keep it a secret)

• Step 3 - Move to be equidistant from both of the people

11


Complex Dynamic Systems Exercise

• What was the aim of this system?• What realizations did you have about the

systems?• What were the interdependencies among

members of the system?• Why did it take so long to accomplish the aim?• How many elements of the system changed

when one person moved?• Can you illustrate a similar experience with in

your organization or organizations you work with?

12


Why go to the trouble of developing this understanding?

“If each part of a system, considered separately, is made to operate as efficiently as possible, the system as a whole will not operate as effectively as possible.”

Russell L Ackoff

13


“Every system is perfectly designed to achieve the results it gets”

» Paul Batalden

14


The Model for Improvement

15

What are we trying toaccomplish?

How will we know that a change is an improvement?

What change can we make that will result in improvement?

Act Plan

Study Do

Langley, et al.


Question 1: What are we trying to accomplish? What is our improvement aim?

• Aim is based on analyses of gaps in service delivery. Use process maps, fishbone analyses, etc.

• Locally determined• Aim is SMART – Specific, Measurable, Achievable,

Relevant, Timely• Focus more narrowly on what is in your power to

change• Don’t try to tackle a really big aim all at once—

rather, break your problems into a series of discrete aims that build on each other

16


Question 2: How will we know a change is an improvement?

• “No data, no improvement”—Measurement is critical to tracking change

• You may need to capture data you don’t normally collect, such as data about a specific step in a process, or information from a specific group of people

• Continually measure, looking at data from each test, to see if things get better—see if your aim is accomplished

17


Question 3: What changes can we make that will result in an improvement?

This is the creative part, where it helps to break out of your usual thought processSources of ideas for changes to test:

Identifying underlying challenges (root cause analysis) People who providing the service Clients Others who do it well Guidelines, literature Creativity methods Change ideas/concepts

18


Five Approaches to Improvement

• Design of a new service or product• Redesign of an existing service or product• Design of a new process• Redesign of an existing process• Improvement of the system as a whole

19


Change concepts

20


Improvement Design

• Through the 3 questions we posed, now we know:– what we want to accomplish (our

improvement aim) – what changes we could make that would

potentially move us towards our aim– and how we’ll know if the change results

in improvement (measurement)

Now we need to test the changes to see if they really lead to improvementPLAN DO STUDY ACT

21

USAID Applying Science to Strengthen and Improve Systems22

P-D-S-A


What makes this cycle work

• Testing on a small scale allows you to know whether your idea can work—gives you a way to try out your ideas to improve the system before deciding to implement

• Allows you to know quickly whether your change will work or not and even to try out ways to make it work better

• Gather data to convince stakeholders that the change will work and get their participation

• The more knowledge one has about how the system under study works, the better the prediction

• Comparing predictions to results is a key source of learning

23


Designing Improvement Work: Aims, Indicators and Content

24



25




Act Plan

Study Do

Langley, et al.


Improvement Design Template

26

Improvement Aim:

System Level or

Care Phase

Child Age

Evidence-based Interventions

Barriers to

achieving aim

Change Ideas

Indicators Coverage (C),

Process (P), Output (OP). Outcome (O)



27

Improvement Aim:

System Level or

Care Phase

Child Age


Barriers to

achieving aim

Change Ideas



USAID Applying Science to Strengthen and Improve Systems28


Developing an Improvement Aim

Start with a simple statement of what we are wanting to accomplish.

Improve school performance of vulnerable children.

29


Aim Statement Checklist

What outcome will improve? More children will pass end of term exams

When will it improve? Over next 12 months

How much will it improve? From 25% to 50%

For whom will it improve? Children aged 0 to 18 years in Nkata District

What tool, method, resource, or system will be used to make the change?Support and strengthen mother’s groups.

30


Aim Statement Checklist

What outcome will improve? When will it improve? How much will it improve? For whom will it improve? What tool, method, resource, or system will be used to make the

change?

Over the next 12 months, we will support and strengthen mother’s groups to increase the percentage of children aged 0 to 18 years living in Nkata District who pass end of term exams from 25% to 50%.

Is the aim SMART? Specific, Measurable, Achievable, Relevant, Timely?

31



32

Improvement Aim: Over the next 12 months, we will support and strengthen mother’s groups to increase the percentage of children aged 0 to 18 years living in Nkata District who pass end of term exams from 25% to 50%.

System Level or

Care Phase

Child Age


Barriers to

achieving aim

Change Ideas




Template

33


System Level or

Care Phase

Child Age


Barriers to

achieving aim

Change Ideas

Indicators Coverage (C), Process (P),

Output (OP). Outcome (O)

Use mother groups to encourage children, especially girls to attend school and study.Mother groups counsel on topics: (1) importance of education for children, especially girls (2) benefits of education (3) need to ensure that domestic chores or selling at market doesn’t prevent them from attending school (4) economic benefits of children staying in school

S. Krishnaratne, H.White and E.Carpenter (2013) Quality education for all, what works in developing countries, working Paper 20. New Delhi: International Initiative for Impact



34


System Level or

Care Phase

Child Age


Barriers to

achieving aim

Change Ideas



Community 0-18 years

Use mother groups to encourage girl-children to attend school and study



35


System Level or

Care Phase

Child Age


Barriers to achieving

aim

Change Ideas






Common Barriers to Delivery of Quality Care

• Weak local leadership• Lack of formal standards• Inaccessible services--financially, geographically, and

culturally• Inadequate inputs/infrastructure• Poorly organized patient care processes• Non-functioning referral/counter-referral mechanisms• Inadequate supervision/lack of continuous training• Poor staff motivation• Attrition of staff• Poor provider competence

36



37


System Level or

Care Phase

Child Age

Evidence-based

Interventions

Barriers to

achieving aim

Change Ideas





Group members not adequately trained.Delays in follow up of children



38


System Level or

Care Phase

Child Age

Evidence-based

Interventions

Barriers to

achieving aim

Change Ideas

Indicators Coverage (C), Process (P),

Output (OP). Outcome (O )



Group members not adequately trained.

Delays in follow up of children


Change concepts

39



40


System Level or

Care Phase

Child Age

Evidence-based

Interventions

Barriers to

achieving aim

Change IdeasIndicators Coverage

(C), Process (P), Output

(OP). Outcome (O)





Develop/conduct training Show community Malawi statistics about increased income that comes with educationSchool notifies member on Wednesday and Friday if child missed school. Member follows up next day.



41


System Level or

Care Phase

Child Age

Evidence-

based Interventions

Barriers to

achieving aim

Change Ideas







Develop/conduct training Show community Malawi statistics about increased income that comes with educationSchool notifies member on Wednesday and Friday if child missed school. Member follows up next day


Types of Indicators

42



43


System Level or

Care Phase

Child Age

Evidence-

based Interventions

Barriers to

achieving aim

Change Ideas

Indicators

Coverage (C), Process (P), Output (OP). Outcome (O)





Develop/conduct training Show community Malawi statistics about increased income that comes with educationSchool notifies member on Wednesday and Friday if child missed school. Member follows up next day

(C ) # children reached by mother’s groups(OP) # members trained(P) % children followed timely(O) % children passing mid term exams


How measurement should work:

• Should be linked to aims• Should be used to guide improvement and test

changes• Should be integrated into daily routine.• Allows for continuous learning for everyone – from

successes and failures• Should concentrate on key measures—doesn’t

overwhelm people with endless data collection and analysis!

44



45




Act Plan

Study Do

Langley, et al.



46

Improvement Aim:

System Level or

Care Phase

Child Age


Barriers to

achieving aim

Change Ideas

Indicators (Coverage, Process,

or Outcome)


Recommended Readings

Associates in Process Improvement (2007). The Improvement Handbook: Model, Methods and Tools for Improvement. Austin, TX.

DiPrete, B.L. (2008). Quality Programs for Orphans and Vulnerable Children: A Facilitator’s Guide to Establishing Service Standards. Published by Pact Inc. and University Research Co., LLC, for the United States Agency for International Development.

Langley G., Nolan K., Nolan T., Norman C., Provost L. (1996, 2009) The Improvement Guide: A practical approach to enhancing organizational performance. Jossey-Bass Publishers: San Francisco.

Lloyd, R (2004) Quality Health Care; A guide to developing and using indicators. Jones and Bartlett Publishers

Massoud, R., Askov, K., Reinke, J., Franco, L.M., Bornstein, T., Knebel, E., and MacAulay, C. (2001). A Modern Paradigm for Improving Healthcare Quality. QI Monograph Series 1 (1) Bethesda, MD: Published for the U.S. Agency for International Development (USAID) by the Quality Assurance Program.

WebsitesUSAID Applying Science to Strengthen and Improve Systems – www.usaidassist.org

Institute for Healthcare Improvement – www.ihi.org

47

http://www.hciproject.org/

http://www.ihi.org/


Thank you!

[email protected]@urc-chs.com

48

mailto:[email protected]

mailto:[email protected]

Documents

1 Design for Improvement: Aims, Indicators & Evidence Diana Chamrad Senior Improvement Advisor, USA Jemimah Owande Improvement Advisor, Kenya Flora Nyagawa