Strengthening Quality of Heath

Services in RBF Projects

Dinesh Nair, Ronald Mutasa,

Rianna MohammedPatron Mafaune,

Kathleen Hill

Presentation Outline

Session objectives and overview

Integrating Quality into RBF projects

Quality in global health

Principles/approaches to improving quality

Integrating quality into RBF projects

Prioritizing health conditions

Selecting standards and defining quality measures

Measuring and verifying quality

Aligning RBF with QI & Health System Strengthening efforts

Country Examples: Liberia & Zimbabwe

Session Objectives

Understand importance of quality and gain

familiarity with improvement approaches

Understand common operational needs and

strategies of RBF projects that incentivize quality

Understand experience to date in RBF projects that

incentivize quality

4

The Issue of Quality in Health Care

High quality care is……

Effective: Adherent with evidence-based

standards

Safe: does not harm patients

Client centered: Respectful of patient

needs, values & preferences

Equitable: Does not vary in quality

because of personal characteristics

(gender, ethnicity, SES, etc)

IOM, 2001, Crossing the Quality Chasm

Preliminary Results from Service Delivery Indicator

(SDI) Survey: Nigeria

Sample of 1,172 health facilities in 6 states (one from each

geo-political zone) end 2013 – not yet public

Diagnostic accuracy less than 40% (judged by vignettes) -

less than half that observed in Kenya

Adherence to guidelines < 1/3rd (by vignettes)

Knowledge of maternal & neonatal complications <

20%, <40% of what it is in Kenya

Closing the Know-Do-Gap Between Proven Best Practices and Actual

Practice in the Places Patients Receive Care

6

Conceptualizing Quality

(Source: Donabedian)

7

Principles Underlying Improvement

Effective teamwork

Understanding how processes of care function within a system and critical bottlenecks

Use of data to continuously measure

and track progress

Understanding and

focus on patient needs

Regular shared learning

for rapid dissemination

best practices

8

Prioritizing What to Improve

Deciding How to Measure if Care Is Improving

9

Choosing improvement

Aims: “What are we trying to

accomplish?”

Defining quality measures:

“How will we know if a change

is an improvement?”

10

Improvement Team Reviewing Quality Indicators

Measuring Quality to Improve Care:

Who Needs What Data for What Purpose?

Facility Staff (managers, providers, staff -hospitals and clinics):

-Need quality measures to assess and continuously improve services.

Is care improving?

Regional/District & Program Managers:

-Need measures to assess and continuously strengthen essential system functions (e.g. competent workforce).

Are essential system functions performing to standard?

--------

Clients (users of care)

National Policy-makers (value, policy)

Global Stakeholders (leadership, advocacy, accountability)

Integrating Quality into RBF Projects: Prioritizing

Health Conditions/Services for Improvement

Focus on:

High-burden conditions in local context (leading causes of mortality and morbidity)

High-burden conditions for which there is strong evidence of effective health care interventions (preventive and curative)

Country government priorities: involve local decision makers and experts

Consider phasing improvement priorities: “impossible to improve everything at once”

Involve local and international experts to:

Review country standards against global evidence: evidence is constantly changing

Distill standards into minimum “intervention bundles”: focus attention on essential high-impact interventions

Illustrative quality of care process measures based on minimum standards:

% cases adherent with standards – “all or nothing adherence” (e.g. % PPH cases managed per minimum standard; % cases pediatric pneumonia treated per standard)

Average % adherence with minimum standards (e.g. average % adherence with newborn sepsis case-management standards; N=30 cases)

Integrating Quality into RBF Projects: Selecting Standards and

Defining Quality of Care Measures

Ilustrative quality measure: Quality of Partogram

Completion (not so simple!)

Quality Measure Operational Definition

% partograms in last quarter

completed per standard

NUMERATOR: Number partograms

documenting cervical dilation, maternal BP,

pulse, temperature at admission and at least

every 4 hours until delivery

DENOMINATOR: Total number of

partograms reviewed

Defining Quality of Care Measures (continued)

Review quality measures being used at country and global level; adapt or develop new measures

Consider including measures of care coordination and performance of essential system functions (e.g. referral/counter-referral; supportive supervision)

Streamline and harmonize indicators: the fewer the better!

Improving adherence with post-partum care best practices: Average

compliance with PNC standards

Herat Province Afghanistan- Nine Health Centers; 2009-2011

16

Measuring Quality: The challenges…..

JAMA Nov 13. 2013

“Quality measurement is in rapid

flux….despite the challenges of a

rapidly expanding number of

quality measures, much of health

care remains poorly measured or

unmeasured.”

Measurement Methods

Observation

Patient interviews & questionnaires (e.g. exit interview; household survey)

Death (and near-miss) audits

Simulations (provider competence)

Provider knowledge/problem-solving (e.g. vignettes/case studies)

Routine information systems (e.g. facility chart & register audits; routine

health information system)

-Regular measurement is a core principle of all improvement, but measurement

alone will not improve care!

18

Common Measurement Challenges

for QI and RBF efforts

Routine HMIS in low-resource settings include few (if any) quality/content indicators

Primary data sources (medical records/registers) often don’t include necessary data

for constructing quality measures

Quality of clinical procedures (e.g. newborn resuscitation) cannot be assessed from a

medical record; creative measurement approaches needed

Few routine indicators of performance of essential system functions (e.g. %

maternities in district with functional neonatal bag & mask at bedside)

Weak staff data management capacity (providers and managers)

Routine Versus Complications Care

Easier to measure routine best practices relevant for every patient: often simple intervention for which a “box” can be checked in a standard record (e.g. ENC, AMTSL; immunization)

More difficult to measure quality of complications care Timely accurate diagnosis

Stabilization and successful timely referral (primary facility)

Prompt and ongoing treatment/monitoring (hospital)

Discharge planning and follow up

20

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

J08 F08 M08 A08 M08 J08 J08 A08 S08 O08 N08 D08

Percentage of pre-eclampsia and eclampsia case management standards achieved

Jan-Dec 2008, average of 120 cases analyzed per monthBased on detection in 120,000 patient contacts at 31 MOH facilities

Measuring Adherence with

Pre-eclampsia/eclampsia Best Practices to Improve

PE/E Care in Niger 31 facilities

21

Changes implemented: •Systematic screening at every contact •Organization daily tasks to prevent stock outs •Standardized emergency case management eclampsia •Standardized referral protocols

Pre-eclampsia/Eclampsia chart audit tool for

primary facility

22

Charts

Evaluation

1

2

3

4

5

1. Blood pressure (BP) recorded

1. Gestational age (GA) recorded (per one of criteria indicated in GUIDE)

1. Urine protein quantified (dipstick +, ++, +++)

1. Danger signs assessed (see chart review guide)

Diagnosis pre-eclampsia or eclampsia recorded if

criteria met 1. DBP > 90 and at least 2+ proteinuria pre-eclampsia (+ seizure if

eclampsia)

First Treatment and referral if primary facility

1. 4 gm loading dose of MgSO4 IV ; monitor for toxicity (reflexes, urine output, respirations)

1. If GA < 34 weeks administer antenatal corticosteroids

1. Transfer with IV access (and provider if possible)

23

Despite challenges it is possible to track quality measures from

(modified) local records (e.g. add columns to registers; stamps to

records/partograms, etc.)

WHO MNCH Quality of Care Consultation Geneva

December 2013: Objectives

Share global and regional experiences in the assessment

and improvement of facility MNCH QoC

Review/agree on a core and supplementary set of global

indicators for accountability - monitoring and reporting on

facility MNCH QoC

Review assessment tools, methodologies and processes

used in measuring facility MNCH QoC

WHO Consultation Draft Set Global Newborn Indicators:

Mix of Structural, Quality of Care Process, Outcome indicators

Proportion of health facilities with maternity services that have functional bag & masks (2

neonatal mask sizes) in the delivery areas

Proportion of newborns who received all four elements of essential newborn care:

immediate and thorough drying

immediate skin-to-skin contact

delayed cord clamping

initiation of breastfeeding in the first hour

Proportion health facilities where Kangaroo Mother Care is operational, by level of facility

Facility neonatal mortality rate disaggregated by birth weight: >4000 g, 2500-3999 g, 2000-

2499 g, 1500-1999 g, < 1500 g

Proportion of health facilities offering maternity services that have BFHI certification and

recertification not older than two years

Aligning RBF with QI and HSS Efforts: Challenges

and Opportunities

RBF projects can be integrated with and/or incentivize QI and HSS efforts

National QI strategy implementation

Pre and in-service training & performance-based supervision

Supply chain interventions

HMIS improvement interventions, etc.

RBF projects can incentivize QI activities and quality performance measures

Regular QI team meetings

District/facility routine collection & analysis of quality measures with action plans

District/facility continuous quality improvement (CQI): setting aims; developing and tracking quality measures; testing changes, sharing learning….

Accreditation, etc.

Country Case Example:

Zimbabwe

Dr. Patron Mafouna,

Provincial Medical Director,

Maniacaland Province

Mr. Ronald Mutasa,

World Bank

• Focus on register completion (“# columns checked”) rather than

processes of clinical care (specific clinical interventions)

• Heavy emphasis on “structural quality” (e.g. appearance of the

facility, medications, supplies, staffing)

• Poorly defined clinical quality indicators (e.g. numerator not

clearly defined), inter-rater reliability issues

• Limited analysis and monitoring of individual checklist items

• Limited link between incentives and clinical quality of care

Zimbabwe Experience With Original Checklist Quality

Items (18 districts; Hospitals and Clinics)

Improving Quality Measurement Over Time:

Illustrative example “Partogram completion” indicator

RBF Project

Phase Indicator

Checklist Guidelines for calculating

indicator

Initial Phase

% partograms

correctly completed

in last quarter

No criteria specified weak quality

measure

Current

Phase

% partograms

correctly completed

in last quarter

Randomly review 10 partograms from last

quarter Calculate % partograms documenting

specific criteria: FHR, cervical dilation, BP, pulse,

temperature documented at admission and at

least every 4 hours until delivery

Beyond Structural Care: Phasing quality incentives for high-burden

MNCH Conditions by project phase

RBF Phase One (9-12 months) RBF Phase Two

Routine MNH/FP

PPH

Obstructed labor

Maternal & Newborn Sepsis

Pediatric pneumonia

Pediatric diarrhea

Pediatric malaria

All Phase One priority conditions

plus:*

Pre-eclampsia/Eclampsia

Newborn Asphyxia

Miscarriage/abortion c

management

Management acute

malnutrition

Zimbabwe: Beyond structural incentives

Innovations to introduce continuous quality improvement and rigorously evaluate

P4Q alone versus

P4Q plus continuous quality improvement

Supporting MOHCC to draft and implement national/regional QI strategy…..align RBF with strategy

Supporting MOHCC to introduce quality indicators into HMIS…..strengthening HMIS

Zimbabwe: Lessons being learned

Technical capacity & substance matters in P4Q, so does political

economy & building ownership by local stakeholders

Complex trade-off between structural and clinical care indicators

Health providers respond to P4Q indicators and incentives

Source documents at provider level matter

Focus on QI principles and emerging global best practices

Country Case Study: Liberia

Rianna Mohammed,

World Bank

• High infection rates and post-surgery

complications

• No systematic use of clinical guidelines

• Accreditation scores on quality are worse

than primary facilities

Low Quality of care at Hospitals:

3

4

Hospitals characterized by:

• Poor infrastructure; inadequate

supply of drugs and equipment.

• Insufficient numbers productive,

responsive, competent staff

• Long waiting times

• Limited internal and external

financing (e.g. 85% of health

expenditure in 2009/10 directed

almost entirely at the PHC-level).

LIBERIA: Neglect of Hospitals...poor quality of care

ALIGNING PBF AND SYSTEM STRENGTHENING: LIBERIA PROJECT SEEKS TO

IMPROVE QUALITY VIA PBF AND COMPETENCY BUILDING

35

Hospital PBF Competency Building

Incentives for:

a) improved quality of care

(i.e. adherence to clinical

protocols)

b) quantity of services

delivered (including in-

service training to health

workers)

• Coaching and verification

support

Better competency to

improve processes of

care

• Support to development of

innovative Graduate

Medical Residency

Program.

• In-service training of

nurses, midwives and PAs

Synergistic

linkage of

components

Improved staff motivation

QUALITY ASSESSMENT/ MONITORING TOOLS

36

1Complicated and assisted delivery

(including C-section)

Any labor that is made more difficult or complex by a deviation from the normal

procedure. Complicated delivery is defined as: assisted vaginal deliveries (vacuum

extraction or forceps), C-section, episiotomy and other procedures.

17

2 Normal deliveries of at risk referralsHigh-risk pregnant women referred by health center to the hospital but delivered

normally. A high-risk pregnancy is defined as: evidence of edema, mal presentation,

increased BP, multi-parity, etc.

17

3Counter referral slips returned to health

facilities

Hospital returns counter referrals letter with feedback on the referred patient to the

referring health center. The counter referral letter is completed in triplicate, with one

also given to the patient, and one retained by the hospital.

2.5

4Newborn referred for emergency

neonatal care treatment and treated

Newborns referred for emergency neonatal care due to: perinatal complications, low

birth weight, congenital malformation, asphyxia, etc.5

6Referred infants and under-fives with

feverAny surgical procedure that does not involve anesthesia or respiratory assistance. 2.5

7 Minor surgical intervention

Any surgery in which the patient must be put under general spinal/anesthesia and

given respiratory assistance. Major surgery in the case of this package of services is

defined as any of the following: Herniarraphy, Appendectomy, Myomectomy,

Sleenectomy, Salpingectomy, Hysterectomy, Thyrodectomy, Mastectomy.

5

8Major surgery (excluding CS, including

major trauma)

Patients transferred from a lower-level facility (health center or health clinic) to the

hospital for emergency treatment. 18

9 Patients transported by ambulance 2.5

10

Number of training sessions held by

faculty for nurses, midwifes and PA

according to in-service curriculum and

defined protocols.

These indicators will incentivize the in-service training activities. 50

11

Number of nurses, midwifes and PAs

that received specialized in-service

training, relevant to benchmarks

10

Verified

Total

EarningsDefinition

Six Hospitals Total

Fee (USD)Indicators Claimed

(c) Quantity Checklist

Actual % Earned Points

1. Obstructed Labor 0.80 3.87 100% 33% 1.29

2. Hemorrhage 1.00 4.84 100% 71% 3.45

3. Maternal Sepsis 1.00 4.84 100% 50% 2.42

4. Eclampsia 0.70 3.39 100% 47% 1.59

5. Neonatal Asphyxia 1.00 4.84 100% 67% 3.23

6. Neonatal Sepsis 1.00 4.84 100% 54% 2.61

7. Prematurity 0.50 2.42 100% 47% 1.14

8. Maternal Newborn Best Practices 1.00 4.84 100% 54% 2.61

9. ETAT 1.00 4.84 100% 33% 1.61

10. Malaria 1.00 4.84 100% 71% 3.45

11. Pneumonia 1.00 4.84 100% 50% 2.42

12. Acute Diarrhea 0.80 3.87 100% 47% 1.82

13. Severe Acute Malnutrition 0.60 2.90 100% 67% 1.94

14. Surgical Safety 1.00 4.84 100% 54% 2.61

100% 60.00 100% 53% 32.20Total/Average

Childbirth:

Maternal-Newborn

Pediatric

(in-patient care)

Surgical Care

Quarter IIII. Process of Care

Detailed Score Checklists

Weight (by

importance)

Point

AllocationMax %

(b) Process of Care

Quality Checklists

Score

1.GENERAL MANAGEMENT (30pt)

2. HUMAN RESOURCES FOR HEALTH (16pt)

3. HYGIENE AND MEDICAL WASTE DISPOSAL (27pt)

4. DRUGS MANAGEMENT (30 pt)

5. EQUIPMENT AND SUPPLIES (84pt)

TOTAL %

Date of Verfication

TOTAL (187pt)

REPUBLIC OF LIBERIAMinistry of Health and Social Welfare (MOHSW)

Hospital Quarterly Quality Assessment

Name of the Hospital

Name of Team Leader of Quality Verification

Verification Period

Quarterly Quality Verification Score

I. Management

II. Structural

(a) Management and

Structural Checklist

Indicators

Max Points

Actual Points

Quarter I

1. General Management 30 2.6

2. Human Resources for Health 16 9

3. Hygiene and Medical Waste Disposal 27 0

4. Drugs Management 30 8

5. Equipment and Supplies 84 48

6. Aggregated Process of Care Score 60 32

Total 247 100

Total Percentage 100% 40%

Total Quality Bonuses (USD) 159,678 64,517

PBF Bonus

Calculation Tool

Business/Operation Plan

Health Worker Bonus

Allocation LHSSP Indices Tool for Bonus Allocation to Individual Health Workers for Hospitals

1 200 50 30 300,000 0 6,944

2 200 70 30 420,000 0 9,722

3 150 80 30 360,000 0 8,333

4 - - -

5 - - -

6 - - -

7 - - -

8 - - -

9 - - -

10 - - -

11 - - -

12 - - -

Quarter:

Total PBF Incentives Earned

% for Individual Bonus

Attendance

points [C]

Hospital Name

Total Individual Bonus

Redemption Hospital

July-Sept 2013

No Name of staffStaff

category

Monthly

salary [A]

Perfor-

mance

points [B]

$50,000

50%

$25,000

Total points =

[A] x [B] x [C]

Indices of

the period

PBF

individual

bonus

Signature of receipt

Min

50%

Max

50%

~60

%

~20%

~20% (1)

Continuous

monitori

ng

(d) Impact Evaluation

\

Quality of Care Checklists for High-burden MNCH Conditions (complications):

Based on National and WHO Standards

Process of Care Checklists

Childbirth: Maternal-Newborn

1. Obstructed Labor 2. Hemorrhage 3. Maternal Sepsis 4. Eclampsia 5. Neonatal Asphyxia 6. Neonatal Sepsis 7. Prematurity

Pediatric (in-patient care)

8. Maternal Newborn Best Practices 9. ETAT 10. Malaria 11. Pneumonia 12. Acute Diarrhea 13. Severe Acute Malnutrition

Surgical Care 14. Surgical Safety 37

38

Chart review elements (see chart review guide for specific criteria) ; each element if

recorded = 1 point

Charts

1. Admission 1 2 3 4 5

1. Cervical dilation recorded at admission (# of cm)

2. Contraction frequency and duration charted at admission

3. Fetal presentation charted at admission

4. Partograph started when cervical dilation 4 cm or greater

Admission Score (x/4)

2. Labor Monitoring (partograph)

1. Cervical dilation recorded at least every 4 hours

2. Frequency and duration contractions recorded at least every 30 minutes

3. Fetal HR recorded at least every 30 minutes

Labor Monitoring Score (x/3)

Standards for Management Obstructed Labor: Illustrative Checklist

Distilling Essential care Items (admission, labor)

Each item has chart review guide that

defines criteria

Five patient charts reviewed:

average score (% adherence best

practices) links with bonus

VERIFYING QUALITY MEASURES: PATIENT CHARTS REVIEWED BY AN

INDEPENDENT COUNCIL

39

Step 1: Find relevant patients

from register (e.g. “Malaria”

patient for Malaria checklist)

Step 2: Record names and

patient numbers

Step 3: Request health workers

to bring charts

• Team of minimum 2 verifiers

from Liberia Medical and

Dental Council (LMDC)

compare charts and

guides/checklists

• 5 charts each of 14 checklists

reviewed quarterly

• Total scores calculated

as % adherence for

each checklist

• Aggregated points are

tied to the performance

bonus

INCENTIVIZING QI ACTIONS AS PART OF HOSPITAL PERFORMANCE:

MANAGEMENT CHECKLIST

40

1.1 Performance management structure operational

• Hospital Board meets every month and problem-solve issues

• Senior Management Team (SMT) meets every month and problem-solve issues

• Quality Improvement (QI) or other relevant Team meets at least every month and problem-

solve issues

1.2 Business (operational) plan updated and implemented

1.3 Performance review

1.4 Reporting and filing of the key data

1.5 Financial Management

1.6 Community Involvement

1.7 Grievance mechanism

8pt

3pt

6pt

2pt

4pt

4pt

3pt

Assessment Components Max

points

INTEGRATING QI ACTIONS AND RBF: QI TEAM AND COACH MOTIVATES HEALTH WORKERS TO

IMPROVE QUALITY THROUGH TRACKING QUALITY MEASURES AND COACHING

41

• Weekly/daily chart review

• Monthly management and

structural checklist review

1 Self-Assessment 2

15 20 30

40

0

50

W1 W2 W3 W4

%

e.g., Sepsis

• Post scores on a wall

• Provide detailed feedback to

staff

Tracking and Feedback

3

• Support improvement activities (e.g., standardize chart,

waste disposal, cleaning)

• Training on the treatment protocol and chart writing

Support to Improvement QI Activities

Multiple levels of performance management at hospital and MOHSW

levels: Incentivizing QI Actions

42

Hospital

QI team

• Responsible for improving specific quality indicators

• Carry out weekly/monthly self-quality assessments and track results at

each ward

• Provide feedback, coach and train health workers

Senior

management

Team

• Develop business plan with supervisors

• Hold supervisors accountable for quality indicators

• (At least ) monthly performance review with QI team & support of

QI team action plans

Hospital

Board

• Provide oversight to hospital management

• Make SMT accountable for results and implementation

of business plan

• Community involvement and grievance

Technical

Committee • Overall oversight of performance trends and

project implementation

Key lessons learned so far

• Measuring quality can be really complicated – finding a right

focus and starting from small is critical (e.g., pre-pilot, five

hospitals only, inpatient only).

• Management strengthening is essential but not easy to do – further

experiments and hands-on coaching will be important.

• Training and re-training on quality measurement is essential in

ensuring that all stakeholders understand how quality will be

measured, and trust the tools and the verifiers.

• Understanding the political economy, and being open and flexible

to change is important in building ownership among stakeholders.

43

THANK YOU