Measurement and Efforts to Improve Quality Gayle Martin and Zelalem Debebe June 30,2015 1

1

Measurement and Efforts to Improve Quality

Gayle Martin and Zelalem DebebeJune 30,2015

www.worldbank.org/SDI www.SDIndicators.org

2

Measurement and Efforts to Improve Quality

Gayle Martin and Zelalem DebebeJune 30,2015

www.worldbank.org/SDI www.SDIndicators.org

Outline

• Large inter- and intra-country variation in key measures of quality

• Effectiveness requires simultaneous availability: in the same place at the same time

• What can be done to improve these measures of quality?

• Without focused effort, improvements in process quality will lag structural quality

3

Large inter- and intra-country variation in key measures of quality:Diagnostic accuracyAdherence to clinical guidelines Treatment accuracy

Malaria with anemiaDiarrhea with severe dehydrationPneumoniaDiabetesTuberculosis

4

Large inter- and intra-country variation in key measures of quality:Diagnostic accuracyAdherence to clinical guidelines Treatment accuracy

5

Effectiveness requires simultaneous availability: in the same place at the same time

6

% facilities with competent providers(46%)

% facilities with steth,+sphyg+sterilizing equip(78%)

Effectiveness requires simultaneous availability: in the same place at the same time

7

Even when conditions are correctly diagnosed they are not necessarily correctly treated

8

Even when conditions are correctly diagnosed they are not necessarily correctly treated

Simulation results: Respiratory Infection cases correctly diagnosed and treated

15.2%

# children with ARI

# children who sought care

# cases correctly

diagnosed

# cases correctly

diagnosed and

treated

364,950

259,114

123,338

55,632

15.2%

10

Simulation results: Respiratory Infection cases correctly diagnosed and treated

Sources: UNPOP for Population data. SDI for provider competence estimates. DHS for prevalence and utilization data.

Kenya Nigeria Uganda Tanzania

512,392 560,120

973,840

364,950

112,33728,967

164,54955,632

# children with ARI

# children who sought care

# cases correctly diagnosed

# cases correctly diagnosed and treated

21.9% 15.2%16.9%5.2%

% of ARI cases correctly diagnosed and treated

11

What can be done to improve these measures of quality?• Decompose clinical guidelines and identify

the relative effect of the components on the probability of successful diagnosis*

• Adherence to history taking and physical examination guidelines increases the probability of diagnostic accuracy by 54 % points

• Estimates are higher for providers with lower medical training (nurses): 36 % points versus 59% points• suggesting a higher return to adhering to

clinical guidelines among nurses compared to doctors and clinical officers

Dep var: Probability of a correct diagnosis

AllDoctors/clinical officers

Nurses

History taking and physical examination

0.540***(0.153)

0.361***(0.085)

0.591***(0.204)

Laboratory test 0.466***(0.063)

0.326***(0.087)

0.540***(0.089)

Observations 3145 1390 1740

R-squared 0.419 0.465 0.416

Number of providers 629 278 348

* We estimate a provider fixed effects model that enables to minimize unobserved heterogeneity due to unobserved time-invariant characteristics (such as innate abilities, intrinsic motivation etc.).

12

Without focused effort, improvements in process quality will lag structural quality

13

Looking to the future …

Providers with weakest skills are located in areas with highest mortality

Service Delivery Indicators