Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Effects of Payment for Performance on
Knowledge, Practice and the Know-Do Gap
Evidence from Pwani,
Tanzania
Josephine Borghi
24th NovemberRBF – a health systems perspective. White Sands Hotel, Dar es Salaam.
Rationale
• P4P is expected to improve quality and service coverage through changed health worker behaviour
• Improvements in health worker knowledge and practice is necessary for optimal health gain
• Knowledge may increase through substitution of health workers; greater investment in training
• Practice (or application of knowledge) is likely to increase through a desire to meet targets, and improved resource availability
• In Rwanda found limited effect on knowledge and improvements in practice, especially among those with higher knowledge levels.
• Aim: examine P4P effects on knowledge, practice and
the gap in Tanzania using data from our evaluation in
Pwani
P4P in Tanzania
Aim: A pilot introduced in 2011 focusing on MCH service coverage to inform a national programme
Location: Pwani region of Tanzania
Implementers: MOHSW and CHAI
Funder: Government of Norway
Scheme Design
Facility level targets:– ANC: IPT2; % HIV+ women on ART – Institutional delivery rate – % of newborns with OPV0 in first 2 weeks– % infants with Penta 3; measles vaccine– % of PNC visit w/n 7 days – CYP – HMIS reports correctly filled and submitted on
time + use of partogramsDistrict – regional level targets:
– % of maternal/perinatal deaths audited on time – % of facilities with stock outs
Study Design
• Design: Controlled before and after study design
– 7 intervention districts
– 4 neighbouring control districts
– Comparable poverty, literacy, rate of institutional deliveries, IMR, pop. per health facility, no. of children < 1 yr
• Timing:
-Baseline in January-February 2012
-Endline in March-April 2013 (13 months)
7 P4P districts 4 districts with no P4P
150 health facilities, 75 in each arm incl.
6 hospitals16 health centres53 dispensaries
1 facility survey at each facility
20 interviews with women who delivered
in past 12 months, from the catchment area of
each facility
Only include facilities eligible for first cycle payment
1-2 health workers at each
facility
Measurement:
Knowledge
• Used a clinical “vignette”: a hypothetical patient case, in this case, a woman attending her first antenatal visit.
• Derived from the World Bank Impact Evaluation Toolkit
• Presented to health workers who regularly provide ANC
• 45 items from the antenatal clinical guidelines were covered in the vignettes, with items corresponding to four dimensions: – patient medical history
– physical examinations
– laboratory investigations
– drugs prescriptions.
• Measure scores for each dimension and in total: number of items mentioned by the total number of items.
Measurement: Practice
• Procedures performed by the provider on patients (adherence to protocol).
• Household interviews with women attending ANC during their current or last pregnancy living within the catchment area of facilities where health workers were surveyed
• A total of 18 items regarding ANC services, 11 match the 45 items in the health workers survey
• Dimensions of care: – physical examinations
– laboratory investigations
– drugs prescription.
• We constructed an additional dimension relating to client counselling and educational services.
Measurement: Gap
• The knowledge–practice gap measures provider efficiency to translate knowledge into actual ANC practice.
• Defined as the difference between the knowledge and practice share of clinical guidelines for an ANC visit.
• Women linked to a given facility by its catchment area are matched to the health workers’ responses for that same facility.
• Take average value for knowledge measure at facility level where more than one health worker was surveyed per facility.
• Measured for 11 items across:– physical examinations
– laboratory investigations
– drugs prescribed.
Analysis
• Used a difference-in-difference identification strategy:
• 𝐾ℎ𝑗𝑡 = 𝛽0 + 𝛽1(𝑃4𝑃𝑗 × 𝛿𝑡) + 𝛽2𝛿𝑡 + 𝛽3𝑍ℎ𝑗𝑡 + 𝛾𝑗 + 휀ℎ𝑗𝑡• 𝑌𝑖𝑗𝑡 = 𝛽0 + 𝛽1(𝑃4𝑃𝑗 × 𝛿𝑡) + 𝛽2𝛿𝑡 + 𝛽3𝑍𝑗𝑡 + 𝛽4𝑋𝑖𝑗𝑡 + 𝛾𝑗 + 휀𝑖𝑗𝑡
• 𝐾ℎ𝑗𝑡 knowlegde share of ANC guidelines by provider h at health facility j in period t
• 𝑌𝑖𝑗𝑡 is the practice share or the gap
• 𝑃4𝑃𝑗 is a dummy variable taking the value of 0 for comparison facilities and 1 for intervention facilities;
• 𝛿𝑡 is a year fixed effects dummy taking the value of 0 at baseline and 1 at endline;
• 𝑍ℎ𝑗𝑡 health worker-level characteristics expected to drive programme outcomes
• 𝑋𝑖𝑗𝑡 are household level characteristics expected to drive programme outcomes
• 𝛾𝑗 is a facility fixed effects to control for facility-level time invariant characteristics; and 휀ℎ𝑗𝑡 is a random error term
• Assumption: pre-trends in outcomes are parallel
Impact on ANC Knowledge
Variables Baseline Impact
P4P Control Diff % effect of P4P
Knowledge shares for each dimension
Medical history taking (% of items known) 26 items
20.2 34.1 -13.9*** 12.5***
Physical examinations (%) 10 items
25.5 42.7 -17.2*** 11.9***
Lab investigations (%) 7 items 26.7 48.4 -21.6*** 18.5***
Drug prescriptions (%) 2 items 63.8 87.1 -23.3*** 16.2***
Total items known (%) 45 items 24.3 40.6 -16.3*** 13.4***
Total items known – gap (%) 11 items
42.0 64.6 -22.7***17.6***
Impact on ANC Practice
Variables Baseline Impact
P4P Control Diff % effect of P4P
Practice shares for each dimension
Client Counselling (% of items done) 7 items
78.6 71.8 6.8*** -3.2*
Physical examinations (%) 6 items 89.4 87.7 1.7 0.2
Lab investigations (%) 3 items 86.8 83.2 3.6* 2.2
Drug prescriptions (%) 2 items 71.4 73.6 -2.2 7.1**
Total items done (%) 18 items 82.6 79.1 3.5*** 0.0
Total items done – gap (%) 11 items
85.3 83.9 1.52.0*
Impact on ANC Know-Do Gap
Variables Baseline Impact
P4P Control Diff % effect of P4P
Gap shares for each dimension
Physical examinations (%) 6 items -54.4 -34.7 -19.7*** 9.2*
Lab investigations (%) 3 items -43.1 -6 -28.0*** 18.6***
Drug prescriptions (%) 2 items -2.5 14.9 -17.5*** 2.8
Total items (%) 11 items -39.1 -17.9 -21.2*** 10.1*
Conclusions
• P4P significantly improved health worker knowledge across all dimensions
• Plausible?• District managers shifted efficient workers to help those struggling to meet
targets.
• Opportunities to upgrade skills with training increased as a result of P4P from health worker survey.
• Knowledge indication of ‘intended behaviour’ – may be more responsive to P4P in the short term than practice
• P4P improves practice in relation to incentivised components of care: drug prescriptions but no evidence of other improvements in adherence to care guidelines
• As knowledge increases and practice generally doesn’t –
the inefficiency gap increases
Limitations
• Imbalance in baseline knowledge between intervention and control
• Unable to assess whether trends in knowledge and practice were parallel prior to P4P; trends in ANC coverage were
• Gap analysis:– Tools not originally intended to pursue this gap analysis – and only a limited
number of items could be compared in this way
• Consider the one-two health workers interviewed as representative of ‘practice’ at a given facility which may not be the case
• Assume households went to their nearest facility for ANC
• Concern that practice found to exceed knowledge at baseline (negative gap)
Acknowledgements
• Josephine Borghi – LSHTM
• Paola Vargas – LSHTM/OPM
• Peter Binyaruka - LSHTM
• Powell-Jackson T - LSHTM
• Patouillard E - LSHTM
• Torsvik G – CMI
• Mayumana I – IHI
• Masuma Mamdani - IHI
• Lange S - CMI
• Maestad O - CMI