Cost-Effectiveness Analysis of Results-Based Financing in
Zimbabwe and Zambia
Donald S Shepard, PhDWu Zeng, MD, PhD
Brandeis University, Waltham, MA
Nov 3, 2016, World Bank Headquarters, Washington, DC
Schema for cost-effectiveness analysis (CEA) for RBF programs
Cost Effects on coverage Effects on quality
Administrator’s costOther donors’ costProvider’s cost(User’s cost)
Household surveyFacility surveyHMIS dataQuality score card
Incremental cost Incremental lives saved, DALYs or QALYs
Incremental cost-effectiveness ratio (ICER)
Inputs
Intermediate results
Component outcomes
Cost-effectiveness outcomes
Costs Financial costs Government and donor
perspective Effectiveness—coverage
Impact evaluation with districts compared
Lives Saved Tool (LiST) software
Literature and country data Effectiveness--quality
Facility surveys Exit interviews Expert opinion (Delphi
panel)
ToolkitWeb: http://documents.worldbank.org/curated/en/2015/09/25069701/cost-effectiveness-analysis-results-based-financing-programs-toolkit
Evaluate cost-effectiveness by the ICER, the price of one unit of good health. The lower the better!
Incremental cost-effectiveness ratio (ICER)
𝐼𝐶𝐸𝑅=𝐷𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒𝑖𝑛𝑐𝑜𝑠𝑡𝑠
𝐷𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑖𝑛h h𝑒𝑎𝑙𝑡 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠Numerator: Added costs of RBF (difference in costs between
RBF and control districts)
Denominator: Added effectiveness or health outcomes (difference in health outcomes between RBF and controls), often expressed as quality-adjusted life years, QALYs
Both quantity (coverage) and quality contribute
CEA of RBF in Zimbabwe
Annual operating costs of RBF program
RBF subsidy payments, $7,045,211
Staff costs, $1,434,096
General administration*, $699,311
Capacity building (meetings, workshops, training)*, $426,644
HQ support costs*, $372,845
Transport costs, $299,412
Capital items for Cordaid*, $298,745
Supplies / equipment for facilities*, $128,209
Total, $10,704,473
$0.00 $0.50 $1.00 $1.50 $2.00 $2.50 $3.00 $3.50
$2.04
$0.41
$0.20
$0.12
$0.11
$0.09
$0.09
$0.04
$3.09
$2.04
$0.41
$0.10
$0.06
$0.05
$0.09
$0.04
$0.02
$2.82
Mature per capita costs
Current per capita costs
Per capita annual operating cost
The number after each category on the left is the current aggregate annual cost for the inter-
vention districts (population 3.46 million). For categories marked with asterisks, half of the
current costs were considered start-up expenses and would be reduced in a mature program. The labels on the right are per capita costs
Aggregate costs from Nat Pharm (US$)*
Group
Pre-period (Jan 2011-Mar 2012)
Post-period (Apr 2012-Jun 2014)
Spending /year (Pre)
Spending/ year (Post) Net difference
Popu-lation
Spending /capita
Control
6,771,163
33,466,940
5,416,930 14,874,196 2,229,897 $6.67
Intervention
6,062,025
29,478,515
4,849,620 13,101,562 3,461,010 $3.79
AdjustedIntervention 13,316,434 -1.63% 3,461,010 $3.85
Difference $0.06
*Source: Nat Pharm data base of drugs distributed representing 92,000 orders to 354 control and 359 intervention customers. Due to the substantial change in expenditure per year, we computed the difference in differences based on the ratios of aggregate expenditure. We calculated the pre-period ratio of intervention to control (0.8953). We computed the “adjusted intervention” by applying this ratio to the control spending/year (post), and computed the net difference by comparing actual and adjusted intervention values.
Financial net costs of current RBF per capita per year (USD)
Cost components Cost SubtotalIncremental RBF operational costs $3.09
Incremental costs at World Bank headquarters $0.10 Subtotal $3.19Net costs of consumables from Nat Pharm -$0.06Less Health Transition Fund payments to control districts (no administrative costs included)
-$0.81
Subtotal -$0.87Net cost $2.32
Quantity of care Institutional delivery: 13.4% Postpartum care: 13.3%
Quality of care
Impact of RBF in quantity and quality of services
Calculated lives and QALYs saved for selected services with RBF and projected from controls:
Quantitative results Annual number of lives saved is 772, i.e., (72+1,471)/2, Equivalent to 18,288 QALYs gained With population of 3.46 million in RBF districts, RBF generates 528 QALYs/100,000 population/year
Site visits suggested coaching strengthened RBF
Effectiveness (RBF vs. control): quality & quantity
Year RBF ControlLives saved
2012 9,705 9,705 0
2013 8,613 9,345 732
2014 8,136 8,875 739
Total 26,454 27,925 1471
Deaths in children under five and lives saved
Year RBF ControlLives
saved
2012 416 416 0
2013 370 414 44
2014 365 393 28Total 1,151 1,223 72
Maternal deaths and lives saved
Contributions of quality and quantity
Quantity (Coverage
improvement); 66%
Quality im-
provement; 34%
Relative shares of quality and quantity
0
100
200
300
400
500
600
350
178
528
Projected health impacts
QAL
Ys g
aine
d pe
r 100
,000
pop
ulat
ion
per
year
Incremental per capita costs: US $2.32QALY impacts (per 100,000 population per year)
Quantity (coverage) alone: 350 QALYsQuality and quantity (effective coverage): 528 QALYs
ICERs ($/QALY gained) Quantity (coverage) impacts alone: $2.32/0.00350 = $663 Combined coverage and quality impacts: $2.32/0.00528 = $439
Cost-effectiveness results: RBF vs. control
CEA of RBF in Zambia
Study design
Incentivized services (indicators) and unit prices
No IndicatorUnit Price
US$
1 Curative consultation 0·20
2 Institutional delivery by skilled birth attendant 6·40
3 Antenatal care (ANC) - prenatal and follow up visits 1·60
4 Postnatal care visit (PNC) 3·30
5 Full immunization of children under one year 2·30
6 Pregnant women receiving 3 doses of malaria intermittent preventive treatment (IPTp) 1·60
7 Family planning (FP) users of modern contraceptive methods 0·60
8 Pregnant women counselled and tested for HIV 1·80
9 HIV+ pregnant women given niverapine (NVP) and zidovudine (AZT) 2·00
Results
RBF vs INP RBF vs CON INP vs CON0.00
2.00
4.00
6.00
8.00
10.00
12.00
HQ costs
Program costs
MSL costsIn
crem
enta
l cos
t per
cap
ita ($
)
Program costs (RBF + input financing) and distribution of incentives
Curative consultations30.0%
Institutional deliveries 14.0%ANC
1.4%PNC6.6%
Full vaccination6.3%
Third dose of IPTp3.7%
Modern FP methods 28.5%
Pregnant women counselled and tested for HIV
9.3%
Pregnant women given NVP and AZT0.2%
Incentive payment
51.4%
Consultancy costs16.3%
Trainings6.9%
Meetings/ Workshops
2.2%
M&E0.9%
Operational costs7.6%
Equipment14.6%
Coverage and quality of key maternal and child health services at baseline and endline
ServicesBaseline Endline DIDs
RBF INP CON RBF INP CON RBF vs INP RBF vs CON
INP vs CON
Coverage of key maternal and child servicesIns Del 68·3% 56·4% 70·9% 80·8% 74·3% 71·2% -5·4% 12·2%** 17·6%***
ANC 97·5% 96·2% 96·3% 98·9% 99·0% 99·1% -1·4% -1·4% 0·0%
PNC 70·3% 56·0% 76·4% 82·4% 73·8% 80·7% -5·7% 7·8%* 13·5%***
BCG 95·6% 97·8% 97·6% 100·0% 99·5% 95·6% 2·7% 6·4%* 3·7%*
DPT 97·1% 95·2% 95·8% 98·6% 97·6% 91·8% -0·9% 5·5%* 6·4%*
HIB 82·5% 88·3% 81·8% 97·9% 88·7% 78·1% 15·0%*** 19·1%*** 4·1%
IPT 92·0% 92·4% 95·1% 98·0% 96·1% 98·1% 2·3% 3·0%** 0·7%
FP∆ 6·5% 9·9% 7·7% 34·0% 15·6% 15·7% 21·8%** 19·5%** -2·3%
Quality index of key maternal and child servicesIns Del 65·5% 66·8% 67·0% 73·5% 74·1% 71·9% 0·7% 3·1% 2·4%
ANC 66·9% 69·1% 68·6% 75·0% 77·2% 73·8% 0·0% 2·9% 2·8%
PNC 66·7% 68·4% 68·3% 74·1% 76·6% 73·4% -0·8% 2·3% 3·0%
Vaccination 78·7% 80·7% 81·7% 81·2% 80·0% 80·4% 3·2% 3·8% 0·6%
FP 77·7% 78·6% 80·6% 81·6% 77·6% 74·8% 4·9% 9·7% 4·8%
QALYs gained from the RBF program in comparison with controls
RBF vs INP RBF vs CON INP vs CON
Mid-point (lower bound-upper bound) Mid-point (lower bound-upper bound) Mid-point (lower bound-upper bound)
PopulationQALYs gained
(unadjusted for quality)
QALYs gained (adjusted for quality)
QALYs gained (unadjusted for
quality)
QALYs gained (adjusted for quality)
QALYs gained (unadjusted for
quality)
QALYs gained (adjusted for quality)
Pregnant women 237 (216-302) 302 (237-345) 475(425-539) 604(539-626) 237(176-302) 302(237-345)
Children under 5 5 088(3 733-6 015) 6 300(4 826-7 323) 11 816(10 480-13 100) 14 574(13 195-15 953) 6728(5 171-8 131) 8 274(6 704-9 843)
Total 5 325 (3 948-6 317) 6 602(5 064-7 688) 12 291(10 905-13 639) 15 178(13 734-16 579) 6 966(5 347-8 433) 8 576(6 942-10 188)
Incremental cost-effectiveness ratios
Comparison
Cost/life saved (US$) Cost/QALY gained (US$)
Mid-point (lower bound- upper bound)
Mid-point (lower bound-upper bound)
RBF vs INP (unadjusted) 39 621 (33 388 - 53 381) 1 674 (1 411 - 2 258)
RBF vs INP (quality adjusted) 31 952 (27 514 - 41 657) 1 350 (1 163 - 1 761)
RBF vs CON (unadjusted) 25 553 (23 024 - 28 767) 1 079 (972 - 1 216)
RBF vs CON (quality adjusted) 20 689 (18 945 - 22 865) 874 (800 - 966)
INP vs CON (unadjusted) 14 786 (12 211 - 19 235) 624 (515 - 813)
INP vs CON (quality adjusted) 12 280 (10 110 - 14 837) 507 (427 - 626)
Discussion
Reference: Zimbabwe’s 2012 GDP/capita was $980If ICER < GDP/capita, program highly cost-effective (WHO)ICER of current RBF
Improved coverage alone: $663 or 0.68x GDP/capita, highly cost-effective
Including quality gains: $439 or 0.45x GDP/capita, highly cost-effective
Mature RBF program would lower cost by 9.0% to $2.11 per capita
Discussion: Interpretation of RBF in Zimbabwe
Calculated average is 528 DALYs vs. mature program (704)Potential increase for mature program: 33%
Discussion: program maturity (Zimbabwe as an example)704
528
0
DALYs per100 population per
year
Phase I periodPhase I period Phase I period
Calculated averageEstimated phase in Mature program
Projected cost per capita $2.11Projected impact is 704 QALYs per 100,000 population per yearProjected ICER is $300
i.e. $2.11 / (704 / 100,000) or 0.31 x GDP/capita
Projected ICER of mature RBF program
Reference: GDP/capita $1,759 (2013)
ICER of RBF Compared to Input-financing: $1,350 or 0.77 GDP/capita, highly cost-effective Compared to pure control: $874 or 0.50 GDP/capita, highly cost-effective
ICER of input financing Compared to pure control: $507 or 0.29 GDP/capita, highly cost-effective
Comparison with Zambia RBF
Reproductive health vouchers in Uganda (African Strategies for Health, 2015)
$302 / QALY or 0.59 x GDP/capita ($510)Simulated maternal community-based health insurance in Uganda (African Strategies for Health, 2015)
$298 / QALY or 0.58 x GDP/capita ($510)RBF is among the very highly cost-effective interventions
Comparison with other maternal-child health programs
1.Use both penalties and rewardsHuman nature: people work hard to avoid penalties
2.Establish a threshold and pay only for activities above ite.g. antenatal care: pay only for incremental coverage over 90%
3.Pay for improvements over last year’s average as well as attainmente.g. Last year’s average 4; this quarter 5; improvement 1
4.Pay a fixed dollar amount for remoteness Current incentives as % of volume are too small
5.Combine RBF with more formative supervision and demand sideHelps providers learn to improve quantity and quality
Potential refinements to RBF
Ministry of Finance, Zimbabwe
Ministry of Health and Child Care, Zimbabwe
World Bank, Zimbabwe
Cordaid, Zimbabwe
Acknowledgments Ministry of Health, Zambia
World Bank, Zambia
World Bank Headquarters
Financial support World Bank Health Results
Innovation Trust Fund
Donald S Shepard, PhD
+1 781 736 3975
www.brandeis.edu/~shepard
ContactsWu Zeng, MD, PhD
+1 781 736 3888
www.brandeis.edu