The effect of an integrated multi-sector model for achieving the Millennium

Development Goals and improving child survival in rural sub-Saharan Africa: a non-

randomised controlled assessment

The Lancet 379 (9827) May 8, 2012, DOI:10.1016/S0140-6736(12)60207-4

Paul M Pronyk PhD1, Maria Muniz MPA1, Ben Nemser MPH1, Marie-Andrée Somers PhD1, Lucy

McClellan MIA1 , Cheryl A Palm1 PhD, Uyen Kim Huynh PhD1, Yanis Ben-Amor PhD1 , Belay

Begashaw PhD2, John W McArthur DPhil1,4, Amadou Niang PhD3, Sonia Ehrlich Sachs MD1, Prabhjot

Singh PhD1, Awash Teklehaimanot PhD1 , Jeffrey D Sachs PhD1 for the Millennium Villages Study

Group

1 The Earth Institute, Columbia University, New York, USA 2 MDG Centre East and Southern Africa, Nairobi, Kenya 3 MDG Centre West and Central Africa, Bamako, Mali 4 Millennium Promise, New York, USA

Millennium Villages Study Group (Columbia University): Principal Investigator: JD Sachs;

Monitoring and Evaluation: CA Palm, PM Pronyk, M Muniz, B Nemser, MA Somers, U Kim Huynh, X

An, S Kaschula, E Quintana; Agriculture: P Sanchez, C Palm, G Nziguheba; Health: SE Sachs, P Singh,

A Liu, Y Ben Amor; Health informatics: A Kanter, P Mechael; Infrastructure: V Modi, E Adkins, M

Berg; Nutrition: J Fanzo, R Remans; Malaria: A Teklehaimanot, H Teklehaimanot; P Mejia;

Economics: JW McArthur

Millennium Villages Study Group (Principal Site Investigators): MDG West and Central Africa

(Bamako): A Niang (director), B Aboubacar, M Coulibaly, S Dapaah-Ohemeng, MA Fripong, D

Guimogo, D Sangare, Y Tankoano; MDG Centre East and Southern Africa (Nairobi): B Begashaw

(director), G Denning, J Aridi, J Murugi, M Wagah, P Wambua, J Wariero; Bonsasso, Ghana: J

Mensah-Homiah, E Akosah; Dertu, Kenya: D Malaam, F Shide; Mayange, Rwanda: Donald Ndahiro, R

Felicien; Mbola, Tanazania: G Nyadzi, M Shemsanga; Mwandama, Malawi: T Mijoya; Pampaida,

Nigeria: B Yunusa, E Ojo, C Woje; Potou, Senegal: S Kandji, M Sene; Ruhiira, Uganda: D Siriri, E

Atuhairwe; Tiby, Mali: B Kaya, T Coulibaly

Corresponding author details:

Dr. Paul Pronyk

Centre for Global Health and Economic Development

The Earth Institute, Columbia University

Suite 401 Interchurch Centre, 475 Riverside Drive, New York, NY 10115

Tele: +1 212 854 0676

Mobile: + 1 917 239 8171

ppronyk@ei.columbia.edu

http://www.thelancet.com/journals/lancet/article/PII%20S0140-6736(12)60207-4/abstractmailto:ppronyk@ei.columbia.edu

The effect of an integrated multi-sector model for achieving the Millennium

Development Goals and improving child survival in rural sub-Saharan Africa: a non-

randomised controlled assessment

Abstract

Background: Simultaneously addressing multiple Millennium Development Goals (MDGs) has the

potential to complement essential health interventions to accelerate gains in child survival. The

Millennium Villages project is an integrated multi-sector approach to rural development operating

across diverse sub-Saharan African field sites. Our aim was to assess the effects of the project on

MDG-related outcomes including child mortality after 3 years of implementation and compare

these changes to local and national reference data.

Methods: Village sites averaging 35,000 people were selected from rural areas across diverse agro-

ecological zones with high baseline levels of poverty and undernutrition. Starting in 2006,

simultaneous investments were made in agriculture, the environment, business development,

education, infrastructure and health in partnership with communities and local governments at an

annual projected cost of $120 per person. We assessed MDG-related progress by monitoring

changes after 3 years of implementation across Millennium Village sites in nine countries. The

primary outcome was the mortality rate of children younger than 5 years of age. To assess

plausibility and attribution, we compared changes to reference data collected from matched

randomly selected comparison sites and national rural trends for the mortality rate of children

younger than 5 years of age from demographic and health surveys. Analyses were done on a per-

protocol basis. This trial is registered with ClinicalTrials.gov, number NCT01125618.

Findings: Baseline levels of MDG-related spending averaged $27 per capita, increasing to $116 by

year 3 of which $25 was spent on the health sector. After three years, reductions in poverty, food

insecurity, stunting and malaria parasitemia were observed across nine Millennium Village sites.

Access to improved water and sanitation increased, along with coverage for many maternal-child

health interventions. Mortality in children younger than 5 years decreased by 22% in Millennium

Village sites relative to baseline (absolute decrease 25 deaths per 1000 live births, p=0.015) and

32% relative to matched comparison sites (30 deaths per 1000 live births, p=0.033). The average

annual rate of reduction in mortality in children younger than 5 years of age was three-times faster

in Millennium Village sites than the most recent 10-year national rural trends (7.8% vs 2.6%).

Interpretation: An integrated multi-sector approach for addressing the MDGs can lead to rapid

child survival gains in rural sub-Saharan Africa.

Introduction

At the United Nations (UN) Millennium Summit in September 2000, world leaders adopted the

Millennium Declaration, committing their nations to a new global partnership to reduce extreme

poverty and address a series of time-bound health and development targets.1 Among these

Millennium Development Goals (MDG) was a pledge to reduce child mortality by two-thirds

between 1990 and 2015.

Despite the priority placed on child mortality within the MDG framework, an estimated 7.6 million

children die every year.2 While important gains have been made in several settings,3 progress in

sub-Saharan Africa has been slow with mortality rates 20 times higher than industrialized

countries and about an eighth of children dying before the age of 5 years.2

Over two thirds of child deaths are preventable through the delivery of effective and low-cost

health interventions.4 The integrated delivery of these interventions has been suggested to be

among the most effective strategies for improving child survival. 5, 6 Although several large-scale

health-sector initiatives to support these efforts have been introduced in sub-Saharan Africa, 7-9 a

number of important challenges remain. Weak and deeply under-financed health systems,10

frequent shortages of medicine and health worker shortages, absence of a supportive policy

environment,;8 an overemphasis on facility-based service provision,7 and access barriers such as

user-fees remain crucial obstacles to achieving universal coverage.11 While coverage is improving

for interventions such as vitamin A or immunisations that can be delivered through single contacts

with health services, persistent challenges remain in areas requiring ongoing engagement with

well-functioning systems, or where behavioral and social changes influence uptake – such as

appropriate infant feeding or modern contraceptive use.12, 13

Equally important, however, has been the uneven progress in addressing wider social and

economic targets articulated in the MDG framework.14 Poverty and food insecurity, low levels of

education, the absence of basic infrastructure, and persistent gender inequalities continue to

undermine gains in child survival.5, 15 While addressing these simultaneous and overlapping

vulnerabilities has theoretical appeal, the design and testing of programs that work across sectors

to achieve the MDGs has thus far been limited.

The Millennium Villages project is a 10-year initiative supporting the integrated delivery of a

package of scientifically-proven interventions with the central aim of achieving the MDGs across

diverse sub-Saharan African sites.16, 17 Local partnerships between the project, communities and

governments coordinate activities across multiple sectors including health, agriculture, the

environment, business development, education and infrastructure. Sites are based in rural areas

where MDG-related progress has been insufficient, representing a range of agro-ecological zones

with corresponding challenges to income, food production, disease ecology, infrastructure and

health system development.16 Project spending is informed by estimates from the UN Millennium

Project suggesting the MDGs can be achieved with sustained annual investments of about $120 per

person ($140, 2008 USD) across all sectors and $40 for health ($47, 2008 USD).18 We aimed to

assess progress towards the MDGs and child survival over the project’s first 3 years and compare

these changes to local and national trends.

Methods

Study design and setting

This study was conducted between 2006 and 2010 in Millennium Village sites in nine sub-Saharan

African countries (Figure 1). Sites represent contiguous villages averaging 35,000 people and were

selected on the basis of several criteria. First, all villages were so-called hunger hot-spots with at

least 20% of children under 5 years of age being malnourished.19 Second, sites were chosen to

represent the agro-ecological zones characterizing more than 90% of farming systems on the

continent. 20. Third, the project was undertaken in countries where national governments are

committed to partnering with the initiative and with the MDGs more broadly.

A core set of interventions for achieving the MDGs have been identified by the UN Millennium

Project.18 These interventions were adapted and flexibly implemented in response to local

conditions after consultation with governments and local communities.16,17 The main components

of the Millennium Village model and the sequence of interventions are shown in Figure 2.

In the health sector, basic services were often unavailable at baseline, requiring major investments

in infrastructure and staffing. Governments were core partners and remained responsible for

employing local professional staff and managing facilities and supply chains. To reduce access

barriers, free primary health care was made available at nearly all sites as even modest co-

payments can restrict access among the poorest.21 An evidence-based package of maternal-child

health interventions was introduced in line with national and WHO guidelines.

In agriculture, improved seeds and fertilizers were subsidized to support high-yielding crop

varieties alongside farmer training on best agronomic practices. Interventions in education

included upgrading buildings and classrooms, making learning materials available, recruiting

qualified teachers and providing school meals. Finally, these efforts were combined with

investments in basic infrastructure to enhance access to improved drinking water and sanitation,

upgrade local roads, promote partnerships to expand mobile-phone coverage, and improve facility

access to grid and solar electricity.

Procedures

To assess MDG-related spending, we examined financial records of the Millennium Village project,

interviewed district government representatives, valued in-kind contributions of materials and

human resources from external partners, and estimated material and labour contributions from

local communities. Non-amortized costs were generated by sector and stakeholder at baseline and

for the first 3 years of the project in eight of the nine countries. We report baseline spending

relative to year 3 which approximates an annual steady-state given low levels of disbursement in

the first project year. Costs are reported in 2008 US dollars and prices for in-kind contributions

were documented using standard imputation methods for multi-center interventions.22

To measure progress towards child mortality and MDG-related outcomes, assessment rounds were

conducted at baseline (2006-07) and after 3 years (2009-10). Within each site, intervention

delivery commenced with about 1000 households before subsequent expansion to a wider area,

representing the target population for longitudinal assessment.

A population census was conducted at baseline to establish sampling frames, after which 300

households were selected at random and proportionally sampled from strata defined by sub-

village, wealth category, and sex of household head. Sample size was determined based on the

ability to detect changes across a range of MDG outcomes, including a 40% reduction in the

mortality rate in children younger than 5 years assuming an intra-cluster correlation coefficient of

0.02 and 200 births per site. Assessments were done during pre-harvest periods. To maintain the

sample size, households lost to attrition were replaced with households from the same baseline

strata.

Local comparison village sites were introduced in the third study year to enhance the plausibility

that recorded changes were the result of intervention exposure.23 Sites were selected at random

from up to three candidates matched on village-level parameters with the potential to influence

child mortality and MDG outcomes. Efforts were made to ensure adequate distance between

Millennium Village and comparison sites to minimize spillover effects (average distance 40km). The

same sampling strategy was employed for the comparison villages, which were assessed on all

outcomes at entry into the study. Additional reference data were derived from demographic and

health surveys from participating countries, with trends in the mortality rate in children younger

than 5 years of age plotted for rural areas from 1990 through 2010.

At each assessment round a household survey was administered to gather information on

demographic characteristics, education, employment, bed net usage, land ownership, agriculture,

food security, assets and access to basic services including water, sanitation, energy, transport and

communication. An adult survey was administered to all individuals aged 15-49 years to examine

health-related MDGs, nutrition, and common causes of child mortality. A section on women’s

reproductive history provides dates of birth for all children and the survival status of each, which is

used to calculate the mortality rate in children younger than 5 years of age. Indicator definitions

were derived from standard MDG assessment guidelines.24

To assess malaria parasitemia, thick and thin peripheral blood smears were collected from eligible

participants. Smears were read by experienced microscopists in a research laboratory in Addis

Ababa using best-practice techniques.25

Anthropometric data were assessed among children younger than 5 years using standard

protocols.26 Recumbent length of children (0-24 months) was read twice to the nearest 0.1 cm on

wooden length boards or mats with sliding head blocks (Shorr Productions, Woonsocket, RI, USA).

Anthropometric indices were calculated using growth references with extreme z-scores excluded.27

At year three, greater efforts were made to ensure all children under 5 were from sampled

households were assessed, resulting in an increase in sample size for anthropometric indicators.

Survey data were collected by enumerators who underwent three weeks of field training. At each

site, the same teams oversaw data collection at baseline and year 3, as well as the enumeration of

Millennium Village and comparison village sites. Masking of enumerators to the intervention was

not feasible. Survey data were double entered using CSPro (version 3.3) and cleaned for structural

and logical errors in both CSPro and Stata (version 10).

The primary study outcome, child mortality, is expressed as the mortality rate in children younger

than 5 years of age - defined as the probability of a child born in a specified year dying before

reaching the age of 5 years subject to current age-specific mortality rates. A range of secondary

outcomes were pre-specified based on effect pathways outlined in the study protocol (Web

Appendix Table A).

Birth-related outcomes were derived from the reproductive histories of female respondents at year

3. Birth histories are used to retrospectively calculate birth-related outcomes for the period before

and after the start of the intervention for Millennium Village and comparison village sites. For the

mortality rate in children younger than 5 years of age, the pre-intervention period includes the 5-

year period before program implementation, while the post-intervention period spans the first 3

project years. For pregnancy-related outcomes, the post-intervention period included births in the

third year of implementation. All post-intervention child-related outcomes are age-constrained and

non-overlapping with the pre-intervention period. Finally, survey methods enumerated up to three

births for skilled birth attendance but only the most recent birth for antenatal and postnatal

outcomes, resulting in variability for these denominators.

Household wealth was estimated through an asset index whereby the first principal component

was extracted from eight indicators of whether or not a household owns a given asset at the time of

data collection (year 3) and 3 years prior (baseline).

All other outcomes were presented for baseline and year 3 in the Millennium Village sites, and for

Year 3 in the comparison village sites. Some outcomes – such as the nutrition indicators – are

defined for age-specific groups (ie, children under 2 years of age) to capture the effect of the

intervention on children conceived or born since the start of the intervention.

In Millennium Village sites, progress towards the MDGs is evaluated based on changes from

baseline to 3 years after program initiation. To assess changes relative to comparison village sites a

variety of strategies are employed. For birth-related outcomes, a difference-in-differences approach

was used to assess whether changes over time in Millennium Village sites were statistically greater

than comparison village sites. For all other outcomes, where comparison village baseline levels

were unavailable, effects were assessed by comparing year 3 outcomes between Millennium Village

and comparison village sites.

A multilevel regression model was used to account for the clustering of observations within sites,

and to adjust for between-group and between-period differences in the recorded characteristics of

households and individuals. The analysis adjusted for differences in the sex of the household head,

whether the household’s main livelihood strategy was farming, and whether the household head

had schooling. For birth-related outcomes, estimates were also adjusted for the mother’s age at

birth, birth order of the child, and child sex; for child outcomes, we also control for the child’s sex

and age. To maximize the number of observations in the analysis, missing values for covariates

were imputed using the dummy variable approach,28 with the percentage of cases with missing data

not exceeding 11%. The analyses are also adjusted for site pairing to account for the study design.

Logistic regression was used for binary outcomes. Two indicators -the mortality rate in children

younger than 5 years of age and survival rate to the last grade of primary education - were

estimated using a discrete time survival analysis, on the basis of probabilities of event occurrence

(death or promotion to the next grade) for different time categories.29 Significance was assessed

using a T test. Cases with missing data on the outcome measure were excluded from the analysis.

All analyses were conducted on a per-protocol basis. The outcome anti-malarial treatment for

children younger than 5 years of age was excluded as new WHO guidelines for rapid testing and

treatment at the household level invalidate questions used to construct this indicator.30 Questions

on exclusive breastfeeding, the introduction of complementary feeding, and appropriate pneumonia

treatment were not captured in our year 3 assessments. Analysis of malaria parasitemia excluded

one site (Rwanda) because of missing data. Reports adhered to the guidelines for Transparent

Reporting of Evaluations with Nonrandomized Designs (TREND).31 Statistical analyses were done in

SAS (version 9.2). Additional detail on statistical models can be found in the Webappendix pp10-19.

The study protocol is registered with ClinicalTrials.gov number NCT01125618.

Role of the funding source

The sponsor of the study had no role in study design, data collection, data analysis, data

interpretation, or writing of the report. The corresponding author had full access to all data and had

final responsibility for the decision to submit for publication.

Results

One site (Ikaram, Nigeria) in the study was lost to evaluation after being absorbed by a separate

government program (the Nigerian Conditional Grants Scheme) with nine of the original ten pairs

included in the final analysis. Response rates at year 3 were comparable between intervention and

comparison sites although more adult women were interviewed in comparison sites (2592 [78%]

of 3310 in Millennium Villages vs 2825 [87%] of 3244 in comparison villages).

Baseline levels of government, non-governmental organisation and community spending on MDG-

related activities were estimated at $27 per head (figure 3). Average annual per head spending at

project year 3 was about $116, of which $25 was spent in health – somewhat below original

projections and in-line with the $43 average heath expenditure per head for countries included in

this study (webappendix p 2). Half of spending was derived from the project, with the remainder

from local governments (30%), non-project stakeholders (14%) and local communities (6%). Major

activities in each sector are summarized in Figure 2.

No significant differences in baseline characteristics between Millennium Villages and comparison

villages were observed for village, household or individual characteristics (table 1). The mortality

rates in children younger than 5 years of age before the intervention were higher in the Millennium

Villages than in the comparison villages but confidence intervals were overlapping (table 1). Site

level differences are shown in webappendix p 3.

Within the intervention sites, 2627 (97%) households were successfully interviewed at baseline,

and 2617 (97%) in year 3. Between baseline and year three, 306 (12%) households were lost to

follow up and replaced with 298 households from the same baseline strata. An additional 77

households were replaced at random to retain a sample of about 300 households per Millennium

Village site. In total, 2617 (97%) of households were successfully interviewed at year 3

(webappendix p 8).

At follow-up, adjusted point estimates of effect for 15 of 17 indicators changed in the postulated

direction with significant differences for 13 outcomes. Reductions in household poverty, food

insecurity and stunting were reported. For child health services, there were improvements in

access to measles immunization, postnatal checks for neonates, and diarrhoea prevalence was

reduced. Large increases in coverage with skilled birth attendance and access to improved water

and sanitation were reported. For MDG 6, levels of antenatal HIV testing and bednet use improved,

and prevalence of Plasmodium falciparum was reduced from 19% to 3%. After 3 years, the

mortality rate in children younger than 5 years was reduced by 25 deaths per 1000 live births, or

22% relative to baseline (p =0.015). When comparing the 5-year period before the intervention

with the 3 years after project initiation (2002-2009), the average annual rate of reduction was

7.8%. No changes were reported in access to antenatal care, or rates of wasting in children and

underweight children younger than 2 years of age. Site-specific data shows mortality reductions in

eight of nine sites (webappendix p 7). For site specific changes in secondary outcomes, the most

consistent improvements were reported for economic and nutritional outcomes, skilled birth

attendance, bednet use, malaria, and access to improved water and sanitation (webappendix p 6).

As a sensitivity analysis, households lost to attrition in year 3 were dropped from the longitudinal

assessment; this does not appreciably affect estimates of change over time with the exception of the

stunting outcome. Unadjusted results were similar in magnitude to adjusted results (webappendix

pp 4-5).

Study outcomes in comparison villages were assessed at year 3 for 2703 (94%) of eligible

households. The intra-cluster correlation was 0.03 for the mortality rate in children younger than 5

years, and ranged from 0.01 to 0.49 for other outcomes. For 14 out of 18 outcomes, changes

occurred in the predicted direction. No significant differences were observed when comparing

poverty, anthropometric outcomes, diarrhea prevalence, measles immunization, newborn care,

antenatal care, or HIV testing in pregnancy between Millennium Village and comparison clusters.

Relative to comparison villages, significantly higher levels of food security, skilled birth attendance,

bednet utilization, and access to improved sanitation were observed. Malaria parasitemia was

lower among Millennium Village sites. Changes in access to improved water and diarrhea treatment

approached threshold levels (p-value= 0.06-0.1). Relative to comparison sites, mortality rates in

children younger than 5 years of age were reduced by 30 deaths per 1000 live births, or a 32%

relative difference (p-value=0.033). Site specific data showed reductions in all Millennium Village

sites relative to comparisons sites (webappendix p 7).

Although our assessment was not powered to assess changes in neonatal and infant mortality, the

greatest reductions were observed in the first month of life, as well as during the 6-23 month age

periods (figure 4).

Analysis of demographic and health surveys for countries included in the assessment shows that

the average annual rate of reduction for mortality in rural areas for the period 1990 - 2010 was

1.6% - with the 1991-2000 annual reduction at 0.5% increasing to 2.6% from 2001–10

(webappendix p 9).

Discussion

This assessment focused on key drivers of child mortality, where progress in sub-Saharan Africa

has been slow, and where cross-MDG synergy remains crucial. Average levels of MDG-related

spending were just $27 per person at baseline, increasing to $116 across all sectors by year 3, of

which $25 was spent in the health sector, which is in-line with current levels of per head health

expenditure for countries included in this assessment (webappendix p 2). 3 years after project

initiation, rural sites across nine sub-Saharan African countries had positive shifts in a range of

MDG-related outcomes including poverty, food security and chronic undernutrition; better

coverage with maternal-child health interventions; lower malaria parasitemia; and improved

access to water and sanitation. Child mortality was reduced relative to baseline levels relative to

matched comparison sites. Finally, the pace of mortality reduction among Millennium Village sites

was three-times greater than the most recent 10-year national rural trends (webappendix p 9)

As a complex intervention operating across many sectors, definitive statements about the specific

mechanisms of mortality reductions are not possible. However, the project placed a strong initial

health sector emphasis on so-called quick wins including optimizing immunization coverage and

bednet distribution to all sleeping sites – with concurrent reductions in malaria parasitemia. Early

efforts to enhance health staffing and facility infrastructure, reduce access barriers such as user-

fees, and cross-sectoral investments to improve roads, emergency transport, and mobile

communication played potentially important parts in improving access to skilled birth attendance.

Although our assessment was insufficiently powered to detect changes in newborn mortality,

reductions in child deaths in the first month of life are encouraging. In the agricultural sector, the

early introduction of fertilizer and improved seeds resulted in a two to three-times increase in

staple crop yields,32 potentially contributing to gains in food security and lower levels in childhood

stunting in Millennium Village sites.33 Finally, major improvements in access to safe water and

sanitation might have generated additional synergies.

Health sector challenges existed in the project’s first 3 years including procurement and supply

chain management, improving health-worker performance, and establishing community health-

worker programs. The presence of these challenges was reflected by the absence of major shifts in

health sector outcomes that characterise the continuum-of-care including diarrhea case

management, antenatal care, and postnatal checks with skilled providers. These factors probably

did not make a substantial contribution to mortality reductions in the early phase of this 10-year

project. In view of the the relatively low starting point for many sites, additional time will likely be

needed to optimise systems and fully extend the reach of services to vulnerable households.

For our assessment, we used longitudinal data from project sites in a range of real-world settings to

assess changes in intervention coverage and MDG-related outcomes. As random site selection

across multiple countries was not feasible, we used a pair-matched design alongside national

reference data to better understand causality and attribution. We opted for this design recognising

that in the context of continent-wide MDG scale-up, many of the same interventions being

introduced by the project are simultaneously being implemented by government and NGO partners,

which could potentially result in understated intervention effects.34 Notably, the consistency of

findings across diverse implementation contexts may serve to enhance generalizability, as factors

such as climate, governance, and economic shifts, which carry the potential to influence MDG-

related outcomes, are likely to vary between settings.

The study also had several limitations that are important to underscore. First, with relatively few

sites, statistical thresholds were difficult to achieve in the absence of large and consistent effect

sizes. Second, the use of historical data from year 3 to calculate preintervention baselines for some

indicators may have led to recall bias and under-reporting. As this study was undertaken similarly

in intervention and comparison groups, this bias would be evenly distributed and result in

conservative estimates of program effects. Third, for a subset of the indicators, regression-to-the

mean cannot be ruled out as a factor explaining estimated gains in the Millennium Village sites. This

would, however, not influence the mortality rate in children younger than 5 years of age, which

were based on one round of data collection. Fourth, sampled households were drawn from an initial

cluster of 1000 households within each site. While the nature and intensity of the interventions

were similar across the site, this sample may not be representative in all cases. Fifth, while political

commitment and community ownership were important prerequisites participation in the

program, we suggest that any large scale development program is unlikely to succeed in their

absence. Finally, spill-over effects between intervention and comparison sites cannot be ruled out,

which again would understate intervention effects.

In summary, early results from the Millennium Villages provides encouraging evidence that

accelerated progress towards the MDGs with reductions in child mortality can be achieved for a

modest cost even in remote rural areas of sub-Saharan Africa (panel). While persistent challenges

to child survival remain in much of the region, we suggest that integrated approaches that deliver

health-sector inputs alongside broader investments in agriculture, nutrition, environment and basic

infrastructure hold great potential. Finally, as a complex initiative with multiple simultaneous

interventions operating across a range of deeply challenging environments, considerable

opportunities for learning remain. Further research to assess the long-term effects of the

programme and improve understanding of barriers, facilitators and synergies to implementation,

and the development of methods and systems to scale-up these lessons learned will be crucial for

achieving the MDGs as 2015 approaches.

Conflicts of interest: The authors declare no conflict of interest

Funding: The research and evaluation for the Millennium Village Project were supported by the

United Nations Human Security Trust Fund, the Lenfest Foundation, Bill and Melinda Gates

Foundation, and BD (Becton Dickinson).

Contributions: CAP and PMP were responsible for study design and interpretation of data; PMP

drafted this manuscript; MM and BN contributed to study design and were responsible for data

collection and interpretation; MAS conducted the statistical analysis; UKH was responsible for

implementation research; LM was responsible for economic costing data; CAP, coordinated the

design and assessment of agricultural interventions; SES, YBA and PS were responsible for the

design of the package of health interventions and their implementation; AN and BB were

responsible for project implementation and oversight in West and East Africa respectively; JWM led

the management team of the project and contributed to the scientific design; AT oversaw the

assessment and implementation of malaria interventions, and analysis of malaria specimens; JDS

was responsible for the overall project conception, design and study oversight. All authors

contributed to the editing of this manuscript and approved the final version.

Millennium Villages Study Group (Columbia University): Principal Investigator: JD Sachs;

Monitoring and Evaluation: PM Pronyk, CA Palm, M Muniz, B Nemser, MA Somers, U Kim Huynh, X

An, S Kaschula, E Quintana; Agriculture: P Sanchez, C Palm, G Nziguheba; Health: SE Sachs, P Singh,

A Liu, Y Ben Amor; Health informatics: A Kanter, P Mechael; Infrastructure: V Modi, E Adkins, M

Berg; Nutrition: J Fanzo, R Remans; Malaria: A Teklehaimanot, H Teklehaimanot; P Mejia;

Millennium Promise: JW McArthur

Millennium Villages Study Group (Principal Site Investigators): MDG West and Central Africa

(Bamako): A Niang (director), B Aboubacar, M Coulibaly, S Dapaah-Ohemeng, MA Fripong, D

Guimogo, D Sangare, Y Tankoano; MDG Centre East and Southern Africa (Nairobi): B Begashaw

(director), G Denning, J Aridi, J Murugi, M Wagah, P Wambua, J Wariero; Bonsasso, Ghana: J

Mensah-Homiah, E Akosah; Dertu, Kenya: D Malaam, F Shide; Mayange, Rwanda: Donald Ndahiro, R

Felicien; Mbola, Tanazania: G Nyadzi, M Shemsanga; Mwandama, Malawi: T Mijoya; Pampaida,

Nigeria: B Yunusa, E Ojo, C Woje; Potou, Senegal: S Kandji, M Sene; Ruhiira, Uganda: D Siriri, E

Atuhairwe; Tiby, Mali: B Kaya, T Coulibaly

References

1. United Nations General Assembly. United Nations Millennium Declaration. http://wwwunorg/millennium/declaration/ares552ehtm; 2000.

2. UN Interagency group for child mortality estimation. Levels and trends in child mortality: report 2011. New York: UNICEF, 2011.

3. Masanja H, de Savigny P, Schellenberg J, et al. Child survival gains in Tanzania: analysis of data from demographic and health surveys. The Lancet 2008; 371: 1276-83.

4. Bhutta ZA, Ahmed T, Black RE, al e. What works? Interventions for maternal and child undernutrition and survival. Lancet 2008; 371: 417-40.

5. UNICEF. State of the World's Children: Child Survival. New York: UNICEF, 2008. 6. Bhutta ZA, Ali S, Cousens S, et al. Interventions to address maternal, newborn, and child

survival: what difference can integrated primary health care strategies make? The Lancet 2008; 372: 972-89.

7. Bryce J, Victoria CG, Habicht JP, Black RE, Scherpbier RW. Programmatic pathways to child survival: results of a multi-country evaluation if Integrated Management of Childhood Illness. Health Policy and Planning 2005; 20(supplement 1): i5-i17.

8. Bryce J, Gilroy K, Jones G, Hazel E, Black RE, Victora CG. The Accelerated Child Survival and Development programme in west Africa: a retrospective evaluation. The Lancet 2010; 375: 572-82.

9. Arifeen SE, Hoque DME, Akter T, et al. Effect of the Integrated Management of Childhood Illness strategy on childhood mortality and nutrition in a rural area in Bangladesh: a cluster randomised trial. Lancet 2009; 374: 393-403.

10. Ekman B, Pathmanathan I, Liljestrand J. Integrating health interventions for women, newborn babies, and children: a framework for action. The Lancet 2008; 372: 990-1000.

11. Xu K, Evans DB, Carrin G, Aguilar-Rivera AM, Musgrove P, Evans T. Protecting households from Catastrophic health expenditures. Health Affairs 2007; 6: 972-83.

12. Countdown coverage writing group. Countdown to 2015 for maternal, newborn, and child survival: the 2008 report on tracking coverage of interventions. The Lancet 2008; 371: 1247-58.

13. Countdown 2008 Equity Analysis Group. Mind the gap: equity and trends in coverage of maternal, newborn, and child health services in 54 Countdown countries. The Lancet 2008; 371: 1259-67.

14. United Nations. Millennium Development Goals Report. Geneva: United Nations, 2010. 15. Fotso JC, Ezeh AC, Madise NJ, Ciera J. Progress towards the child mortality millennium

development goal in urban sub-Saharan Africa: the dynamics of population growth, immunization, and access to clean water. BMC Public Health 2007; 7(218): doi:10.1186/471-2458-7-218.

16. Sanchez P, Palm C, Sachs J, et al. The African Millennium Villages. Proceedings of the National Academy of Sciences 2007; 104(43): 16775-80.

17. The Earth Institute, Millennium Promise. Harvests of Development in rural Africa: the Millennium Villages after three years. New York: The Earth Institute, Columbia University, 2010.

18. UN Millennium Project. Investing in development: a practical plan to achieve the Millennium Development Goals. Overview London: Earthscan, 2005.

19. UN Millennium Project. Halving hunger: can it be done. New York: UN Millennium Project Task force on Hunger, 2005.

20. Dixon J, Gulliver A, Gibbon D. Farming Systems and Poverty. Rome: FAO; 2001.

http://wwwunorg/millennium/declaration/ares552ehtm

21. Ansah EK, Narh-Bana S, Asiamah S, Dzordzordzi V, Biantey K, al. e. Effect of removing direct payment for health care on utilisation and health outcomes in Ghanaian children: A randomised controlled trial. PLoS Med 2009; 6(1): e1000007. doi:10.1371/journal. pmed.

22. Grieve R, Cairns J, Thompson SG. Improving Costing Methods in Multicentre Economic Evaluation: the Use of Multiple Imputation for Unit Costs. Health Economics 2009; DOI: 10.1002/hec.1531.

23. Habicht JP, Victora CG, Vaughan JP. Evaluation designs for adequacy, plausibility and probability of public health programme performance and impact. Int J Epidemiol 1999; 28(1): 10-8.

24. United Nations Development Group. Indicators for monitoring the Millennium Development Goals: Definitions, rationale, concepts and sources. New York: UNDG, 2003.

25. WHO. Bench aids for malaria diagnosis. Geneva: World Health Organization, 2000. 26. Cogill B. Anthropometric indicators measurement guide. Washington D.C.: Food and

Nutrition Technical Assistance, 2001. 27. WHO. WHO Anthro 2005 software and macros. Geneva: WHO, 2005. 28. Allison P. Missing Data (Quantitative Applications in the Social Sciences). Thousand Oaks,

CA.: Sage; 2001. 29. Willett JB, Singer JD. Applied Longitudinal Data Analysis: Modeling Change and Event

Occurrence Oxford, UK: Oxford University Press; 2003. 30. WHO. Guidelines for the treatment of malaria. Geneva: WHO, 2010. 31. Des Jarlais DC, Lyles C, Crepaz N, Group atT. Improving the reporting quality of

nonrandomized evaluations of behavioral and public health interventions:the TREND statement. American Journal of Public Health 2004; 94: 361-6.

32. Nziguheba G, Palm CA, Berhe T, et al. The African Green Revolution: Results from the Millennium Villages Project. Advances in Agronomy 2010; 109: 75-115.

33. Remans R, Pronyk PM, Fanzo J, et al. A multi-sector intervention to accelerate reductions in child stunting: an observational study from nine sub-Saharan African countries. American Journal of Clinical Nutrition 2011; doi: 10.3945/ajcn.111.020099.

34. Victora CG, Black RE, Boerma JT, Bryce J. Measuring impact in the Millennium Development Goal era and beyond: a new approach to large-scale effectiveness evaluations. Lancet 2010; July 9. 2010(DOI: 10.1016/S0140-6736(10)60810-0).

35. Jones G, Steketee RW, Black RE, Bhutta ZA, Morris SS. How many child deaths can we prevent this year? Lancet 2003; 362: 65-71.

36. Briggs CJ, Garner P. Strategies for integrating primary health services in middle- and low-income countries at the point of delivery. Cochrane Database Systematic Reviews 2006; 2(CD003318).

37. Rutherford ME, Mulholland K, Hill PC. How access to health care relates to under-five mortality in sub-Saharan Africa: systematic review. Tropical Medicine and International Health 2010; 15(5): 508-19.

Research in Context: Panel

Systematic Review

We searched PubMed and Google scholar for reports published in English between Jan 1, 2001 and

Jan 1, 2011, with the search terms “child mortality” and “Africa”. We identified no previous reviews

or assessments of integrated initiatives that aimed to achieve the full range of MDGs, or of programs

to reduce child mortality by combining health and non-health sector inputs, especially in the

African context. There is, however, extensive evidence from systematic reviews supporting the

efficacy of a range of discrete, low-cost health and nutrition interventions for improving child

survival in low-income settings.4,35 Additional reviews have examined the effectiveness of systems

to integrate and deliver these interventions at the primary care and household level. 6,36 While

strategies such as community health workers hold great promise, few studies have reported

outcomes across the continuum of care or have assessed programs that work on a large scale.

Finally, although a few assessments have attempted to address access barriers and socio-cultural

factors that influence demand for services, the reduction of user-fees, mass-media campaigns,

conditional cash transfers and community mobilisation have been linked to improvements in child-

health outcomes in some settings 6,36,37

Interpretation

Our analysis suggests that the integrated delivery of interventions across multiple sectors is

feasible for a modest cost, that substantial progress towards the Millennium Development Goals

(MDGs) can be achieved in a relatively short 3-year period, and that the combination of

interventions can lead to reductions in child mortality at a pace sufficient to achieve MDG 4 in areas

of rural sub-Saharan Africa. Although health sector interventions such as immunization and malaria

control were potentially important drivers, efforts outside the health sector (agricultural inputs to

improve food security and nutrition; interventions to reduce access barriers such as the elimination

of user fees and the upgrading of roads, transport and communication; and basic improvements in

water and sanitation) probably contributed to reported improvement in child survival.

Table 1: Characteristics of Millennium Villages and comparison villages

Millennium Village sites (N=9) Comparison villages sites (N=9)

Mean 95% CI Mean 95% CI

Village characteristics (at Year 0)

Land area (square km) 133.2 (102.2 − 164.1)

128.2 (97.2 − 159.1)

Site has electricity 0.0% N/A 0.0% N/A

Site has cellular coverage 78% (39% − 95%) 78% (39% − 95%)

Distance to nearest main town (km) 11.9 (8 − 15.8) 12.6 (8.7 − 16.5)

Distance from center of village to nearest paved road (km) 14.8 (0.8 − 28.7) 14.5 (0.5 − 28.4)

Number of months road not accessible to vehicles 2.3 (2 − 2.7) 2.5 (2.2 − 2.8)

Distance to clinic (km) 5.6 (1.8 − 9.5)

10.2 (6.3 − 14.1)

Number of NGOs/partners per site 1.3 (0.8 − 1.9) 1.4 (0.9 − 2)

Number of facilities per 10,000 capita Markets 0.7 (-0.4 − 1.7)

1.4 (0.4 − 2.5)

Primary schools 5.6 (-0.4 − 11.5)

8.6 (2.6 − 14.5)

Secondary schools 0.0* N/A

0.0* N/A

Clinics 0.7 (-0.8 − 2.1)

1.3 (-0.1 − 2.7)

Site has no irrigation of cultivable land 33.3% (10% − 69.1%)

33.3% (10% − 69.1%)

Religion - % of population that is Christian 47% (32.7% − 61.4%)

38% (23.4% − 52.1%)

Characteristics of households (at Year 3) Household head has no primary education 87.1% (83.1% − 90.3%)

87.9% (84.1% − 90.9%)

Household head is a woman 14.3% (10.2% − 19.7%) 11.3% (7.9% − 16%)

Household head's main livelihood strategy is farming 81.9% (77.2% − 85.9%) 85.1% (80.9% − 88.5%)

Household size 7.1 (5.7 − 8.6) 5.9 (4.5 − 7.3)

Dependency ratio 138.2 (132.6 − 143.7) 131.9 (126.3 − 137.4)

Age of adult female household members 33.0 (32.3 − 33.8) 31.9 (31.1 − 32.7)

Baseline outcomes (at Year 0) Asset-based wealth index 41.0 (38.6 − 43.4)

39.0 (36.7 − 41.5)

Skilled birth attendance 32.6% (26.6% − 39.1%)

25.9% (20.7% − 31.8%)

Access to antenatal care 45.3% (29% − 62.8%) 46.0% (29.5% − 63.4%)

Under 5 mortality rate 113 (99 − 128) 90 (77 − 103)

NOTES: Village infrastructure information is from the village matching checklist. The characteristics of households are from the Year 3 household survey. Baseline outcomes are calculated based on reproductive/pregnancy histories collected from women at Year 3. The asset-based wealth index is scaled to have a mean of 50 (SD 25). *Interval has zero width because there is no variance in this characteristic across sites.

Table 2: Study outcomes in Millennium Village intervention sites and comparison village sites

Millennium Village Sites (9 sites) Comparison Village Sites (9 sites)

Millennium Villages vs comparison villages in Year 3

Absolute

Absolute Absolute

Observation Year 0 Year 3

change

Year 0 Year 3

change

difference

Indicator unit (N) (N) (95% CI) p-value (N) (N) (95% CI) p-value (95% CI) p-value

MDG I: Poverty and Nutrition

Asset-based wealth index Household* 41.0 60.3

19.3

Millennium Village Sites (9 sites) Comparison Village Sites (9 sites)

Millennium Villages vs comparison villages in Year 3

Absolute

Absolute Absolute

Observation Year 0 Year 3

change

Year 0 Year 3

change

difference

Indicator unit (N) (N) (95% CI) p-value (N) (N) (95% CI) p-value (95% CI) p-value

MDG 5: Maternal Health

Access to antenatal care Births* 45.3% 41.5%

-3.8% 0.422

46.0% 40.3%

-5.7% 0.230 1.9% 0.773

(194) (460) (-13.3 − 5.6) (191) (443) (-15.2 − 3.7) (-11.3 − 15.1)†

Skilled birth attendance Births* 32.6% 57.2%

24.7%

Figure 1: African Millennium Village Project Study Sites

Figure 2: Millennium Villages Project: Intervention activity time line (nine sites)

Figure 3: Non-amortised spending on the Millennium Development Goals per head by sector, constant 2008 USD (eight sites, all stakeholders)

$0

$20

$40

$60

$80

$100

$120

$140

$160

$180

Bas

elin

e

Yea

r 3

Bas

elin

e

Yea

r 3

Bas

elin

e

Yea

r 3

Bas

elin

e

Yea

r 3

Bas

elin

e

Yea

r 3

Bas

elin

e

Yea

r 3

Bas

elin

e

Yea

r 3

Bas

elin

e

Yea

r 3

Bas

elin

e

Yea

r 3

MV

P M

od

el

Ghana Kenya Rwanda Tanzania Malawi Nigeria Uganda Mali 8-Site Avg

Management

Other

Infrastructure

Agriculture

Health

Education

Figure 4: Absolute Change in the mortality rate of children younger than 5 years of age from baseline to year 3 by age category

MILLENNIUM VILLAGE = Millennium villages CV = Comparison villages

Web Appendix Table A: Study outcomes, descriptions and hypothesized direction of change

Outcome Description Hypothesized direction of

change

MDG 1: Poverty and Nutrition

Household poverty Asset-based wealth index† Increase

Food insecurity Proportion of households reporting not enough food for at least 1 of past 12 months

Decrease

Wasting Proportion of children under 2 years of age with weight for height Z-score < -2.SD

Decrease

Underweight Proportion of children under 2 years of age with weight for age Z-score

Web-appendix Table B: Per capita GDP and total health spending by participating country

* World Bank. World Development Indicators. GDP per capita (current US$). http://data.worldbank.org/indicator/NY.GDP.PCAP.CD † World Health Organization. World Health Statistics 2011. Geneva, WHO Press, 2011.

GDP/capita in 2008, current US$*

Total health spending/capita in

2008, current US$†

Ghana $1,226 $55

Kenya $781 $33

Malawi $291 $18

Mali $604 $39

Nigeria $1,375 $73

Rwanda $471 $45

Senegal $1,121 $62

Tanzania $502 $22

Uganda $461 $44

AVERAGE $759 $43

http://data.worldbank.org/indicator/NY.GDP.PCAP.CD

Web-appendix Table C: Comparison of Millennium Village and comparison village characteristics in each site

Bonsaaso Potou Pampaida Tiby Mbola Mayange Ruhiira Mwandama Dertu

Ghana Senegal Nigeria Mali Tanzania Rwanda Uganda Malawi Kenya

Village characteristics (at Year 0)

Land area

0 + + + N/A − − − −

Site has electricity

0 0 0 0 0 0 0 0 0

Site has cellular coverage

0 0 0 0 0 0 0 0 0

Distance to nearest main town

+ 0 0 0 + − 0 0 0

Distance from village center to nearest paved road

+ − − + − − + − −

Number of months road not accessible to vehicles

− + − − − − 0 − 0

Distance to clinic

0 − 0 − + 0 − 0 0

Number of NGOs/partners per site

0 + 0 − 0 0 − 0 0

Number of facilities per 10,000 capita

Markets

0 + 0 − 0 − 0 0 −

Primary schools

+ + − + + − − − −

Secondary schools

0 0 0 0 0 0 0 0 0

Clinics

0 0 0 + 0 + − 0 −

Site has no irrigation of cultivable land

0 0 0 0 0 0 0 0 0

Religion - % of population that is Christian

+ 0 − 0 + N/A 0 + 0

Characteristics of households (at Year 3)

Household head has no primary education

0 0 − * 0 − * 0 + * 0 0

Household head is a woman

0 + * 0 0 0 0 0 0 − *

Main livelihood strategy is farming

0 0 0 0 0 − * 0 − * 0

Household size

+ * 0 + * + * 0 0 0 + * + *

Dependency ratio

0 + * + * 0 0 0 + * 0 0

Age of adult female household members

+ * − * + * 0 0 0 + * + * 0

Baseline outcomes (at Year 0)

Asset-based wealth index

0 0 0 + * + * 0 0 + * 0

Skilled birth attendance

+ * 0 0 0 0 0 0 0 + *

Access to antenatal care

0 0 0 0 0 0 0 0 0

Under 5 mortality rate

0 0 0 0 0 0 0 0 0

NOTES: Village infrastructure information is from the village matching checklist. The characteristics of households are from the Year 3 household survey. Baseline outcomes are calculated based on reproductive/pregnancy

histories collected from women at Year 3. N/A = not available. For village infrastructure: "0" = same for MILLENNIUM VILLAGE and CV sites; "+" = MILLENNIUM VILLAGE site value is greater; "-" = MILLENNIUM VILLAGE site

value is lower. Statistical tests of the difference cannot be conducted because there are only two data points and these characteristics are measured at the village level. For other characteristics: "0" = Difference is not

statistically significant; "+*" = MILLENNIUM VILLAGE site value is statistically greater at 5% level; "-*" = MILLENNIUM VILLAGE site value is statistically lower at 5% level.

Web Appendix Table D: Study outcomes in Millennium Village intervention sites and Comparison Village sites

(Not adjusted for Household and Individual Characteristics)

Year 0 Millennium Village Sites (9 sites) Comparison village Sites (9 sites)

Millennium Village vs Comparison Village in Year 3: Simple difference (SDIFF) or difference-in-difference (DD)

based on

Absolute

Absolute

Absolute

Observation Year 3 Year 0 Year 3

change

Year 0 Year 3

change

difference

Indicator unit recall? (N) (N) (95% CI) p-value (N) (N) (95% CI) p-value Type (95% CI) p-value

MDG I: Poverty and Nutrition

Asset-based wealth index Household Yes 41.0 60.3

+19.3***

Year 0 Millennium Village Sites (9 sites) Comparison village Sites (9 sites)

Millennium Village vs Comparison Village in Year 3: Simple difference (SDIFF) or difference-in-difference (DD)

based on

Absolute

Absolute

Absolute

Observation Year 3 Year 0 Year 3

change

Year 0 Year 3

change

difference

Indicator unit recall? (N) (N) (95% CI) p-value (N) (N) (95% CI) p-value Type (95% CI) p-value

MDG 6: HIV, TB, and Malaria

Antenatal HIV testing Births Yes 27.5% 69.9%

+42.4%***

Web Appendix table E: Site specific changes in secondary outcomes, MILLENNIUM VILLAGE sites baseline to year 3

# of sites

Direction of difference and statistical significance

Expected in right

Bonsaaso Potou Pampaida Tiby Mbola Mayange Ruhiira Mwandama Dertu

Indicator direction direction Ghana Senegal Nigeria Mali Tanzania Rwanda Uganda Malawi Kenya

MDG I: Poverty and Nutrition

Asset-based wealth index + 9

+ *** + *** + *** + *** + *** + *** + *** + *** + ***

Food insecurity − 7

− *** + ** − *** + ** − *** − *** − *** − *** − ***

Wasting − 5

− + + − + − − * + −

Underweight − 7

− + − − + − − − −

Stunting − 8

− + − − − *** − − − −

MDG 4: Child Health

Diarrhea prevalence − 6

+ − + *** − ** + − *** − − −

Diarrhea treatment + 6

+ * + + − *** + + + − −

Measles immunization + 7

+ + + − + *** − + ** + + *

Postnatal check + 7

+ + − + + + + + −

MDG 5: Maternal Health

Access to antenatal care + 3

+ − − − − + + − −

Skilled birth attendance + 9

+ *** + * + + + + *** + *** + +

MDG 6: HIV, TB, and Malaria

Antenatal HIV testing + 8

+ + + * + + ** + + *** + ** −

Bed net utilization + 9

+ *** + *** + *** + *** + *** + + *** + *** + **

Malaria prevalence − 7

+ − *** − ** − *** − N/A − *** − ** −

MDG 7: Environmental Health

Access to improved water + 9

+ *** + *** + *** + *** + + *** + *** + *** +

Access to improved sanitation + 8

+ *** + *** + + *** − + *** + *** + + *

NOTES: *p-value

Web Appendix Table F: Site-specific changes in under 5 mortality: Year 0 to Year 3

Absolute values per 1000 livebirths

Millennium Village

Comparison village Difference-

change change in-

Site (births) (births) difference

Primary Outcome

Bonsaaso (Ghana) +3.2 +9.5

-6.3

(821) (701)

Potou (Senegal) -15.0 -0.7

-14.4

(881) (940)

Pampaida (Nigeria) -12.2 +47.8

-60.1

(3628) (1886)

Tiby (Mali) -77.8 +10.6

-88.5

(829) (956)

Mbola (Tanzania) -18.3 +12.8

-31.1

(640) (841)

Mayange (Rwanda) -24.4 -23.7

-0.7

(614) (715)

Ruhiira (Uganda) -54.7 +0.6

-55.3

(826) (763)

Mwandama (Malawi) -86.1 -58.4

-27.7

(692) (691)

Dertu (Kenya) -23.0 -4.2

-18.7

(1310) (533)

NOTES: The unit of observation is births. The difference-in-difference (DD) estimate is equal to the difference between Millennium Village and CV sites in Year 3, minus the pre-intervention difference between groups in Year 0. Equivalently, it is also equal to the change over time in MILLENNIUM VILLAGE sites minus the change over time in the CV sites. Estimates are adjusted for household and respondent characteristics. Rounding may cause slight discrepancies in calculating differences.

Web Appendix Figure A: Sample flow chart and response rates

Web Appendix Figure B: National trends in Under 5 Mortality in rural areas for countries where MILLENNIUM VILLAGE sites are located (1990-

2010)

Web Appendix Statistical Models

Technical Appendix: Statistical Models

This appendix describes the analyses conducted to estimate the quantities presented in the outcomes tables. The statistical model used for the analysis depends on the type of indicator:

The first category of indicators are those where a baseline value for the outcome of interest is available for the MILLENNIUM VILLAGE sites, but not the comparison village (CV) sites.1 For these indicators, the tables in the paper present estimates of (1) the change in outcomes from Year 0 to Year 3 in the MILLENNIUM VILLAGE sites, and (2) the difference between outcomes in MILLENNIUM VILLAGE sites and comparison village (CV) sites at Year 3 (which labeled “SDIFF” in the tables).

The second category are indicators for which we are able to calculate a baseline

value for the MILLENNIUM VILLAGE and CV sites, based on recall items in the Year 3 survey. This category of indicators includes assets2 and pregnancy-related outcomes such as skilled birth attendance.3 For these outcomes, the outcomes tables present: (1) the change in the indicator from baseline to follow-up in the MILLENNIUM VILLAGE sites, (2) the change in the CV sites, and (3) the amount by which the change in the MILLENNIUM VILLAGE sites differs from the change in the CV sites (difference-in-difference estimate, which is labeled DD in the tables).

The third category is the under-5 mortality rate (U5MR). This indicator is a special

case of the second category – baseline values can be retrospectively calculated. However, the estimation of the U5MR is based on life tables, so the statistical model is based on a survival analysis.

The statistical models used for each of these types of outcome are described below. We use the following terms in this appendix:

Site: A “site” is a village cluster. There are 18 sites in the analysis – 9 intervention sites and 9 comparison sites.

Pair: A “pair” is the MILLENNIUM VILLAGE site and its matched comparison site.

There are 9 pairs in the analysis.

1 Baseline data were not collected in the CVs, because the comparison villages were not added to the study until Year 3. 2 Baseline asset values are calculated based on a set of survey items about household assets 3 years ago. 3 Baseline values for these indicators are constructed from women’s reproductive histories collected at Year 3.

I. Outcomes without a baseline value for the CV sites This section applies to the following indicators: food insecurity, wasting, underweight, stunting, diarrhea prevalence and treatment, measles vaccination, bed net utilization, malaria prevalence, access to improved water, and access to improved sanitation. A. Change over time in MILLENNIUM VILLAGE sites (Year 3 vs. Year 0) The following logistic model is fit to a pooled dataset that includes observations for the MILLENNIUM VILLAGE sites at baseline and Year 3:

(1)

Where the variables are defined as follows:

= Outcome for person i in site j

= Dummy indicator for individuals observed at Year 3 (=1 if person i is observed at Year 3; =0 if observed at baseline)

= A set of dummy indicators for the sites (=1 if an individual is in site m; =0 otherwise)4

= A set of k person-level and/or household-level characteristics for individuals in the sample, measured at baseline or Year 3

And where the residual has two levels, to account for the clustering of observations within sites:

= Site-level error term for each site j (unexplained site-level effects, to

adjust the standard errors for clustering)5

= Person-level error term for person i in site j (unexplained individual-

level within-site effects) Given this model specification:

= The expected value of outcome Y (in log odds) for MILLENNIUM VILLAGE sites in Year 0

= The estimated change in the log odds of outcome Y for the MILLENNIUM VILLAGEs, from Year 0 to Year 3

A t-test is used to evaluate whether is statistically different from zero.6

4 Also called “site fixed-effects”. Their purpose is to account for the paired nature of the study design (each MILLENNIUM VILLAGE has two data points – one at Year 0 and one at Year 3). They also increase the precision of estimated change. 5 Also called “site random-effects”.

For the purposes of the outcomes tables, the estimate of (the change over time in log

odds) is converted to a probability (percentage) scale; this value is presented in the “Absolute Change” column for the MILLENNIUM VILLAGE sites.7 The standard error for the change is then used to calculate a confidence interval. B. Simple Difference between MILLENNIUM VILLAGE and CV sites in Year 3 (SDIFF) The following logistic model is fit to a pooled dataset that includes observations for the MILLENNIUM VILLAGE and CV sites at Year 3:

(2)

Where the variables are defined as follows:

= Outcome for person i in site j

= Dummy indicator for observations in MILLENNIUM VILLAGE sites (=1 if an individual is in an MILLENNIUM VILLAGE site; =0 if in a CV site)

= A set of dummy indicators for the site pairs (=1 if an individual is in pair m; =0 otherwise).8

= A set of k person-level and/or household-level characteristics for individuals in the sample, measured at Year 3

And where the residual has two levels, to account for the clustering of observations within sites:

= Site-level error term for each site j (unexplained site-level effects, to

adjust the standard errors for clustering)9

= Person-level error term for person i in site j

Given this model specification:

= The expected value of outcome Y (in log odds) for CV sites in Year 3

= The estimated difference (in log odds) between MILLENNIUM VILLAGEs and CVs in Year 3

6 Because the model controls for sites, this is a paired t-test.

7 This is done by using the parameters estimates from Model 1 to evaluate the predicted log odds for the average

person in the sample at Year 0 and at Year 3. These two values are then converted back to a probability

(percentage) scale. The difference between them is the change over time on a probability scale.

8 Also called “pair fixed effects”. Their purpose is to account for the paired nature of the study design (each MILLENNIUM VILLAGE site is compared to its CV site). They also improve the precision of the estimated difference between MILLENNIUM VILLAGE and CV sites. 9 Also called “site random-effects”.

A t-test is used to evaluate whether is statistically different from zero.10

For the purposes of the outcomes tables, the estimate of (difference between

MILLENNIUM VILLAGE and CV sites in log odds) is converted to a probability (percentage) scale.11 This value is presented in the “Absolute Difference” column. The standard error for the difference is then used to calculate a confidence interval.

II. Outcomes with a baseline value for the CVs This section applies to the following indicators: the wealth-based asset index, newborn care, antenatal HIV testing, skilled birth attendance, and access to antenatal care. For these indicators, the following “difference-in-difference” model is fit to a pooled dataset that includes observations for the MILLENNIUM VILLAGE sites and CV sites:

(3) (Note: For the four pregnancy-related outcomes – which are dichotomous – a logistic model is used, i.e. the left-hand side of the log odds of outcome Y). The variables in Model 3 are defined follows:

= Outcome for person i in site/cluster j

= Dummy indicator for observations in MILLENNIUM VILLAGE sites (=1 if observation is in an MILLENNIUM VILLAGE site; =0 otherwise)

= Dummy indicator for observations from Year 3 (=1 for observations in Year 3 ; =0 otherwise)12

= A set of dummy indicators for the matched pairs (=1 if an individual is in pair m at baseline; =0 otherwise).

Given this model specification:

= The expected value of outcome Y for MILLENNIUM VILLAGE sites in

10

Because the model controls for sites, this is a paired t-test.

11 This is done by using the parameters estimates from Model 2 to evaluate the predicted log odds for the average

person in the MILLENNIUM VILLAGE sites and CV sites at Year 3. These two values are then converted back to a

probability (percentage) scale. The difference between them is the simple difference between MILLENNIUM

VILLAGE and CV sites on a probability scale.

12 For pregnancy-related outcomes, POST=1 for births that occurred in the 3rd year of project implementation, and POST=0 for births that occurred prior to the start of project implementation. For assets, POST=1 for the assets of households at Year 3, and POST=0 for the assets of households at baseline (three years ago).

Year 0

= The expected value of outcome Y for MILLENNIUM VILLAGE sites in Year 3

= The expected value of outcome Y for CV sites in Year 0

= The expected value of outcome Y for CV sites in Year 3

From these values, we can get the change over time in the MILLENNIUM VILLAGEs and CVs:

= The estimated change in the MILLENNIUM VILLAGEs from baseline to Year 3

= The estimated change in the CVs from baseline to Year 3

These two values can then be used to obtain the difference-in-difference estimate (DD). The DD estimate is equal to the change over time in the MILLENNIUM VILLAGE sites minus the change over time in the CV sites:

= Difference-in-difference estimate

A t-test is used to evaluate whether these estimates are statistically different from zero; their standard errors are used to construct confidence intervals. For the pregnancy-related outcomes (which are dichotomous), estimates of change and of the difference-in-difference are converted to a probability (percentage) scale for the outcomes tables.13

III. The Under-5 Mortality Rate (U5MR) The analysis of under-5 mortality rates is conducted using a life table approach; in practice, this is implemented using a discrete time survival regression. Using a regression-based approach makes it possible to adjust the results for the characteristics of households, and to adjust the standard errors for the clustering of children within sites. This section describes the data that were used to estimate the U5MR, how the U5MR was calculated, and how this information was used to estimate change over time and the difference-in-difference estimates presented in Table 3. A. The Dataset The survival analysis is based on a panel dataset that measures whether or not a child was alive during different age/time periods. As explained in the paper, birth histories collected

13

This is done by using the parameters estimates from Model 3 to evaluate the predicted log odds for the average

person in the MV sites and CV sites at Year 3, and at Year 0. These four values are then converted back to a

probability (percentage) scale, and used to obtain the estimated change and difference-in-difference on a

probability scale.

from women in the villages at Year 3. Women were asked to provide the date of birth and death of all their live births. From this information, it is possible to determine whether a child has died, to what age they survived, and whether they were alive before and/or after the intervention started. In the panel dataset, time is measured using the following 8 discrete age categories:

0 – 1 months (0-30 days) 1-3 months (31-91 days) 3-6 months (92-182 days) 6-12 months (183-365 days) 12-24 months 24-36 months 36-48 months 48-60 months

Each child has T lines, where T is the number of age categories during which a child was alive during the study period (with T having a maximum of 8). The first age category for a child is the one at which they entered study period; the last age category is the one at which they exited the study period or died. For the purposes of the analysis, the “baseline” period is defined as the 5 years before the intervention started; the “follow-up” period is the first 3 years of implementation. Therefore, the study period is 8 years. In the panel dataset, for each time period t, a child is coded as being either alive or dead at the end of the period. Each period in a child’s life is also coded as happening either “prior” to the start of the intervention (POST=0), or after the start of the intervention (POST=1). B. Statistical Model The following survival model was fit to the panel dataset:

(4) Where t denotes the age category and the variables are defined as follows:

= Dummy indicator for whether child i has died by the end of age

category t

= Set of dummy indicators for the 8 age categories (=1 for observations in period t; 0= otherwise)

= Dummy indicator for children in MILLENNIUM VILLAGE sites (=1 if child i is in an MILLENNIUM VILLAGE site; =0 otherwise)

= Dummy indicator for children in CV sites (=1 if child i is in a CV site; =0 otherwise)

= Dummy indicator for observations at baseline (=1 for observations during the baseline period; 0 otherwise)

= Dummy indicator for observations at follow-up (=1 if for observations during the follow-up period; 0 otherwise)

= A set of dummy indicators for the matched pairs (=1 if child is in pair m at baseline; =0 otherwise).

= A set of k person-level and/or household-level characteristics for children in the sample

And where the residual has two levels, to account for the clustering of observations within sites:

= Site-level error term for each site j (unexplained site-level effects, to

adjust the standard errors for clustering)

= Person-level error term for person i in site j

C. Calculating the U5MR for each time period (baseline and follow-up) and for each group (MILLENNIUM VILLAGE, CV) With this model specification, we can estimate the log-odds of dying during age category t, by time period and by intervention group:

= The log-odds of dying in age category t for MILLENNIUM VILLAGE sites (baseline)

= The log-odds of dying in age category t for MILLENNIUM VILLAGE sites (follow-up)

= The log-odds of dying in age category t for CV sites (baseline)

= The log-odds of dying in age category t for CV sites (follow-up)

These log-odds are then converted to probabilities as follows:

= The probability of dying in age category t for MILLENNIUM VILLAGE sites (baseline)

= The probability of dying in age category t for MILLENNIUM VILLAGE sites (follow-up)

= The probability of dying in age category t for CV sites (baseline)

= The probability of dying in age category t for CV sites (follow-up)

These probabilities are called hazard rates in the survival analysis literature. By multiplying these probabilities, we can calculate the probability of surviving to age 5 for each period (baseline and follow-up) and by group (MILLENNIUM VILLAGE or CV):

= The probability of surviving to age 5 for MILLENNIUM VILLAGE sites (baseline)

= The probability of surviving to age 5 for MILLENNIUM VILLAGE sites (follow-up)

= The probability of surviving to age 5 for CV sites (baseline)

= The probability of surviving to age 5 for CV sites (follow-up)

To get the U5MR – or the probability of dying before age 5 – we just need subtract these survival probabilities from 1:

= The probability of dying before age 5 for MILLENNIUM VILLAGE sites (baseline)

= The probability of dying before age 5 for MILLENNIUM VILLAGE sites (follow-up)

= The probability of dying before age 5 for CV sites (baseline)

= The probability of dying before age 5 for CV sites (follow-up)

D. Change over time and difference-in-difference estimates Next, we use these estimates to get the change over time in the MILLENNIUM VILLAGEs and CVs:

= The estimated change in the MILLENNIUM VILLAGEs from baseline to follow-up

= The estimated change in the CVs from baseline to follow-up

We can also get the difference-in-difference estimate, or the change over time for the MILLENNIUM VILLAGEs minus the change over time for the CVs:

= Difference-in-difference estimate

To conduct hypothesis testing for these values, we need a standard error for the U5MR in each time period/group. These standard errors are a function of the standard error of the death probabilities at each age category:14

= Standard error for U5MR for MILLENNIUM VILLAGE sites (baseline)

= Standard error for U5MR for MILLENNIUM VILLAGE sites (follow-up)

= Standard error for U5MR for CV sites (baseline)

= Standard error for U5MR for CV sites (follow-up)

These standard errors are then used to calculate the standard errors for the parameters of interest (change over time and difference-in-difference):

= Standard error for change in

MILLENNIUM VILLAGE sites

= Standard error for change in CV

sites

14 Standard errors are based on Greenwood’s formula (see Singer and Willett, 2003, Applied Longitudinal Data Analysis):

(a)

where At is the probability of death in period t (hazard rate) and nt is the number of children still alive at the start of period t. The variance of the hazard rate is equal to:

(b)

Substituting (b) into (a) and rearranging, we get:

This re-expression of the Greenwood formula is convenient, because it is based on the standard error of the death probabilities -- which are provided directly by SAS and adjusted for clustering via the regression model.

= Standard error for difference-in-difference

These standard errors are then used to test whether changes over time and the difference-in-difference estimate are statistically significant (based on a t-test) and to construct confidence intervals.