13
LIMITATIONS OF THE DHS & MICS IN LOW- and MIDDLE-INCOME COUNTRIES Robyn Schreiber November 2014 Rutgers University Edward J Bloustein School of Planning and Public Policy

Limitations of DHS and MICS

Embed Size (px)

Citation preview

Page 1: Limitations of DHS and MICS

LIMITATIONS OF THE DHS & MICS

IN LOW- and MIDDLE-INCOME COUNTRIES

Robyn Schreiber

November 2014

Rutgers University Edward J Bloustein

School of Planning and Public Policy

Page 2: Limitations of DHS and MICS

Limitations of the Demographic and Health Survey (DHS) and

Multiple Indicator Cluster Survey (MICS) in Low- and Middle-income Countries

In the majority of low- and middle-income countries, civil registration systems are inexistent or

inadequate, resulting in an overall “Scandal of Invisibility” due to absence of reliable data for

births, deaths and health status. The lack of accurate data has serious implications for vital

statistics, human and constitutional rights, policy and national development, the allocation of

external funding as well as monitoring the World Health Organization’s Millennium Development

Goals (MDGs). In the absence of civil registration systems, standardized household-based

surveys are conducted to estimate levels of population, health and nutrition. The two most

prominent international household surveys are the Demographic and Health Survey (DHS) and

the Multiple Indicator Cluster Survey (MICS), largely funded by the United States Agency for

International Development (USAID) and the United Nations International Children’s Emergency

Fund (UNICEF) respectively. Due to their large influence on data collection in low- and middle-

income countries, the DHS and MICS programs collaborate closely to ensure survey tools,

methodologies and analysis are harmonized to improve comparability across surveys and to

avoid duplications of efforts.

Background MEASURE DHS is the international program, within USAID, that assists countries in

implementing a DHS. MEASURE DHS was established in 1984 and since then has facilitated the

completion of over 300 nationally representative surveys in over 90 countries. DHS is considered

as the golden standard of household surveys in low- and middle-income countries. MICS,

established in 1995, was originally developed by the World Summit for Children to be a simple,

quick measure progress towards global goals. Due to criticism of early methodology, MICS

subsequently became more complex and mimicked DHS survey tools and implementation

protocols. More than 240 MICS surveys have been conducted in over 100 countries. Since their

inception, DHS and MICS have adapted their methods to minimize error and meet emerging data

needs. This summary will focus on MEASURE DHS Phase III (2008-2013) and MICS Round 4

(2009-2012) because they are the most recent phase for which the organizations have published

a methodology toolkit and program evaluation.

DHS and MICS collect a wide range of quantitative and qualitative data. Both surveys focus on

indicators of fertility, maternal and child health and mortality and include optional survey

modules on a variety of specific issues. MEASURE DHS aims to conduct DHS in a country every

five years whereas UNICEF attempts to conduct MICS every three years. Some data collected

differs between two surveys; DHS focus on using biomarkers for quantifying health and collects

information on sexually transmitted diseases, domestic violence, women’s empowerment and

reproductive health and family planning whereas MICS focuses on child labor, child discipline,

early childhood development and knowledge of danger signs for illness. In addition to the

Standard DHS, MEASURE DHS has developed several other surveys focused on specific public

health issues such as the AIDS Indicator Survey, Malaria Indicator Survey and Service Provision

Assessments.

Page 3: Limitations of DHS and MICS

MEASURE DHS and UNICEF staff do not arrange or perform survey activities themselves; rather,

the responsibility for executing a survey lies with a single implementing agency within the

country being surveyed. This agency may be governmental (National Statistical Office), non-

governmental (a family planning organization) or private-sector (private research firm). Other

institutions may contribute to the survey efforts by taking on certain responsibilities such as

providing staff with survey experience, office facilities or vehicles. If services are provided by

several sources, a Memorandum of Understanding is created. The majority of participating

countries receive funding from USAID (DHS) or UNICEF (MICS) with additional funding from The

World Bank, The Global Fund, United Nations Population Fund, Joint United Nations Programme

on HIV/AIDS, World Health Organization, host countries or other donors.

MEASURE DHS and UNICEF staff are responsible for developing survey tools and providing

technical assistance to implementing agencies. Generally, technical staff is provided through

short-term visits depending on availability, skill and expertise of the implementing agency; they

provide assistance with assessing feasibility of conducting a survey, survey design, sampling, field

staff training, fieldwork monitoring, data processing, data analysis, report writing and

dissemination activities. Staff and the implementing agency develop a protocol and submit it for

review to an institutional review board or ethics review panel and the ICF International Review

Board. Once protocols are approved and local field staff are hired and trained, the implementing

agency can begin the survey process.

Sampling Both DHS and MICS surveys follow a multi-stage cluster sample design. The first stage

utilizes the country’s most recent sampling frame, typically the most recent census enumeration

areas (EAs). When a country does not have a census or master sample, the implementing agency

may use alternative sample frames such as electoral zones or a gridded satellite map. Population

clusters are then selected from the sampling frame using probability proportional to size

sampling technique. MESURE DHS and UNICEF suggest sample sizes in accordance to total

population size, however sample size is ultimately limited by available funding. Clusters may be

divided into survey domains such as urban and rural groups. Multi-level stratification is used to

divide the population into first-level strata according to certain criteria, typically geographic, and

then is subdivide into second-level strata depending on what the implementing agency is

interested in analyzing. Second-level strata may include separation according to female literacy

or presence of health facilities.

After clusters are sampled and stratified, the second stage of sampling beings. MEASURE DHS

and UNICEF recommend that a household listing operation should be implemented before

surveying. In said operation, a household listing operation team visits each cluster and creates a

map of the location listing every household. From this list, households are randomly selected to

be surveyed. DHS and MICS survey methods do not allow teams to survey any additional

households and household substitution is not permitted. In the selected households, all women

of reproductive age (15-49) are individually interviewed. Sub-samples of men may be included by

Page 4: Limitations of DHS and MICS

interviewing all men in every second or third household. Unlike DHS, MICS require field workers

to interview any care takers in the house, regardless of sex or age.

Sample Weighting DHS and MICS samples are not self-weighting due to the need for specific

regions or areas to be over-sampled. Both design and sampling weights must be utilized for

household and individual data.

Questionnaires Standard DHS and MICS collect information on fertility, mortality and maternal

and child health. DHS collects maternal and child health information on the past five years from

biological mothers only; MICS interviews any household care taker on data from the past two

years. Both surveys are largely dependent on participant response and so the majority of survey

data is subject to response bias. To avoid translation errors, it is suggested that questionnaires

are translated into all applicable languages prior to data collection. Current editions of DHS and

MICS questionnaires total around two hours of interviewing. Several optional modules may be

added to the standard surveys including specific modules on domestic violence, female genital

cutting and health expenditures for the DHS and child labor, insecticide treated nets and hand

washing for MICS.

DHS MICS1

Standard topics

Anemia Child Health Education Environmental Health Family Planning Fertility and Fertility Preferences Gender/Domestic Violence HIV/AIDS Knowledge, Attitude and Behavior HIV Prevalence Household and Respondent Characteristics Infant and Child Mortality Malaria Maternal Health Nutrition Tobacco Use Unmet Need Wealth Women’s Empowerment

Mortality Nutrition Child Health Water and Sanitation Reproductive Health Child Development Literacy and Education Child Protection HIV/AIDS and Sexual Behavior Access to Mass Media and Use of Information/Communication Technology Subjective Well-Being Tobacco and Alcohol Use

Modules Domestic Violence Female Genital Cutting Maternal Mortality Fistula Health Expenditures

1. MICS Indicator List does not distinguish between standard and module indicators

Page 5: Limitations of DHS and MICS

Biomarkers MEASURE DHS and UNICEF use biomarkers, objective physical or biological measures

of health conditions, to complement questionnaire data. Standard DHS anthropometric

measurements of height and weight of children under five years of age is used to assess overall

health. Both DHS and MICS standard protocol include malaria testing; field workers test children

and women’s blood samples are tested for anemia. Since 2001, anonymous HIV testing has been

included in the standard DHS questionnaires using a dried blood spot (DBS) method. Both

women and men are tested in every second or third household in the sample.

Wealth Index Generally, two wealth indexes are used to quantify the wealth of a household. The

DHS Wealth Index is a survey-specific measure that is based on the analysis of household and

service amenities in a country at a particular point in time. This measures the relative economic

status of households to one another, but cannot be used for comparison between other

countries or trend analysis. The second wealth index, the Comparative Wealth Index, uses the

World Bank’s Gross National Income per capita as a baseline to develop an index that is

comparable across surveys and time. Both indexes may be utilized by DHS/MICS to assess the

general wealth of a household for later analysis.

Pretest A pretest is critical for testing survey processes such as checking translations, applying

biomarkers and other survey procedures. If an implementing agency conducts a pretest, a

separate household listing operation is completed and a small sample of households are

selected. The pretest experience is the basis on which survey questions and manuals are revised;

field staff document issues and these problems are address before the official survey is

implemented.

Data Collection Prior to data collection, MEAUSURE DHS and UNICEF suggest to include some

public relations activities to building understanding and support for the survey. The approach to

data collection is team based: a team consists of one supervisor, one field editor and ideally 3-4

female interviewers and 1-2 male interviewers. Depending on the type and complexity of

biomarkers the team may include a health technician. Number of teams is dependent on sample

size, duration of data collection, number of languages spoken in the country and available

funding. To reduce non-response error, field workers are instructed to return to a household

three times, over two days, before indicating them as a non-response.

Heavy emphasis is placed on data quality control. Several levels of supervision are outlined in

DHS and MICS methodology materials to improve data quality. The first level of supervision is the

team supervisors and field editors, who monitor the team, periodically conduct short re-

interviews, observe all biomarker data collection and review questionnaires for completeness.

The second level of supervision consists of the implementing agency’s central office visits to the

field. The survey director, field coordinators and DHS/MICS staff visits teams to evaluate each

team. Field control tables are also produced by the field teams periodically to analyze data that

has already been entered for response rate, age distribution and level of missing values.

Page 6: Limitations of DHS and MICS

Data Processing Data processing is the responsibility of the implementing agency. MEASURE DHS

and UNICEF outline suggestions regarding data entry limitations, verifications and sample

weighting. Once the final dataset is complete, a set of tables for the preliminary and final reports

are created.

Analysis and Reports Three reports are produced for each DHS survey. The first is a brief

preliminary report, consisting of 15-20 tables on key indicators, produced within three months

after the end of data collection. It is typically produced jointly by the implementing organization

and MEASURE DHS staff and has limited distribution. The second report, a final report, is based

on a set of DHS tabulations modified by the MEASURE DHS country manage and implementing

agency staff to fit the questionnaires used in their specific survey. Individual chapters of the final

report are written by local staff and edited by MEASURE DHS staff. For most surveys, an

additional Key Findings Report is produced with intent to present the data in an easily accessible

format for wide dissemination. The Key Findings Report is produced with the main survey report

but focuses on the most important findings. The Key Findings Report is drafted by the DHS staff

and reviewed by the implementing agency’s staff.

Generally, two reports are produced for each MICS survey. The first report is a Key Findings

Report, which is produced by the implementing agency and is typically completed three to six

months after the completion of fieldwork. If required, UNICEF employees will review and finalize

the report. The second report, the Final Report, is also produced by the implementing agency

following a general template provided by the MICS program.

In addition, MEASURE DHS and UNICEF may assist the implementing agency in further analysis

resulting in research papers, published journal articles, presentations for professional meetings

or short statements that can be used to respond to policy issues.

Dissemination Dissemination of survey results is critical to MEASURE DHS and UNICEF efforts.

Survey completion, a one-day national seminar takes place to present the findings to policy

makers, program managers, researchers, non-governmental organizations, donor organizations

and mass media. In-country dissemination activities may include: regional seminars, specialized

reports, workshops with media outlets, meetings with political leaders and mini-seminars with

special groups. All DHS and MICS survey data is made available in online data archives:

www.measuredhs.com and http://data.unicef.org/. The data archives include final reports,

survey datasets, publications and information on methods.

Use DHS and MICS make up the baseline data for funding, global decision making for health

planning and implementation and monitoring and evaluation of health programs. Managing

human capital facilitates good governance and allows resources to be allocated effectively. DHS

and MICS play key role in tracking the progress of programs focused on elimination inequalities

and disparities and help assess the global Millennium Development Goals. Survey data is often

used in academia to quantify, analyze and compare rates in various countries. This research,

Page 7: Limitations of DHS and MICS

combined with program evaluation and basis for funding promotes health reform in the

countries where DHS and MICS survey.

LIMITATIONS

Sampling

Multi-stage cluster sample design

Scientific sampling methods required each element of the target population to have a known,

non-zero probability of selection. One of the biggest problem with DHS and MICS sampling

methods is finding an up-to-date, reliable sampling frame for a country.

Available sampling frames – A study conducted by researchers Gupta, Shuaib, Becker, Rahman

and Peters used a 1979 Afghani census for a 2003 MICS sampling frame. 2004 pre-census data

was used to generate sampling weights, however power to detect change from these estimates

is low.

Sampling error/Coverage bias

Both surveys focus primarily on women of reproductive age (15-49) and children under five years

old. This limits knowledge of all other subpopulations within the country and is therefore not

suitable for comparison to men or the ageing population. Analysis of the national population is

therefore limited.

In addition, the population being sampled is limited to those who live in fixed households. DHS

and MICS exclude individuals living in group quarters (such as the military), the homeless,

nomadic and those who are institutionalized.

Sampling Weights

Due to the nature of the sampling technique, sampling weights are required before analysis of

DHS and MICS data. Unfortunately, there is a high potential for sampling weights misuse by the

implementing agency or other researchers. There is a lack of consensus on where to use

multivariate analysis when weighting sampling and discrepancies can lead to drastically different

data analysis.

Questionnaires Design

Length of Interview

Current DHS and MICS questionnaires take two hours to complete. The excessive length of the

interview encourages interviewers to skip questions or search for shortcuts such as age

displacement (discussed below). The interview should engage participants and encourage

complete answers, but avoid participant and interviewer boredom or fatigue.

Page 8: Limitations of DHS and MICS

Breadth

Recently, both DHS and MICS are collecting more data on communicable and non-communicable

disease, however data is still lacking. Excluding AIDS and Malaria, there are few disease

indicators recorded likely due to the difficulty of assessing disease prevalence through self-

reporting questionnaires.

Translation

In some cases, there may be issues with translation of questionnaires. For example, in several

languages there may be difficulty translating questions asking whether or not a child was offered

fluids or whether or not a child was given fluids.

Community Case Management

DHS and MICS are the only available means of obtaining data on treatment coverage. Although

data on care seeking and treatment coverage is available, neither survey collects data on the

treatment source.

Data Collection

Measurement error

Specific questions related to health indicators may not be an appropriate measurement of said

indicators. Complex health indicators, such as nutrition or disease, are limited to a few questions

regarding behavior or symptoms. Verbal autopsies, in which interviewers ask participants various

questions about an individual’s cause of death, may not be precise enough for evaluation of the

impact of health interventions or the assessment of cause-specific mortality.

Reporting/Recall bias

Excluding anthropometric measurements and biomarkers, all survey information comes from the

respondent. This makes DHS and MICS susceptible to recall bias. In a study analyzed DHS data in

Rural China, researchers concluded that there was a positive correlation between event

distinctiveness and recall accuracy. Issues regarding memory and accurate answers are

imbedded into the nature of questionnaire surveying.

Social-desirability bias

Some respondents may respond to questions in a socially acceptable direction rather than telling

interviewers the truth. This response bias occurs mainly for questions that deal with sensitive

subjects. Only certain individuals will exhibit this bias.

Non-response bias

Page 9: Limitations of DHS and MICS

Non-response bias occurs when the answers of respondents differ in meaningful ways from the

potential answers of those who did not answer. MEASURE DHS and MICS boast less than 10%

non-response bias.

“Don’t Know”

There are significant proportion of missing responses of month of birth and exact calendar year

of birth for women and children. The proportion of “missing” or “don’t know” responses is often

high for date and age variables for births, marriages and deaths, all of which are central to the

estimation of fertility and mortality rates.

In addition, analysis comparing literate and illiterate DHS responses in Iran shows a positive

correlation between mother’s literacy and reported child morbidity. Conversely, there was a

positive association between mother’s illiteracy and child mortality. This suggests that there may

be socially, educationally patterned differential recall bias and reporting that DHS and MICS

surveys are not accounting for.

Misclassification Bias

Biological vs Non-biological mothers

DHS collects maternal and child health information on the past five years from biological

mothers only; MICS interviews any household care taker on data from the past two years. This

disparity in data collection may affect comparability between the two surveys.

There are also cultural factors to consider regarding maternal reporting. In some cultures,

women may consider their nieces, nephews or adopted children as their own and may respond

to the questionnaire as if they are the biological mothers. It is integral, therefore, to consult with

individuals who have intimate knowledge of the culture and customs in the area so

questionnaires can be modified as such.

Age Heaping/Displacement

Analysis of survey data concludes that certain ages are often heaped together at certain digits,

particularly ages which end in a 0 or a 5. Oftentimes child deaths will heap at one year old. In

addition, a comparison with the female-to-male sex ratio reveals that women’s ages are being

displaced into younger than 15-19 and older than 45-49 categories. This places the women

outside of interviewing age. Similarly, many children are displaced to the >5 years old age group

to avoid certain sections within the questionnaire. It is also suggested that early neonatal deaths

may be omitted due to emotional or cultural factors.

Sibling Survivorship

DHS and MICS often ask women to report the sex, age, and (if applicable) date and cause of

death of each of their siblings who has lived to the age of 15 years old. This information is used

to calculate both male and female adult mortality rates and assess cause of mortalities.

Page 10: Limitations of DHS and MICS

Oftentimes issues arise regarding recall bias, “don’t know” responses, misclassification of who is

considered their sibling and cultural factors.

Deviation from MEASURE DHS/UNICEF Methodology

The MEASURE DHS program is staffed with approximately 70 individuals consisting of

demographers, data processing specialists, physicians, public health professionals, geographers,

biomarker specialists, data analysts, laboratory technicians, qualitative research experts, data

dissemination specialists, editors and report production staff. The degree of technical assistance

provided to a country is highly dependent on the availability of the limited staff.

The UNICEF MICS program has a highly decentralized organizational structure, leaving

implementing agencies to make critical decisions regarding survey implementation. The MICS

Phase III Evaluation reports that significant data quality lapses were noted in several countries as

well as deviations from recommended procedures in sampling and fieldwork implementation.

Country-level authority makes deviation from practices very common. There is no written

documentation to assert the adherence to survey guidelines. Sampling techniques and

performing household listing operations may be disregarded completely. Field staff may

interview households that are not chosen for the sample, or they may substitute households.

Field team composition and fieldwork quality controls may differ greatly from recommendations.

During data collection, interviewers may not conduct themselves according to their technical

training, thus disrupting the comparability of survey results. Scheduling fieldwork during a

difficult season, or during political or social unrest may require intense modification of surveying

technique. Data processing, the use of sampling weights and final reports are briefly reviewed by

MEASURE DHS and UNICEF, so there is potential for deviation in that aspect as well.

Overall, funding issues affect the entire survey implementation. Limited funding affects sample

size, staff, time, quality of data and data processing and overall survey reports.

Use

Millennium Development Goals

Both DHS and MICS data are used to track the progress of the United Nations Millennium

Development Goals, specifically goals 4 (Reduce child mortality) and 5 (Improve maternal

health). The lack of accurate baseline estimates for child mortality and maternal health, and the

potentially inaccurate DHS and MICS measurements threaten the validity of tracking progress.

Without a valid measurement of progress, the Millennium Development Goals lose their overall

impact and purpose.

Comparability

Due to DHS and MICS dependence on a country’s implementing agency, there is no purposeful

coordination of survey timing. DHS and MICS suggest surveys be completed every 5 and 3 years

respectively, however actual timing is determined by funding and availability of an implementing

Page 11: Limitations of DHS and MICS

agency. When considering comparisons between countries, it is important to note that surveys

do not occur simultaneously so comparisons may not reflect accurate differences.

Academia

A systematic review found that DHS and MICS data are cited heavily in the academic field.

Between the years 1984 to 2010, 1117 peer-reviewed publications referenced DHS data in over

200 academic journals. MICS data is utilized in a great variety of publications in journals such as

Journal of Health, Population and Nutrition, Tropical Medicine and International Health,

International Journal of Preventative Medicine, International Journal of Epidemiology, Maternal

and Child Health Journal and International Journal for Equity in Health. The heavy use of DHS and

MICS data as a basis for academic analysis is worrisome because inaccurate data leads to

inaccurate conclusions and theories.

Capacity Building

Some countries believe that, after one or two previous surveys, they are capable of technically

conducting a survey. Unfortunately, no countries are able to conduct a survey without financial

support from external funding.

Civil registration and vital statistics systems. DHS and MICS are only proxy population

measurements done in the absence of adequate civil registration and vital statistics systems.

MEASURE DHS and UNICEF attempt to build capacity to conduct future household surveys;

supplies such as scales or GPS units remain in each country for future use and the organizations

host workshops on the use of data and statistics. Whereas these actions are beneficial for

conducting future household surveys, MEASURE DHS and UNICEF do not focus on establishing an

in-country civil registration system.

Funding

There is a noted positive association between number of publications containing specific DHS

indicator data and United States funding for said specific international health domains. Funding

is the basis for meaningful change and if funding is reflective on inaccurate data, then programs

and supplies will be allocated incorrectly and cannot optimize the funding potential.

Page 12: Limitations of DHS and MICS

References

Ahmed, S., Ali, D., Bisharat, L., Hill, A., LaFond, A., Morris, L., . . . Richter, K. (2009). Evaluation of UNICEF

Multiple Indicator Cluster Surveys Round 3 Final Report.

Attaran, A. (2005). An Immeasurable Crisis? A Criticism of the Millennium Development Goals and Why

They Cannot Be Measured. PLos Med, .

Boerma, J., & Sommerfelt, E. (1993). Demographic and Health Surveys: Contributions and Limitations.

World Health Statistics Quarterly, 222-226.

Bryce, J., Arnold, F., Blanc, A., Hancioglu, A., Newby, H., Requejo, J., & Wardlaw, T. (2013). New Findings,

New Strategies and Recommendations for Action. PLOS Medicine: Measuring Coverage in

Maternal, Newborn and Child Health.

Corsi, D., Neuman, M., Finlay, J., & Subramanian, S. (2012). Demographic and Health Surveys: A Profile.

International Journal of Epidemiology, 1-12.

Eisele, R., Rhoda, D., Cutts, T., Keating, J., Ren, R., Barros, A., & Arnold, F. (2013). Total Survey Error and

the Interpretation of Intervention Coverage Estimated from Household Surveys. PLOS Medicine:

Measuring Coverage in Maternal, Newborn and Child Health, 124-130.

Fabic, M., Choi, Y., & Bird, S. (2012). A systematic review of Demographic and Health Surveys: data

availability and utilization for research. Bulletin of the World Health Organization.

Fischer Walker, C. L., Fontaine, O., & Black, R. E. (2013). Current Indicators for Measuring Coverage of

Diarrhea Treament Inerventions and Opportunities for Improvement. PLOS Medicine: Measuring

Coverage in Maternal, Newborn and Child Health, 101-106.

Gupta, S., Shuaib, M., Becker, S., Rahman, M., & Peters, D. (2011). Multiple Indicator Cluster Survey 2003

in Afghanistan: Outdated Sampling Frame adn teh Effect of Sampling Weights on Estimates of

Maternal and Child Health Coverag. Journal of Health, Population and Nutrition, 388-399.

Hazel, E., Requejo, J., David, J., & Bryce, J. (2013). Evaluation of Community-Based Treatment of

Childhood Illnesses through Household Surveys. PLOS Medicine: Measuring Coverage in Maternal,

Newborn and Child Health, 131-137.

Manesh, A., Sheldon, T., Pickett, K., & Carr-Hill, R. (2007). Accuracy of child morbidity data in demographic

and health surveys. International Journal of Epidemiology, 194-200.

MEASURE DHS. (2012). Biomarker Field Manual. Calverton, Maryland: ICF International / Demographic

and Health Surveys.

MICS Model Reports: Key Findings Report, Final Report. (2014, June). Retrieved from UNICEF Child Info:

http://www.childinfo.org/mics5_modelreports.html

MICS Survey Tools: Questionnaires and Indicator List, Sampling, Planning the Survey, Data Collection, Data

Processing, Tabulation Plan. (2014, February). Retrieved from UNCIEF Child Info:

http://www.childinfo.org/mics5_tools.html

Page 13: Limitations of DHS and MICS

Plowman, B. A., & Fotso, J. C. (2013). UNICEF Evaluation of the Multiple Indicator Cluster Surveys Round 4.

Pullum, T. W. (2006). DHS Methodological Reports No. 5 An Assessment of Age and Date Reporting in the

DHS Surveys. Calverton, Maryland: Macro International Inc.

Pullum, T. W. (2008). DHS Methodological Reports No. 6 An Assessment of the Quality of Data on Health

and Nutrition in the DHS Surveys. Calverton, Maryland: Macro International Inc.

Rutstein, S. O., & Staveteig, S. (2014). DHS Methodological Reports No. 9 Making the DHS Wealth Index

Comparable. Rockville, Maryland: ICF International.

Salaam, D. (2009). Regional Workshop on Civil Registration and Vital Statistics Systems in Africa. The

African Statistical Journal, 475-478.

United Nations Population Division. (2011). Mortality Estimates from Major Sample Surveys: towards the

design of a database for the monitoring of mortality levels and trends. New York: United Nations.

US AID. (2012). Demographic and Health Survey Sampling and Household Listing Manual. Calverton,

Maryland: ICF International .

US AID. (2012). Survey Organization Manual for Demographic and Health Surveys. Calverton, Maryland:

ICF International.

Victora, C. G., Black, R. E., Boerma, J., & Bryce, J. (2011). Measuring impact in the Millennium

Development Goal era and beyond: a new approach to large-scale effectiveness evaluations. The

Lancet, 85-95.