Assessing Information Needs and Survey Alternatives

1

Assessing Information Needs and Survey Alternatives

Kathleen Beegle, DECRGPoverty and Inequality CourseModule 1: Multi-topic Household SurveysJanuary 23, 2008

2

The Demand for Data

1. Performance-based management has created pressures on developing countries to improve the quantity and quality of their macro and micro-data:

Is the public sector delivering good services?

Are country policies/poverty reduction strategies reducing poverty?

Is aid supporting poverty reduction?

In the World Bank: “Results-based” CASs, for example

3

The Demand for Data

1. Performance-based management2. Millennium Development Goals (MDGs)

reflects the strong commitment on the part of the international development community and national governments to monitor and evaluate the results of their policies and programs (United Nations, 2000).

4

The Demand for Data

1. Performance-based management2. MDGs

3. HIPC, Poverty Reduction Strategies (PRSPs)

The HPIC (debt reduction initiative for heavily-indebted, low-income countries) requires

a plan for poverty reduction that can be measured and monitored access to concessional lending from the World

Bank and International Monetary Fund (IMF/IDA, 1999a and 1999b).

5

The Demand for Data

Much of the increased demand for data has been focused on household-level data. It is not possible to monitor PRSPs or MDGS

without solid household data Measurement of welfare and other key social

indicators. (Muñoz and Scott, 2005). Decisions on appropriate policies to reach the

MDGs or the targets in the PRSPs need household data.

Although they do not exclusively need/require only household-level data, although they often imply survey data

6

What these demands look like

PRSPMeasure welfare/poverty Identify problems--magnitude, causesAlternative policiesCost/benefitMonitorEvaluate

7


PRSP MDGs

MDG 1: Eradicate extreme poverty and hungerMDG 2: Achieve universal primary educationMDG 3: Promote gender awareness, empower womenMDG 4: Reduce child mortalityMDG 5: Improve maternal healthMDG 6: Combat HIV/AIDS, malaria and othersMDG 7: Ensure environmental sustainabilityMDG 8: Develop a global partnership for development

8


PRSP MDGs General Demand

Poverty and Inequality Benefit Incidence Analysis Public services Determinants of observed outcomes Assessment of alternative policies Impact Evaluation Inputs to Program Design

9

Recent Efforts to Increase Data: Quantity and Quality

Partnership in Statistics for Development in the 21st Century (PARIS21) was initiated to “…to act as a catalyst for promoting a culture of evidence-based policymaking and monitoring in all countries, and especially in developing countries.” (PARIS21 web site). 1999

Trust Fund for Statistical Capacity Building (TFSCB), managed by the World Bank, set up in 2000 to help build the capacity of statistical systems/ statistical plans

10

Recent Efforts to Increase Data: Quantity and Quality

World Bank line of credit for Statistical Capacity Building to support countries in the implementation of these statistical master plans.

The Program to Improve Surveys of Living Conditions in Latin America (MECOVI), (co-sponsored by the IADB, ECLAC, WB, regional program to improve quality and data

SPARC- Eastern Carribean Initative to improve surveys, UNDP, CDB, WB IADB, OECS, inter alia. (2004)

LSMS Phase IV: Methodological Research on measurement, field work and technological advances

11

Focus of Presentation

Assessing Information Needs Sources of data Need for multiple sources of data Role of household surveys

12

Assessing Information Needs

Inputs Financial & physical indicators of inputs (monthly)

Internal Outputs Achievement/performance

indicators (annually)

Outcomes Benefits/usage indicators

(annually)

External

Impact Indicators of improvements in living standards (~ 5 years)

13

Data Sources National accounts Current public expenditure statistics Program of Price collection (cons./prod.) Administrative Records (from line

ministries) Qualitative Work Surveys:

Multi-topic w/ welfare focus (LSMS/IS) Monitoring (CWIQ, PS) Income and Expenditure (IES, HBS) Single topic (Labor Force Surveys (LFS) Demographic and Health (DHS)) Enterprise Facilities

14

Surveys and Policy Analysis

Social GoalsSocial Goals

Increase enrollment

Increase female LFP

Lower infant mortality

Gov’t ProgramsGov’t Programs

Conditional Cash Transfers

Day care centers

Public Health Campaign

Households

Individuals

Firms

Surveys: Going Beyond Rates

Understanding secondary school enrollments, 12-18 year olds, Albania 2002

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

AveragePercent

• In almost all countries we have a single statistic: mean enrollment at the national level. In this case it is 61%.

•This is interesting for monitoring purposes, but it doesn’t say much about poverty or other factors.

•... A regional disaggregation would be useful

Understanding secondary school enrollments, 12-18 year olds,

Albania 2002

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

AveragePercent

• In some countries we have regional breakdowns, with marked contrasts

•The contrast between urban and rural rates emphasizes the disadvantages faced by rural communities.

• Other breakdowns would be useful

Urban

Rural


Albania 2002

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

AveragePercent

• …possibly, official statistics can add the gender dimension

•…the figures show that, in urban areas, there is no gender differential but a large gap in rural areas.

•But we still don’t know much about who sends their children to school

Urban

RuralMale

Female

Male

Female


Albania 2002

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Q1 Q2 Q3 Q4 Q5

Percent

•…With a survey we can show enrollment rates broken down by consumption level--and thus understand an additional dimension

Consumption quintile

Female, urban

Male, urban

Male, rural

Female, rural

Average

19

Gathering information through surveys (household or other)

There is a range of options They can be ordered along two main

dimensions: degree of representativeness subjective/objective dimension

Casestudy

Purposive selection

Quotasampling

Small prob.sample

Large prob.sample

Census

Degree of Representativeness

Direct measurement

Questionnaire(quantitative)

Questionnaire(Qualitative)

Structuredinterview

Openmeetings

Subjective assessments

Conversations

Casestudy

Purposive selection

Quotasampling

Small prob.sample

Large prob.sample

Census

Subjective/Objective Dimension

Direct measurement



Structuredinterview

Openmeetings


Conversations

Casestudy

Purposive selection

Quotasampling

Small prob.sample

Large prob.sample

Census

Wonderful World of Surveys

Windscreen Survey

Participant observation

Beneficiary Assessment

Participatory Poverty Assessments Sentinel Site

Surveillance

Household Budget Survey

Census

CWIQ/PSLSMS/ IS

Community Surveys

Direct measurement



Structuredinterview

Openmeetings


Conversations

Casestudy

Purposive selection

Quotasampling

Small prob.sample

Large prob.sample

Census

Wonderful World of Surveys: “Statistical Surveys”

Household Budget Survey

Census

CWIQ/PSLSMS/ IS

24

Features of a Statistical Survey

Structured Questionnaire Random/Probability Sample

25

Tradeoffs to Consider When Planning a Survey as Part of a System of Surveys

Overall scope Single vs. Multi-topic Probability vs. Purposive Sampling Sampling vs. Non-Sampling Errors Time vs. Cost Data vs. Capacity Building Surveys over time: repeated cross

sections, panels, rotating

26

Summary

Surveys are one source of information among many (system of information) Consider all the key elements of a National

Poverty Monitoring System

27

Key Elements of a National Poverty Monitoring System

Timely Annual National Accounts Current Public Expenditure Statistics Program of Consumer and Producer Price

Statistics In-depth Welfare Survey (LSMS/IS, IES?) ‘Light’ Annual Monitoring Surveys (CWIQ, PS) Longitudinal Studies Qualitative work on key topics Specific tools for project/program/policy

monitoring and evaluation

28

Summary

Surveys are one source of information among many (system of information)

No one survey can meet all data needs: System of Household Surveys

29

System of Household Surveys Goal: System able to respond to

evolving needs: not produce data X or survey YDetermine data needs before they are

URGENT Identify appropriate instruments, Implement them properly, timely

fashion,Analyze the resulting data

30

Improving the SHS Linking Users and Producers Providing adequate resources Continuous Survey Program

Not necessarily permanent survey Benefits

Avoid loss of capacity Create greater levels of capacity (building on

existing) Economies of scale Policy makers know when data will be available Protects NSO from pressures for ad hoc surveys Ongoing system actually allows more flexibility

and responsiveness

31

Summary

Expanding demand for timely, relevant data

Need to determine the range of data needs to begin to define a system of information

Surveys are one, important, source of information among many

No one survey can meet all data needs: System of Household Surveys

32

References

United Nations (2000). “United Nations Millennium Declaration.” United Nations’ General Assembly, Fifty-fifth Session, New York, New York.

International Monetary Fund and International Development Association (1999a). “Building Poverty Reduction Strategies in Developing Countries.” Report to the Board of Directors, International Monetary Fund and International Development Association, Washington, D.C.

International Monetary Fund and International Development Association (1999b). “Heavily Indebted Poor Countries (HIPC) Initiative: Strengthening the Link between Debt Relief and Poverty Reduction.” International Monetary Fund and International Development Association, Washington, D.C.

Muñoz, Juan and Kinnon Scott (2005). “Household Surveys and the Millennium Development Goals.” Paris21, processed.

33

References

DECRG, (2006) “LSMS IV: Research for Improving Survey Data”, processed

LSMS Web page: http://www.worldbank.org/lsms/

ISLC/ MECOVI web site: http://worldbank.org/lac

then search on MECOVI

Documents

Assessing Information Needs and Survey Alternatives