The Mexican Family Life Survey within the Surveys´ contexts regarding the evaluation of social programs in Mexico

Luis Rubalcava

OBJECTIVE

The role of population based panel surveys as a tool to evaluate social programs and policies over time

Its link to administrative records

roadmap

1. Panel Surveys2. Advantages of a Panel for evaluation3. On going panel survey in Mexico, MxFLS4. Example of Evaluation of Health Insurance Program5. Example of Evaluation of OPORTUNIDADES (CCT)

1. Panel survey

Panel study is one in which the unit of analysis is interviewed in different periods of time

Rotative panel studies in which the sample is followed only for a limited period of time. The sample is refreshed every certain period

Longitudinal studies that only go back to original locations (interview whoever is there)

Longitudinal studies that do follow up of people who leave the original household

Longitudinal studies over generations (PSID)

2. Advantages

Important tool for the evaluation of natural experiments Economies of scale-- it is useful for many evaluations and policies, as opposed to

evaluations done purposefully for one program Long term exposure analysis—if you have a long enough panel and low attrition Difficult to do in randomized trials

2. Advantages

Excellent tool to study transitions Health transition Labor transitions Poverty transitions

2. Advantages

Important for the study of migration Domestic migration International migration Return migration

3. MEXICAN FAMILY LIFE SURVEY

CAMBS/UIA/CIDEMéxico

GOAL

Collect information on the welfare of Mexican households Representative of the population at the national, urban-rural and regional level Multi-thematic Longitudinal Community component

Characteristics

Conducts in-person interviews (to all household members) Collects objective health information (anthropometric measures and

biomarkers) with specialized health personnel Comprises information on ONE single instrument (households and individuals) Follows individuals overtime, regardless of their place of residence Re-contacts in Mexico Re-contacts in the USA

Characteristics

First round (baseline survey) in 2002 Second round in 2005-2006 Third round in 2010-2011 Original sample: 35,000 individuals

Topics Household Level

Economic characteristics (income, expenditure, wealth, savings, credits) Transfers inside and outside the household Victimization Agricultural land use and management (rural households)

Topics INDIVIDUAL Level

Human capital (schooling, cognitive ability and health status) Employment and time use Migration (national and international) Reproductive health Marriage history Anthropometric measures and biomarkers Reproductive health, contraceptive methods and pregnancy history Health perception, mental health

HEALTH

Objective health information (anthropometric measures and biomarkers) of all household members: Weight Height Waist circumference Hip circumference Blood pressure >15 years old Hemoglobin >6 months old

HEALTH

Objective health information (anthropometric measures and biomarkers) of all household members:

2005 + cholesterol LDL, cholesterol HDL, total glucose and dried blood samples

In 2010+Hba1c, leg length

Topics COMMUNITY Level

Collects information on the infrastructure of the communities where the sample households live:

Schools Health centers Small health providers Local authorities Commercial establishments

Advantages Evaluation of Public Policies

The MxFLS allows: Analyzing socioeconomic and demographic indicators of the same individuals over time PANEL For example, who is in poverty over an extended period? Who can leave poverty? Who falls

into poverty? MULTIPURPOSE Studying the relationship between different variables For example, what is the relationship between poverty and migration to the US? What is the

relationship between economic productivity and health? Conducting causal analyses.

Advantages Evaluation of Public Policies

The MxFLS allows: For example, how does having a US migrant affect mental health? Dynamic Analysis: Mobility of the same individuals over time. Do they remain in the formal sector?

Do they exit the formal sector to go to the informal? Local Infrastructure and its impact on household welfare Example: What is the relationship between the local infrastructure and economic well-being? Does the

lack of public schools affect educational levels? By how much?

Advantages Evaluation of Public Policies

The MxFLS allows: Program Evaluation of large programs represented in the sample. For example, Evaluation of Prospera (formerly Oportunidades). Evaluation of Seguro

Popular, health insurance program.

4. Example: Evaluation of seguro popular

Background

By 2000, 50 million of the Mexican population did not have health insurance => Catastrophic health expenditures were covered directly by the uninsured affecting the poorest

families.

Seguro Popular (SP) represents the effort of the Mexican government to: Reduce catastrophic health expenditures. Reduce health inequalities Improve health status of the uninsured. Government goal to achieve by 2012 universal coverage.

Background

Fig1. Number of individuals affiliated to the three main public health institutions in Mexico

Millionsof

people

Study Objective

Since SP is close to achieve the goal of universal coverage, we evaluate the impact of SP on: Health services utilization Health Status Out-of-pocket health expenditure

Using longitudinal data from the Mexican Family Life Survey (MxFLS) implemented in 2002 and 2009-2011.

DATA

Advantages of using MxFLS Baseline was conducted in 2002, before the period in which SP was expanded, Provides socio-economic and demographic characteristics of the individuals before the

implementation of the program, which can be used as controls to analyze the impact of the programs. Provides information on several health outcomes before and after the implementation of the

program.

Hypotheses

Non-beneficiaries eligible for coverage

<Hypothesis 3

SP BeneficiariesOut-pocket health expenditure

Health Status

SP Beneficiaries Non-beneficiaries eligible for coverage>

SP Beneficiaries Non-beneficiaries eligible for coverage

>Hypothesis 2

<

Hypothesis 1

Health services utilization

METHODS

Difference-in-Difference Propensity Score Matching Estimate propensity scores

P(i) = Pr (Di =1|i) (1)

Xi Observable characteristics before treatment

Di =1 if at least one household member is a beneficiary of the SP

Match propensity scores Estimate difference-in-Differences

ATTDD = E(1-0|T1 =1) - E(1-0|T1 =0) (2)

results

Health services utilization Increase of 6% in the probability of visiting

a doctor in the case of SP beneficiary men living in rural areas.

0%1%

2%3%

4%5%

6%7%

6%

Beneficiary men living in rural areas

results

Health services utilization Increase of 47% in the number of visits to

the doctor of SP beneficiary women living in urban areas.

No impact for children from 0 to 5 years old.

0%10%

20%30%

40%50%

47%

Beneficiary women living in urban areas

resultsGraph2. Impact of SP on Health Status of Rural Women 20-60 years old

Relative Self-Reported Health (Very Good, Good)

ADLS for age>50: Can not Stand Up from the floor

Rural Women 7% -17%

7%

-17%-20%

-15%

-10%

-5%

0%

5%

10%

resultsGraph3. Impact of SP on Health Status of Urban Men 20-60 years old

Cold Cough Abdominal pain Heart Disease

Urban Men -6% -5% -4% -3%

-6%

-5%

-4%-3%

-7%

-6%

-5%

-4%

-3%

-2%

-1%

0%

resultsGraph6. Impact of SP on Out-of-Pocket Household Health Expenditure

National- Total Reported Household Health Expenditure

Urban- Contraceptive Methods

Households -56.0% -100.0%

-56.0%

-100.0%-100.0%

-90.0%

-80.0%

-70.0%

-60.0%

-50.0%

-40.0%

-30.0%

-20.0%

-10.0%

0.0%

5. Example: Evaluation of OPORTUNIDADES (CCT)

“Do Conditional Cash Transfer Programs Improve Work and Earnings among its Youth Beneficiaries? Evidence after a Decade of a Mexican Cash Transfer Program.”

Susan W. Parker, CIDE

Luis N. Rubalcava, Spectron/Cambs

Graciela M. Teruel, Universidad Iberoamericana

introduction

CCT Program Longer-term effects can be directly measured only after a significant number of

years of program operation and follow-up. Little is known about the longer-term impacts of CCTs on work and earnings of

youth beneficiaries.

objective

We estimate long term impacts of Oportunidades on youth education and labor market outcomes in rural areas using 3 rounds (2002, 2005, 2009) of the longitudinal Mexican Family Life Survey.

Does additional education induced by CCT’s improve employment and salaries as youth beneficiaries enter adulthood?

Intergenerational effects on reducing poverty?

objective

Sample: Rural children age 10 to14 when Oportunidades began in 1997-1998, followed until the last round of the MxFLS (2009) when they were in their early to mid twenties. Impacts observed in 2002, 2005 and 2009.

hypotheses

Schooling: Transfers reduce the shadow wage of children’s time in activities other than school.

Work: Initially school subsidies reduce time spent in work. (substitutes)

But with more schooling, youth receive higher wages offers. Eventually should increase working as well as wages/job conditions.

DATA

Use MxFLS--Low attrition, migrants followed and interviewed.

Administrative data on Oportunidades receipt at community level merged to MxFLS.

Definitions of groups

Treatment group: “Early” beneficiaries, youth in communities incorporated in 1997 or 1998 (T1998)

Control group: “Late” beneficiaries, youth in communities incorporated in 2004 or afterwards (T2004)

results

Initial results confirm impacts of Oportunidades on accumulating additional education. 0.

18

0.47

0.37

Grades of completed schooling

results

Some apparent increases in labor market participation of Oportunidades youth. No impact in 2002

0

0.13

0.11

Currently working

results

No impacts on income per hour or job benefits.