Measuring socio-economic Measuring socio-economic impact of post-disaster impact of post-disaster

sheltershelterDeveloping a standardized methodologyDeveloping a standardized methodology

Shelter Meeting 09a, GenevaMay 2009

Simone van Dijk Alexander van Leersum

Intro Intro Measuring the impact of post-disaster shelter: short- or long-term?

Status Quo: ◦ Long term (i.e. impact) vs short-term evaluation

Experience of case studies◦ Vietnam◦ Indonesia

Development of standardized methodology

Objectives of the presentationObjectives of the presentation1) Long term effects of housing 2) Aims of a standardized methodology3) A proposed methodology

◦ Before-after◦ With-without

4) Measuring socio-economic dimensions◦ Direct measurement◦ Indirect measurement (e.g. Wealth-index; PCA)◦ Self-assessment

5) Cross-programme comparison

1) Long term effects of housing1) Long term effects of housing

What kind of a relation can be distinguished?

Intertwined elements, problems with causalities and counterfactual analysis

Housing conditions(e.g. construction

materials, size, structural

strength, cultural appropriateness)

Protection from: cold, heat, rain

Satisfaction with life:- sense of belonging

- sense of empowerment

Vulnerability to storms & floods:

- Human safety- Safety of possessions

Repair and maintenance costs of

housing

Living Standards &allocation of

household resources: - food

- consumer durables- motivation to work

Health conditions: physical and mental

I ncome generating activities

Economic independence:

- loans and debts- income levels

Condition Direct effects Indirect effects

2) Aims of a standardized 2) Aims of a standardized methodology (1/2)methodology (1/2)

What is the added value?

Examples of expected gained insights/added values◦ Proportion of income spent on

housing adaptations◦ How long does a HH uses a

house, how do they use it?◦ Is the house cultural

appropriate?

2) Aims of a standardized 2) Aims of a standardized methodology (2/2)methodology (2/2)

Developing a methodology for a long-term impact study that allows for comprehensive analysis of a household’s socio-economic situation.

Developing a general approach and methodology for long-term socio-economic impact studies irrespective of location.

Developing a general methodology that is suitable for the evaluation of different types of post-disaster housing programs.

What do we want to achieve?

3) A proposed methodology (1/4)3) A proposed methodology (1/4)

How to tackle causality questions?

With-without (intervention vs control)◦ Selection of groups

Before-after (impact of intervention)◦ Timing; project cycle

3) A proposed methodology (2/4)3) A proposed methodology (2/4)

Before and after: The importance of timing

Living standards

2000 2007

Control group (i.e. non-beneficiaries)

Impact of intervention

Beneficiaries

Time

With-without & Before-after effect research

Comparability of groups

3) A proposed methodology (3/4)3) A proposed methodology (3/4)With-without: selection of intervention- & control group(s)

Control group, intervention group(s)

Socio-economic position should be comparable between groups with respect to: Average age of the household Labor force Average size of the household Predominant sources of income Educational level

3) A proposed methodology (4/4)3) A proposed methodology (4/4)

Different (housing) interventions, e.g. in Ache

Living standards

Dec. 2004 2008

No-support

Time

With-without & Before-after effect research

Permanent Housing (P.H.)

Transitional Shelter (TS)

Only T.S.

T.S. + P.H.

Impact o

f

inte

rventio

n?!

4) Measuring socio-economic 4) Measuring socio-economic dimensionsdimensions

What kind of data is required and feasible?Type of measurement

Discipline Data processing

Type of data

Direct measurement •Income & consumption•Access to social services

- -

Indirect measurement

• Housing characteristics• Durable assets• In-house services

Wealth index (Factor analysis)

Objective

Self-assessment HousingEconomic Living StandardsSocial standards

5-point scale Subjective

4) Measuring socio-economic 4) Measuring socio-economic dimensionsdimensions

What kind of data is required and feasible?Type of measurement

Discipline Data processing

Type of data

Direct measurement •Income & consumption•Access to social services

- -

Indirect measurement

• Housing characteristics• Durable assets• In-house services

Wealth index (Factor analysis)

Objective

Self-assessment HousingEconomic Living StandardsSocial standards

5-point scale Subjective

What type of latrine does your household uses?

Flush toilet (setting) [ ] 1

Toilet (standing) [ ] 2

Other, unhygienic latrine [ ] 3

No latrine [ ] 4

Do you have days of food deficits per week?

No [ ] 1

Sometimes [ ] 2

Yes [ ] 3

The quality of your present house is...

Very bad [ ] 1

Bad [ ] 2

Ok [ ] 3

Very good [ ] 4

Good [ ] 5

5) Cross-programme comparison 5) Cross-programme comparison (1/3)(1/3)

(How) can shelter programmes be compared among each other?

Examples of questions which can makecomparison possible

Only relative comparison is possible; comparison of different socio-economic status of groups

E.g. Proportion of income spent on housing adaption/extension/repair

Standardization of methodologyPartly standardization of questions

5) Cross-programme comparison 5) Cross-programme comparison (2/3)(2/3)

Why a standardized methodology?

Strength of method lies in the combination of◦ Focus: socio-economic dimensions related to housing◦ Approach: intervention + control group(s)◦ Timing: different moments in time, to measure impact◦ Data collection: objective and subjective◦ Data collection: quantitative and qualitative

5) Cross-programme comparison5) Cross-programme comparison(3/3)(3/3)

What are the underlying aims of a standardized methodology?

What can we expect from insights of impact studies?

1)Support from donors 2)Improvement of the program design3)Improvement of the (shelter)design

Thank you

6) Extra6) ExtraMeasurement of impact further explained

Answer scale’s

Measuring mean-difference between groups

Statistically significant different?(e.g. T-test, ANOVA, Cross-tabs)

5% overlap (1st order error, Alpha) 95% Confidence Interval

Intervention Group

Control Group

H0 =µintervention ˃ µcontrol

6) Extra6) Extra

PCA further explained

Data type Analyzed variables

Actual variables to construct 'Wealth Index',

based on Principle Component analysis

Housing rooms/person, type of wall, floor, roof

floor type

Consumer Durables radio, refrigerator, bicycle, TV, motorbike, car, ploughing machine/tractor, other agricultural equipment, mobile phone, fixed telephone, sewing machine, electric fan, rice cooker

radio, bicycle, TV, electric fan, rice cooker

Services water, sanitation, electricity, fuel for cooking

electricity, sanitation

Wealth index values ranging between 0 to 5(most poor towards least poor)