Assessing Public Expenditures on Social Protection:

Some Methodological Suggestions

Kathy Lindert, World BankQualidade do Gasto Publico no Brasil

June 26-27, 2003

Outline: Key QuestionsSector-Wide View What are the goals of the sector? Where does the money come from (financing)? How much is spent? Where does the money go? (composition of

spending) Basic inventory Mapping against key vulnerable groups Gaps, duplications, horizontal inequities across

programs

Program View (what do you get for the money?) Institutional aspects Performance indicators

Sector-Wide View

Goals of the Social Protection “Sector”

SOCIAL ASSISTANCE-To provide support to the

extreme poor (structural poor)-To help households cope with

shocks (transient poor)- To reduce inequality (redistributive goals)

- To reduce future poverty via human capital conditions (CCT)

- To provide social services to particularly vulnerable groups

(Generally in form of non-contributory transfers – cash or

in-kind; conditional or not)

SOCIAL INSURANCE

- To help individuals, households mitigate the

adverse effects of risks and shocks, such as: old age, disability, health shocks,

unemployment, etc.

(Generally in form of contributory payments)

To assist individuals, households, and communities to better manage risks, and to provide support to the critically poor

Goals, continuedGoals depend largely on profile of poverty, risk and vulnerability: Profile of Poverty:

Poverty headcount, gap, trends Characteristics of the poor Degree of chronic vs. transient poverty (what share are structurally

poor vs. poor due to shocks?) Profile of risks and vulnerability

Sources of risk and vulnerability Specific vulnerable groups (e.g., disabled, street children, child

laborers, indigenous, etc.)

Goals also depend on the extent to which private transfers, insurance are available and accessible to the poor (formal, informal)

Where do funds come from? (sources of financing)

General RevenuesOther sources (mainly for social insurance): Payroll taxes/contributions

Employer/employee contributions Self-employed contributions

Penalties from employers Interest income Other – fees for service

Other sources (not always counted): Donors, NGOs Counterpart contributions

Financing Issues

Earmarked or not (e.g., fondo de pobreza)

Federal, state, local

Open entitlement vs. fixed budget allocation

Reliability, pro-cyclical or counter-cyclical

How much is Spent? (1)Not an easy question to answerSpending spread across multiple programs, multiple agencies, and multiple levels of governmentNeed an inventory of programs & spending: At federal level (across agencies) At state/municipal level:

Probably can’t do complete inventory for all states/municipalities

Select sample (some with high capacity/spending, some with low capacity/spending) to gauge overlaps, complementarities with federal spending

...How much is spent? (2)With inventory of spending on each of the (main) programs, can estimate how much is spent on SPCalculate as % of GDP, total public spending, total social spending

Expenditure Estimates on Social Protection (SP), Social Insurance (SI) and Social Assistance Programs (SA) Country SP SI SA

% of GDP % of GDP % of SP % of GDP % of SP Argentina 5.0 4.1 82.0 0.9 18.0 Brazil 10.8 10.3 95.4 0.5 4.6 Mexico 4.3 3.2 74.4 1.1 25.6 Costa Rica 5.4 3.7 68.5 1.7 31.5 El Salvador 5.9 4.2 71.2 1.7 28.8 Guatemala 3.0 1.9 63.3 1.1 16.9 Honduras 4.1 1.6 39.0 2.5 61.0 Nicaragua 6.5 5.4 83.1 1.1 16.9

Source: World Bank, various reports. Data mostly for 1999/01 period, central government spending only.

Composition of Spending (1): Basic inventory of programs

Social insurance: Pensions (general old age, survivor, disability, civil service) Unemployment insurance [Health insurance]

Social assistance: Cash transfers

Conditional (linked to health, education sectoral goals) Non-conditional (pure cash transfers)

In-kind transfers (food – such as school feeding, other) Subsidies (food, energy, agricultural, housing) Workfare (transfers in exchange for public works) Active labor market programs

Type of Program Number of Programs

Budget 2000 (million pesos)

Percent of total

Budget

Major Beneficiaries

1. Social Insurance —Social Security —Negative Income Tax

4 3 1

170,539.0 158,687.0 11,760.0

76.1 70.8% 5.2%

- Formal sector employees - Formal sector employees

2. Sectoral Social Assistance —Education —Health —Housing credit —Other

29 18 5 2 4

15,861.9 6,622.8 4,740.7 3,779.6 718.8

7.1% 3.0% 2.1% 1.7% 0.3%

- Poor, low educated - Rural poor - Public sector employees - Various vulnerable groups

3. Income Transfers and Subsidies —Progresa (conditioned income T) —Food Programs

7 1 6

14,765.2 9,635.0 5,130.2

6.6% 4.3% 2.3%

- Rural poor - Poor

4. Income Generation —Temporary Employment —Labor Training —Rural Development

54 1 2

51

15,531.8 3,997.7 1,683.9 9,850.2

6.9% 1.8% 0.7% 4.4%

- Poor unemployed - Low income - Rural communities

5. Social Infrastructure 5 2,250.1 1.0% -- Communities with low access to basic infrastructure

6. Natural Disaster Protection 1 4,839.9 2.2% -- Communities hit by natural disasters

7. Other 5 202.8 .09% -- Poor communities TOTAL 105 223,990.7 100%

Composition of Spending (1): Basic inventory of programs: Mexico Example

Composition of Spending (2):Mapped Against Key Vulnerable Groups

One quick and practical way to analyze the mix of programs is to map spending on the main programs against key vulnerable groups using a “life-cycle” approachAdvantages of this “life-cycle”mapping: Easy, quick overview of where funding goes (main groups) Can identify possible overlaps, spending biases (e.g., in favor of

elderly rather than youths), and possible gaps – for broad groups

Some disadvantages: Not sure of actual gaps, duplications at household level Life-cycle approach mainly focuses on individual risks, ignores that

these individuals are actually part of households

Population Group/

Main Risk Leading Risk Indicators Indicator Value, Urban Area Indicator Value, Rural Area

Poorest 10 percent

Poorest 30 percent

All Poorest 10 percent

Poorest 30

percent

Poorest 60 percent

All

Ages 0–5: Stunted development

Chronic Malnutrition (stunting) 11%. 30%

ECD coverage n.a. n.a. Preschool attendance (age 5

only) 53% 71% 82% 44% 52% 62% 66%

Ages 6–14 Low human capital development

Primary school enrollment (ages 6–11)

92% 94% 95% 86% 91% 93% 94%

Age-for-grade* (ages 6–11) 1.07 1.04 1.01 1.13 1.12 1.10 1.08 Lower secondary school

enrollment (ages 12–14) 41% 55% 68% 24% 24% 29% 37%

Age-for-grade (ages 12–14) 1.19 1.13 1.07 1.36 1.37 1.31 1.24 Child employment (ages 12–14) 16% 13% 9% 40% 30% 28% 26% Violence/crime Inactivity—neither work nor

attend school (ages 12–14) 21% 12% 7% 9% 14% 14% 13%

Composition of Spending (2, cont’d):Identifying Vulnerable Groups: Mexico Example

(Highlighted areas signal prominent at-risk groups)

Population Group/

Main Risk Leading Risk Indicators Indicator Value, Urban Area Indicator Value, Rural Area

Poorest 10 percent

Poorest 30 percent

All Poorest 10 percent

Poorest 30 percent

Poorest 60 percent

All

Ages 15–24 Low human capital development

Upper secondary enrollment (ages 15–17)

19% 29% 46% 6% 6% 7% 13%

University enrollment (ages 18–24)

2% 4% 15% 0.7% 0.2% 0.9% 2%

Low income Unemployment (ages 15–24) male

female

24% 12%

23% 12%

13% 10%

4.4% 0%

3.2% 6.9%

5.2% 5.3%

4.9% 5.2%

Violence/crime Inactivity (ages 15–17) 37% 28% 18% 31% 30% 29% 27% Inactivity (ages 18–24) 49% 43% 28% 29% 36% 36% 35% Ages 25–64 Low income Unemployment: male

female 6.5% 3.5%

4.9% 2.8%

3.2% 1.8%

0.6% 0.0%

0.9% 1.5%

1.5% 0.8%

1.5% 0.6%

Part-time job (as % of all employed)

37% 32% 26% 52% 50% 46% 42%

No education or incomplete primary (low skills) (ages 25–40)

49% 36% 17% 70% 68% 61% 49%

No education or incomplete primary (low skills) (ages 41–64)

78% 69% 42% 93% 90% 90% 85%

Ages 65+ Low income Receives pension 2% 7% 22% 0% 0.1% 0.2% 7% General population

Low-quality housing

No piped water 18% 15% 7% 55% 53% 48% 38%

No piped sewerage 50% 37% 18% 96% 93% 89% 79% No electricity 5% 3% 0.9% 29% 24% 20% 14%

Mexico Vulnerable Groups …

Population Group Leading Risk Indicators Indicator Value, Urban Area Indicator Value, Rural Area Isolated rural villages (population 100 or less, total 2.6 million people) Indigenous (total population 11,500,000 people, of which 80 percent (9,200,000) among the extreme poor, and 1 million monolingual)

Lack of access to basic infrastructure, social services

High dependency ratio (large proportion of children and elderly relative to working-age population)

Systematically higher

poverty rates and lower social development indicators

Not applicable

(all indicators for indigenous are given in next column—data do not allow rural/urban breakdown)

- Without access to: electricity (59 percent), sewerage (90 percent), primary school (40 percent), secondary school (100 percent), health mobile unit (30 percent), access to temporary employment program (94 percent), free tortilla (99 percent), etc. - These villages represent 2.9 percent of total population, but 3.3 percent of youth (0–14) and 3.4 percent of elderly (over 65), vs. 2.6 percent of working age population (25–64). -Illiteracy rates, age 15 and over: 49 percent (women), 27.8 percent (men). In communities with over 70 percent indigenous populations (total pop. 4,000,000), 28 percent of children do not attend school, most living in communities with 100 or fewer inhabitants. Over 1 million school-age children speak indigenous language, 250,000 of which are monolingual. Only 38 percent of schools in indigenous communities offer all 6 grades of primary education.

The life-cycle faces its limits and so we move beyond it:

Social risk among specific population groups in Mexico

Population Group/

Main risk Program Coverage

Indicator Value, Urban Area

Indicator Value, Rural Area

Poorest 10 percent

Poorest 30 percent

All Poorest 10 percent

Poorest 30 percent

Poorest 60 percent

All

Ages 0–5: Stunted development

Family receives Progresa transfers

n.a

ECD coverage n.a. Ages 6–14 Low human capital development

Receives scholarship to attend school (6–11)

2.8% 2.0% 2.0% 3.1% 3.4% 3.9% 4.1%

Violence/crime Receives scholarship to attend school (12–14)

1.6% 1.7% 2.2% 3.4% 5.6% 5.0% 4.5%

Ages 15–24 Low human capital development

Receives scholarship to attend school (ages 15–17)

3.6% 1.5% 3.6% 1.9% 1.3% 1.3% 1.3%

Low income Receives scholarship to attend school (ages 18–24)

0% 5.6% 5.0% 0% 0% 3.3% 2.8%

Ages 25–64 Low income Has attended training

program 0% 2% 12% 0% 0% 0.2% 2%

Has access to social security system

9% 19% 43% 0.2% 0.9% 3% 11%

Receives ‘ayuda alimentaria o dispensa’

1.6% 4% 12% 0% 0.1% 0.7% 3%

Ages 65+ Low income Receives pension 2% 7% 22% 0% 0.1% 0.2% 7% General population

Low-quality housing

Access to housing credit 0.7% 4% 16% 0% 0.1% 0.4%

Low access to savings/credit facilities

Taking the next step: mapping programs onto risk groups

Incidence of programs targeted to key social risks by decile and region

Composition of Spending (3, cont’d)Gaps, duplications, horizontal inequities

across programsAnother important aspect in analyzing the mix of social programs is the extent to which they incur duplications or gaps in coverage, which result in horizontal inequitiesExamine which households receive:No benefits (if poor, a gap)Benefits from one programBenefits from multiple programs

Composition of Spending (3, cont’d)Gaps, duplications, horizontal

inequities across programsData needed: Representative household survey data with

comprehensive listing of main (nation-wide) programs

Such data are not regularly collected in Brazil PNAD do not include comprehensive list of main

social programs POF survey will yield some such data for first time

since 1996... an important opportunity for policy feedback

Overlap between Social Risk Management Arrangements

File: tt_hh.dta ( 7 Feb 2002 ) 8 Feb 2002

Venn Diagram N = 7276

Social_Protection Private_Transfers

(76 %) (28 %)

1306 (18 %)

1649 23 %

408 6 %

3913 54 %

% of total File: tt_hh.dta ( 7 Feb 2002 ) 8 Feb 2002

Venn Diagram N = 7276

Social_Insurance Social_Assistance

(8 %) (75 %)

1714 (24 %)

484 7 %

4968 68 %

110 2 %

% of total

File: tt_hh.dta ( 7 Feb 2002 ) 8 Feb 2002

Venn Diagram N = 7276

Local_Remittances Foreign_Remittances

(15 %) (8 %)

(13 %) Charity

5219 (72 %)

45 1 %

401 6 %

699 10 %

10 0 %

448 6 %

308 4 %

146 2 %

% of total File: tt_hh.dta ( 7 Feb 2002 ) 8 Feb 2002

Venn Diagram

N = 7276

School_Feeding

Scholarships

( 39 %)

(3 %)

( 21 %) (6 %)

School_Material_Assistance

Other_Social_Assistance

4164 ( 57 %)

41 1 %

49 1 %

1223 17 %

64 1 %

111 2 %

118 2 %

1235 17 %

11 0 %

13 0 %

8 0 %

32

0 %

13

0 %

2 %

167 0 %

1

26 0 %

% of total

Composition of Spending (3, cont’d) Gaps, duplications, horizontal inequities across programs, example

How Adequate?... A Judgment CallJudgments: Does the program mix have the appropriate blend of social

assistance and social insurance? Appropriate mix of public – private provision/financing? Given the country’s profile of poverty and vulnerability, does

the program mix provide an adequate balance of efforts to assist:

The chronic poor (structural poor) The transient poor (due to shocks) Special vulnerable groups (young children, youths, disabled, etc.) Of formal/informal sectors?

Are there big gaps in intervention? Significant overlaps, duplication, fragmentation? Is the overall level of effort sensible? Too high? too low?

Program Analysis: What do you get for the money?

Indicators to Evaluate(for each of main programs)

Institutional Aspects: Objectives Institutional Arrangements & Delivery Mechanisms Sustainability

Performance Indicators (What do you get for the spending?) Adequacy (Coverage, benefit levels) Equity Efficiency Impact (poverty, inequality, human capital, etc.)

Institutional AspectsObjectives of Program: Ideally should evaluate program against these Often, programs have multiple objectives

Institutional Arrangements & Delivery Mechanisms: Agencies responsible for design, implementation Administrative structures Resources and systems (adequacy) Incentive structures Targeting mechanisms Delivery of benefits

Sustainability: Is the burden on the budget sustainable? How would predicted

demographic, poverty or fiscal changes affect this?

Adequacy of ProgramsCoverage Who benefits from the spending? disaggregated as relevant: urban/rural, poverty groups,

region, formal/informal

Adequacy of benefit level What is the average transfer? Benchmarks vary by program, e.g.,:

Average pensions compared to average wages Unemployment insurance to average wages Social assistance to poverty line, etc.

Data sources: Institutional data Household survey data (coverage)

Adequacy: Coverage (of poor)

Figure 12.3 – Coverage of SA Programs by Poverty Group (ENCOVI 2000)

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%S

ha

re o

f p

opu

lati

on

in

rec

ipie

nt

ho

us

eh

old

s

Non Poor All Poor Extreme Poor

Adequacy: Benefit LevelsFigure 12.1 - Average Benefit Levels and Dispersion among Social Protection Programs

0

2500

5000

7500

10000

12500

15000

17500

20000

Pen

sio

ns

Su

rviv

ors

hip

Alim

on

y

Sch

ool

Fe

edin

g

Sch

ola

rsh

ips

Sc

hool

ma

teri

als

Sc

hoo

ltr

ans

port

Ele

ctr

icit

ysu

bsid

y

Oth

er

SA

Qz

/ H

ou

seh

old

/ Y

ea

r

Note: Vertical bands indicate the dispersion of benefit levels, from 5th to 95th percentile. World Bank calculations using the ENCOVI 2000, Instituto Nacional de Estadísticas – Guatemala

Poverty Line

Equity Examine: Who receives how much? (distributional incidence of

benefits received across deciles/quintiles) Errors of exclusion, inclusion By welfare group (decile, quintile) Also by other pertinent groups (urban/rural,

informal/formal, gender, race etc.)

Data Needed: Nationally representative household survey data that

includes questions on receipt of program benefits (and how much received)

Such data are currently weak in Brazil (PNAD don’t include social programs)... POF opportunity

Equity: Distributional IncidenceGuatemala: Targeting Outcomes of SP Programs

0%10%20%30%40%50%60%70%80%

Soc

ial

Insu

ranc

e

Soc

ial

Ass

ista

nce

Sch

ool

Fee

ding

Sch

olar

ship

s

Tra

nspo

rtS

ubsi

dy

Ele

ctric

ityS

ubsi

dy

% o

f ben

efits

rec

eive

d by

poo

rest

(Q

1) a

nd r

iche

st (

Q5)

qui

ntile

s

Q1

Q5

Overall Effectiveness: Coverage, benefits, equity

Figure 12.6 - Effectiveness of Social Assistance in Reducing Poverty

School Feeding

Scholarships

School materials

assistance

Electricity subsidy

Other SA

School transport subsidy

-10%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

-20% -10% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Coverage of the poor: % of ex-ante poor receiving the benefit

Note: The size of the bubbles is proportional to the relative importance of the transfer in the consumption of the recipient

Targ

etin

g In

cid

enc

e: %

fund

s g

oin

g t

o e

x-a

nte

poo

r

World Bank calculations using the ENCOVI 2000, Instituto Nacional de Estadísticas – Guatemala

Efficiency

Specific indicators vary by program. Some examples: Social assistance: administrative costs Unit costs: how compare with international

practice or local benchmarks? Pensions: Effective rate of return All programs: does intended budget reach

beneficiaries or are there indications of resources being siphoned off for unintended uses?

Impact on labor markets (discourage work?)

ImpactImpact on relevant outcomes: Changes in poverty, inequality Changes in employment Human capital outcomes (e.g., do more kids

attend school due to conditional transfer?) etc.

Numerous methodologies for assessing these Simulations using household survey data

Simple simulations (given transfer amount received) Simulations taking into account behavioral effects

Ex post impact evaluations (with/without; before/after; control/treatment groups)

ConclusionsImportant to look both at individual programs Efficiency, effectiveness, impact

But also at spending across programs in sector Gaps, duplications, fragmentation Appropriate mix or major biases Multiple providers:

Levels of government (federal, state, local) Various agencies / ministries

Uses of such analyses: Management and planning feedback Possibly suggestive of needed overhaul, integration,

rebalancing of safety net