Economic impact research

Survey of international innovations and best practices

Economic impact 101

Focus: The full effects of a spending action on the commercial activity of a given area

– Eg. For each dollar invested in youth sport Ontario, how many dollars of knock-on economic activity are generated?

Scope: Not just direct impact, but also follow-through (effects on suppliers and consumers)

Limitations: Economic impact is NOT social impact; Economic Impact Analysis (EIA) does not measure effectiveness and is only one half of the value-for-money equation

2

Priority questions

1. How can OTF measure the economic impact of its community investments?

2. How can OTF demonstrate the economic impact of the Not-for-profit (NFP) sector in Ontario?

3. How can OTF provide leadership and vision to facilitate development-oriented commissioning and evaluation throughout the Ontario NFP sector?

3

What do we want to

change / do?

Who can change / do

it?

How much do they need

/ get?

What happened?

What is the new reality?

Where EIA fits in the big picture

4

ApplicationEvaluation

What is that change worth?

1. Economic Impact Analysis: tools, options, and best practices

5

The economic impact equation

Goods with

market prices

Goods with no market price

Total Economic

Impact

• Initial investment

Direct Impact

• Purchasing / supplier value added

Indirect

Impact

• Value added to products / consumers

Induced

Impact

6

7

1a. Measuring the Economic Impact of Goods with Market Prices

Input-Output: The

Whole Economy

Aggregated

Spend Analysi

s: 1

Vertical

ROI: 1 firm, 1

product

Goods with market prices8

Return on Investment Analysis

• Measures profits as a percentage of total costs

–Eg. $50 revenue on $25 investment = 100% ROI • Many firms set ROI threshold ranges for making

investment decisions

–Eg. 0%>10% = “No”, 10%>25% = “Maybe”, 25< = “Yes” • Static indicator / snapshot of one moment in time • One Stakeholder: only values benefits to investor

9

Goods with market prices

ROI is a good starting point for evaluating the monetary impact of a single investment

to one stakeholder at one point in time

10

Goods with market prices

Aggregate Spend analysis is an easy, inexpensive way for OTF to approximate

the direct (1st order) impacts of its spending

Aggregated Spend Analysis (current OTF practice)

• Measures expenditure information for comparable agencies to discern direct economic impacts

PRO: Simple, low-cost approach• Low compliance costs, highly flexible

CON: Non-standardized, unreliable, limited in scope• Some stakeholders: measures benefit to investor and

agency, touches on some benefits to some suppliers

11

Goods with market prices

IO Modelling is the international governmental standard for Economic Impact Analysis, multiplier

calculation, FTE equivalency

Input Output Modelling (IO) (Ministry of Finance standard) • Comprehensive analysis of how shocks to one sector of

the economy affect all other sectors

Pro: Standardized, comprehensive approach

–Rigorous, reliable, and highly instructive

Con: Compliance enforcement

–Requires accurate, uniform expenditure and production info • All Stakeholders: IO modeling accounts for all value added to

all stakeholders, directly and indirectly

Basic Input Output Model

Goods with market prices

Goods / services made & sold

Goods / services bought & used

What one sector makes, another sector uses to make something elseWhen goods are discounted or given away, their true value is

not reflected in these tables; Most NFP sector output DISAPPEARS!

12

Source: Input-Output Economics, 2nd Ed., By Wassily Leontieff (Oxford, 1986)

13

1b. Valuing the Economic Impact of Goods with no market price

14

Goods with no market price

We know that good health and clean water have high intrinsic value, but how can we measure this value when they are seldom bought or sold?

Market v. non Market valuation

Market valuation Can only be applied to

goods bought and sold at equilibrium rates

Value determined by price (revealed preference)

Tells only half of the story (ie. Value of commoditised goods only)

Non-market valuation Can be applied to goods

bartered, discounted, or given away

Value determined by possession or deprivation

Tells only other half of the story (ie. Value of non-commoditisable goods only)

15

Value Determined by Willingness to pay, as

reflected in price

In absence of price, how do we determine value?

Market v. non Market Valuation

Market valuation Q: What’s it worth to you? A: Revealed preference

(the price you paid)

Simple Economic Value Q: What’s it worth to the

economy? A: Net added value of

outputs over inputs

Non-market valuation Q: What’s it worth to you? A: Contingent Valuation

(eg. replacement cost)

Blended Economic Value Q: What’s it worth to

society? (mkt + non-mkt) A: Net value of all

desirable outcomes over all inputs for all stakeholders

16

Valuing Social Goods

Social Return on Investment (SROI) We live in societies, not just economies (Profit, People,

Planet) Social goods have intrinsic value – ALL value is

measurable Social goods carry different value to different people Social goods like volunteer time are not only outcomes

of community investments, they are also inputs SROI is a robust methodology used by governments (eg.

UK), and respected by Nobel Prize-winning economists

17

In commerce, value = price; SROI values stakeholder inputs and outcomes by deducing price information without transactions

Valuing Social Goods

Social Return on Investment (SROI) – How to

1. Identify all stakeholders

2. Determine what each stakeholder puts into a project, and what they get out of it

3. Determine what inputs and outputs are worth to each stakeholder (“Contingent Valuation”: replacement value, willingness to pay, proxy value, substitution premium, etc.)

4. Determine degree of change you can take credit for

5. Calculate net benefit (degree of positive change)

6. Calculate SROI (apply values to net benefit)

18

SROI is an international standard for measuring the intrinsic value of social goods, which are valuable but not commoditised

Valuing social goods

1. Are there any market-priced goods that can serve as accurate proxies for the value of social goods?

2. What is the replacement cost of a social good?

3. How much would you pay to continue enjoying the benefits of a given thing?

4. What would you pay to avoid giving something up?

5. At what rate do you value your time?

Some examples of Contingent Valuation (CV)

In 1993, Nobel-winning economists Robert Solow and Kenneth Arrow used a primitive CV approach to estimate the cost of the 1988 Exxon

Valdez oil spill. Using a Willingness to Pay framework, they valued the costs of the spill between $2.8 - $4.2 billion, based on the price the average American was willing to pay towards programs that would prevent a future spill. Current methods are far more sophisticated.

19

Sample SROI Analysis: I20

Source: A Guide to Social Return on Investment, (2012) The SROI Network:

http://www.thesroinetwork.org/publications/doc_details/241-a-guide-to-social-return-on-investment-2012

Sample SROI Analysis: II21

Source: A Guide to Social Return on Investment, (2012) The SROI Network:

http://www.thesroinetwork.org/publications/doc_details/241-a-guide-to-social-return-on-investment-2012

Blended valuation =

Value of commoditised goods +

Value of social goods

22

• Initial investment

Direct Impact

• Purchasing / supplier value added

Indirect

Impact

• Value added to products / consumers

Induced

Impact

Market value create

d

Social / conting

ent value

created

Total Economic

Impact

Blended valuationF

or

all s

take

ho

lder

s / s

ecto

rs23

2. Social and Economic Indicators of vibrant and resilient communities

24

Why resilience?

Resilience-oriented community developers commission projects that remediate specific areas of vulnerability so that communities become empowered to take care of themselves, come what may

Resilience analysis is at the cutting-edge of needs-based commissioning, providing focus, detail, performance targets and measurement indicators for effective community development

Resilience = The capacity to recover quickly from difficulties, challenges, and unexpected changes of circumstance

25

Recipe for resilience

Resilient Community

Level of Objective Wellbeing

Level of Subjective Wellbeing

Changes / Trends in Levels

of WellbeingSustainability of

Level of Wellbeing / Probability of

Continuity

Asse

ts

(vul

nera

bilit

ies)

Warning system

26

Relating wellbeing to resilience

Is absolute level of wellbeing high?

Is this level stable/ sustainable?

Yes

Vulnerable to shocks

Resilient Community

No

Yes

Develop resilience capacity

Can community adapt / respond / rebound?

No

Yes

Vulnerable to shocks

No

27

Choosing indicators

Top-Down (theoretical model)1. Identify people reporting highest

levels of life satisfaction

2. Isolate objective and subjective indicators of wellbeing with strongest statistical correlation to life satisfaction

3. Identify local threats to wellbeing and local assets that could mitigate effects of unexpected shocks

4. Commission projects that bolster existing defences or build new ones

Bottom-Up (empirical model)

Example: Wellbeing and Resilience Measure (WARM) – Young Foundation, UK

1. Ask people what the most important characteristics of wellbeing are

2. Develop indicator set that suitably captures these characteristics and is capable of accurately tracking them over time

3. Identify areas which perform well against the set and those that don’t, broken down by indicator

4. Commission projects to address poor indicator outcomes

Example: Canadian Index of Wellbeing – University of Waterloo, Ontario

28

Indicator Set

Top-Down approaches

Regression / Sensitivity Analysis (Discern which indicators have

highest impact on life satisfaction)

29

Bottom-Up approaches

Public Input / Definition of Wellbeing

Find / develop indicators for tracking priorities

30

Sample indicators: objective

Living Standards: Income differential, wages vs. cost of living, number of vacancies, number of workless households, incidence of persons with low income, etc.

Health: Life expectancy, infant mortality, percent obese / smokers, physicians/10,000 people, etc.

Community: Crime rate, volunteerism, proximity to relatives, commuting distance, proximity to leisure, etc.

Governance: Voter turnout, ration of registered to eligible voters, newspaper subscription, number of public consultations, etc. NB: Some indicators are positive (as they increase,

so too does wellbeing) while others are negative (as they increase, wellbeing declines)

31

Sample indicators: subjective

Living Standards: Feeling of economic security, job satisfaction, empowerment, inflation expectations, marital status, caring for disabled, etc.

Health: Ability to perform regular tasks, self confidence, incidence of depression or anxiety, perceived availability of care or assistance, etc.

Community: Perceived safety, amount of contact with neighbours, degree of social integration, gentrification, volunteerism, etc.

Governance: Responsiveness, representativeness, opportunities for input, etc.

32

Canadian Index of Wellbeing (composite)

Bottom-up tracking

Community Vitality Indicator

33

Source: Canadian Index of Wellbeing, Community Vitality Indicator http://ciw.ca/_images/other/community_headline_indicators_EN.png

Source: Canadian Index of Wellbeing, Composite Index, http://ciw.ca/en/CanadianIndexOfWellbeing/CompositeIndex.html

Top-down tracking

Wellbeing and Resilience Measure

34

Source: Taking the Temperature of Local Communities, The wellbeing and resilience measure, Authors Nina Mguni and Nicola Bacon, The Young Foundation, (2010 )http://www.youngfoundation.org/publications/reports/taking-temperature-local-communities

Resilience mandala

IndividualSupports (networks)Systems and structures (environment)

35

Analyzing wellbeing

Universal level

Domain level

Targeted level

Overall wellbeing of whole population

Focus on one area of wellbeing / aspect of life experience for

whole population

Measures wellbeing of specific groups and sub-groups within an

area or population

36

International Variations

Country Thought Leader Type Mission

Canada Canadian Index of Wellbeing

Bottom-Up, some top-down

National Indicator development and tracking effort

Germany, Spain, Netherlands

Parliamentary Committees

Bottom-Up Roundtable exercise to determine most appropriate measurement systems

UK NEF/Young Foundation/ Office of National Statistic

Top-down and bottom-up

Policy Focussed scanning, development and evaluation schema

France Institut national de la statistique et des études économiques / OECD / Stiglitz Commission

Top-down and bottom-up

Resolve tension between GDP and quality of life indicators

US National Academy of Sciences

Top-down Key National Indicators System (KNIS)

Australia Australian Bureau of Statistics

Bottom-Up and Top-down

Measures of Australia’s Progress (MAPS)

37