Ireland

Evaluating the impact of enterprise/industrial policy

supports: developing new methods and approaches

Dr Helena LenihanDepartment of EconomicsKemmy Business School

University of Limerick

Helena.lenihan@ul.ie

Our world has changed!

Industrial Policy in Europe

• A stronger European Industry for Growth and Economic Recovery (10th Oct 2012)

• The pillars for a reinforced industrial policy – Investment in innovation– Better market conditions– Access to capital– Human capital and skills

Progress report from the ‘Action Plan’ (Q1-2012):

• Launch of a new Potential Exporters Division in Enterprise Ireland

• €150 million Partial Credit Guarantee Scheme.

Progress report from the ‘Action Plan’ (Q1 and Q3-2012):

• A second call under Innovation Fund Ireland

• Introduction of funding programmes targeting female entrepreneurs and also for business clustering

• Pilot Industry-led clustering programme

Progress Reports on the Action Plan for Jobs

Departments and agencies have delivered to completion:

Q2-2012 -A completion rate of 94%.Q3-2012-A completion rate of 87%

Old wine in new bottles?

In 2011, 82 Enterprise Ireland-backed entrepreneurship programmes took place

Specifying Policy objectives

Lack of Enterprise policy evaluation nationally

Compared to the UK, Ireland has been very poor in building datasets to inform policy

design and evaluation (Ruane, 2012)

Defining Policy Evaluation

• Exploring the counterfactual• What works best under what circumstances?• Comprises analytically rigorous attempts to

determine the impact of public policy initiatives.

• Key challenge to isolate the impact of a particular policy intervention.

Why Evaluate?

• Impact

• Value for money

• Leads to policy improvements

Lots of calls recently for Policy Evaluation (1)

2012 Action Plan for Jobs – “We will complete evaluations of all agency

programmes and schemes to ensure that they are adequately addressing the needs of

today’s manufacturing businesses”

(Action 7.1.4).

Policy Evaluation

Let’s keep it on agenda

- Let’s do it properly this time–not just another fad

(1) Current evaluation frameworks/metrics are generally lacking vis-à-vis the evaluation of ‘newer’ enterprise policies coupled with a move from ‘old’ towards ‘new’ business models.

(2) Illustrate how ‘LOGIC’ models can be extended to account for broader ‘societal’ impacts.

The ‘world’ for the firm has also changed (1)

• Old business model characterised by:– desire for business stability– focus on core business activity– efficiency attainment via standard

procedures– focus on developing in-house

specific competencies

The ‘world’ for the firmhas also changed (2)

• ‘Evolutionary’ approach– Multi-level perspective– No firm is an island

• Enabling conditions for enterprise start-up and growth.

• R&D; innovation, commercialisation of capital, networking and clustering, entrepreneurship and education at the heart of enterprise policy.

“New” enterprise/business model focuses on:

• Innovation and novelty and flexibility• Lack of predictability• Operates as part of a network• Combines internal & external ‘know how’• Operates as part of supply chain models• Invests in Human as opposed to physical

capital• Knowledge based economy and rests on

intangible assets

What does the new approach mean for policy evaluation? (1)

Evaluation metrics should include as many as possible of the following:

• Socio-economic fabric of locality or region• Extent of additionality• Opportunity cost issues• Impact on ‘wider publics’ (Public Interest

Evaluation’) and a concern with social impact and value

What does the new approach mean for policy evaluation? (2)

• Impact on sustainable development• Impact on average local/regional wage

rates within economy.• Displacement and crowding out effects

across firms.• Capacity building• Linkages between firms• Multiplier impacts

What does the new approach mean for policy evaluation? (3)

• Knowledge creation through institutional diversity and pluralism.

• Impact of the environment shaping the economy upon the dynamics of entrepreneurship/enterprise activity.

Logic model-potential to make a worthwhile methodological contribution?

• Provide a guide as to what should be assessed and measured.

• Incorporate multi-level (layered) effects.• Compare ideal against actual operation in

a programme.• Not only look at impact at level of firm but

has potential to consider wider economy and even society effects.

InputsPers

on

nel

Mon

ey

Tim

e

Tech

nolo

gy

Part

ners

Exp

eri

en

ce

Events & Work-shops

MeetingEmail

Co-opInterfirmAcademicAgencies

Change in Mindset:Increased Collaboration:Learning: Skills

Risk with sharing info

Increased knowledge;Innovation;

Firm Profitability

Firm Survival.Increased

Attractiveness of the Local

Economy; Jobs

Figure 1: Logic model (chain of cause-and-effect)

Unproductive relationships;

Box 1

Box 2

Box 3

Adapted from Lenihan (2011)

ActivitiesShort-Term &

Medium-Term Outcomes

Long-Term Outcomes

Increased Network & Economic activity;

Improved Knowledge

TransferLocal, Regional

National

Figure 1: Logic model (chain of cause-and-effect)

Positive on society

Improved infrastructure;

Negative on society

increased traffic congestion;

consumer prices

Box 4

Box 5

Box 6

Adapted from Lenihan (2011)

UnproductiveNetworks:

Displacement

Entrepreneurial activity in Society

Box 7

Economic Development Innovation;

Welfare Effects

Box 8

Public Policy interventions

Source: Lenihan, H. (2011) ‘Enterprise policy evaluation: Is there a ‘new’ way of doing it?’Evaluation and Program Planning, 34 (4), 323-332.

Conclusion and way forward for policy evaluation (1)

• Challenge: move away from evaluation methodologies that are purely concerned with narrowly defined economic impacts and measure impact purely at level of assisted firms.

• Bottom line: firm and economic activity more broadly do not operate in a silo but operate as part of a ‘holistic’ system.

Conclusion and way forward for policy evaluation (2)

• Greater investment should be made with regard to ex-ante evaluation.

• Evaluation as a ‘science’ is lacking when it comes to theory.

• Caution against the use of datasets originally collected for different purposes.

Conclusion and way forward for policy evaluation (3)

• Old Chestnut: ‘How’ to evaluate (single biggest challenge)

• Logic model-potential

• March 2012-Irish Government Economic and Evaluation Service (IGEES)

Conclusion and way forward for policy evaluation (4)

• Make the ‘Evaluations’ publicly available

• Most important specify policy objectives

The way we evaluate enterprise policy impact must also change!

Thank you

Comments/Questions?

Helena.Lenihan@ul.ie