Embed Size (px)
DESCRIPTION
Mirjam Harmelink14 March 2012
Citation preview
Harmelink consulting [email protected]
Classical evaluation frameworks for energy
efficiency programs
IEA – SEAI Workshop on “Evaluating the Multiple Benefits of Energy Efficiency”14th March 2012, IEA, ParisMirjam Harmelink (Harmelink consulting)
Harmelink consulting [email protected]
Program/Policy Planning Cycle
PolicyObjective
s
PortfolioDesign
ProgramImplementati
on
Program Monitoring
and Evaluation
Program
Design
Harmelink consulting [email protected]
Types of evaluation carried out in the field of energy efficiency programs
Evaluation type
Description Uses
Impact evaluation
Quantifies direct and indirect benefits of the program/policy
Determines amount of energy savings, emission reductions and in some cases possible co-benefits
Process Evaluation
Indicates how the program/policy implementation procedures are performing from both administration and participant perspective
Identifies how program/policy process can be improved
Market Effects Evaluation
Indicates how the overall supply chain and market have been affected by the program
Determines changes that have occurred in markets and whether they are sustainable with or without the program/policy
Cost-effectiveness evaluation
Quantifies the cost of program implementation and compares program/possible benefits
Determines whether the energy efficiency program/policy is a cost-effective investments as compared to other programs and energy supply resources Source: National Action Plan for Energy Efficiency Leadership Group
(2007)
Harmelink consulting [email protected]
What do we typically evaluate in impact evaluations?
4
Ex ante
Ex post
Target achiev. /
Effectiveness
Impact/Effectiveness
Efficiency
€
€
To what extent did policy instruments make a
difference in meeting the targets compared to the
situation without the policy instruments in
place?
What was the cost effectiveness of
the policy instruments, and
could targets have been reached against lower
costs?
To what extent are policy
targets being met?
Harmelink consulting [email protected]
En
erg
y s
avin
gs/
CO
2 r
ed
uct
ions
Baseline (no policies)
With policies
Autono-mouschange
Net policy impact G
ross C
han
ge
MeasurementPolicy launch
Main challenge: setting the baseline
Harmelink consulting [email protected]
Methods applied in policy evaluationTop down methods
– End use or sector indicators – Economic / econometric modelling
Bottom-up methods– Direct measurement– Analysis of energy billing or sales– Modelling (based on stock and market statistics and surveys)– Enhanced engineering estimates– Mixed deemed and ex-post estimate– Deemed estimate– Theory based policy evaluation– Logic modelling– Equipment indicators
6
Harmelink consulting [email protected]
Energy efficiency programs often have multiple objectives
0 2 4 6 8 10 12
Energy savings only
CO2-reduction
Innovation
Competiveness/employment
Poverty control
Other
Number of case studies
Source: Ecofys, Lund University, Politecnico Milano, Wuppertal Institute (2007) www.aid-ee.org
Harmelink consulting [email protected]
Examples on exploring (co-)benefits (2)
Source: RIVM, ECN (2010) Co-impacts of climate policies on air polluting emissions in the Netherlands
Harmelink consulting [email protected]
Main challenges• Developing good evaluation framework > linking
energy efficiency improvements and co-benefits in a plausible way (theory)
• Developing a sound baselines• Defining suitable indicators• Results should spur the right discussion > but
remember – Tinbergen Rule: “for each and every policy target there
must be at least one policy tool. If there are fewertools than targets, then some policy goals will not be achieved”
Harmelink consulting [email protected]
11
Thank you for your attention!
For more information please contact
Mirjam Harmelink
T: +31 6 42342483
