# Ex Post evaluation of cohesion policy intervention 2000-2006 financed by the Cohesion Fund (incl. ISPA) WP C – CBA of Environmental Projects Workshop

#

Ex Post evaluation of cohesion policy intervention 2000-2006 financed by the Cohesion Fund (incl. ISPA)

WP C – CBA of Environmental Projects

Workshop with Member States,Brussels 3 February 2011

Session 2: Introduction and findings of 10 Environment Projects of WP C

Christina van Breugel, Tine Skyggebjerg, Szabolcs Szekeres, Birgitte Holt Andersen (COWI)

Davide Sartori, Silvia Vignetti (Csil)

13 Feb 2011 Ex post eval cohesion fund_DG Regio

#

Overview of the 10 case studies


Waste management

Water management

Waste water management

01 Bulgaria

03 Pilsen

17 Barcelona

06 Crete

09 Zaragosa

27 Hungary

22 Dublin

13 Madrid

29 Poland

50 Portugal

#

Outline of presentation

1. Presentation of the Ex ante situation

2. Reporting of results of ex post CBA

3. Main differences from ex ante to ex post

4. Typical components and typical benefits

Waste management projects

Water Management projects

Waste water projects

5. Main findings related to:

Ex ante CBA

Ex post CBA


#

1. Ex ante situationMain driver for project initiation and project context


CASES Lega

l com

plia

nce

Envi

ronm

enta

l

Heal

th

Risk

miti

gatio

n

Oth

er

Self

cont

aine

d

Upg

radi

ng e

xisti

ng

infr

astr

uctu

re

Part

of

'Mas

ter P

lan'

01 Bulgaria/Waste Mngt X X X X03 Czeck Rep/Pilsen: Combined water X X X X06 Greece/Crete: Waste Mngt X X09 Spain/Zaragoza: Water supply X X X X13 Spain/Madrid: Waste/Energy recovery X X17 Spain/Barcelona: Waste water X X X X22 Ireland/Dublin: Waste water X X X X27 Hungary/Szeged: Waste water X X29 Poland/Szczecin: Waste water X X X X X50 Portugal/Lipor: Waste management X X

8 6 1 1 2 4 3 8

Main driverProject context

MAIN DRIVER

Legal compliance 8Environmental 6

Health 1Risk mitigation 1

Other 2

No of projects

PROJECT CONTEXT

Self contained 4Upgrading existing 3

Part of 'Master Plan' 8

No of projects

#

1. Ex ante situationEx ante assumptions, financial and economic analysis


CASES01 Bulgaria/Waste Mngt03 Czeck Rep/Pilsen: Combined water06 Greece/Crete: Waste Mngt09 Spain/Zaragoza: Water supply13 Spain/Madrid: Waste/Energy recovery17 Spain/Barcelona: Waste water22 Ireland/Dublin: Waste water 27 Hungary/Szeged: Waste water29 Poland/Szczecin: Waste water50 Portugal/Lipor: Waste management

QUALITY OF FINANCIAL ANALYSIS

High quality 2Medium quality 7

Poor quality 1

No of projects

Very

reas

anab

le

Reas

onab

le

Wea

k or

misl

eadi

ng

High

qua

lity

Med

ium

qua

lity

Poor

qua

lity

High

qua

lity

Med

ium

qua

lity

Poor

qua

lity

X X XX X X

X X XX X X

X X XX X XX X X

X X XX X X

X X X2 6 2 2 7 1 3 2 5

Ex ante assumptions

Quality of ex ante financial

analysis

Quality of ex ante

economic analysis

EX ANTE ASSUMPTIONS

Very reasanable 2Reasonable 6

Weak or misleading 2

No of projects

QUALITY OF ECONOMIC ANALYSIS

High quality 3Medium quality 2

Poor quality 5

No of projects

#

1. Ex ante situationThe use of CBA for project formulation and decision

One thing is the quality of the ex ante – another thing is HOW the CBA was actually used for project formulation and decision making

CBA used for project formulation?

– NO - Why not?

(a) since most projects emerged to comply with legislation

(b) project is part of a larger context (Master Plan)

CBA used as basis for decision making?

– Not really, perhaps due to the timing of CBA


'The city authorities opted

for this project largely

because they were told

that only such an integrated project would be

eligible for financing'

#

1. Ex ante situationThe timing of the CBA


CBA

Project identificationProject formulation

EIA process/develpment consentsApplication

TenderingConstruction Operation

3-5 years 3-5 years 30 years

The project is usually already a part of a process or part of a regional or local Master Plan

The Project needs to be seen in this context

#

2. Reporting on results of ex post analysis

Most projects investigated appear to be sensible investments as they provide for fundamental EU environmental infrastructures ('needs to have'/legal compliance)

In economic terms however only one project generate positive ex post ENPV meaning that economic benefits justify the costs

Few problems with either over-capacity or under-capacity

The implemented technical solutions are with one exception sensible and reasonable

Size of investment costs appear more or less reasonable according to our technical experts

There are however unrealised benefits in some of the projects

Wider benefits include: improved environmental awareness among citizens, political benefits, enabling benefits, etc.


#

2. Reporting on results of ex post analysisOutcome of the projects (1/2)

93 March 2010 Ex post eval cohesion fund_DG Regio

Project outcomes

Fulfillment of objectivesScore 1-5

Financial sustainability Direct benefits Unrealised benefitsHindrances for

realizing benefits

Other project outcomes/unintended

effects

01 Bulgaria 4

Only the operational cost can be financed by the waste management tax No re-investement are included for e.g. sealing of the cells

Reduction of illegal waste disposal _ _A first step toward a modern waste handling management system

03 Czeck Rep/Pilsen: Combined water

4

Yes due to the tariffs and since the affordability analysis of the total tariffs showed no risk of payment problems among household consumers.

Much improved quality of drinking water + reliability of supply of tapped waterReduced pollution of the river due to reduced risk of overspill of sewage systemConnection of approximately 1000 households to the central sewage system

Upgrading of the river environment (underway)

Perhaps slowed down now due the financial crisis

Enabled other projects to emerge, e.g. GreenwaysInstitutional learning, e.g. establishing of UKEP

06 Greece/Crete: Waste Mngt

4UncertainOP cost much higher than budgetRevenues lower than budget

Reduction of illegal waste disposal of 90000 tonnes of waste a yearRecycling of 30.000 tonnes of waste a year

Use of compost for soil enrichmentReduced use of pesticidesRefined sorting of waste100% waste management

Lack of awareness among farmersLack of education of households to sort waste at source

Waste management know-how to benefit rest of CreteIncreased environmental awareness among citizens

09 Spain/Zaragoza: Water supply

5The project is sustainable thanks to full cost recovery tariffs

Significant improvement in water quality _ _

Reduction of potabilization treatment costs for municipalities. the project has also indirectly induced the municipality of Zaragoza to undertake actions that have radically reduced water losses and modified the behavior of the users, with beneficial effects on environment and safety of water supply.

13 Spain/Madrid: Waste/Energy recovery

5

The energy generation component generate a surplus. While the other components require operational funding form the municipality.

Closing off illegal landfill - stopping negative environmental impactsEnergy generation

Unutilized synergies (energy plant) with other facilities of the combined Madrid management facility

Contractual bindings of current contract with different operators

An important part of the combined Madrid Waste Management facility

#

2. Reporting on results of ex post analysisOutcome of the projects (2/2)

103 March 2010 Ex post eval cohesion fund_DG Regio

Project outcomes

Fulfillment of objectivesScore 1-5

Financial sustainability Direct benefits Unrealised benefitsHindrances for realizing

benefits

Other project outcomes/unintended

effects

17 Spain/Barcelona: Waste water

5

Based on the financial indicators, the project is not sustainable. No income is generated. However; the economic assessment shows a sustainable investment.

Reduced risk of flooding (giving positive effects on quality of bathing water, traffic congestion and reduction of damage costs)Improved know how of water management. Qualitative benefits includes access to ground water for irrigation and street cleaning, improved ecosystem in marine water, public information activitiesand cleaner streets.

_ _

A lot of knowledge and experience has been gathered. The project has many visits from delegations from other cities and countries. There has been a lot of awareness rising among the population.

22 Ireland/Dublin: Waste water

3

The Project capacity is based on the information from the time of the application - and now it requires an upgrading to cater for unexpected increased demand - therefore it is not financially sustainable and requires the identification of further investment funds. Further, a revision of the user fee structure is needed.

The direct benefits include an increase in housing permissions and recreational benefits, in particular in connection with cleaner Dublin Bay beaches.

_

The design capacity of the new infrastructure is too low and it does not comply with the requirements of the newly designated sensitive waters. Hence, further investments are needed to fully reap the benefits.

_

27 Hungary/Szeged: Waste water

5

The project is financially sustainable despite its poor financial results, attributable to the high capital costs. But the fixed assets are not in the books of the operator. Tariffs are sufficent to cover operating and reinvestment costs and are inflation indexed.

The direct benefits are the extension of the sewage system to new users, who therefore avoid the cost of dealing with their own individual sewage solutions.

The objective of a cleaner Tisza River is unrealized because the dillution effects of the river are so strong (2400:1). The effects of the project might be more directly felt in the future, when incoming river water will be cleaner (icluding that of the Maros, which arrives from Romania)

_

The project is excellently run. There has been some awareness rising among the population.

29 Poland/Szczecin: Waste water

5

The project appears to be financially sustainable as user fees have been allowed to increase in line with the increase in the quality of service provision.

Direct benefits to the inhabitants of Szczecin due to improved quality of waste water and water supply services, and from improved environmental conditions of the Oder River.

Since the new infrastructure has only been in operation for aroumnd a year, it is still early days for estimating e.g. possible impacts on business and housing developments.

_Szczecin is no longer a HELCOM hotspot

50 Portugal/Lipor: Waste management

5

Financial sustainability is insured either by green or "guaranteed" administrative prices and close control of costs

Comprehensive and efficient management of urban solid waste generated in the second largest conurbation of the country with a population in excess of one million inhabitants

-

Usual implementation delays of a fairly large and multisectoral investmentAcceptance of compost quality by farmersAwareness to waste source separation + recycling of the public at large

Not detected

#

3. Main differences from ex ante to ex post

Financial results

– higher operational costs

– lower operational income

– project delays

– investment costs overrun


Economic results

– overestimated or arbitrary economic benefits

– unrealised benefits

#

3. Main differences from ex ante to ex postResults of financial and economic analysis (1/2)


EX ANTE

•The investments has risen due to budget overruns – some ex-post CBA are only done for some of the component and are therefore not comparable

•The FNPV are small or even negative – the negative results are larger in the ex-post analyses - no good business cases have been identified

•The ENPV are more positive in the ex-ante analyses than the ex-post CBAs – When complying with legislation a less positive ENPV could be accepted as it is part of a higher level objective of generally improving the environment

#

3. Main differences from ex ante to ex postResults of financial and economic analysis (2/2)


• Few B/C ratios above 1

• A number of the benefits have only been qualitatively described in the CBAs due to lack of benefit estimates

•A number of projects have experienced overcapacity

• Barcelona – improved marine water quality

•Hungary – covers just a section the Tisza river

•Bulgaria - is part of a master plan of national waste handling

#

4. The typical components and typical benefits of the different types of projects

Waste management

Drinking water management

Waste water treatment


# 153 Feb 2011 Ex post eval cohesion fund_DG Regio

Waste Management


Water supply


Waste Water

#

5. A) Main findings related to ex ante CBA

The main CBA ex ante issues:

not integrated in the decision process

not looking at individual components

missing the bigger pictures (e.g. synergies, risk un-realised benefits)


Recommendations:

Do the CBAs much earlier in the process. A solid CBA should precede doing the final technical design of the project

CBA thinking should apply to the selection of alternatives, prior to the final design of the project

CBA to be related to the Master Plan context

Cost efficiency analysis could be considered for 'need to have' projects

#

5. B) Main findings related to ex post CBA

This study has used CBA for ex post impact assessment and have drawn some lessons:

1) starting from individual components is the easiest way to identify the benefit elements

2) concentrate on valorisation of the main benefit elements, if too uncertain the result is altered unreasonable

3) the wider benefits are often important outcomes but are difficult to quantify


Considerations

1) break-even analysis might be more convincing to illustrate economic surplus/deficit

2) develop a price and benefit catalogue (Excel tool) to support CBA in practice

3) CBA to be combined with other qualitative impact assessment methodologies to improve the capturing of wider benefits


THANK YOU FOR YOUR ATTENTION!

Documents

# Ex Post evaluation of cohesion policy intervention 2000-2006 financed by the Cohesion Fund (incl. ISPA) WP C – CBA of Environmental Projects Workshop