United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity
Building in Italy
Martina Bolli, Silvia Ciampi, Francesco Giordano & Laura Tagle
Italy’s National Evaluation System
Content
1.Cohesion policy in Italy: long-term
background & contrasting
directions
2.The Italian evaluation policy
3.Efforts & (unlikely) achievements…
plus some drawbacks
COHESION POLICY IN ITALY: LONG TERM BACKGROUND & CONTRASTING DIRECTIONS
2007-2013 - Evaluation as one of the pillars of a (much needed) renewal of Italy’s regional policy
• Citizen-centered policies
• Strong focus on essential services—rather than on procedures & expenditure
• Emphasis on local knowledge
• Responsibility to regional governments
• Links with national policies
• Greater importance to administrative capacity
The regional (= cohesion) & rural development policy in Italy
• Intentionality of territorial goals
• Additionality vis-à-vis ordinary policy
Resources come from EU & National funds
Levels of programming: a complex framework
• National “umbrella” strategies for regional & rural development policies
• 7 Ministries develop their own programs• Each of the 21 Regions develops
– its multi-sector strategy– 5 Programs (each program is funded by one source)
“Naturally”, each of the 5 programs is usually managed by a separate Directorate
“Programs” end up being bureaucratic constructs rather than guides for action
Over time, silos have been created
Sources of money
• EU Regional Fund• National funds • EU Social Fund• EU Rural Development
Coordination organisms
• Ministry for economic development
• Ministry for agriculture• Ministry for labour
Other departments
….Beyond silos
2007-2013: intentional efforts to bridge gaps & introduce policy dialogue among and within
authorities in a new perspective
At the same time, EU regulations ask for separate
programmes by EU source of money (& within Italy the
momentum for integration dwindles)
The National Strategy for 2007-2013 tried to make the various sources of funding work together through common:
• Programming
• Implementation
• Monitoring
• Surveillance
• Evaluation
21 Regions (70%) 6 Ministries (30%)
Centre-North: 15% South: 85%
3bln€ to be given to Southern Regions on the basis of performance in waste, water, education and social services
Regional development strategyBetween 2007 & 2010: lower national resources (other than national cofinancing)•different distribution North-South•the instruments needed to implement nationally-relevant innovations (among which greater integration) dwindle
THE ITALIAN EVALUATION POLICY
Evaluation capacity building/1
From EU evaluation requirements towards a national policy
• Institutional building:– Creation of Evaluation Units within executive branch of
Regional authorities – Network of Evaluation Units
• Specialized: – Regional and Rural Development policies only
• Both demand & supply side– Improve regional authorities’ ability to Demand for & Use
evaluations
– Improve Evaluation Units ability to Manage and Conduct evaluations
Evaluation capacity building/2
• Organized guidance & support (National Evaluation System)
• Enabling approach:– No rigid prescriptions– Reputational mechanisms
• Legitimize & support innovators at regional level
Evaluation policy /1
• Nationally-funded interventions must be evaluated--not only those funded by EU
• Objects: – effects of a well-defined (however complex) intervention or group
of interventions on a problem, a social group, or an area—rather than a bureaucratic construct (e.g., a multisectoral “program”—tipically operating on a large area, affecting millionsof people)
– across bureaucratic borders: the evaluation object may have been financed by more than one source
– selectivity: choose to answer defined questions about “crucial & controversial issues”
Evaluation policy/2
• Each Region drafts an Evaluation Plan:– What is evaluated (and what is not)
– Who (internally or externally) performs each evaluation
– When
– Organization of evaluation function
– Resources (human and financial) devoted to evaluations
The National Evaluation System• UVAL – Ministry of Economic Development
(Coordinates)
• INEA – Ministry for Agricultural, Food and Forestry Policies
• ISFOL – Ministry of Labour
• Department for Equal Opportunities
• Evaluation Units of Regional Authorities
Its composition shows a clear intention to work together across silos
& involving Regions in a federal function
The National Evaluation System Each component differs from the others
• Central units:– Have differing rules, interpretations, goals, tasks &
status
• Regional evaluation units:– Conduct or manage evaluations…– … but also provide project selection, planning,
knowledge management to the Region– Have background and skills closer to regional fund
policies (or national funds) rather than rural development or active labour policies
EFFORTS & (UNLIKELY) ACHIEVEMENTS… PLUS SOME DRAWBACKS
Common guidance - include & adjust for differing rules, traditions, goals
Personalized support to Regions (all managing authorities together)
Ensure quality & evaluation of evaluations (....)
Observe processes & disseminate information (jointly)
Facilitate dissemination (conference-like meetings, workshops, create & maintain eval database www.dps.tesoro/valutazione.it)
A “federal” function performed across silos & by central & regional units
Some results
• All Regions developed an Evaluation Plan http://www.dps.tesoro.it/uval_linee_valutazione.asp#10
• Evaluations have been launched:– and even completed: 50! (in addition to the mandatory ex
post evaluations of rural development)
– and a lofty 60% available on the internet
• Evaluation units (established between 2000 & 2002) have started conducting internal evaluations
Still far from perfection but …
Why we are happy anyway…
• Activities started (and 50 concluded) even though – (or just because?) there are no enforceable
requirements (no sanctions) – there is still strong inertia in the system
• old evaluations fulfilled needs of managing authorities, national coordinators, and Commission rapporteurs
• restricted market with “marriages” between managing authorities & evaluators
– still little pressure from social partners & general public
• Innovations go together: evaluations seem to be used as an instrument by Regions willing to improve
• Innovations implemented where there has been more support from central state (not only on evaluation)