D5.1 Guidelines for Projects Monitoring and Evaluationec.europa.eu/energy/.../files/...projects_monitoring_and_evaluation_en.pdf · IEE/07/710 – EGS: WP5 D5.1: Guidelines for Projects

IEE/07/710/S12.499412

D5.1 Guidelines for Projects

Monitoring and Evaluation

Work package: WP5 – Finalisation in the Participant Schools

Partner responsible for D5.1.:

Multidisciplinary European Research Institute Graz (MERIG)

Author: Johann Laister

Version–Date: v2.0 – 10/09/2009 (final version)

Disclaimer: The sole responsibility for the content of this publication lies with the authors. It does not necessarily reflect the opinion of the European Communities. The European Commission is not responsible for any use that may be made of the information contained therein.

IEE/07/710 – EGS: WP5 D5.1: Guidelines for Projects monitoring and evaluation 2

Index

Executive Summary ................................................................................................................................ 3 Introduction ............................................................................................................................................. 4

EGS – Background of WP5................................................................................................................. 4 EGS Work Organisation.................................................................................................................. 4

WP5 Overview .................................................................................................................................... 5 Task 1. Monitoring and Evaluation Guidelines............................................................................... 5 Task 2. Feasibility Plans.................................................................................................................. 5 Task 3. Implementation and Monitoring of the Pilot Actions......................................................... 6

WP5 Roles and Responsibilities.......................................................................................................... 6 WP5 Organisation Chart ..................................................................................................................... 7

Evaluation Guidelines for the Forums..................................................................................................... 8 Background ......................................................................................................................................... 8 Goals of the Guidelines ....................................................................................................................... 8 Indicators............................................................................................................................................. 9

Potential to Initiate Change ........................................................................................................... 10 Reproducibility.............................................................................................................................. 10 Transferability ............................................................................................................................... 10 Efficiency ...................................................................................................................................... 11 Publicity ........................................................................................................................................ 11 Achievability within the EGS Project ........................................................................................... 12 Impact on Target Groups............................................................................................................... 12 Active Involvement ....................................................................................................................... 12 Coherence with Policies ................................................................................................................ 13 Independency from External Resources........................................................................................ 13

Usage of Indicators – Priority Check ................................................................................................ 13 Priority Description ....................................................................................................................... 14 Indicator Check List ...................................................................................................................... 14 Responsibilities ............................................................................................................................. 14

Feasibility Studies ................................................................................................................................. 15 Monitoring of Pilot Actions .................................................................................................................. 16

Type of Pilot Actions ........................................................................................................................ 16 Guidelines for Monitoring................................................................................................................. 17

Annex .................................................................................................................................................... 18 Priority Check Template ................................................................................................................... 19 Pilot Monitoring Template -- Monthly short check .......................................................................... 20 Pilot Monitoring Template -- Final Monitoring Report .................................................................... 21


Executive Summary

This document presents the Guidelines for Projects Monitoring and Evaluation (deliverable D5.1) of the project Energy Education Governance Schools (EGS) which is co-financed by the European funding programme Intelligent Energy Europe. The Guidelines for Projects Monitoring and Evaluation discuss the whole framework of work package 5 (WP5) but specifically focus on the guidelines and indicators which are used to find priorities in the different selection processes (selection of priorities concerning “energy” in and beyond the school forums established in EGS).

The Introduction on the one hand gives an overview about the whole EGS project and outlines how work package 5 is positioned and interrelated to other work packages. The main aim is to provide tools and methods which support the forums (established in WP4) to select the most suitable priorities, support the elaboration of feasibility studies concerning the selected prioritized approaches, support the selection of at least one pilot action per forum (13 in total) and support the monitoring of the pilot actions.

The main part of the document are the Evaluation Guidelines for the Forums. These guidelines introduce the backgrounds and the indicators which are used to decide which priorities shall in a first step be selected to be further elaborated in a feasibility study and which can lead in the following step to pilot actions. We present and describe 10 indicators which can be used to guide the evaluation and decision process; but we appeal for a flexible usage of the indicators in the forums to open space for the introduction of other indicators which might be relevant for a specific topic or local requirements and to open space for weighting/prioritising the indicators in the school forums. The guidelines outline a practice oriented concept which can easily be used by the persons involved in the selection and decision process and give practical advice how to implement the process.

Chapter Feasibility Studies introduces briefly what is planned for task 5.2, deliverable D5.2 of EGS whereas clear instructions for the forum leaders will be elaborated by the responsible partner UVVG. The feasibility studies report (D5.2) summarises a total of at least 39 short feasibility studies (three from each of the 13 forums). The topics for the feasibility studies are selected in the discussion and selection processes in and around the forums. The feasibility studies will help to take the decision if a certain priority / project idea is worth to be followed and developed further and to be implemented. We propose to use a similar set of indicators also for the feasibility studies.

The last chapter Monitoring of the Pilot Actions outlines how the pilot actions will be observed and guided by monitoring. Each forum will select at least one pilot project out of the three feasibility studies. This pilot project shall be activated, implemented and monitored within the framework of EGS to contribute to more energy efficiency in schools but to test also the selection and monitoring processes and guidelines.

The Annex contains three checklist which will be used: The first checklist is a simple to use template for priority-check and can also be used to document how the forum ranked priorities. The second and third template will be used to monitor and report the progress of the pilot actions.


Introduction

EGS – Background of WP5 The project Energy Education Governance Schools (EGS) aims at involving local communities in tackling energy issues and improving energy efficiency. The EGS project hypothesis is that the main actor that can contribute to involving the whole community and at the same time train the younger generation is the school. Consequently the main target group of the project are schools. Some consortium members are schools and additional schools will be involved in the project activities, so all partner countries are covered. Beyond the primary target group schools (teachers, students, administrative staff) other related groups are addressed by the project and will benefit from the project results: parents/families, local entrepreneurs, local communities and stakeholders. When using the term “EGS target group” in this document we always refer to this wider definition of the target group.

Besides the preparation of an energy state of the art assessment in schools, general awareness raising and mainstreaming of the project results and good practices found, the project plans to develop/integrate new educational content for students, to elaborate and support training for schools staff and to initiate pilot activities. These pilot activities shall be selected and defined by a bottom up approach involving all different target groups (teachers, students, administrative staff, parents/families, local entrepreneurs, interested locals, community stakeholders etc.) addressed by the EGS project.

The main outputs of EGS will be a manifesto in which schools declare their willingness to improve in terms of “energy”, what can have very different implications. This manifest shall be promoted widely. Furthermore energy management systems and audits in schools will be initiated, training for students and teachers elaborated. Local forums will be initiated, which outline and describe priorities that shall lead to the implementation or/and clear documentation of at least one pilot project in each local community/school. In general the monitoring and documentation of good practices is of high relevance in the project EGS because especially well documented good practice examples can be easily adapted and transferred also beyond the project consortium.

EGS Work Organisation The project is organised in seven work packages whereas WP1, WP6 and WP7 are dedicated to management and local as well as transnational dissemination activities.

WP2 organises the initial survey to define the state of the art in terms of energy in general and energy education in the participating and interested schools.

WP3 will raise the schools capacities by initiating new educational offers for students, teachers and schools staff.

WP4 and WP5 are closely related: In WP4 local community forums (local communities at schools) will be build up inviting all relevant target groups to participate in a discussion process in which action plans targeting important challenges and highlighting priorities shall be defined.

In WP5 on the one hand methods and tools will be defined on how to select the most feasible priorities and topics for more detailed feasibility studies and finally the pilot actions for which also monitoring and evaluation methods will be elaborated. On the other hand the pilot actions will be implemented by the forums, local communities or responsible stakeholders during WP5.


WP5 Overview The main aim of work package 5 (WP5) is to provide tools and methods which

• support the forums established in WP4 to select the most suitable priorities.

• support the elaboration of feasibility studies concerning the selected prioritized approaches.

• support the selection of at least one pilot action per forum (13 in total) which shall be implemented during the project.

• support the monitoring of the pilot actions.

Task 1. Monitoring and Evaluation Guidelines The monitoring and evaluation guidelines contain the following actions:

• to draft of the guidelines including indicators

• to discuss these guidelines among the partnership

• to produce the final version of the guidelines D5.1

Aims: + Define an efficient monitoring system for the project.

+ Support the process of finding priorities in the forums.

+ Define meaningful indicators for pilot action selection process / preparation for Task 2.

Output until July 2009 (PM11):

Deliverable: D5.1 Guidelines for monitoring and evaluation

One report/paper (approx. 20-30 pages)

Task 2. Feasibility Plans The feasibility plans contain the following actions:

• Forums: Selection of priorities to be implemented

• Feasibility studies for at least 3 priorities in each forum

• Feasibility studies reporting D5.2

• Identification of at least one pilot action in each forum

Aims: + Find out which topics/subjects are most relevant AND realisable for piloting.

+ Enable the process of starting 13 pilot actions (1 in each forum).

Output until June 2010 (PM22):

Deliverable: D5.2 Feasibility studies report

One report describing the 39 feasibility studies (approx. 40-50 pages)


Task 3. Implementation and Monitoring of the Pilot Actions The implementation and monitoring of the pilot actions contains the following actions:

• Enable at least one project having the EGS/IEE resources available.

• Identify new economic/human resources for completing the activities, for buying eventual renewable energy technologies or installing energy saving systems.

• Identify national fiscal and/or economic incentives.

Aims: + Implement relevant pilot actions

+ Support the implementation process with quality assurance tools

Output until December 2010 (PM28):

Deliverable: D5.3 Pilot actions monitoring & evaluation report

One report describing the selection process, the 13 pilot actions, their results and the evaluation outcomes (approx. 20-30 pages)

WP5 Roles and Responsibilities

WP / Task

Responsible Organisation

Role Supporting Organisations

Roles

WP5 MERIG WP leader, coordination of activities, reporting and communication

All partners Contribution to monitoring and evaluation

T 1 MERIG Draft guidelines, facilitate the discussion process and produce D5.1

13 Partners organising the forums / all partners

Contribute to discussion and definition of indicators, provide outputs of WP4

T 2 UVVG Guide the selection process for feasibility plans (13*3), the feasibility studies and the selection of pilot action topics, produce D5.2


Contribute to discussion, implement the feasibility studies and select one pilot topic per forum

T 3 MERIG Guide the implementation of the pilot actions, introduce monitoring and evaluation, produce D5.3


Implement 13 pilot actions, contribute to monitoring and reporting


WP5 Organisation Chart The main objective of this document is to present the Guidelines for Projects Monitoring and Evaluation (D5.1) what means to discuss the whole framework of WP5 but specifically focus on the guidelines and indicators which are used to find priorities in the different selection processes. The following chart graphically presents the framework of WP5 which will be elaborated in more details in this document.

04/09 05/09 06/09 07/09 08/09 09/09 10/09 11/09 12/09 01/10 02/10 03/10 04/10 05/10 06/10 07/10 08/10 09/10 10/10 11/10 12/10 01/11

D5.1 D5.2 D5.3

Forum Action Plan (WP4)

Priority 1

Priority 2

Priority 3

P4 P5

P6 P7

At least 3 prioritiesare selected in each forum (in total 39)

13 forum action plans developed

in WP4

WP5: Guidelines and tools for the selection of suitable priorities

Feasibility Study 1

Feasibility Study 2

Feasibility Study 3

WP5: Instructions for feasibility studies

One feasibility study for each

selected priority(in total 39)

WP5: D5.2Feasibility

Studies Report

Pilot Action

At least 1 pilot action will be selected & implemented by

each forum/local community (in total 13)

WP5: Guidelines and monitoring indicators for pilot actions

WP5: D5.3Pilot

Actions Evaluation

Report

WP5: D5.1 Monitoring

and Evaluation Guidelines

Problem 1

Problem 2

Problem 3

Problem 6 Problem 5

Problem 4

WP4

: Des

crip

tion

of

EGS

Loca

l Pro

ject

s


Evaluation Guidelines for the Forums

The evaluation guidelines for the forum action plans will introduce the backgrounds and the indicators which are used to decide which priorities shall be selected to be further elaborated in a feasibility study. We try to reduce complexity as far as possible to provide a practice oriented concept which can easily be used by the persons involved in the selection and decision process.

Background In the EGS project 13 local forums1 will be established where the different target groups of the project will participate. Each of the forums will elaborate a Forum Action Plan where also different priorities are highlighted addressing challenges which have been discussed in the forum.

Basically the organisation of the forums is a formalised approach including clearly defined steps and documentation. Nevertheless also in activities which not directly relate to the EGS project (for examples meetings of EGS target group representatives for other purposes) priorities and actions related to the topic energy in schools might be discussed in addition to the forums. The evaluation guidelines can also be used for priorities defined in that way, also leading to feasibility studies and pilot actions as described in the chart above.

Target Groups

Like described above the EGS target group consists of teachers, students/pupils, administrative school staff, school authorities, parents/families, other stakeholders in the local community such as local/regional policy makers, entrepreneurs and in general the interested public. Representatives of these groups will participate in the local forums and the discussion processes.

The main target group for the evaluation guidelines are those who guide the processes in the local forums and are also involved in other discussion and decision making processes besides the EGS project. So mainly those persons who organise, implement and guide the local forums are addressed by these guidelines (i.e. EGS project partners representatives, forum leaders, forum steering groups etc.).

Goals of the Guidelines The evaluation guidelines we propose in this chapter should address requirements which are summarised in the following items:

• To provide a framework for the selection of the “best-fitting” priorities:

The presented framework should outline the background information and refer to methods which are proposed for the selection process. The aim is to provide a framework which supports and guides decision processes. The decisions have to be taken by individuals/groups

1 More details about the organisation and management of the forums and the Forum Action Plans are outlined in D4.1 Common methodology for the organization and management of the forums.


based on their subjective experiences, backgrounds, information etc. Based on objective information, indicators have to be weighted and prioritised based on the goals of EGS and the local experience and requirements in the forums (or of the users). So the framework we propose here can be used flexible and has to be adapted and prioritised also based on local conditions.

• To provide indicators which can be used to decide on priorities:

Part of the whole framework will be qualitative and quantitative indicators. Since these evaluation guidelines shall address a wide range of different activities and priorities, the indicators have to be generic enough to not exclude priorities which shall be checked but at the same time have to be specific enough to provide meaningful information. The set of indicators shall be limited and of reduced complexity to allow a first check. More details on the indicators and if necessary additional indicators can be elaborated and applied in the feasibility studies.

• To provide evaluation tools for practical application:

Tools will be proposed which can easily be used and shall guide the decision processes in order to make the indicators applicable. The tools will mainly be checklists which can be used to evaluate a certain idea or priority and to document the decision processes. We intend to make these tools as easy to use and self-explaining as possible and follow a more practice oriented and less academic approach in evaluation. So a clear priority is the usefulness of the results for the school, forums and local community.

• To provide a structure to report findings:

The used/filled in checklists are one component for reporting the decision processes. Additionally a template for reporting the decision processes will be elaborated based on the same criteria as the tools: easy to use and self-explaining.

Indicators In this chapter the main indicators which will be used to assess ideas or priorities will be described. The list of indicators and their description can be used as a reference but is not exhaustive. The definition of additional indicators which can be based on regional (e.g. special regional policy priorities), organisational (e.g. school internal regulations), legal (e.g. national laws) or other necessities, might be added by the users to allow a selection and evaluation process which is as complete as possible.

General preliminary remarks concerning the indicators

• Some of the indicators are interdependent what underlines that the addressed subject can be addressed on more dimensions.

• Some indicators might not address a certain topic or field of intervention. These indicators should not be considered in that case. As mentioned above vice versa additional indicators might be defined if the forum decides that there are other points that have to be considered in the decision process.


Potential to Initiate Change This indicator addresses medium to long term effects of the priority and its’ potential to initiate change in the behaviour and habits of people. It is based on the assumption that we want to initiate change processes to tackle energy issues and improve energy efficiency in schools and local communities. Other indicators are overlapping, so if a certain priority has no potential to initiate change it might also have low potential in the other area. The potential is low when we can not expect any change processes by following the priority and developing certain concrete actions or activities.

Related questions are for example:

• Has the priority medium to long term effects?

• Will it change the behaviour or habits of its’ target group?

Example: The priority “to raise the energy efficiency of the school building” might have good impacts in terms of energy consumption if it leads to activities like “install new windows”, “install new insulation” etc. It has medium to long term effects but the potential to change behaviour might be limited.

Reproducibility This indicator is based on the assumption that a certain problem or challenge can not be solved in one go. So we should observe if we can reproduce a certain priority and use it again as the starting point for new discussions which lead to concrete actions. The “reproducibility” is high if the problem addressed is always evident (or might appear again). We differentiate reproducibility, which addresses the same domain, and transferability, which addresses the transfer to other domains. (Transferability is tackled with the next indicator).


• Can the priority be set again (and again)?

• Is the problem/challenge addressed evident or “solved” in one go by a certain activity?

Example: The priority “to raise the energy efficiency of the school building” can be reproduced because there are numerous actions which can be taken to raise the efficiency.

Transferability To transfer means in this case to apply in another context. This could be another location (for example school) or another problem or challenge. The indicator is based on the assumption that we intend to develop good practice examples which can easily be adapted and used in other contexts. A low transferability might appear for priorities which are defined for very specific (unique) and location based problems.


• Is the priority only valid for this problem or can it also be set for other problems?

• Is the priority bound to a specific location?


Example: The priority “to raise the energy efficiency of the school building” has the potential to lead to concepts and actions which are transferable to other schools. (What can be done? How can it be done? How can it be financed? etc.).

Efficiency This indicator is based on the assumption that we intend to select those priorities which suggest the most favourable input – output relation. Thereby we can consider different types of energy efficiency like economic efficiency (expressed by cost, time etc.), efficiency related to education and training (e.g. training effects, multiplication effects) etc. The efficiency is high if we expect with a certain resources input a “maximum” output.


• Does the priority favour a more efficient use of resources?

• Is the priority to be considered a very efficient way to inform or train persons and raise awareness about energy issues?

Example: The priority “to raise the energy efficiency of the school building” intends to raise efficiency per se. Nevertheless we might compare it to other priorities which might have a higher “efficiency” potential.

Publicity It is a very important criterion for the success of a project if it reaches its’ target group, can raise public awareness and gain attention. Already for the priorities it should be considered if there is a good potential to reach publicity for example by addressing very important topics, topics with a good visibility, media impact and high local visibility. So this indicator is based on the assumption that the selection of priorities which suggest high publicity also raise the potential for good publicity of later implemented pilot projects and of the EGS project in general (and therefore also the Intelligent Energy Europe programme).


• Are the priorities of public interest or do they only address a very narrow circle?

• Are the priorities per se oriented to publicity or do they require specific marketing activities to be promoted?

Example: The priority “to raise the energy efficiency of the school building” could gain good publicity because might lead to rather large projects which can be used as good practices. But publicity is not “included” in related activities but is bound to awareness raising (articles, info about concepts to other schools, etc.)


Achievability within the EGS Project This indicator is based on the intention to realise pilot projects which address our priorities within the framework of EGS. The achievability is high if the potential that we can implement and finalise pilot activities which address a certain priority is high. This should not limit the selection process to only short term priorities. The intention is to highlight that also “smaller” priorities which are close to achievement are important for EGS.


• Can we expect to achieve tangible results addressing this priority within the EGS project?

• Are there potentials to realise them within the given time-frame and budget?

Example: The priority “to raise the energy efficiency of the school building” might have good impacts in terms of energy consumption if it leads to activities like “install new windows”, “install new insulation” etc. But it is not achievable within the EGS budget and might hardly be achievable within the time foreseen for pilot activities. Nevertheless single activities related to the priority like new financing concepts, energy audits etc. could be realised.

Impact on Target Groups This criterion addresses the potential of a certain priority to generate impacts on the EGS target groups. It should be clear that all proposed priorities have the potential to generate effects on the target group whereof they benefit or where they are involved actively. This criterion is related to the following “active involvement” but includes also the passive component.


• Are the target groups effected passively or actively by actions which can be initiated by the priority?

Example: The priority “to raise the energy efficiency of the school building” has impacts on the target groups by for example a better room climate in classrooms (direct impact on students and teachers), lower energy consumption etc.

Active Involvement EGS intends to actively involve different target groups (from pupils to the interested public) whereby the “centre” of the activities are always schools. The indicator is based on the assumption that we intend to select those priorities which do not only generate a high impact in terms of energy efficiency but also involve the target groups as far as possible. Thereby a specific focus is the involvement of the target groups which are close to schools (students, teachers, other schools staff, parents). The “active involvement” is high when a certain priority has the potential to generate activities which allow a active participation of the EGS target group.


• Has the priority the potential to lead to actions which allow active involvement or does it only involve a limited group of decision makers?

• Can different groups be involved or is only one group addressed?


Example: The priority “to raise the energy efficiency of the school building” might have rather low active involvement of different target groups if it leads to activities like “install new windows”, “install new insulation” etc.

Coherence with Policies There might be no priorities which are not coherent to policies in general but this indicator is based on the assumption that we intend to enhance those priorities which refer to local and regional policy priorities concerning energy (efficiency) if existing.


• Is the priority coherent to policies in general?

• Does it specifically address regional policy priorities?

Example: The priority “to raise the energy efficiency of the school building” might be perfectly addressing regional priorities if these priorities foster the thermal renovation of (public) buildings.

Independency from External Resources This indicator is based on the assumption that the target groups of EGS are the driving force for more energy efficiency and that EGS activates and supports their efforts. So we might favour priorities which safeguard a certain independency from external resources (e.g. money, manpower, know chow) and empower the target group itself. This should not mean that external counselling, help and support should be excluded but it should underline the fact that the know how of EGS project partners and the involved target groups should be the driving force for setting priorities and implementing the following pilot activities.


• Can activities related to the priority be implemented by the EGS target group?

• Is the realisation highly dependent on external resources?

Example: The priority “to raise the energy efficiency of the school building” might have a limited independency from external resources depending on the involvement of schools in the planning process, in the decision on financial issues etc.

Usage of Indicators – Priority Check We propose to use the priority check template which can be found in the annex, containing the

• priority description and the

• indicator check list.


Priority Description Each priority which is discussed in the forums or which is suggested in discussions with stakeholders should be described in short outlining the title, the main aims, goals and benefits for the schools or/and other stakeholders. This short description should not exceed 10 lines of typed text. Additional documentation could be annexed.

Indicator Check List As outlined above the operational use of the indicators should be made as easy as possible. So we propose to use the short check list which is annexed. This check list can be used by individuals or in groups where a discussion about how a certain priority addresses each of the ten indicators will lead to a conclusion and rating.

The set of indicators shall be seen as a guideline; so if a certain indicator does not fit to a priority formulated by the forum, this indicator simply should be not used. If on the other side another indicator is seen as very useful which is currently not in the list, it can be added.

We propose an easy rating scale with only three points in form of a traffic light for the first check, documentation and rating of priorities.

RED (0) means that the priority does not contribute to the indicator or contributes negatively

YELLOW (1) means that there are doubts but there might be potential to contribute positively

GREEN (2) means the priority contributes positively to the indicator / supports it sufficiently

As outlined in the indicator description it is up to discussions in the forum if some indicators gain higher priority then others. This can be based on local necessities, preferences, policies etc.

Responsibilities The project partners who are responsible for the establishment and implementation of the forums are also responsible to keep a full documentation and reporting of the forums’ activities and results.

The priority check template is on the one hand a tool to select the best fitting priorities and project ideas which can be further developed and shall lead to a pilot action. On the other hand it is also a way of documentation and reporting on the forums’ decision making process.


Feasibility Studies

Deliverable D5.2 of EGS is a feasibility studies report which summarises a total of at least 39 short feasibility studies (three from each of the 13 forums). The topics for the feasibility studies are selected in the discussion and selection processes in and around the forums. These processes are supported by the evaluation guidelines for the forums (priority check) and are documented supported by the template for priority check (annex to this document) and the template for EGS Local Project description (annex to D4.1).

The feasibility studies will help to take the decision if a certain priority / project idea is worth to be followed and developed further and to be implemented. The priority check is the first step in the selection process, the feasibility study the second step. Thereby similar indicators will be used but will be described in more details or/and will be quantified where possible. In this proof of concept the question shall be answered for which projects (or parts of projects / actions) it is feasible to be implemented within the EGS project duration and framework.

Following indicators can be used for the feasibility definition, which will be further elaborated in the preparation phase of work package 5, Task 2: Feasibility Plans:

Organisational feasibility

• Potential to initiate change

• Active involvement

• Impact on target groups

• Publicity

Economic feasibility and availability of resources

• Efficiency

• Independency from external resources

• Achievability within the EGS project (time, resources)

Legal feasibility

• Coherence with policies

Feasibility concerning exploitation

• Transferability

• Reproducibility

Technical feasibility


Monitoring of Pilot Actions

Each forum will select at least one pilot project out of the three feasibility studies which shall be activated, implemented and monitored within the framework of the EGS project to contribute to more energy efficiency in schools but to test also the selection and monitoring processes and guidelines.

According to the project planning the selection process will be closed in spring 2010 with the final document D5.2 Feasibility Studies Report. So at latest the preparation phase for the pilot projects will start mid 2010 what gives at least a period of six month for the implementation and documentation of the pilot activities. In many cases pilot activities will start earlier because the decision making process has been made quicker, but for all pilot activities a limited amount of time and financial resources are available in the EGS project. (Other sources might be raised but this is uncertain and up to partners’ aptitude and fortune by finding external and adding additional internal resources.)

Type of Pilot Actions Concerning the envisaged pilot actions there are several open questions which can not be answered at this stage of the EGS project, because the selection process is in its’ starting phase and it is not clear which pilot actions can be expected:

• Size of pilot projects in terms of resources (time, effort, money, etc.)

Considering what was said above we expect as pilot projects rather small energy efficiency projects or sub-projects (projects which are part of a larger energy efficiency project). But there is also potential for larger pilot actions depending on additional resources.

• Topics of pilot actions

Since the forums are not bound to a specific topic the context in which pilot actions can be selected and implemented is very wide – basically the topic “energy efficiency” has to be addressed but there are numerous very different pilot project possibilities which all could find place in the framework of EGS.

So the range of pilot actions which can appear in the framework of EGS can for example be as large as

• “building a new and more efficient heating system for the school” or

• “adding thermal insulations and new windows to the school building”

on the one side to

• “initiating carpools of students which are brought to school by their parents” or

• “initiating waste avoidance processes in classrooms”

on the other side.


Guidelines for Monitoring The guidelines for monitoring the pilot actions can only provide a framework for documentation of the implementation in order to have a standardised reporting scheme. It will consist of the initial documentation of the pilot project which is defined in the feasibility study (including the initial targets), a monthly reporting checklist (targeting quantitative and qualitative indicators) and a final reporting template.

We again refer to our initial list of indicators and the descriptions and propose the following indicators to be used for monitoring the pilot activities. Thereby for very complex pilot action additional indicators might be needed and defined, indicators might be exchanged or for several pilot activities indicators might not be used. So we again propose an initial set of indicators which can be adapted to the field of intervention.

After a short description of the action and a check of criteria like the relation between planned and used costs and time constraints the following indicators will be used for monthly checking and for the final check:

Reproducibility

• Plans to start the action again

• Pilot has been mainstreamed to become “regular programme”

Transferability

• Plans for transfer made

• Transfer already in preparation or implemented

Efficiency

• Energy saved (in %, KWh, litres, etc.)

• Renewable energy used

• Resources used more efficient

Impact on target groups

• Number of persons (students, teachers, staff, parents, etc.) involved directly

• Number of persons on which the pilot has direct effects

Publicity

• Number of persons reached and informed

• Press/media coverage

The checklists will be used by the responsible partner for the respective pilot action and a short report on the pilot action with attached checklists will be prepared and sent to MERIG who is responsible for WP5 until November 2010. Deliverable 5.3 Pilot Actions Monitoring and Evaluation Report which summarises and documents progress and results is based on the reports from the partners implementing the pilot actions.


Annex

Annex 1 Priority Check Template

Annex 2 Pilot Monitoring Template -- Monthly Short Check Pilot Monitoring Template -- Final Monitoring Report

IEE/07/710 Energy Education Governance Schools http://www.egs-project.eu

Priority Check Template Done by/Name:

Place/Date:

Priority title:

Main aims and goals: (max 4 lines)

Main benefits for the schools or/and other stakeholders: (max 4 lines)

Indicators 2 1 0 Potential to initiate change

Short note:

Reproducibility Short note:

Transferability Short note:

Efficiency Short note:

Publicity Short note:

Achievability within the EGS project Short note:

Impact on target groups Short note:

Active involvement Short note:

Coherence with policies Short note:

Independency from external resources Short note:

Total: Green Yellow Red Sum (G*2+Y)


Pilot Monitoring Template -- Monthly short check

Done by/Name: Place/Date:

Pilot title:

Activities are implemented on time Yes No

Short explanation: Pilot implementation is in the frame concerning efforts (cost, time) Yes No

Short explanation: Indicators check Reproducibility

• Plans to start the action again Yes No

• Pilot has been mainstreamed to become “regular programme” Yes No

Short explanation:

Transferability

• Plans for transfer made Yes No

• Transfer already in preparation or implemented Yes No

Short explanation: Efficiency If “Y” Quantification [kwh, €, …]

• Energy saved Yes No

• Renewable energy used Yes No

• Resources used more efficient Yes No

Short explanation: Impact on target groups If “Y” Quantification [Number of particip. …]

• Persons (students, teachers, staff, etc. ) involved directly Yes No

• Persons on which the pilot has direct effects Yes No

Short explanation: Publicity If “Y” Quantification [Numbers]

• Persons reached and informed Yes No

• Press/Media coverage Yes No

Short explanation:


Pilot Monitoring Template -- Final Monitoring Report Pilot title:

Short description of the pilot action: (aims, objectives, target groups, …) (max 15 lines)

The main goals have been reached? Yes No Explanation:

Activities have been implemented on time Yes No Short explanation:

Pilot implementation finalised in the frame concerning efforts (cost, time) Yes No Short explanation:

Please describe feedback you’ve received from the target group (max 5 lines)


Pilot Monitoring Template -- Final Monitoring Report (p2) Indicators check Reproducibility

• Plans to start the action again Yes No

• Pilot has been mainstreamed to become “regular programme” Yes No

Short explanation:

Transferability

• Plans for transfer made Yes No

• Transfer already in preparation or implemented Yes No

Short explanation:

Efficiency If “Y” Quantification [kwh, €, …]

• Energy saved Yes No

• Renewable energy used Yes No

• Resources used more efficient Yes No

Short explanation:

Impact on target groups If “Y” Quantification [Number of particip. …]

• Persons (students, teachers, staff, etc. ) involved directly Yes No

• Persons on which the pilot has direct effects Yes No

Short explanation:

Publicity If “Y” Quantification [Numbers]

• Persons reached and informed Yes No

• Press/media coverage Yes No

Short explanation:

Documents

D5.1 Guidelines for Projects Monitoring and Evaluationec.europa.eu/energy/.../files/...projects_monitoring_and_evaluation_en.pdf · IEE/07/710 – EGS: WP5 D5.1: Guidelines for Projects