Upload
dinhthu
View
217
Download
0
Embed Size (px)
Citation preview
Horizon 2020 support
to Smart, Green and
Integrated transport
Studies and reports
EUROPEAN COMMISSION
Directorate-General for Research and Innovation Directorate H — Transport Unit H.1 — Transport Strategy
Contact: Roberta Zobbi
E-mail: [email protected]
European Commission B-1049 Brussels
EUROPEAN COMMISSION
Horizon 2020 support to
Smart, Green and Integrated
transport
Studies and reports
Directorate-General for Research and Innovation
2017 Smart, Green and Integrated Transport
LEGAL NOTICE
This document has been prepared for the European Commission however it reflects the views only of the
authors, and the Commission cannot be held responsible for any use which may be made of the information
contained therein.
More information on the European Union is available on the internet (http://europa.eu).
Luxembourg: Publications Office of the European Union, 2016.
PDF ISBN 978-92-79-71792-5 doi: 10.2777/274467 KI-01-17-922-EN-N
EPUB ISBN 978-92-79-71792-5 doi: 10.2777/274467 KI-01-17-922-EN-N
© European Union, 2017.
Reproduction is authorised provided the source is acknowledged.
Cover images: © Lonely, # 46246900, 2011. © ag visuell #16440826, 2011. © Sean Gladwell #6018533,
2011. © LwRedStorm, #3348265. 2011. © kras99, #43746830, 2012. Source: Fotolia.com
EUROPE DIRECT is a service to help you find answers to your questions about the European Union
Freephone number (*): 00 800 6 7 8 9 10 11
(*) The information given is free, as are most calls (though some operators, phone boxes or hotels may charge you)
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 5
PREFACE Horizon 2020 is the European Union’s largest Research and Innovation programme for the period
2014-2020 and the Transport Challenge is allocated a budget of €6339 million.
The objective of the interim evaluation is to assess the overarching technical, political and
operational objectives of the programme by looking at the achievements of Horizon 2020 and the
Societal Challenged 4 (SC4) in terms of relevance; effectiveness; coherence (internal and external);
efficiency and EU added-value.
The approach followed takes into account the increased emphasis on outcomes (i.e. the impacts for
stakeholders, expected and unexpected) rather than on outputs from research projects and
programmes. It has also looked at both FP7 and Horizon 2020 as there is an overlap between the
project completion of the former and latter. At the time of this interim evaluation most of the
Horizon 2020 projects are in the initial phases of development and none have been finished.
The methodology for this evaluation combines quantitative and qualitative data, analysis and
evidence validated with relevant stakeholders. The methodology used consists of:
- Document and statistical analysis;
- Internet based surveys for different target groups, but mainly the respondents were project
coordinators;
- Questionnaire based individual and group interviews with selected interested parties;
- Informal exchanges and information gathering; and
- Analysis of available EC data (partly through data and text mining).
The Expert Group made an effort to verify all information received from individuals1 against different
sources and where possible and appropriate to draw its own conclusions as experts. Care was taken
to ensure that the sampling was representational (across modes, instruments and
players/stakeholders and also in terms of member states, types of projects and programmes). To
gain additional insights, the group performed individual and group interviews. Where indicated that
a view is ‘shared’, this means that this view was mentioned in the survey and by those that have
been interviewed. When ‘many’ are referred to it means that the comment has been supported by a
number of survey respondents, interviews and by the Expert Group. Where the comment or view has
only been put forward by one or two interviewees the comment or view has been triangulated
against other evidence or included because this view is shared by the majority of the Expert Group.
Single or unsupported views have not been included in this report. The representativeness of
opinions and conclusions was further assured by the fact that interviewees and participants in the
stakeholder hearing were invariably high level representatives of relevant organisations or
committees. They could be trusted to speak with care and with due regard for their affiliation
although they had the liberty to vent personal opinions as well. In most cases the interviewees
explicitly mentioned it if they were speaking in a personal capacity. The online survey among project
coordinators, though yielding less response than was hoped for, was considered by the Expert Group
1 Sources were promised confidentiality both individually and for their affiliated institutions.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 6
to be representative in numbers. The response to another survey among Programme Committee
members was so low that it was not considered representative and was therefore discarded.
The Expert Group realises that a number of opinions and recommendations may give food for further
investigation. Due to time constraints the Group was not able to pursue in detail all potentially
interesting remarks, especially in those cases that interviewees came up with spontaneous
comments that could not be checked against other material. The Expert Group took on those
comments at its discretion only if they considered them plausible enough and worthwhile
mentioning.
The Expert Group also puts forward a set of recommendations that can be considered as lessons
learnt and suggestions for consideration or improvement in the new round of research activities for
transport (FP9).
The report has been prepared by the following independent experts Joris Al, Heather Allen, Griet De
Ceuster, Ivonne Herrera and Jesús Monclús as part of an interim evaluation of the Societal Challenge
4 (Smart, Green and Integrated Transport) for the European Commission and not for general
publication. The information comes from a number of sources including an Internet based
questionnaire, interviews with key stakeholders, a stakeholder hearing and the collecting of data
(some of which were provided by the Commission). Comments from these sources are confidential
and stakeholder views were only considered when they were also shared by the Expert Group and
validated by other evidence to support this opinion. It is indicated in the text when this is not the
case.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 7
THE EXPERT GROUP The report has been prepared by the following independent experts Joris Al, Heather Allen, Griet De
Ceuster, Ivonne Herrera and Jesús Monclús.
Joris Al (NL) is chair of the Expert Group. He was a civil servant with several ministries in the
Netherlands and later became a consultant several years before he retired in 2014. During the last
10 years of his career, he was Director of Road and Navigation Research and subsequently Director
for International Affairs with Rijkswaterstaat. He was president of FEHRL from 2010 to 2014 and
chaired the Executive Board of CEDR in 2013.
Heather Allen (UK) is an international expert on sustainable urban transport, climate change and
gender. She developed the Sustainable Development Charter for UITP (the International Association
of Public Transport) and their diversity programme and also led the Sustainable Transport Team at
one of UK’s leading research centres, Transport Research Laboratory (TRL).
Griet De Ceuster (BE) is general manager of Transport & Mobility Leuven. She is involved in long term
forecasts, policy evaluations and indicators for sustainable mobility. She is leading transport- and
emission modelling studies across Europe. She was involved in several research programmes on road
congestion indicators, long term transport prognoses, pricing issues, external costs and transport
technologies. Griet De Ceuster holds a MSc in traffic engineering and studied economy at the KU
Leuven and the TU Delft.
Ivonne Herrera (NO) has degrees in Electrical Engineering, a Master in Aeronautical Maintenance
and Production and a PhD in Safety Management and Resilience Engineering. She has more than 20
years of experience in the industry and research on avionics engineering, maintenance, safety and
risk analysis for aviation and petroleum industries. She is currently Senior Scientist in the area of
Information and Communication Technology and Adjunct Associate Professor teaching Master
courses addressing risk governance, safety management and resilience.
Jesús Monclús (ES) holds a PhD in Mechanical Engineering and a MS in Transportation Safety. He has
experience as National Contact Point and National Delegate for the FP7 Transport Programme
Committee and for the Clean Sky 1 National Representatives Group. He is now professionally
involved in injury prevention programmes, including road safety, at international level.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 8
TABLE OF CONTENTS PREFACE ................................................................................................................................................... 5
THE EXPERT GROUP ................................................................................................................................. 7
TABLE OF CONTENTS ............................................................................................................................... 8
GLOSSARY .............................................................................................................................................. 11
1 EXECUTIVE SUMMARY .................................................................................................................. 17
2 INTRODUCTION ............................................................................................................................. 22
2.1 Context ................................................................................................................................... 22
2.2 Objectives and intervention logic .......................................................................................... 23
2.2.1 Objectives ...................................................................................................................... 23
2.2.2 Interim Evaluation.......................................................................................................... 26
2.2.3 Intervention logic ........................................................................................................... 27
3 IMPLEMENTATION STATE OF PLAY [Key data provided centrally by Commission to Expert group]
32
4 RELEVANCE .................................................................................................................................... 33
4.1 Is Horizon 2020 Transport tackling the right issues? ............................................................. 33
4.1.1 Relevance to the situation in Europe when Horizon 2020 Transport was designed and
how it is addressing Europe’s challenges ...................................................................................... 34
4.1.2 Areas of relevance of specific European interest .......................................................... 40
4.1.3 Horizon 2020 Transport and its international outreach ................................................ 43
4.2 Addressing specific stakeholder needs, including emerging issues ...................................... 45
4.2.1 Suggestions for improvements from a stakeholder perspective ................................... 51
4.3 Other issues relevant to Relevance ....................................................................................... 52
4.4 Lessons learnt and areas of improvement ............................................................................ 53
5 EFFECTIVENESS .............................................................................................................................. 56
5.1 Short-term outputs from the programme and early success stories .................................... 56
5.1.1 Success stories ............................................................................................................... 56
5.1.2 TRL levels........................................................................................................................ 57
5.1.3 Human capital development ......................................................................................... 58
5.1.4 Knowledge transfer ........................................................................................................ 59
5.1.5 Building research, development and innovation capabilities ........................................ 59
5.1.6 Competitiveness & exploitation .................................................................................... 60
5.2 Expected longer-term results from the programme ............................................................. 61
5.3 Progress towards attaining the specific objectives of SC4 - Smart, green and integrated
transports .......................................................................................................................................... 62
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 9
5.4 Progress towards the overall Horizon 2020 objectives ......................................................... 64
5.5 Lessons learnt/ Areas for improvement ................................................................................ 64
6 EFFICIENCY .................................................................................................................................... 65
6.1 Introduction ........................................................................................................................... 65
6.2 Overview of funding allocations ............................................................................................ 65
6.2.1 Budget of the overall programme ................................................................................. 65
6.2.2 Budget per activity area ................................................................................................. 66
6.2.3 Balance between instruments and funding schemes .................................................... 67
6.2.4 Description of topics ...................................................................................................... 69
6.2.5 Projects have equal size ................................................................................................. 70
6.2.6 Additional remarks on topic definition .......................................................................... 71
6.3 Programme's attractiveness .................................................................................................. 73
6.3.1 Analysis of the Participants ............................................................................................ 73
6.3.2 Oversubscription on some topics .................................................................................. 74
6.3.3 Adequacy of funding rules ............................................................................................. 79
6.3.4 The cost of submitting a proposal ................................................................................. 80
6.3.5 Two-stage approach ...................................................................................................... 81
6.4 Cost-benefit analysis .............................................................................................................. 82
6.5 Other issues related to efficiency .......................................................................................... 83
6.5.1 Evaluation process & selection criteria .......................................................................... 83
6.5.2 Length of process ........................................................................................................... 84
6.5.3 Administrative burden ................................................................................................... 85
6.6 Lessons learnt/ Areas for improvement ................................................................................ 86
7 COHERENCE ................................................................................................................................... 87
7.1 Internal coherence ................................................................................................................. 87
7.1.1 Internal coherence of instruments to implement Horizon 2020 Transport .................. 87
7.1.2 Internal coherence with other Horizon 2020 intervention areas .................................. 90
7.1.3 Ensuring that every euro spent counts twice ................................................................ 95
7.2 External coherence ................................................................................................................ 97
7.2.1 Coherence with other EU funding programmes ............................................................ 97
7.2.2 Coherence with other public support initiatives at regional, national and international
level 102
7.3 Lessons learnt/Areas for improvement ............................................................................... 104
8 EU ADDED VALUE ........................................................................................................................ 106
8.1 Understanding added value ................................................................................................. 106
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 10
8.2 Horizon 2020 transport programme demonstrating EU Added Value ................................ 106
8.2.1 European integration ................................................................................................... 106
8.2.2 Increasing European competitiveness ......................................................................... 108
8.2.3 Europe’s role in standardisation and harmonisation .................................................. 109
8.2.4 Building stronger Europe-wide research communities and level of expertise ............ 110
8.2.5 Delivering research that feeds into European policy objectives ................................. 112
8.2.6 Linking calls and topics more directly with policy expectations .................................. 112
8.3 Success of different instruments to deliver EU Added Value .............................................. 113
8.3.1 ERA-NET ....................................................................................................................... 113
8.3.2 CEF ............................................................................................................................... 113
8.3.3 PPPs and JUs ................................................................................................................ 113
8.4 Other issues related to EU Added value .............................................................................. 115
8.5 Lessons learnt and areas for improvement ......................................................................... 117
8.5.1 Increased integration ................................................................................................... 117
8.5.2 Formal evaluation and follow-up of projects .............................................................. 118
8.5.3 Future research directions organizing for innovation ................................................. 119
8.5.4 Dissemination and exploitation aspects ...................................................................... 119
9 SUCCESS STORIES FROM PREVIOUS FRAMEWORK PROGRAMMES ........................................... 123
10 LESSONS LEARNT / CONCLUSIONS .............................................................................................. 124
10.1 Relevance ............................................................................................................................. 124
10.2 Effectiveness ........................................................................................................................ 128
10.3 Efficiency .............................................................................................................................. 129
10.4 Coherence ............................................................................................................................ 130
10.4.1 Internal coherence ....................................................................................................... 130
10.4.2 External Coherence ...................................................................................................... 132
10.5 Added value ......................................................................................................................... 134
10.6 Conclusions and recommendations ..................................................................................... 136
LITERATURE .......................................................................................................................................... 140
ANNEX: SURVEY QUESTIONS FOR PROJECT COORDINATORS ............................................................. 142
ANNEX: INTERVIEW QUESTIONS FOR STAKEHOLDERS ........................................................................ 157
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 11
GLOSSARY € euro
3D 3 dimensions
5G 5th generation of mobile communications
ACARE Advisory Council for Aviation Research and Innovation in Europe
ACEA Association des Constructeurs Européens d’ Automobiles
ACEM Association des Constructeurs Européens de Motocycles
ALICE Alliance for Logistics Innovation through Collaboration in Europe
ARF Access to Risk Finance
ART Automated Road Transport
ATM Air Traffic Management
BBI Bio-Based Industries, a Joint Undertaking
CARE Community database on road accidents
CEF Connecting Europe Facility
C-ITS Collaborative ITS (intelligent transport systems)
CIVITAS City, Vitality and Sustainability, a European Initiative launched in FP7 to
promote sustainable transport in cities
CO2 Carbon dioxide
COM Commission européenne / European Commission
COP 21 Climate Change conference 21st
CORDA data COmmon Research DAta Warehouse
CP Collaborative projects, a funding scheme in FP7
cPPP Contractual public-private partnership
CS Clean Sky, a Joint Undertaking
CS2 Clean Sky 2, a Joint Undertaking
CSA Coordination and Support Action, a funding scheme in Horizon 2020
CSJU Clean Sky Joint Undertaking
CSO Civil Society Organisation
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 12
DG Directorate General
DG MOVE Directorate General Mobility and Transport
DG RTD Directorate General Research and Innovation
EAEC European Atomic Energy Community
EBSF European Bus System of the Future
EC European Commission
eCALL Emergency Call
ECSEL Electronic Components and Systems for European Leadership, a Joint
Undertaking
ECTP European Construction Technology Platform
ECTRI European Conference of Transport Research Institutes
EE Energy efficiency
EFSI European Fund for Strategic Investment
EG Expert Group
EGVI European Green Vehicles Initiative
EGVIA European Green Vehicles Initiative Association
EIB European Investment Bank
EIP European Innovation Partnership
EIT European Institute of Innovation & Technology
ELENA European Local ENergy Assistance, a fund of the EIB
ERA European Research Area
ERA-NET European Research Area Network
ERC European Research Council
ERDF European Regional Development Fund
ERRAC European Rail Research Advisory Council
ERTICO European Road Traffic Information Company
ERTMS European Rail Traffic Management System
ERTRAC European Road Transport Research Advisory Council
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 13
ESFRI European Strategy Forum on Research Infrastructures
ESIF European Structural and Investment Funds, a fund of the EIB
ETP European Technology Platform
ETRA Spanish traffic engineering company www.etra.es
EU European Union
EU13 European Union, 13 countries joining the European Union in 2004, 2007 and
2013 (Bulgaria, Croatia, Cyprus, Czech Republic, Estonia, Hungary, Latvia,
Lithuania, Malta, Poland, Romania, Slovakia, Slovenia)
EU28 European Union, all 28 countries that are members of the European Union
Euratom European Atomic Energy Community
EURICS Energy Union Integrated Strategy on Research, Innovation and
Competitiveness
EV Electric vehicle
ExGr Expert Group
FCH2 Fuel Cells and Hydrogen 2, a Joint Undertaking
FEHRL Forum of European Highway Research Laboratories
FET Future and Emerging Technologies
FIA Fédération Internationale de l'Automobile
FIM Fédération Internationale de Motocyclisme
FP7 7th Framework Programme
FP9 9th Framework Programme
FTI Fast Track to Innovation
GDP Gross Domestic Product
GHG Greenhouse gas
GNSS Global Navigation Satellite System
GV Green Vehicles
IA Innovation Action, a funding scheme in Horizon 2020
ICT Information and Communications Technologies
IMG Industry Manufacturing Group
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 14
IMO International Maritime Organisation
INEA Innovation and Networks Executive Agency
Io(T)T Internet of (Transport) Things
IoT Internet of Things
ITS Intelligent Transport System
JTI Joint Technology Initiative
JU Joint Undertaking
KIC Knowledge and Innovation Community
KPI Key performance indicator
LEIT Leadership in Enabling and Industrial Technologies
LIFE EU’s funding instrument for the environment and climate action
LNG Liquefied natural gas
LRIT Long-Range Identification and Tracking
M€ Millions of Euros
Marco Polo Logistics efficiency funding programme during FP7
MG Mobility and Growth
MIT Massachusetts Institute of Technology
MoU Memorandum of understanding
MS Member State
MSCA Marie Sklodowska-Curie Actions
n Number
NCP National Contact Point
NMBP Nanotechnologies, Advanced Materials, Biotechnology and Production
NMP Nanotechnologies, Advanced Materials and Advanced Manufacturing and
Processing
P2P Peer to peer
PDA Project Development Assistance, part of the Energy programme
POLIS Network of European cities and regions working together to develop
innovative technologies and policies for local transport
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 15
PPI Public Procurement for Innovative Solutions
PPIS Public Procurement in Innovative Services
PPP Public-private partnership
R&D Research and development
RDI Research, development and innovation
RI Research Infrastructures
RIA Research and Innovation Action, a funding scheme in Horizon 2020
RIS River Information System
RIS3 Regional Research and Innovation Strategies for Smart Specialisation
RPO Research performing organisations
RRI Responsible Research and Innovation
S2R Shift2Rail
SC1 Horizon 2020’s Societal Challenge 1, Health, Demographic Change and
Wellbeing
SC2 Horizon 2020’s Societal Challenge 2, Food security, sustainable agriculture and
forestry, marine, maritime and inland water research and the bioeconomy
SC3 Horizon 2020’s Societal Challenge 3, Secure, clean and efficient energy
SC4 Horizon 2020’s Societal Challenge 4, Smart, green and integrated transport
SC5 Horizon 2020’s Societal Challenge 5, Climate action, environment, resource
efficiency and raw materials
SC6 Horizon 2020’s Societal Challenge 6, Inclusive Societies
SC7 Horizon 2020’s Societal Challenge 7, Secure Societies
SDGs Sustainable Development Goals
SES Single European Sky
SESAR Single European Sky ATM Research, a Joint Undertaking
SEWP Spreading Excellence and Widening Participation
SME Small and medium enterprise
SRIA Strategic Research and Innovation Agenda of ACARE
SSN Safe Sea Net, vessel traffic monitoring and information system
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 16
STEER Intelligent Energy Europe, a funding programme during FP7
STRIA Strategic Transport Research and Innovation Agenda
SUMP Sustainable urban mobility plan
SWAFS Science with and for Society
TAG Transport Advisory Group
TEB Transit Elevated Bus
TEN-T Trans-European transport network
TP Technology platform
TRA Transport Research Arena Conference
TRIP Transport Research Innovation Portal, formerly known as the Transport
Research Knowledge Centre (TRKC)
TRIVALUE Ex-post evaluation of Transport Research and Innovation in the FP7
‘Cooperation’ Programme
TRKC Transport Research Knowledge Centre (currently known as Transport Research
Innovation Portal , TRIP)
TRL Technology Readiness Level
TSI Technical standard
UAV Unmanned aerial vehicle
Waterborne TP Waterborne Technology Platform
WP Work Programme
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 17
1 EXECUTIVE SUMMARY Horizon 2020, the European Union’s largest Research and Innovation programme for the period
2014-2020, was adopted in 2013 to help address the challenges Europe faces. It is built on three
main pillars: excellent in science, societal challenges and industrial leadership. These remain the core
values of Horizon 2020 today, although it is also recognised that some of the challenges may have
changed since its inception.
The Transport Challenge is allocated a budget of €6339 million for the period 2014-2020 and the
programme is structured into four broad lines of activities.
Horizon 2020 addresses seven societal challenges. The specific objective of the Societal Challenge 4
(SC4) ‘Smart, green and integrated transport’ is to boost the competitiveness of the European
transport industries and ‘to achieve a European transport system that is resource-efficient, climate-
and environmentally-friendly, safe and seamless for the benefit of all citizens, the economy and
society’.
This interim evaluation of the Smart, Green and Integrated Transport Societal Challenge of Horizon
2020 is part of the overall Horizon 2020 interim evaluation that is to be completed by the European
Commission by mid-2017. This report has been prepared during the period June – October 2016 by
an Expert Group (consisting of five members) as input to the EC internal evaluation of transport
research that will be conducted by DG RTD and DG MOVE.
The objective of the interim evaluation is to assess the overarching technical, political and
operational objectives of the programme by looking at the achievements of Horizon 2020 and the
Societal Challenged 4 (SC4) in terms of:
- Relevance;
- Effectiveness;
- Coherence (internal and external);
- Efficiency;
- EU added-value.
The approach followed by this evaluation differs from previous ones (such as the ‘Commitment and
Coherence, Ex Post evaluation of FP7’) as it takes into account the increased emphasis on outcomes
(i.e. the impacts for stakeholders, expected and unexpected) rather than on outputs from research
projects and programmes. It has also looked at both FP7 and Horizon 2020 as there is an overlap
between the project completion of the former and latter. At the time of this interim evaluation most
of the Horizon 2020 projects are in the initial phases of development.
It should be noted that this interim evaluation covers a period in time when the Commission
introduced a greater focus on bringing research findings closer to the market and during which the
expectation of the role of industry in policy delivery through participation in research has also
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 18
changed. Examples of these trends include the creation of Public Private Partnerships (PPP)2 and
Joint Undertakings (JU), the more open character of research calls, and the combination of the ERA-
NET models, supporting the shift in policy objectives. As the PPPs and JUs are subject to a separate
evaluation, the present evaluation does not encompass those evaluations in any great depth.
However, it does include them where they intersect with the objectives of this research programme
and SC4.
The methodology for this evaluation combines quantitative and qualitative data, analysis and
evidence validated with relevant stakeholders. The methodology used consists of:
- Document and statistical analysis;
- Internet based surveys for different target groups, but mainly the respondents were project
coordinators;
- Questionnaire based individual and group interviews with selected interested parties;
- Informal exchanges and information gathering; and
- Analysis of available EC data (partly through data and text mining)
The Expert Group made an effort to check all information received from individuals3 against different
sources and where possible and appropriate to draw its own conclusions as experts.
Care was taken to ensure that the sampling was representational (across modes, instruments and
players/stakeholders and also in terms of member states, types of projects and programmes). To
gain additional insights, the group performed individual and group interviews.
It should be noted that there were certain constraints in the preparation of this report, such as the
timing of the interim evaluation and the access to certain information such as data on the balance of
gender in projects. Also the analysis of newcomers has not been made in any great depth due to the
lack of information available.
The Expert Group looked at the relevance of Horizon 2020 to establish if it is tackling the right issues,
and pursuing the right (relevant) objectives and they investigated to what extent the programme will
yield relevant research results in terms of impact, deployment (especially looking at innovations) and
how the findings can respond to the stated societal challenge SC4 on smart, green and inclusive
transport.
Transport has a high impact and relevance not only on the daily lives of citizens and society but also
on economic growth and jobs (interpreting competitiveness in its widest sense). There are numerous
issues that are relevant both to transport and the specific objectives of Horizon 2020. The Expert
Group also looked at the extent that feedback loops can positively influence policy development at
European level.
It is considered that the overarching research objectives of the programme remain relevant although
it may be questioned if the total of contracted research will meet the ambitions stated in the societal
2The use of the term PPP in the text refers to the many different structures of public, private partnerships
rather than a legal entity except where more specifically we have used other terms such as Joint Undertaking
(JU) or Joint Technology Initiative (JTI) 3 Sources were promised confidentiality both individually and for their affiliated institutions.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 19
challenges. The programme also remains extremely (and possibly increasingly) relevant in terms of
its international outreach and relevance. The topics of climate change, road safety and
standardisation were highlighted.
Effectiveness has been interpreted as ‘measurable impacts in relation to the overarching objectives
of Horizon 2020, SC4 and specifically to the project objectives’. The Expert Group also included
investigating the wider levels of effectiveness in terms of human capital development, knowledge
transfer (especially in respect to communication and dissemination of results) and, to some extent,
the take up of tools and recommendations by those not directly involved with projects. It would have
been ideal to associate these results with the predetermined objectives such as competitiveness, job
creation and societal impact but this went beyond the scope of the Terms of Reference for this
group.
A number of projects as yet unfinished are likely to exceed their initial Technology Readiness Levels
(TRLs). This is in line with the outcome and closeness to the market ambitions of Horizon 2020
(compared to FP7) and its objective to help research make the transition towards deployment and/or
further research.
The vast majority of stakeholders and participants agree that Horizon 2020 is a unique programme
and its outcomes would have been very difficult to obtain should it not have existed. It is likely that
the quality and quantity of outputs is expected to be similar for Horizon 2020 as for FP7. Some early
success stories were identified to illustrate a good (equivalent to FP7 or higher) level of effectiveness.
However, despite this and based on the data available, the number of knowledge transfer activities
are expected to be lower for Horizon 2020 compared to FP7.
The impact on research, development and innovation capabilities is expected to be similar for
Horizon 2020 compared to FP7. Almost all projects have undertaken deployment activities, or are
planning to do so, and direct contact with stakeholders is the most popular activity, in both in FP7
and Horizon 2020.
However, project coordinators state that they plan to undertake less activities overall in Horizon
2020 to help deploy the results than in FP7. The reasons are unclear, as one would expect a higher
number given the focus on more market-oriented research, and the Expert Group found this a
surprising result.
A number of concerns were raised in respect to the focus on the higher end TRLs as this was stifling,
rather than engendering innovation. Cost effective solutions in early stages of TRL readiness are seen
as being desirable additions to the overall work programme especially in respect to the inherent
inertia in the transport sector for change and the slow progress seen in many areas, especially in
decarbonisation and alternative fuels.
Horizon 2020 is expected by respondents to yield more long-term outcomes than FP7 but as few
projects are finished, it is not possible to draw this conclusion yet. Nonetheless Horizon 2020 is seen
to contribute to strategic cooperation in research going beyond FP7. In relation to SC4 concerns were
raised on the bias towards technology over socio-economic and behaviour research.
Efficiency was considered as the relationship between the resources used by an intervention and the
changes generated by the intervention (which may be positive or negative). The benefits of the
outputs and outcomes from the research should be achieved at a reasonable cost. It was noted that
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 20
there have been several improvements to the Horizon 2020 process compared to FP7, which are
appreciated by the participants: the current balance between instruments and funding is considered
good. The feedback that was received from the majority of stakeholders was that the overall budget
was large but still insufficient to reach the desired objectives. Also, in some areas the balance of
allocation of funds could be better. It was also recognised that there are no other European or
International research programmes that are better funded than Horizon 2020.
The management of the projects has been improved with the outsourcing of these activities to INEA.
However, the most debated areas were in relation to the call texts, the two-stage process and
scientific and innovation management. More transparency associated with definition in the calls,
more focus in topics and increasing the level of success of proposals, seen today as frustratingly low,
were widely requested. The two-stage process is considered to bring extra cost burdens to those
preparing the proposals and has not delivered substantial gains in shortening the time from call to
contract. It is also not certain that the new system is allowing researchers from all member states to
participate, as there is a growing bias for those who are already leaders in some areas to build strong
consortiums that may exclude smaller players who may have innovative ideas.
Internal and external coherence was thoroughly studied. Internal coherence looked at the coherence
of the programme and its instruments and, in particular, with regard to Public Private Partnerships
(PPPs), with other Societal Challenges and with other areas of Horizon 2020, (such as ERC, EIT-KICS).
External coherence looked at the relationships with other funding mechanisms of the EU (structural
funds, European Back for Investments) and with other national and regional funding programmes for
innovation.
Overall the structure and goals of the differing programmes and instruments are seen as being
coherent with the ambition of Horizon 2020 and the stated objectives of the Union for transport
research. However in some areas improvement in the implementation and how the instruments
work both individually, and especially as a complement to each other are suggested.
Coherence with other EU funding programmes is considered to be extremely important and
necessary to close the innovation divide between member states and increase lagging areas. Given
the importance of transport for economic growth and equity, it is clear that research and innovation
should be well connected with education and training employment policies.
From the Expert Group’s outreach and analysis there is widespread and enthusiastic appreciation of
the EU added value to transport research under this programme. This is not unique to Horizon 2020
as it was also felt for the previous Framework Programmes. Based on interview material and survey
results (PPMI survey and experts survey) overall Horizon 2020 plays a useful and important role in EU
research, boosting the necessity and strategic coordination of the research process.
The collaborative nature of Horizon 2020 is also thought to deliver more than the sum of fragmented
national programmes. It delivers clear European added value contributing to the excellence of
science, enhancement of scientific reputation and improving scientific quality. Areas where Europe is
perceived to be in a leadership position, e.g. in aviation or automotive, should continue to receive
support to keep in front as competition from outside Europe e.g. China, Korea or USA is strong.
Demonstrator projects undertaken by multi-players and pilot projects with various countries
participating are seen as adding particularly high value as, in the majority of cases, it would not be
possible to conduct them at national level.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 21
There are important areas that require attention such as ensuring that project results and European
research knowledge is maximised. The number of outputs in terms of deliverables from FP7 and
Horizon 2020 is large but it is not easy for outsiders to be able to use this information or for those
who have not worked on the projects to retrieve it – a case of quantity over quality.
Overall from the information gathered by the Expert group, there is enthusiastic and broad support
for Horizon 2020 and for EU research programmes in general. This comes from all stakeholders,
including industry (private sector), member state governments, civil society and the research
community. Recognising that transport is unique in terms of its societal value, economic and
environmental impacts and the need for coordination of research in this field, there is definite
support for a new Framework Programme post Horizon 2020.
Looking more closely at the 5 criteria set out by the Commission to frame our work, and considering
the Societal Challenge 4 as requested, the Expert Group also puts forward a set of recommendations
that can be considered as lessons learnt and suggestions for consideration or improvement in the
new round of research activities for transport.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 22
2 INTRODUCTION
2.1 CONTEXT
Horizon 2020, the European Union’s largest Research and Innovation programme was adopted in
2013 to help address the challenges Europe faces. It is built on three main pillars: excellence in
science, societal challenges and industrial leadership. These remain the core values of Horizon 2020
today although it is also recognised that some of the challenges may have changed since its
inception.
The EU 2020 strategy presented innovation and research as a key element in order to kick-start
economic growth and stressed the positive relation at an aggregate level between investment in
research and development (R&D) and socio-economic growth. It also highlights the positive relation
between such investments and productivity. Transport research, therefore, is a fundamental
constituent of the EU’s efforts to create and sustain economic and social growth, and this has rightly
been reflected by its overall position and funding within the Horizon 2020 Programme.
Transport is one of the seven “societal challenges” that form the basis of European policymaking
since 2009 and have become an established theme for European Research. Transforming transport
from being 95% dependent on fossil fuel, reducing its contributing of some 25% of Europe’s
Greenhouse Gas emissions (GHG) and causing 100,000 premature deaths from local air pollution into
a ‘Smart, Green and Integrated’ system is seen as being the key objective for Horizon 2020 transport
research actions4. To meet this the Union has allocated significant budget to transport research
(€938 million for the period 2016-2017 and overall earmarking nearly 8% of the Horizon 2020 budget
over the 7 years of the programme up to 2020). Within these seven challenges there are four that
can be considered to be of transversal interest (cross-cutting) with transport. These include “Secure,
clean efficient Energy; Climate action, environment, resource efficiency and raw materials; Europe in
a changing world – inclusive, innovative and reflective societies and Secure societies – protecting
freedom and security of Europe and its citizens.
It is also recognised that Horizon 2020, as an essential part of Europe’s research strategy, should
contribute to the overarching objectives and European ambitions of open innovation, open science
and being open to the world. Funding for research and innovation also aims to make Europe’s
transport system sustainable, seamless, competitive and responsive.
The period of 2012-2016 saw a number of global developments that have impacted the policy and
research landscapes. These include a number of new, important international political agreements
such as texts of the Sustainable Development Goals and the landmark Paris Climate Change
Agreement as well as an increase in the threat of terrorism, with major incidents occurring in several
member states, an unforeseen influx of migrants from a number of conflict and disadvantaged
regions and huge step-change progress in some areas of technological developments (examples
include 3D printing, smart phone applications and bottom up citizen centred innovative solutions
such as Uber).
4 Halving road fatalities by 2020 is also seen as being a key transport objective.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 23
2.2 OBJECTIVES AND INTERVENTION LOGIC
2.2.1 OBJECTIVES
Horizon 2020 is the European Union's Research and Innovation Framework Programme for the
period 2014-2020 that is designed to help tackle current challenges by providing excellence in
science and cutting edge innovation. Horizon 2020 has been designed to address a number of critical
weaknesses in its science and innovation system (especially those identified in the Ex-ante Impact
Assessment of Horizon 20205). Europe had a number of early and promising leads in many key
technology areas, but in the face of growing global competition this advantage has not translated
into the strong innovative and competitive leadership positioning in line with this potential.
The EU regulation No 1291/2013 of the European Parliament and Council of 11 December 2013
states that Horizon 2020 helps to deliver the Union's objectives:
- to strengthen its scientific and technological basis by achieving a European Research Area
(ERA) in which researchers, scientific knowledge and technology circulate freely;
- to encourage the Union to advance towards a knowledge society; and
- to become a more competitive and sustainable economy in respect of its industry and to
ensure that the conditions necessary for the competitiveness of Union industry exist.
As stated in the Note on the Horizon 2020 Monitoring Report 20146 when quoting the EC’s
Communication Europe 20207. “Investment in research and innovation is essential for Europe’s
future and it is at the heart of the Europe 2020 strategy for smart, sustainable and inclusive growth
and of the priorities of the Juncker Commission”. Horizon 2020 is meant to contribute to building a
society and an economy based on knowledge and innovation in all member states of the European
Union through cutting edge research and development and by leveraging additional research,
development and innovation funding and by contributing to attain research and development
targets, including the target of 3% of GDP for research and development across the Union by 2020.
Preface to Open Innovation, Open Science, Open to the World
The words of Jean-Claude Juncker, President of the European Commission, could not be clearer in his
preface to the “Open Innovation, Open Science, Open to the World – a vision for Europe” booklet8:
“Most of the political priorities set for my mandate as President of the European Commission depend
to a greater or lesser extent on research and innovation. Research and innovation create investment
opportunities for new and better products and services and therefore increase competitiveness and
employment”.
5 https://ec.europa.eu/research/industrial_technologies/pdf/Horizon-2020-ex-ante-impact-assessement-
nmp_en.pdf 6 http://ec.europa.eu/newsroom/Horizon2020/document.cfm?doc_id=15108
7 Communication from the Commission, Europe 2020: A strategy for smart, sustainable and inclusive growth,
COM(2010) 2020 final. 8 https://ec.europa.eu/digital-single-market/en/news/open-innovation-open-science-open-world-vision-
europe
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 24
Horizon 2020 supports the implementation of the Europe 2020 strategy, which sets out the Union’s
objectives of smart, sustainable and inclusive growth, highlighting the role of research and
innovation as key drivers of social and economic prosperity, environmental sustainability and other
Union policies9. In addition it sets itself ‘the goal of increasing spending on research and development
in order to attract private investment of up to two thirds of total investments, thereby reaching an
accumulative total of 3 % of gross domestic product (GDP) by 2020 while developing an innovation
intensity indicator’.
Horizon 2020 is organised into three pillars and subdivided into seven major societal challenges to
address Europe’s main challenges. The Transport Challenge is allocated a budget of €6339 million for
the period 2014-2020 and the programme is structured in four broad lines of activities aiming at:
- Resource efficient transport that respects the environment;
- Better mobility, less congestion, more safety and security;
- Global leadership for the European transport industry; and
- Socio-economic and behavioural research and forward-looking activities for policy making.
The specific objective of the Societal Challenge 4 ‘Smart, green and integrated transport’, and the
objective of this interim evaluation, is to boost the competitiveness of the European transport
industries and ‘to achieve a European transport system that is resource-efficient, climate- and
environmentally-friendly, safe and seamless for the benefit of all citizens, the economy and society’.
Horizon 2020 is also expected to contribute to the Juncker Commission’s Agenda for Jobs, Growth,
Fairness and Democratic Change (2014-2019), focused on ten priorities of strengthening Europe’s
competitiveness and stimulating investment for the purpose of job creation. In his mission letter10 to
the Commissioner for Research, Science and Innovation he states that the Commission as a whole
needs to aim to be more than the sum of its parts, by working together in strong teams in new
collaborative ways, ‘cooperating across portfolios to produce integrated, well-grounded and well
explained initiatives that lead to clear results’ with a focus on ensuring effective implementation and
follow-up on the ground.
Commissioner Moedas’ three goals or strategic priorities for EU research and innovation policy11 are:
- Open Innovation: more actors, more investment (more venture capital and a European Fund
of Funds, real synergies with the Structural Funds via a “Seal of Excellence”), better
regulatory environment for innovation and a new European Innovation Council
- Open Science, including a European Research Integrity Initiative
9 Research and innovation are also key factors for other flagship initiatives of the Europe 2020 strategy, notably
'Innovation Union', 'Resource-efficient Europe', 'An industrial policy for the globalisation era', and 'Digital
Agenda for Europe', and other policy objectives, such as climate and energy policy. Moreover, the objectives of
the Europe 2020 strategy relating to research and innovation include cohesion policy, which has a key role to
play through capacity-building and providing a stairway to excellence. 10
http://ec.europa.eu/commission/sites/cwt/files/commissioner_mission_letters/moedas_en.pdf 2014 11
Open Innovation, Open Science, Open to the World – a vision for Europe. Directorate-General for Research
and Innovation. ISBN 978-92-79-57346-0 - doi:10.2777/061652 - KI-04-16-263-EN-N.
https://ec.europa.eu/digital-single-market/en/news/open-innovation-open-science-open-world-vision-europe
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 25
- Open to the World, with more engagement in science diplomacy and global scientific
collaboration and leading a global research area
Commissioner Moedas12 also specifically mentions three major challenges for research, which are:
- EU research rarely succeeds in getting research results to market. Technologies developed in
Europe are most of the time commercialised elsewhere.
- In some areas Europe falls behind on the very best science, although it generates more
scientific output than any other region in the world. At the same time, every part of the
scientific method is becoming an open, collaborative and participative process.
- Europe punches below its weight in international science and science diplomacy.
In the recent Commission Staff Working Document from July 2016 on the implementation status of
the European Transport White Paper13, the Commission states that five of the ten President’s
priorities are particularly relevant for transport: (1) a new boost for jobs, growth, investment, (2) a
deeper and fairer internal market with a strengthened industrial base, (3) a resilient energy union
with a forward looking climate change policy, (4) a connected digital agenda and (5) the EU as
stronger global actor. The Commission also stresses that these strategies encompass various policy
areas and reflect a more cooperative and Horizontal approach of the Commission to addressing main
challenges, and that all five priorities serve as a new impetus for various transport policy initiatives.
To a large extent, it is evident that President Juncker’s priorities concentrate on taking Europe out of
its current economic crisis. Horizon 2020 serves as a strong anchoring point for research and
innovation efforts in many countries and as a source of inspiration for optimising national and
regional research and innovation systems.
In addition to its contribution to the European Commission objectives, Horizon 2020 also aims at
addressing other issues raised in the framework of the evaluation of previous framework
programmes, notably FP714 and take up key lessons learned from these programmes.
Comparing Horizon 2020 objectives to the five FP7 objectives indicates the continuity from FP7 to
Horizon 2020:
1. Promoting excellence in research,
2. Fostering competitiveness and economic growth,
3. Contributing to solving social challenges,
4. Strengthening the human potential and researchers’ mobility,
5. Fostering transnational research cooperation.
How Horizon 2020, and to some extent FP7, is lining up with these goals are dealt with in this report,
especially under the topics of relevance and coherence. The general objectives of Horizon 2020
12
European Commission - Speech - Open Innovation, Open Science, Open to the World. 22 June 2015. Carlos
Moedas – Commissioner for Research, Science and Innovation. Brussels, ‘A new start for Europe: Opening up to
an ERA of Innovation’ Conference. http://europa.eu/rapid/press-release_SPEECH-15-5243_en.htm 13
http://ec.europa.eu/transport/themes/strategies/doc/2011_white_paper/swd(2016)226.pdf 14
Ex-post Evaluation FP7 and Horizon 2020 Ex-ante Impact Assessment
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 26
transport can easily be mapped against President Juncker’s top priorities15, showing their relevance.
In addition there is an increasing relevance for some international topics such as climate change and
topics that could help the Union and member states to address aspects of the Sustainable
Development Goals, a number of which are relevant in particular to the ambitions of SC4 – Smart,
Green and Integrated Transport. The main difference between the programme lies with the
objective of bringing the research findings closer to the market and how they may be exploited more
readily than those from FP7.
2.2.2 INTERIM EVALUATION
This interim evaluation of the Smart, Green and Integrated Transport Societal Challenge of Horizon
2020 is part of the overall Horizon 2020 interim evaluation that is to be completed by the European
Commission by mid 2017. This report has been prepared by an Expert Group (consisting of five
members16 as input to the EC internal evaluation of transport research that will be conducted by DG
RTD and DG MOVE.
The scope of the evaluation is defined in the Terms of Reference for this Expert Group and it covers
Societal Challenge 4 of Horizon 2020 activities from the start of the programme (2014) until
September 2016. In addition to the evaluation, the Expert Group was asked to provide
recommendations for the remainder of the Horizon 2020 Programme and future Framework
Programmes. These are provided at the end of this report.
The Horizon 2020 Smart, Green and Integrated Transport Societal Challenge Interim Evaluation
includes as far as possible, an assessment of the longer term impacts of previous Framework
Programmes (in particular FP7) and their contribution to the Europe 2020 Strategy. Therefore, it
considers the results of an additional ca.100 finished FP7 projects, corresponding to an additional
€300 million investment since the previous evaluation. Moreover, the present evaluation
methodology17 entails that the projects still under way have also been subjected to a scrutiny of their
prospective results (ca. 100 more projects, corresponding with ca. €650 million).
The approach followed by this evaluation differs from previous ones (such as the ‘Commitment and
Coherence, Ex Post evaluation of FP7’) as it takes into account the increased emphasis on outcomes
(i.e. the impacts for stakeholders, expected and unexpected) rather than on outputs from research
projects and programmes. The shift of focus to outcome and impact is considered as a major
transition and the increased budget for Horizon 2020 compared to FP7 is also noted as another key
difference.
The primary reference points are the policy objectives that Horizon 2020 aims at supporting in the
field of transport. These are:
- resource efficient transport that respects the environment, minimising transport's systems'
impact on climate and the environment (including noise and air pollution) by improving its
efficiency in the use of natural resources, and by reducing its dependence on fossil fuels.
15
http://ec.europa.eu/priorities/index_en 16
Due to unforeseen circumstances a fifth expert was added to the group in September 2016 17
In the specific survey set up for his evaluation, project managers of both finished and running projects were
targeted
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 27
- better mobility, less congestion, more safety and security. The aim is to reconcile the growing
mobility needs for seamless transport, through innovative solutions for integrated, inclusive,
affordable, safe, secure and robust /resilient transport systems.
- reinforcing the global leadership, competitiveness and performance of European transport
manufacturing industries and related services including logistic processes and retain areas of
European leadership (such as aeronautics).
- socio-economic and behavioural research and forward looking activities for policymaking.
The aim is to support improved policy making which is necessary to promote innovation and
meet not only today’s but also future transport challenges and the societal needs related to
it.
In previous evaluations of FP7 and from earlier surveys (ex post evaluation FP7, TRIVALUE evaluation
of FP7, TAG report June 2016) a number of recommendations were made that Horizon 2020 should
address. The most important changes or improvements include:
- strengthen the relation between research and policies;
- create more conditions for the uptake of research results;
- improve measurement of impacts, with more emphasis on breakthrough innovations, and
consider how to increase competitiveness through research, international cooperation in
networks; and
- increase the engagement of civil society.
This interim evaluation has taken these into account where appropriate.
2.2.3 INTERVENTION LOGIC
The objective of the evaluation is to assess in relation to the overarching political and operational
objectives of the programme by looking at its achievements in terms of:
- Relevance;
- Effectiveness;
- Coherence (internal and external);
- Efficiency; and
- EU added value.
The five assessment aspects are the main criteria for the evaluation, determining whether the
Horizon 2020 SC4 is on track to deliver its objectives following the intervention logic, as designed by
the European Commission.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 28
Figure 1
It should also be noted that this evaluation has taken place during a period when there has been a
shift in focus within the Commission (as referred to in section 1) and during which the expectation of
the role of industry and the private sector in policy and research has changed.
Examples include the creation of Public Private Partnerships (PPP) and Joint Undertakings (JU), the
more open character of research calls, the combination of the ERA-NET models, etc. These changes
are meant to reflect and support the shift of accent in the policy objectives. As the PPP’s and JU’s are
subject of a separate evaluation, the present evaluation does not encompass those evaluations in
any great depth, although it is recognised that their existence may well influence the working of
Horizon 2020. Relevant PPPs such as European Green Vehicle Initiative are included in this report.
Unless stated otherwise in the text, the generic term of Public Private Partnership (PPP) is used to
describe the various types of official PPPs and other public-private contracts between the EC and
third parties18. This interim evaluation has not gone into any great depth in its evaluation of PPPs as
this was out of the scope of this work but it has considered them when they have intersected with
the Horizon 2020 programme and especially in relation to aspects of SC4.
For the sake of this evaluation the terminology used in the intervention logic is defined as follows:
- Inputs: funds from FP7 (last part) and Horizon 2020 (until 2016 and excluding PPPs and JUs),
and the estimated leverage effect for national contributions and industry
- Activities: the FP7 projects completed after the ex post evaluation of FP7 (in lieu of Horizon
2020 projects being completed), the still running FP7 projects, the Horizon 2020 calls and
their selected projects until 2016
18
The term PPP in the text refers to many different structures of public-private partnerships rather than a legal
entity except where more specifically we have used other terms such as Joint Undertaking (JU) or Joint
Technology Initiative (JTI)
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 29
- Outputs: research reports, conferences, demonstration sites and pilot projects, articles in
scientific papers, patents etc. It must be noted that Horizon 2020 has not yielded many
actual (final) outputs yet.
- Results: proven and applied knowledge that is recognised as output such as described in the
achievements report, implemented research findings, working demonstrators etc. Again it
must be noted that Horizon 2020 has not yielded any direct results yet as research is still
going on for the first calls. Any changes from the expectations of FP7 results in respect to
Horizon 2020 (and SC4) results were noted where possible.
- Impacts: the mostly qualitative estimates of the effect of results (see achievements report),
or results expected on the basis of ex ante impact assessment of project proposals have been
considered.
Evaluation methodology
The methodology for this evaluation combines quantitative and qualitative aspects, complemented
by expert judgement based as much as possible on evidence and data. The data has been prepared
by the European Commission and supplied to the Expert Group, but it has not been extensively
analysed.
Figure 2: Methodology logic
The structure of the report follows the template provided by the Commission and the evaluation
concept in the following steps:
1. Inventory of the five criteria to establish the main challenges and questions per criterion.
2. Refine to identify key data and statistics, analyses and expert input.
3. Information gathering (document and data searches, surveys, interviews, stakeholder hearing,
examples of success stories)
4. Aggregate information
5. Conclusions and recommendations per criterion
6. Lessons learned and recommendations
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 30
With this in mind the Expert Group decided on an approach that focussed on evidence-based
information from multiple sources rather than statistical analysis. The methodology used consists of:
- document and statistical analysis (see literature list in annex),
- Internet based surveys for different target groups (see Annex for further information on the
questionnaire), of which the most response was received from project coordinators
- a number of semi-structured interviews with selected parties chosen to be representative for
the different subsectors of transport and covering the various profiles of participants,
- information gathering and analysis of available EC data (partly through data and text mining),
a hearing with interested parties and stakeholders who represented business and industry,
the research community, and governments (but not the Commission), and two formal
meetings with the Commission (one in September 2016 and a final one in October).
- informal exchanges,
Triangulation was used as rationale for using multiple qualitative and quantitative source of evidence
(Yin, 200919). The use of multiple source of evidence far exceeds other approaches limited to one
type of measurement. The most important advantage of triangulation is the development of
converging lines of inquiry. Thus, findings and conclusions are likely to be more convincing and
accurate if they are based on several sources of information.
To gain additional insights to the survey in terms of qualitative inputs, the Expert Group performed
individual and group interviews. These interviews were conducted using a list of questions related to
the each of the evaluation criteria, but the interviews mostly took place as a dialogue where the
interviewees had an opportunity to offer their own input on central topics. The profile of those
interviewed consists of a strategic selection of representatives are from different nationalities from
EC Officials, Programme Committee Members, Technology Platforms, Private Public Partnerships,
Joint Undertakings, Project Coordinators and other relevant stakeholders.
Overall a total of 24 people were interviewed during the period August – September 2016. In
general, each interview lasted 1.5 hours. As the interviewees had backgrounds from different
positions representing different key areas of transport, and they spoke on the behalf of the subsector
or stakeholder group that they represented. The results were considered by the Expert Group to be
very rich and informative, but due to time and resource limitations, the essential elements of these
interviews are presented in a condensed form. There has not been an opportunity to conduct an in-
depth analysis or for this to be entirely triangulated against all other sources.
Useful input was also collected from the Internet survey(s) and the stakeholder hearing. It was
nonetheless noticed that the survey was possibly the least effective method for data collection as
people are suffering from being over subscribed to take part in surveys. Despite this the Expert
Group feel it worthwhile to mention the high level of interest to take part when requesting
participation and that people were overall willing to take part and generous with their time.
However, despite several attempts to reach out to the rail sector the Expert Group did not get
spontaneous or enthusiastic responses to participate in this review. It is not entirely clear why this
was the case but therefore to some extent the comments collected are based on comments from
19
Ref: Yin, R.K. (2009). Case study research: design and methods. (4th
ed.) Thousand Oaks, CA: Sage
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 31
those outside of the rail sector. It is the opinion of the Expert Group from their own observations
that the isolation of the rail sector from other transport modes might be symptomatic for that sector.
The key areas of evaluation and analysis include the following:
Commission requested criteria:
- Relevance
- Coherence
- Effectiveness (including simplification as indicated in FP7 evaluation)
- Efficiency
- Added Value
Inward looking aspects:
- New collaborative ways of working
- Cooperation across portfolios
- Clarity of objectives of initiatives
- Results based outcomes (with policy feedbacks)
- Integration of modes and sectors
Outward looking aspects:
- Climate resilience and low carbon economy
- Job creation and competiveness
- Inclusion and equity (including safe and seamless)
- Open and excellence in Science
- Open Innovation
- Open to the World (including relevance to internationally)
The Expert Group would like to note that there were certain constraints in the preparation of this
report that has resulted in some areas not being able to be addressed in depth. This includes the
timing of the interim evaluation, which meant that no Horizon 2020 projects have actually been
completed and the limitations on scope and time of the contract. In addition some key areas of
information have not been able to be accessed in any depth such as the balance of gender in projects
and analysis of newcomers, due to the lack of information available. Therefore it has only been
possible to compare the procedures of allocation of grants, the call timings and text (terminology
used) between the two programmes as no impact of results of Horizon 2020 can be considered.
Despite this, it was interesting and useful to compare the two programmes, as there have been a
number of structural improvements made under Horizon 2020 and the Expert Group’s approach has
tried to capture bottom up experience that can be used to make further improvements and input
into the preparations of FP9.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 32
3 IMPLEMENTATION STATE OF PLAY [KEY DATA
PROVIDED CENTRALLY BY COMMISSION TO EXPERT GROUP] PLEASE NOTE THAT THE INFORMATION FOR THIS CHAPTER IS TO BE PROVIDED BY THE COMMISSION
AND IS CONSTANTLY BEING UPDATED.
THE EXPERT GROUP HAS RECEIVED A NUMBER OF UPDATES (THE LAST ONE 10.10.2016) AND HAS
TAKEN THE RELEVANT INFORMATION FROM THIS INFORMATION INTO CONSIDERATION IN THEIR
EVALUATION. THE ‘CHAPTER’ IS PROVIDED IN AN ANNEX FOR REFERENCE PURPOSES
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 33
4 RELEVANCE As stated in the note on the Horizon 2020 Monitoring Report 201420 when quoting the EC’s
Communication Europe 202021: “Investment in research and innovation is essential for Europe’s
future and it is at the heart of the Europe 2020 strategy for smart, sustainable and inclusive growth
and of the priorities of the Juncker Commission”. This certainly applies to Horizon 2020 transport
research.
4.1 IS HORIZON 2020 TRANSPORT TACKLING THE RIGHT ISSUES?
Transport is unique in terms of its societal value and therefore relevance to SC4, as it touches
everyone’s daily lives and is a unique combination of:
- Its economic importance, not only for the number of jobs it offers in industries, companies
and services directly working in this sector but also because its services are required by all
other sectors (except for, up to some extent, digital virtual goods) to transport people and
goods facilitating trade between countries and regions.
- Its impact on the environment in terms of local and air quality and global pollution, energy
consumption and security, noise and land take.
- Its impact on the quality of life of every European citizen every single day: with trips to
schools, to work, for leisure or health reasons and for social contact but we also need to
recognise the negative impacts on society from traffic accidents, air pollution and the
severance of communities.
The following table summarizes the specific objectives of Horizon 2020 Transport challenge as well as
the four broad lines of activities of Horizon 2020 that provide the backbone of the work
programmes22.
Table 1: Horizon 2020 Transport specific objectives and lines of activities
Horizon 2020 Transport specific objectives Horizon 2020 Transport - Lines of activities
To boost competitiveness of the European
Transport industries
Global leadership for the European transport
industry
To achieve a European transport system that is
resource efficient, climate- and environmentally-
friendly
Resource efficient transport that respects the
environment
To achieve a European transport system that is
safe and seamless for the benefit of all citizens,
the economy and society
Better mobility, less congestion, more safety and
security
Socio-economic and behavioural research and
forward looking activities for policy making
20
http://ec.europa.eu/newsroom/Horizon2020/document.cfm?doc_id=15108 21
Europe 2020: A strategy for smart, sustainable and inclusive growth, COM(2010) 2020 final. 22
http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32013R1291&from=EN. REGULATION (EU)
No 1291/2013 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 11 December 2013 establishing
Horizon 2020 - the Framework Programme for Research and Innovation (2014-2020)
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 34
As the European Regulation establishing Horizon 2020 states23“investing in research and innovation
for a greener, smarter and fully integrated reliable transport system will make an important
contribution to the objectives of the Europe 2020 strategy and of its flagship initiative 'Innovation
Union'. Activities will support the implementation of the White Paper ‘Roadmap to a Single European
Transport Area - Towards a competitive and resource efficient transport system’. They will also
contribute to the policy goals outlined in the flagship initiatives 'Resource Efficient Europe', 'An
Industrial Policy for the Globalisation Era' and 'Digital Agenda for Europe' and interface with the
relevant Joint Programming Initiatives”.
In general, Horizon 2020, and especially the Transport Societal Challenge, is more oriented towards
solving the most pressing needs of society than its predecessor FP724, which was more focused on
science and technology (European Commission’s “Transport research and innovation achievements
report, 31 March 2016”25).
The Expert Group looked to establish if Horizon 2020 and the research calls of SC4 were tackling the
right issues, and if Horizon 2020 pursues the right (relevant) objectives. In addition, will the
programme yield relevant research results that will have the desired impact and to what extent is
innovation – in terms of potential and real impact on society, jobs and growing the economy –
treated? In other words, how does it contribute to responding to the stated societal challenges and
how can it help achieve policy objectives (in line with the requirement for it to become outcome
rather than output focussed).
4.1.1 RELEVANCE TO THE SITUATION IN EUROPE WHEN HORIZON 2020 TRANSPORT
WAS DESIGNED AND HOW IT IS ADDRESSING EUROPE’S CHALLENGES
The rationale and Union added value that represented the basis for the Horizon 2020 Transport, as
described in the European Regulation establishing Horizon 2020 can be summarized as follows:
- “European transport industry was facing increasingly fierce competition from other parts of
the world. Breakthrough technologies would be required to secure Europe's future
competitive edge and to mitigate the drawbacks of our current transport system.
- It was essential to reduce the environmental impact of Transport through targeted
technological improvement.
- Congestion was an increasing problem; systems were not yet sufficiently smart; alternative
options for shifting towards more sustainable modes of transport were not always attractive;
road fatalities remained dramatically high; citizens and businesses expected a transport
system that were accessible to all, safe and secure. The urban context posed specific
23
http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32013R1291&from=EN. REGULATION (EU)
No 1291/2013 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 11 December 2013 establishing
Horizon 2020 - the Framework Programme for Research and Innovation (2014-2020) and repealing Decision No
1982/2006/EC. 24
www.transport-research.info/programme/transport-including-aeronautics-Horizontal-activities-
implementation-transport-programme 25
Internal report of the European Commission made available to the Expert Group.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 35
challenges and provides opportunities to the sustainability of transport and for a better
quality of life.
- Within a few decades the expected growth rates of transport would drive European traffic
into a gridlock and make its economic costs and societal impact unbearable, with adverse
economic and societal repercussions.
The problems of pollution, congestion, safety and security were common throughout the Union
and called for collaborative Europe-wide responses… objectives could not be achieved through
fragmented national efforts alone… Emphasis would be placed on priority areas that match
European policy objectives where a critical mass of effort would be necessary, where Europe-
wide, interoperable or multimodal integrated transport solutions could help remove bottlenecks
in the transport system, or where pooling efforts transnationally and making better use of and
effectively disseminating existing research evidence could reduce research investment risks,
pioneer common standards and shorten time to market of research results.
Research and innovation activities should include a wide range of initiatives, including relevant
public-private partnership, that cover the full innovation chain and follow an integrated approach
to innovative transport solutions. Several activities were specifically intended to help bring results
to the market: a programmatic approach to research and innovation, demonstration projects,
market take-up actions and support for standardisation, regulation and innovative procurement
strategies all serve this goal. In addition, using stakeholders' engagement and expertise would
help bridge the gap between research results and their deployment in the transport sector.
Investing in research and innovation for a greener, smarter and fully integrated reliable transport
system would make an important contribution to the objectives of the Europe 2020 strategy and
of its flagship initiative 'Innovation Union'. The activities would support the implementation of the
White Paper "Roadmap to a Single European Transport Area - Towards a competitive and
resource efficient transport system". They would also contribute to the policy goals outlined in the
flagship initiatives 'Resource-efficient Europe', 'An Industrial Policy for the Globalisation Era' and
'Digital Agenda for Europe'. They would also interface with the relevant Joint Programming
Initiatives”.
In spite of a slower increasing pace of transport activities due to the long-lasting economic crisis and
some achievements in the reductions in traffic fatalities, the situation in Europe has evolved in the
direction of exacerbating the need for action as presented above: even fiercer international
competition, a larger share of anthropogenic impacts on environment due to transport, additional
transport services demand due to internet shopping, increased awareness of the impacts of
transport on health and social inclusion. The wording of the SC4 objectives is broad enough to cover
the additional challenges since the inception of Horizon 2020 (see table 6 in paragraph 4.2.)
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 36
Horizon 2020 Transport also continues to be clearly relevant for the following flagships initiatives of
the Europe 2020 strategy presenting in 201026:
- The Smart Growth goal: Digital Agenda for Europe, Innovation Union and Youth on the move
- The Sustainable Growth goal: Resource Efficient Europe and An industrial policy for the
globalisation era
- The Inclusive Growth goal: An agenda for new skills and jobs and a European platform
against poverty.
Transport is clearly a major factor in the first two initiatives linked to Smart Growth (digital
transformation of the transport sector and ‘Innovation Union’). It also plays a strong role in the
initiatives fostering inclusive growth, connected markets, logistics, PPPs (public-private partnerships)
and external policy instruments27. However, the connections between Horizon 2020 transport calls
and these flagships initiatives are not clear. Both in the hearing and in several interviews the cross-
cutting nature of transport was mentioned. Comments were made that this was not being specifically
picked up and that the horizontal aspects of the European Union policy identified in these flagship
initiatives were not sufficiently being picked up either. This is also the opinion of the Expert Group
based on their analysis.
With regard to more recent developments in EU strategy, the general objectives of Horizon 2020
transport can easily be mapped against President Juncker’s top priorities28, showing that they are
highly relevant. The relevance of Horizon 2020 transport can be clearly observed in the following
table, showing the evolution of top-level European priorities between 2010 and 2014. Horizon 2020
Transport priorities fit perfectly with them.
Table 2: Horizon 2020 Transport specific objectives compared with Europe 2020 and Juncker’s
priorities
Year 2010
Europe 2020
Year 2013
Horizon 2020 Transport
Year 2014
Juncker’s priorities
Smart growth Competitiveness of the European
Transport industries
Jobs, Growth and Investment
Sustainable growth A European transport system that is
resource efficient, climate- and
environmentally-friendly
Energy Union and Climate
Inclusive growth A European transport system that is
safe and seamless for the benefit of all
citizens, the economy and society
Justice and Fundamental
Rights
It is clear that transport has a high impact and relevance on economic growth and jobs (interpreting
competitiveness in its widest sense) and this is reflected in the following facts and figures from the
Horizon 2020 Transport Work Programmes, among other documents:
26
http://ec.europa.eu/europe2020/europe-2020-in-a-nutshell/flagship-initiatives/index_en.htm 27
http://ec.europa.eu/europe2020/europe-2020-in-a-nutshell/eu-tools-for-growth-and-jobs/index_en.htm 28
http://ec.europa.eu/priorities/index_en
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 37
- In 201329: the transport sector directly represented 6.95% of the Union’s Gross Domestic
Product (GDP). The whole industry accounted for 7.03% of total employment in the EU,
corresponding to more than 14 million people in absolute terms. It is estimated that around
12 million jobs are in the automotive sector (both directly and indirectly). The aviation sector
alone generates around 2% of EU’s GDP (or some €110 billion). It directly employs between
1.4 million and 2 million people and indirectly supports between 4.8 million and to 5.5
million jobs. Its combined positive impact, including tourism, is thought to be as much as
€510 billion through the multiplier effect30. Both sectors are considered to be high-tech
sectors, employing highly qualified people and leading technologies in line with Europe’s
ambition to be a global leader. In addition there are technology spillovers from the work in
these areas that can be successfully transferred to other industrial sectors31.
- Freight transport and logistics keeps much of Europe’s economy moving. In 2012 (EU28)
freight transport was responsible to close to 3.8 trillion tonne-kilometres. 45% of this was
transported by road, 40% by sea, 9% by rail and about 3% by inland waterways. However,
Eurostat surveys estimate that 24% of road freight vehicles in the EU are running empty.
Flow imbalances are seen as being part of the problem and improvements in efficiency in the
EU logistics sector are estimated to be able to provide savings of between €100 billion and
€300 billion per year32.
In the Commission Staff Working Document of July 2016 on the implementation status of the
European Transport White Paper33, the Commission states that five of the President ’s ten priorities
are particularly relevant for transport. These are namely: (1) a new boost for jobs, growth,
investment, (2) a deeper and fairer internal market with a strengthened industrial base, (3) a resilient
energy union with a forward looking climate change policy, (4) a connected digital agenda and (5)
ensuring that the EU is a stronger global actor. The Commission also stresses that these strategies
encompass various policy areas and reflect a more cooperative and horizontal approach of the
Commission to addressing main challenges, and that all five priorities serve as a new impetus for
various transport policy initiatives.
Of particular relevance is the link between transport research and climate. This is clearly seen in the
data provided by the European Commission as part of its cross-cutting reporting34. Based on the EC’s
data, the share of EC contribution to Climate Change (all projects) is 23.9% while for the entire
Horizon 2020 it is 60.1%35. It can be concluded from these two values, that transport is more relevant
for climate change than for Horizon 2020 in general. Only “SC3 Secure, clean and efficient energy” is
more relevant than transport with a percentage of 99.1%. Interestingly, “SC5 Climate action,
29
http://ec.europa.eu/transport/themes/strategies/doc/2011_white_paper/swd(2016)226.pdf 30
Brussels, 7.12.2015.COM(2015) 598 final. An Aviation Strategy for Europe.http://eur-lex.europa.eu/legal-
content/EN/TXT/PDF/?uri=CELEX:52015DC0598&from=EN 31
https://ec.europa.eu/research/participants/data/ref/Horizon 2020/wp/2014_2015/main/Horizon 2020-
wp1415-transport_en.pdf 32
10% to 30% improvement would yield these savings
http://ec.europa.eu/research/participants/data/ref/Horizon 2020/wp/2016_2017/main/Horizon 2020-
wp1617-transport_en.pdf 33
http://ec.europa.eu/transport/themes/strategies/doc/2011_white_paper/swd(2016)226.pdf 34
Based on a set of Key Performance Indicators KPIs 35
Data provided by the EC for the purposes of this interim assessment. Date of data extraction: 01/09/2016.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 38
environment, resource efficiency and raw materials” has a percentage of 47.1%, smaller than in the
case of transport.
A similar analysis based on the same information can be offered in the case of the share of EC
contribution to Sustainable Development (all projects). Here transport is 85.5%, while for the entire
Horizon 2020 it is 46.1%. In fact, transport is the fourth largest area of Horizon 2020, after “SC2 Food
security, sustainable agriculture and forestry, marine, maritime and inland water research and the
bioeconomy”, “SC3 Secure, clean and efficient energy” and “SC5 Climate action, environment,
resource efficiency and raw materials”. This relates also to section 8.2.5 concerning Horizon 2020
delivering research that provides feedback to policy (to be found in the section 8 on added value).
The 2011 White Paper36 is one of the initiatives foreseen to deliver on the resource-efficient Europe
flagship37 and the relevance of Horizon 2020 transport to the European transport policy presented in
it38 is undeniable.
The following table maps the links between the Transport White Paper goals and the specific
objectives of the Horizon 2020 transport challenge. Horizon 2020 transport research is especially
relevant by contributing to new technologies, developing new ways of implementation, pilots and
demonstrations, and for socio-economic and behavioural research.
Table 3: Horizon 2020 Transport specific objectives compared with Transport White Paper goals
Transport White Paper Horizon 2020 Transport specific objectives
To boost competitiveness of the European Transport industries
GENERAL GOAL: a 60 % reduction of CO2 by 2050 compared to 1990 levels and by around 20% by 2030 compared to emissions in 2008. Operation goal 1. Cars; aim to halve the use of 'conventionally-fuelled' cars in cities and have virtually CO2-free city logistics in major urban centres by 2030. Operation goal 2. Low-carbon fuels in aviation should reach 40 % by 2050 and CO2 emissions from maritime bunker fuels should be reduced by 40 % by 2050 compared to 2005 levels.
To achieve a European transport system that is resource efficient, climate- and environmentally-friendly
OPERATIONAL GOALS 3 TO 10. Operation goal 3. Freight; modal shift from road transport. Operation goal 4. EU-wide high-speed rail network. Operation goal 5. Multimodal TEN-T core network. Operation goal 6. Long-term comprehensive network. Operation goal 7. Traffic-management systems in all modes. Operation goal 8. Multimodal transport information. Operation goal 9. Close to zero fatalities in road transport. Operation goal 10. Towards ‘user pays’ and ‘polluter pays’
To achieve a European transport system that is safe and seamless for the benefit of all citizens, the economy and society
36
http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52011DC0144&from=EN 37
http://ec.europa.eu/resource-efficient-europe/pdf/resource_efficient_europe_en.pdf 38
http://ec.europa.eu/transport/themes/strategies/2011_white_paper_en.htm. Commission White Paper
"Roadmap to a Single European Transport Area – Towards a competitive and resource efficient transport
system" (COM(2011)0144).
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 39
Of the above operation goals, developing and deploying new and sustainable fuels and propulsion
systems, optimising the performance of multimodal logistic chains and increasing the efficiency of
transport and of infrastructure use with information systems are particularly related to Horizon 2020
Transport objectives.
In July 2016, the Commission presented its Implementation Assessment of the White Paper39. The
main conclusions are that there are no indications of substantial change in the main transport trends,
identified in 2011 White Paper, in recent years. ‘Transport activity continues to raise concerns about
its negative externalities, notably GHG emissions, air pollution, noise, congestion, safety. Road
remains a predominant mode of transport and source of emissions, while aging and urbanisation
remain major demographic trends for transport. Still some new societal developments and
technological advancements have gained on pace and are likely to shape the functioning of transport
services already in the near future’.
As a general conclusion, the Commission states that ‘the 2011 White Paper defines a long-term vision
until 2050 for a transport sector that continues to serve the needs of the economy and of the citizens
while meeting future constraints’.
For instance, increased action is needed at EU level in connection with road safety, a key societal
challenge showing stagnation and a deviation from the projected trend. This is an area where
research is playing40 a key role.
Figure 3: Road fatalities in the EU since 2001.
Source: DG MOVE, CARE database (provisional data for 2015)
39
http://ec.europa.eu/transport/themes/strategies/doc/2011_white_paper/swd(2016)226.pdf 40
The Transport Work Programme 2014-2015 introduced a specific topic titled “MG.3.4-2014. Traffic safety
analysis and integrated approach towards the safety of Vulnerable Road Users” and the Transport Work
Programme 2016-2017 includes several topics related to road safety such as the topic “MG-3.2-2017:
Protection of all road users in crashes”.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 40
The future Strategic Transport Research and Innovation Agenda (STRIA), as part of the upcoming
communication on Energy Union Integrated Strategy on Research, Innovation and Competitiveness
(EURICS), will help to streamline the research and innovation efforts and focus them on the most
pressing challenges and the most beneficial actions for transport.
In general, and based on interviews, hearings and our own expert opinion, it can be stated that the
aim of Horizon 2020 Transport to address the broad number of important issues and deliver
excellence and impactful relevant research and development outcomes under one programme
constitutes a large challenge. Several stakeholders questioned whether the work programmes are
too broad and if they should rather focus or the more pressing needs or market failures instead of
trying to cover all aspects.
4.1.2 AREAS OF RELEVANCE OF SPECIFIC EUROPEAN INTEREST
Horizon 2020 transport research is certainly relevant for many important European initiatives,
policies and communications, but this varies depends on the topic as this section presents.
(A) DIGITAL SINGLE MARKET PRIORITY
It is more complex to find direct relevance with President Juncker’s priorities than to align Horizon
2020 Transport specific objectives with the new priorities set up by Commissioner Moedas41: Open
Innovation, Open Science and Open to the World. This is mainly due to the more general character of
these three priorities affecting, in one way or another, all research and innovation domains and
challenges.
For instance, within the Digital Single Market priority, the need for affordable parcel delivery costs is
specifically mentioned42 since more than 90% of e-shoppers consider low delivery prices and
convenient return options as important when buying online. In fact, 62% of companies that are
willing to sell online say that high delivery costs are a problem for them. The EC’s 2015
communication “A Digital Single Market Strategy for Europe” actually dedicates an entire section to
affordable high-quality cross-border parcel delivery43. Research and innovation in this area is clearly a
relevant area for Horizon 2020 Transport but it has not yet been addressed to the extent that a
digital single market requires.
The Horizon 2020 Transport Work Programme 2014-2015 makes an explicit reference to the Moedas
priority of Open Science: ‘a novelty in Horizon 2020 is the Open Research Data Pilot which aims to
improve and maximize access to and re-use of research data generated by projects’44.
With regard to the Open to the World priority, there is a clear and very important link with Horizon
2020 transport and this is presented later in a specific section on international outreach.
According to the European Commission’s data, cross-cutting KPIs reporting on the share of EC
contribution to the Digital Agenda (all projects) in the Societal Challenge SC4 Smart, green and
integrated transport come in at 17.6% while for the Horizon 2020 as a whole the percentage reaches
41
http://europa.eu/rapid/press-release_SPEECH-15-5243_en.htm 42
https://ec.europa.eu/priorities/sites/beta-political/files/dsm-factsheet_en.pdf 43
http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52015DC0192&from=EN 44
https://ec.europa.eu/research/participants/data/ref/Horizon 2020/wp/2014_2015/main/Horizon 2020-
wp1415-transport_en.pdf
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 41
24.1%45. This can be considered to be below optimal levels to achieve the desired Digital Agenda
objectives.
(B) RESOURCE EFFICIENT EUROPE
In the case of the flagship initiative “Resource Efficient Europe”, the European strategy indicates that
the Commission will work46:
- To present proposals to modernise and decarbonise the transport sector thereby
contributing to increased competitiveness. This can be done through a mix of measures e.g.
infrastructure measures such as early deployment of grid infrastructures of electrical
mobility, intelligent traffic management, better logistics, reduction of CO2 emissions for road
vehicles as well as for the aviation and maritime sectors…
- To accelerate the implementation of strategic projects with high European added value to
address critical bottlenecks, in particular cross border sections and intermodal nodes (cities,
ports and logistic platforms)
The Resource Efficient Europe communication also highlights that47:
- Technological improvements, a significant transition in energy, industrial, agricultural and
transport systems, and changes in behaviour as producers and consumers is required to
achieve a resource-efficient Europe;
- Production processes (such as Just-in-time or the collection of waste and recycling) designed
to reduce the energy may conversely require more transport.
- The EU’s industrial policy for the globalisation era specifically indicates that the Commission48
should ensure that transport and logistics networks enable industry to have effective access
to the Single Market and the international market beyond. Trans-European transport
networks should be aligned with the requirements of a resource-efficient, low-carbon
economy.
- Policy measures to improve resource efficiency and overall economic competitiveness must
place greater emphasis on 'getting prices right' and making them transparent to consumers,
for instance in transport, energy. There is a need for new models focused on specific policy
areas and sectors such as energy and transport.
More specifically in the low carbon and climate change areas, Horizon 2020 Transport Research and
innovation also represents a necessary contribution towards the objectives set in several European
Commission communications:
a) COM(2011) 112 final. A Roadmap for moving to a competitive low carbon economy in 205049:
reducing CO2 emissions by at least 60 % in transport by 2050 compared to 1990 levels.
b) COM/2015/080 final. A Framework Strategy for a Resilient Energy Union with a Forward-
Looking Climate Change Policy50: by 2030, at least 27% share of renewable energy consumed
45
Data provided by the EC for the purposes of this interim assessment. Date of data extraction: 01/09/2016. 46
http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2010:2020:FIN:EN:PDF 47
http://ec.europa.eu/resource-efficient-europe/pdf/resource_efficient_europe_en.pdf 48
http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2010:2020:FIN:EN:PDF 49
http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2011:0112:FIN:en:PDF
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 42
in the EU and at least 27% improvement in energy efficiency. At least 40% domestic
reduction of greenhouse gas emissions by 2030 and 60% by 2040. Reaching and exceeding
these intermediary objectives will allow the EU to pursue the goal of a 80-95% decrease in
greenhouse gas emissions by 2050. This Energy Union Communication includes the
announcement of a “Strategic Transport Research and Innovation Strategy” STRIA to be
presented in the next months.
The Expert Group, supported by comments in several interviews and the hearing, found it difficult to
establish clear links between these high-level numerical policy objectives and the specific
contribution expected from some topics, in particular those less prescriptive (for instance, what will
be the contribution to “an 80-95% decrease in greenhouse gas emissions by 2050” that will be
provided by a certain project concentrating of improving powertrain efficiency). Topics do
correspond to the general objectives but quantifying its impact towards them is more complex. The
future Strategic Transport Research and Innovation Agenda (STRIA) should fill this gap by producing
roadmaps and looking more precisely to what can be performed in research to achieve the
overarching objectives. STRIA is considered a key component designed to streamline the research
and innovation efforts and focus them on the most pressing challenges and the most beneficial
actions for transport.
Probably the most recent document that shows the relevance of Horizon 2020 Transport research
and innovation is the European Commission’s communication “Accelerating Europe’s transition to a
low-carbon economy”51. This communication stresses, among other important issues, the role of the
EU’s cities in the fight against climate change at local level and the key role of the energy sector but
also that of the industry, buildings, transport, waste, agriculture, land-use and forestry.
(C) GLOBAL INITIATIVES
In addition to all the previous European Union initiatives, there is a very relevant number of
international agreements that have been signed over the past 12 months, notably the Paris Climate
Change agreement of 201552 and the new Sustainable Development Goals (SDGs) adopted last year
201553.
In preparation of the International Climate Change conference (COP 21), held in Paris in December
2015, countries submitted national climate action plans54. The European Union announced 55 that the
50
http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=COM:2015:80:FIN 51
Brussels, 20.7.2016.COM(2016) 500 final. Communication from the Commission to the European Parliament,
the Council, the European Economic and Social Committee and the Committee of Regions. Accelerating
Europe's transition to a low-carbon economy. Communication accompanying measures under the Energy
Union Framework Strategy: legislative proposal on binding annual greenhouse gas emissions reductions by
Member States from 2021 to 2030, legislative proposal on the inclusion of greenhouse gas emissions and
removals from land use, land use change and forestry into the 2030 climate and energy framework and
communication on a European Strategy for low-emission mobility.
https://ec.europa.eu/transparency/regdoc/rep/1/2016/EN/1-2016-500-EN-F1-1.PDF 52
http://ec.europa.eu/clima/policies/international/negotiations/paris/index_en.htm 53
Transforming our World: The 2030 Agenda for Sustainable Development
United Nations, 2015.https://sustainabledevelopment.un.org/post2015/transformingourworld/publication 54
INDCs – (Intended) National Determined Contributions
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 43
EU and its Member States are committed to a binding target of at least 40% domestic reduction in
greenhouse gas emissions by 2030 compared to 1990; at least a 27% share of renewable energy
consumption and at least 27% energy savings compared with the business-as-usual scenario56 with its
new 2030 Framework for climate and energy, an increase of targets stated in the White Paper. These
commitments are economy wide and transport is only one of the sectors that will contribute to these
reductions. Horizon 2020 Transport Research and innovation represents a necessary contribution
towards these low carbon objectives and those set out in several European Commission
communications.
This is supported with the data provided by the European Commission as part of its cross-cutting KPIs
reporting. The relevance of transport in the EC contribution to Climate Change and to Sustainable
Development was already mentioned in section 4.1.1, with a percentage of 85.5%. This is largely
coherent with the figures provided by the European Commission as part of its cross-cutting KPIs
reporting. Based on the EC’s data, the percentage of budget of topics in the Work Programme
mentioning at least one third country or region is 28.5% in the case of transport, while it is only
14.8% for the entire Horizon 202057. It can be concluded from these two values, that international
cooperation is much more important for transport than for Horizon 2020 in general.
It is also clear that these international agreements send a clear signal to all stakeholders, investors,
businesses, civil society and policy-makers that the global transition to clean energy and a shift away
from fossil fuels is here to stay58. The role of transport in achieving this is a major one.
By contributing to the economy, competitiveness and job creation, the transport sector is also
relevant to delivering other key SDGs such as poverty, social exclusion and social inequity. This
should be able to be measured rather than assumed. Working on new indicators that more clearly
show how sustainable transport improves equity, inclusion and uplifts people out of poverty would
fit with the evolution of challenges mentioned in the previous section, and is considered by the
Expert Group and others to be an interesting topic for developing research questions in the future.
Therefore it is considered that Horizon 2020 Transport has the potential to help Europe and its
member states to deliver on the Sustainable Development Goals and the International Climate
Change Agreement.
After having analysed the two Horizon 2020 Transport Work Programmes published so far, the
Expert Group agrees that there has been some progress, and the Horizon 2020 Transport Research
agenda is relevant towards achieving a Resource Efficient Europe in terms of technology, energy
transition and to some extent transformation of transport systems.
4.1.3 HORIZON 2020 TRANSPORT AND ITS INTERNATIONAL OUTREACH
One of the top Commissioner Moedas priorities is ‘Open to the World’, so it is worthwhile revisiting
how Horizon 2020 transport is directly linked to this priority.
55
www4.unfccc.int/Submissions/INDC/Published%20Documents/Latvia/1/LV-03-06-EU%20INDC.pdf 56
www.consilium.europa.eu/uedocs/cms_data/docs/pressdata/en/ec/145397.pdf 57
Data provided by the EC for the purposes of this interim assessment. Date of data extraction: 01/09/2016. 58
http://ec.europa.eu/clima/policies/international/negotiations/paris/docs/qa_paris_agreement_en.pdf
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 44
Firstly, the international dimension in Horizon 2020 transport is seen as being more strategic, and
reinforced, as became clear from the interviews, for instance. The link of the Horizon 2020 transport
Societal Challenges and the Open to the World priority is clearly presented in the 2016-2017 Horizon
2020 Transport Work Programme59:
“International cooperation will have a key role to play. Global challenges such as CO2 and
polluting emissions, oil dependency, transport safety and security, noise pollution, and
standardisation of many services, products and procedures will benefit from global
solutions...”
As one specific example of these challenges, it is emphasized that transport safety will be a priority
for a Euro-African science diplomacy partnership. Since FP7 transport research has a long tradition of
international cooperation activities in aviation, renewable fuels, smart mobility and safety. The
openness to the world dimension of the Transport Work Programmes is also reflected in the
following sentence taken from the 2016-2017 Work Programme:
“All new priorities in the Work Programme… have a two-fold aim: addressing key challenges
that Europe faces, and making our industry more competitive and cooperative through
transferring these solutions and standards worldwide, as other regions are confronted with
similar challenges”.
The Horizon 2020 Monitoring Report 2014 also indicates60:
“Identifying suitable themes and partners for targeted international cooperation activities
was an important part of the preparation of the first Horizon 2020 work programmes. The
impact of this approach is reflected in the increase from 12% (FP7 baseline) to 22% (October
2015) in the indicator on the budget share of Work Programme topics mentioning
international cooperation or a specific third-country or region”.
Based on the specific analysis conducted for this report, it is estimated that around 24% of the
budget of the first two rounds of Horizon 2020 transport calls (without including PPPs) is related to
international cooperation, either because the topic directly addresses it or because international
cooperation is mentioned.
This percentage is therefore comparable to the average (22%) reported in the Horizon 2020
Monitoring Report 2014, and is also largely coherent with the figure provided by the European
Commission as part of its cross-cutting KPIs reporting. As mentioned and based on the EC’s data, the
share of EC contribution to International Cooperation topics is 28.5% and from this it can be
concluded, that international cooperation is much more important for Transport than for Horizon
2020 in general.
On the other hand, and based on the data available to as to 1st of August 2016, the share of EC
Contribution to Participations in Signed Grants represents a 0.1% for the entire Horizon 2020
59
http://ec.europa.eu/research/participants/data/ref/Horizon 2020/wp/2016_2017/main/Horizon 2020-
wp1617-transport_en.pdf 60
http://ec.europa.eu/newsroom/Horizon2020/document.cfm?doc_id=15108
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 45
Transport, a 0.2% for projects excluding the SME Instrument and the JUs, a 0.4% in the SME
Instrument and a 0% in JUs61.
4.2 ADDRESSING SPECIFIC STAKEHOLDER NEEDS, INCLUDING EMERGING
ISSUES
Horizon 2020 Transport offers comprehensive consultation mechanisms with stakeholders that are
designed to help the programme to adapt to new scientific and socio-economic developments
including:
- Consultations with Member States through the Transport Programme Committee
(Comitology procedures62)
- Transport Advisory Group
- Consultations with key stakeholders such as industry groups, research associations,
technology platforms and other networks such as POLIS, ERTICO…
- Open or public consultations
- Internal consultations…
This evaluation relies partly on an extensive consultation process, including contacts with technology
platforms (e.g. ERTRAC, ERRAC, Waterborne, ACARE, and ECTP), support actions (e.g. ALICE),
association (e.g. POLIS, ECTRI, FEHRL, EREA, ECVIA), key SMEs (e.g. ERTICO), JU’s and industry
associations (e.g. IMG, ACEA, ACEM), modal representative associations (e.g. FIM, FIA), large or key
participants in projects, and of course member states. The roadmaps produced by these groups are
of particular relevance for transport and many of the topics or areas for research are continuously
being considered in the calls.
In its Scoping Paper for the preparation of the next 2018-2020 Work Programme63, the Commission
reports that the Transport Advisory Group (TAG) pointed out the relevance of both the content of
the previous Work Programmes and the priority challenges identified in the first TAG report in 2014
and in its follow-up report submitted to the European Commission (May 2016). They stressed the
need to continue with a disruptive rather than incremental approach to research in order to respond
rapidly to shifting transport paradigms with new revolutionary technologies, business environment
and mobility patterns. In addition, the TAG identified a number of new issues, which have emerged
since 2014, for instance:
- The positive impact of the COP21 agreements providing the transport sector with a clear
agenda and mandate. An increasing impact of climate change and extreme weather
phenomena risks calls for new approaches towards preparedness and resilience, particularly
for major transportation nodes and mass transport systems.
61
Data provided by the EC for the purposes of this interim assessment in the document “Interim Evaluation -
Projects - EU.3.4 - Transport.xlsx” 62
http://eur-lex.europa.eu/summary/glossary/comitology.html 63
First Orientations towards the scoping paper for Horizon 2020 Societal Challenge 'Smart, Green and
Integrated Transport'. Internal document of the European Commission that has been made available to the
Expert Group.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 46
- A slow growth world full of uncertainties contributing to accumulated delays in building and
maintaining infrastructure and mass transit systems, with accompanying negative impacts on
accessibility, congestion, safety and the overall reduced efficiency of transport systems.
- Security threats targeting Europe and its logistic and strategic centres,
- Lower oil prices affecting many economic drivers (although it may have a positive impact on
air transport and the auto industry) it impacts on alternative fuel research and development,
and may lead to delays in Europe’s transition towards a carbon-free economy.
- Large-scale migration flows impact border management, hamper mobility, increase costs,
lower accessibility and accentuate regional differences.
- The explosion of new business models related to digitalisation, disruptive technologies and
new actors from other non-transport sectors.
Formal consultations with key stakeholders such as industry groups, research associations,
technology platforms and other networks such as POLIS, ERTICO took place in 2016 as part of the
preparatory work for the drafting of the Work Programme 2018-2020. Approximately 40
organisations from all transport modes and including research organisations, industry associations,
public bodies and users of transport as well as citizens associations participated in the stakeholder
consultation from 23/3/2016- 11/5/2016. A number also took the trouble to prepare position papers
(e.g. ECTRI and ETRA) to not only to set out their priority areas, but also to voice concerns on present
calls, processes and programmes. The priority areas of research underpinning the current strategic
objectives highlighted by the stakeholders were64:
- Fighting climate change and the slow progress to decarbonisation passenger and freight
transport
- The resilience of transport systems
- Electrified transport, including energy harvesting and storage systems
- Seamless and integrated transport (with particular emphasis of public transport),
multimodality, interoperability
- A single market for mobility as a service
- Digitalisation, including multi-model trip information, digital transport areas, big data and
transport, Internet of (Transport) Things Io(T)T
- Safety and security
- Environment and health
- Governance and regulation
New areas (inter alia) identified by the stakeholders that they would like to see included in the next
work programme were:
- Automated transport: not only road vehicles, but including drones, vessels, infrastructure for
automated transport and covering new terms such as big data, drones, etc.65
- Transport and health (obesity and active transport, mainly)
- Circular economy and its relation to transport
64
WP 2018-2020. Summary of replies to public consultation. Internal doc prepared by DG RTD, Transport. 65
Brussels, 7.12.2015.COM(2015) 598 final. An Aviation Strategy for Europe.http://eur-lex.europa.eu/legal-
content/EN/TXT/PDF/?uri=CELEX:52015DC0598&from=EN
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 47
- New nanotechnology materials with self-reporting and self-healing capabilities, for instance
for infrastructure
- Hyper-connected transport (including 5G systems for rail communications), physical Internet,
disruptive technologies
- Vulnerabilities and new security threats (including cyber-security and personal data
protection)66
Other more general proposals from the stakeholders for improving the programme include:
- To increase focus on bottom-up technology (lower TRLs) R&D in the Work Programme, since
JUs are already dealing with higher TRLs.
- Few innovative options can be integrated easily into today’s transport systems without major
step changes occurring either within or from outside the sector, highlighting increased effort
towards ‘low tech’ cost effective solutions that can bridge the gap between now and post
2020 transport systems.
- Basic research into desirable futures and transition pathways, social impacts and evaluation
or assessment of the effects of changes (such as emissions, climate change), technological
transformation (e.g. the sharing economy).
- Increase integrated approaches with other sectors and working arrangements with other
DGs such as energy (regarding climate change, for instance), ICT or Nanotechnologies,
Advanced Materials and Advanced Manufacturing and Processing (NMP).
- Request for more clearly defined and specific call topics and descriptions rather than the
current rather broad approach to calls (that can be widely interpreted). This is confirmed
with frustration expressed by stakeholders in the interviews especially in respect to the
evaluations of proposals, as it is felt that experts are not be able to evaluate effectively so
many perspectives.
- More continuity in the research topics (a topic is included only once in the whole plan
period) and also in the terminology used.
- “Emergency” calls in between (larger) programmed calls.
- Stronger emphasis on user perspectives, user education and behaviour, social inclusion,
reduction of inequities, and user outreach and implementation issues (governance, user
behaviour, stakeholder resistance to change, new funding models…) and less bias towards
technology.
- Include disruptive approaches, game changers, and the study of market failures in all modes.
- Reinforcement of international cooperation via bilateral and multilateral funding
mechanisms in support of Europe’s global role and the competitiveness and access to
markets of the European stakeholders (industries, research institutes, etc.)
- Increased clarity on how stakeholder and expert guidance offered to the Commission is
treated in respect to developing calls and responding to new challenges (see above game
changers / new concepts etc.).
Many of the proposals from the stakeholders represent a desire for increased attention to a medium
to long term vision, and are perfectly aligned with EU’s high level objectives. These include
transport’s impact on climate change, improving the energy efficiency and sustainability of transport,
66
Probably a cross-cutting issue with other Horizon 2020 societal challenges
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 48
exploiting innovative technologies, deploying more Intelligent Transport Systems, tackling safety and
security, and more generally fostering the competitiveness of the European transport industry in
global markets etc.
In line with this, the “Scoping Paper” from the TAG proposed the following strategic orientations for
2018-202067.
Table 4: Strategic orientations proposed by the Transport Advisory Group for the future Work
Programme 2018-2020
Technology oriented research and innovation
increasingly complemented with a strong focus
on user needs, societal issues and regulatory
framework.
Accounting for the people: demand, needs and
behaviours; inclusion and access.
Promotion of an innovation strategy aimed to
decarbonisation, digitisation and human-centred
transport system in an increasingly integrated
manner.
Technologies transforming the transport system
Towards an integrated, sustainable and robust
transport system.
Competitiveness, business models and markets.
This is somewhat validated by the project coordinators surveyed as part of this assessment, who
think that the Horizon 2020 Transport programme objectives take into account the latest scientific,
socio-economic, political developments in the field of transport research and innovation at the
national/European and international level68 but also expressed dissatisfaction on how the results
were the taken forward.
The responses to the survey are shown in the table below.
Table 5: Do the Horizon 2020 Transport work programme objectives take into account the latest
scientific, socio-economic, political developments in the field of transport research and innovation at
the national/European and international level? Answers to the project coordinator survey.
0 (not taken into account)
1 2 3 4 5 (fully taken into account)
I don’t know
Scientific 0% 0% 4% 9% 61% 22% 4%
Socio-economic
0% 0% 4% 13% 52% 17% 13%
Political 0% 0% 4% 9% 39% 13% 35%
Results show that the majority agreed (78%) that Horizon 2020 Transport takes the latest socio
economic trends into account from a scientific perspective, however the feedback loop to policy was
less well established or clear to them. Only 50% felt that this was somewhat satisfactory while 34%
67
First Orientations towards the scoping paper for Horizon 2020 Societal Challenge 'Smart, Green and
Integrated Transport'. Internal document of the European Commission that has been made available to the
Expert Group. 68
Survey to project coordinators design and distributed as part of the present assessment
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 49
did not know. The latter is of most interest as it is clearly not seen by those answering the survey as
being of critical importance and their focus of interest is clearly more technology rather than policy
development biased.
Finally, the appreciation of increased regional relevance on Horizon 2020 Transport over FP7 was
also mentioned in the stakeholder hearing (September 2016) but the participants further validated
that there was a not a strong link to policy development especially at regional levels.
As the Transport Work Programme 2014-2015 explains in the case of aviation, the proposed Topics
are in line with the Strategic Research and Innovation Agenda (SRIA) of the Advisory Council for
Aviation Research and Innovation in Europe (ACARE). They are also aligned with the goals of the
Report of the High Level Group on Aviation Research published in 2011 under the title “Flightpath
2050”. In fact, the aviation research and innovation activities within Horizon 2020 Transport are seen
as being perfectly aligned with the aviation strategy for Europe presented by the European
Commission in December 201569.
Another example is the following excerpt from the 2016-2017 Work Programme: “Waterborne topics
reflect an assessment of research needs which takes into account the current priorities of the
stakeholders…”.
From a transport policy perspective and the July 2016 Implementation Assessment of the Transport
White Paper, the current situation is seen as having changed from that of 2011 and new
socioeconomic challenges relevant to Horizon 2020 Transport include:
Demographic and urbanisation trends differing to those predicted in published literature as
guidance for development.
Increased complexity of business and new Circular Economy approaches
Deep changes in supply and value chains with the growing importance of software,
globalisation, ICT and 3D printing.
Fragmented and incomplete framework conditions for “smart transport” in areas such as
standardisation, interoperability and data exchange.
Growth of digitalisation and mobility as a service.
Automation and connected vehicles challenges beyond technology (human factors).
Unexpected interest and technology readiness levels (TRL) in new concepts such as drones.
Strong growth in the collaborative and sharing economy (cloud funding, bottom up solutions
and sharing rather than owning concepts).
Increasing importance of the role of active modes in the urban transport mix (especially in
response to growing concerns on transport related impacts on health).
Growing dissatisfaction of European citizens of contentious negative externalities of
transport (air quality, safety of vulnerable users, noise, congestion, land take etc).
Increasing security threats, including cyber-security.
Unresolved social issues such as the recent migration challenges.
69
Brussels, 7.12.2015.COM(2015) 598 final. An Aviation Strategy for Europe.http://eur-lex.europa.eu/legal-
content/EN/TXT/PDF/?uri=CELEX:52015DC0598&from=EN
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 50
As can be seen, many of these challenges listed in the revision of the Transport White Paper have
also been identified in interviews, by survey responses, informally and at the hearing (September
2016). The list above also reflects directly the proposals for new developments and emerging issues
proposed by stakeholders above (TAG report, Scoping Paper…). The following table shows how the
new challenges identified in the Interim Assessment of the Transport White Paper can be perfectly
tackled within the existing Horizon 2020 Transport specific objectives.
Table 6: Link between the Horizon 2020 Transport specific objectives and the new socioeconomic
challenges identified by the various stakeholders
New socioeconomic challenges identified in the Interim Assessment of the Transport White Paper
Horizon 2020 Transport specific objectives
• Increased complexity of business and new Circular Economy approaches
• Unexpected growth in the collaborative and sharing economy (cloud funding, bottom up solutions and sharing rather than owning concepts)
• Deep changes in supply and value chains with the growing importance of software, globalisation, ICT and 3D printing
• Growth of digitalisation and mobility as a service.
• Unexpected interest and technology readiness levels (TRL) in new concepts such as drones.
To boost competitiveness of the European Transport industries
• Increasing importance of role of active modes in the urban transport mix (especially in response to growing concerns on transport related impacts on health)
• Increasing security threats, including cyber-security.
• Fragmented and incomplete framework conditions for “smart transport” in areas such as standardisation, interoperability and data exchange.
To achieve a European transport system that is resource efficient, climate- and environmentally-friendly
• Demographic and urbanisation trends differing to those predicted in published literature as guidance for development.
• Automation and connected vehicles challenges beyond technology (human factors).
• Growing dissatisfaction of European citizens of contentious negative externalities of transport (air quality, safety of vulnerable users, noise, congestion, land take etc).
• Unresolved social issues such as the recent migration challenges.
To achieve a European transport system that is safe and seamless for the benefit of all citizens, the economy and society
The purpose of this table is to show that all the previous emerging issues can be accommodated in the existing
specific Transport objectives. An analysis of how they are being tacked in the Work Programmes or what is the
actual room for action from the Horizon 2020 Transport challenge would require a deeper analysis.
One important concluding remark at this point is that Horizon 2020 Transport objectives are quite
clearly fully capable of including all the above new trends and issues and are still fully relevant. In
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 51
other words, that Horizon 2020 Transport objectives do not need to be adapted to the changed
scenario and challenges but, instead, that new and emerging issues can be embraced by existing
objectives
4.2.1 SUGGESTIONS FOR IMPROVEMENTS FROM A STAKEHOLDER PERSPECTIVE
Horizon 2020 Transport has sound procedures in place for stakeholder consultation and flexibility of
the Work Programmes is shown, for instance, by the introduction of a large specific call on
Automated Road Transport in WP 2016-2017 with €114 million budget including full demonstration
activities. However, a number of participants in the interviews and the hearing mentioned the need
for added flexibility and quicker processes to analyse the relevance and effects of new, quickly
emerging needs (for example: the consequences of autonomous driving). At present, new and
“urgent” topics sometimes need to undergo a long process until they find their way into the text of
the calls. This is considered by the Expert Group to diminish some aspects of relevance.
From the survey, comments offered by project coordinators included a specific concern that there
was some inflexibility especially in respect to the technology based EU Road Maps. The speed of
innovation is traditionally difficult to anticipate over any length of time70. Few people are able to
correctly estimate the take up and success of these disruptive and counter-intuitive technologies and
business models. The recommendation in this case may be to insist on swift mechanisms for annual
updates of those technology maps.
A minority of stakeholders. mainly from technology platforms, also voiced their opinion on a lack of
transparency or information in connection with the preparation of the work programmes71. In
particular they were suggesting more information on how the final process of prioritizing proposed
topics in the Work Programme is implemented (in other words, what is included and what not). This
should be taken into consideration as a different issue to the generally well-accepted and
appreciated process of collecting information from the stakeholders and the consultation process.
These remarks are in connection with how the individual and collective inputs to this process are
treated which is unclear to stakeholders currently. The Expert Group suggests that a solution could
be found with increased efforts in communicating on how work programmes are prepared and how
inputs are treated (as well as more clarity on how actors (new and non-traditional) can get involved
into the process to ensure that new ideas are brought to the table). This increased effort could come
from the EC itself (for instance with a short section on the Work Programmes explaining what inputs
where received and how they were treated generally) and from the National Contact Points (NCPs)
when presenting the work programmes to the national constituencies.
An important issue regarding the involvement of stakeholders that recurrently arose during the
interviews for this interim assessment is the limited involvement of representatives from the softer
transport modes. It is recognised that this may partially be due to the fact that they are not
constituted in well-defined groups, as the majority of other more traditional transport modes are.
Stakeholders such as civil society organisations that represent citizens at large, pedestrians,
passengers of all transport modes, union representatives were mentioned.
70
Note the explosive success of AirBnB or Uber over the past five years 71
Impacting both relevance and coherence
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 52
As part of its cross-cutting KPI reporting, the European Commission monitors the percentage of
Responsible Research and Innovation (RRI) projects where citizens, Civil Society Organisations (CSOs)
and other societal actors are involved72. While for the entire Horizon 2020 the share of signed grants
flagged as RRI-relevant (all projects) is 6.9%, in the case of transport the share is only of 3.6%. This is
considered to be low and should be strengthened.
It is also felt by the Expert Group, based on the analysis of the work programmes and comments
from interviews and the hearings, that this has led to a dominance of some aspects of the road maps
from the larger actors (industries) rather than looking for the best solution to address the challenges
facing Europe and that Horizon 2020 Transport is designed to address.
4.3 OTHER ISSUES RELEVANT TO RELEVANCE
Overall the increased attention in Horizon 2020 and to the transport component in relation to
strengthening close-to-market activities is considered to be an important and positive change,
increasing the potential for European research to be cutting edge and world beating.
A preliminary analysis of the 2014-2015 and 2016-2017 Transport Work Programmes concludes that
particular reference to pilots, demonstrations are included in topics that represent a 17.5% of the
budget of the calls (a budget of €317.8 million out of a total of €1819.9 million for the first two
Transport calls). In this regard, putting more emphasis during the evaluation of proposals on the path
to market of innovations (as it is already the case in the SME Instrument) might provide a clear
message to innovators. This higher emphasis could have two dimensions: specific briefing to the
evaluators insisting on the importance of close-to-market project activities or even changing the
evaluation criteria in order to give larger weight to the impact criterion.
Results from the survey show that few patents have materialised directly from the last batch of FP7
projects but the number of prototypes and major demonstration projects is much more promising.
The fundamental role the Commission plays in funding major demonstration projects was confirmed
in the Stakeholder hearing (September 2016).
But, as the Commission presents in its Horizon 2020 Monitoring Report 2014, bridging from discovery
to market application is also achieved through innovation actions73. In this respect, the report
indicates that:
“Overall, 202 innovation action projects have been signed in 2014, with a requested EC
contribution of €1071 million. This represents 4.20% of the total number of successful projects
signed related to calls closed in 2014 (4809) and 12.65% of the total EC contribution allocated
to these successful projects (€8467.83 million)”.
In the case of transport, and based on the analysis of the budget per topic conducted for this report,
the budget allocated to Innovation Actions in the 2014-2015 and 2016-2017 Work Programmes
reaches the 25.9%, much higher than the indicated overall 12.65%.
72
Data provided by the EC for the purposes of this interim assessment. Date of data extraction: 01/09/2016. 73
http://ec.europa.eu/newsroom/Horizon2020/document.cfm?doc_id=15108
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 53
New tools such as the Common Exploitation Booster74, supported through the Horizon 2020
Common Support Centre, and obviously the centre in itself, are valuable tools for project participants
in their path towards commercialization.
Particular attention is deserved here on the reflection of the balance between incremental research
and radical innovations and integrating them into policy (see also previous point on timeliness and
relevance). As this sentence from the Work Programme 2014-2015 summarizes, “crafting the right
rail innovation strategy will require a step well beyond just technology”. It is very often mentioned
that incremental research will not be enough to respond to the difficult challenges ahead and that
breakthrough innovations are indeed needed regardless of the transport mode at stake. The
inflexibility of the programme is brought into question here.
Aviation research, for instance, includes a small-scope topic on radical or ‘Breakthrough innovation
for European aviation’ which is intended to make proof of concepts and technologies in the TRL
range 1-2. Obviously longer-term research has a privileged place in the line Socio-economic Research
and Forward Looking Activities of the Transport call, but short-term interests should not be side lined
or marginalised.
On the other hand, high-risk research of a more interdisciplinary nature is addressed in the Future
and Emerging Technologies (FET) under the Excellent Science part of Horizon 2020. Despite this, the
links between FET and transport calls are seldom explained and, more importantly, utilized.
It is interesting to note that some aspects of the international trend in transport research and
innovation can be better defined as “radical innovations for the short to medium term”: Tesla over-
the-air software upgrades of its electric vehicles or new concepts such as Hyperloop75, Space
Tourism76, Uber and the use of drones for parcel delivery by Amazon or the Chinese Transit Elevated
Bus (TEB)77. It gives sometimes the impression that Europe is still deciding what the barriers for
innovations are, while real large-scale demonstrations are already taking place in other parts of the
world. A clearer view should be established when it comes to differentiate between research and
deployment: it is clear from the examples above that demonstrating radical innovations take less
time than deploying them, so it might be worthwhile supporting the former.
4.4 LESSONS LEARNT AND AREAS OF IMPROVEMENT
Generally Horizon 2020 Transport is seen as being enormously relevant for EU environmental
sustainability, society at large, competitiveness and boosting the European economy. From the
transport policy point of view, the transport actions under Horizon 2020 were largely based on the
Transport White Paper and as long as it is still pursued and relevant, so is Horizon 2020 transport. It
retains, and may increase, in relevance as additional objectives arise in fields such as security.
74
http://exploitation.meta-group.com/Pagine/About-Us.aspx 75
https://hyperloop-one.com 76
www.virgingalactic.com 77
https://en.wikipedia.org/wiki/Transit_Elevated_Bus
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 54
Its goals are clear for stakeholders: namely to improve competitiveness of the European transport
industry and to provide European citizens the transport system of the future that is green, efficient
and safe.
Horizon 2020 and Horizon 2020 transport has undergone reform when compared to FP7 (or other
previous Framework programmes) in terms of impact and implementation orientation, including
reorienting focus on innovation, demonstration and the introduction of new financial instruments.
The programme has also been highly relevant during the current economic crisis, not only in terms of
funding for research but it has also served to better identify priorities and to focus available
resources at national level. For example it was timely, in particular, for the automotive sector and
helped avoiding job losses in the field of research. A clear example of this alignment of priorities can
be found below in connection with Clean Sky, but it was also specifically mentioned by two of the
interviewees from the Transport Programme Committee. In one case, it was metioned that even if
there were not a national research programme specifically dealing with transport issues, the Horizon
2020 Transport priorities had been included in the general research programme.
Relevance of the work programmes is reassured by a sound work programme preparation and in the
call topics: such as the example of the Automated Road Transport (ART) call in Work Programme
2016-201778. Relevance can also include a national or regional dimension (several of the stakeholders
interviewed referred to “national relevance” of Horizon 2020 Transport) and, in this regard, the links
of Horizon 2020 Transport with national and other regional research programmes also helps to
underpin its relevance. The example from the Aviation Sector on signing Memorandum of
Understandings are thought to help underpin the potential of regional and European cooperation as
they bring together synergies between Horizon 2020 Transport and the regions’ interests. The
regions participate in defining Clean Sky calls and JUs give input to regional programmes to avoid
duplication and to perform complementary work. It is seen as being highly strategic and CS has
signed 11 MoUs and other are in the pipeline.
New research concepts such as “Open innovation” must be better explained and need to be better
understood by transport research participants. Openness seems to sometime conflict with
competitiveness and particular caution should be given, for instance, when transferring IPRs outside
the EU79. The transport sector is a very competitive sector, both inside Europe and outside (for
instance, new Asian companies entering the rail /road sectors) therefore too much openness is not
considered to universally wise in this sector. The concept of Open Science also collides frequently
with patents, the latter being a KPI for Horizon 2020. To some extent, those participating in Horizon
2020 transport sometimes question the relevance of investing in a programme that is too open.
The Expert Group found, based on interviews and hearings, that it is difficult to establish clear links
between high-level policy objectives and particular project objectives. In this regard, it is expected
that the future Strategic Transport Research and Innovation Agenda (STRIA) should fill this gap by
producing roadmaps and looking more precisely to what can be performed in research to achieve the
78
This “thematic” call included both an ICT Infrastructure topic directly introduced by the EC as a necessary
cornerstone of the ART system and also a specific topic on safety and end-user acceptance. 79
Art. 30.3 of the Model Grant Agreement already obliges beneficiaries in certain cases to notify the EC about
an intended IPR transfer outside the EU and gives the Commission the right to object such a transfer
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 55
overarching objectives. STRIA is considered as a key component designed to streamline the research
and innovation efforts and focus them on the most pressing challenges and the most beneficial
actions for transport.
Overall the transport research agenda of Horizon 2020 Transport enormously and clearly is relevant
to addressing the Societal Challenges outlined in SC4. It is seen as being fully relevant for achieving
key European goals such as competitiveness, environmental sustainability and social liveability.
However several areas of improvement have been identified in this interim evaluation.
One important remark at this point is that Horizon 2020 Transport objectives are quite clearly fully
capable of including all the above new trends and issues. In other words, that Horizon 2020
Transport objectives do not need to be adapted to the changed scenario and challenges but, instead,
that new and emerging issues can be embraced by existing objectives.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 56
5 EFFECTIVENESS Effectiveness has been interpreted as measurable impacts in relation to the overarching objectives of
Horizon 2020 and more specifically to programme and project objectives. The Expert Group also
included the wider levels of effectiveness in terms of communication, dissemination of results (also
covered in greater length in the chapter on EU Added Value) and the take up of tools and
recommendations beyond the boundaries of the project especially by those not directly involved
with projects. Ultimately it would have been ideal to associate these results with the predetermined
objectives such as competitiveness, job creation and societal impact but this went beyond the scope
of the Terms of Reference for this group.
5.1 SHORT-TERM OUTPUTS FROM THE PROGRAMME AND EARLY
SUCCESS STORIES
The number of deliverables from FP7 and Horizon 2020 is impressive but extracting relevant and
useful knowledge from them is not easy. This aspect of knowledge ‘draining’ was mentioned in many
discussions and interviews with respondents involved in this evaluation process.
What happens after the end of any project is out of control of the programme, although in the Expert
Group’s opinion (and this point was also validated in several of the interviews) this might be one of
the most important outcomes of working towards achieving the goals. The impression from some
project participants80 is that the results are “put on the shelf”, and lack any practical or substantive
analysis of results or innovation, despite the original project goal to exploit the results. It is also
feared that the broad scope of the topics as presented in Horizon 2020 might lead to an increase of
such ‘valueless’ projects. In the paragraphs below, some topics were highlighted.
5.1.1 SUCCESS STORIES
As Horizon 2020 has just started and few projects have been finalised, examples of promising
projects for the Expert Group to investigate were few and far between. Many FP7 projects are now
finished, providing good examples of projects delivering results, and there is no evidence to date that
suggests outputs from Horizon 2020 will be any different (as the conditions are similar) from those of
FP7. Good examples can be found in the areas of the Green Vehicle Initiative and the urban mobility
calls (some are documented as in ERTRAC From Collaborative Research Projects to Market
Deployment with 12 success stories). Other programmes such as the CIVITAS programme is also
considered a good example (see later in Chapter on European Added Value). Such examples have
helped Europe to become a world reference in urban mobility (e.g. Sustainable Urban Mobility Plans
- SUMPs) and related ICT applications and provide a boost to exporting technology and knowledge to
third countries.
Other examples of commercialisation of results have also been found:
- TrafficFlow81 started with initial funding made available by the Regional Government of
Tuscany in the framework of the EraSME project ORUSSI (Era-Net - FP7)
80
In the survey and the interviews. 81
http://cordis.europa.eu/project/rcn/200334_en.html
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 57
- CELSO82 on Automated Vehicle Monitoring systems
- Horizon 2020: Magenta Lab83,
More recently appreciation was expressed for the funding of the SETRIS Project, which brings
together five Transport European Technology Platforms (ETPs) – road, rail, air, water and logistics –
and a balanced representation of all transport modes with a variety of industrial/ commercial
players. Their aim is to deliver a cohesive and coordinated approach to research and innovation
strategies for all transport modes in Europe especially in line with the White Paper “Roadmap to a
Single European Transport Area”, and it is seen that this will help fill gaps and smooth any overlaps.
5.1.2 TRL LEVELS
The Horizon 2020 programme is set up to bring research, with higher technology readiness levels
(TRL84) closer to market making the transition towards deployment and/or further research more
likely than in FP7.
This is confirmed by the project coordinators, as can be seen in the figure below showing the TRL
levels of a sample set of FP7 and Horizon 2020 projects. It is clearly visible that Horizon 2020 aims at
a higher TRL levels than FP7 and this is also confirmed by the approach of PPPs to research.
Figure 4: Technological Readiness Level (TRL). Source: Survey among project coordinators, September
2016, n=21 for FP7, n=15 for Horizon 2020.
The higher TRL levels require a different approach in Horizon 2020 compared to FP7. There is a need
to improve the transition to deployment beyond traditional dissemination/exploitation activities and
this is seen by all stakeholders as one of the big challenges in Horizon 2020. Horizon 2020-funded
research should not only project transport futures, but also seek to influence them in various ways.
More effort should be put in market readiness through diffusion and involving targeted clients.
A stronger emphasis on the road to market, including combination of other funding sources outside
Horizon 2020 during the evaluation phase would send a clear message about the possibilities for
these external opportunities. In addition a more detailed section in the work programme for
additional funding opportunities and synergies, or something similar, would be helpful in this
82
http://cordis.europa.eu/project/rcn/204322_it.html 83
www.magentalab.it 84
https://ec.europa.eu/research/participants/data/ref/Horizon 2020/wp/2014_2015/annexes/Horizon 2020-
wp1415-annex-g-trl_en.pdf
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 58
respect, as currently those involved in projects have little motivation to go searching for more
funding once a project is finished.
The converse of this is that a number of concerns were raised in respect to the focus on the higher
end TRLs and that this was stifling, rather than engendering, innovation. Cost effective solutions in
early stages of TRL readiness were seen as being desirable additions to the overall work programme
especially in respect to the inherent inertia in the transport sector for change and the slow progress
seen in many areas, especially in decarbonisation and alternative fuels. This seemed to be most
prevalent in the complex sectors of road and rail.
5.1.3 HUMAN CAPITAL DEVELOPMENT
The human capital development is expected to be similar for Horizon 2020 as for FP7. This can be
derived from a survey among project coordinators. The two graphs below show that the inputs, in
terms of numbers of researchers involved (both male and female) are expected to be slightly lower
while the outputs in terms of impact on mobility and gender equality are expected to be slightly
higher.
Figure 5: Average number of human resource related activities targeted or achieved at the end of the
project. Source: Survey among project coordinators, September 2016, n=30..32 for FP7, n=20.22 for
Horizon 2020.
Figure 6: Assessment of human capital impact targeted or achieved at the end of the project. Source:
Survey among project coordinators, September 2016, n=26..29 for FP7, n=18..19 for Horizon 2020.
An important point was brought up specifically by some participants in the stakeholder hearing,
notably from the research community, that there was little room for failure. It is not suggested that
poor projects should be supported – but the knowledge and human capital development value of a
project that has encountered major unexpected barriers is not valued under the current system. It is
thought that the experience of not being fully successful is thoroughly undervalued and it should find
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 59
a place in the system – after all Edison made 1,000 unsuccessful attempts at inventing the light bulb
and in his opinion ‘the light bulb was an invention with 1000 steps’.
5.1.4 KNOWLEDGE TRANSFER
The number of knowledge transfer activities is expected to be lower for Horizon 2020 than for FP7.
This can be derived from a survey among project coordinators, which is shown in the graph below. A
possible explanation could be the focus of Horizon 2020 on bringing the transport research closer to
the market. This has an influence on the areas of knowledge transfer as conferences and
publications, which are rather academically oriented.
Figure 7: Average number of knowledge transfer related activities targeted or achieved at the end of
the project. Source: Survey among project coordinators, September 2016, n=31..32 for FP7, n=21.22
for Horizon 2020.
5.1.5 BUILDING RESEARCH, DEVELOPMENT AND INNOVATION CAPABILITIES
The impact on research, development and innovation capabilities is expected to be similar for
Horizon 2020 compared to FP7. This can be derived from a survey among project coordinators,
which is shown in the graph below.
Figure 8: Assessment of impact on research, development and innovation capabilities targeted or
achieved at the end of the project. Source: Survey among project coordinators, September 2016,
n=31..32 for FP7, n=19..21 for Horizon 2020.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 60
5.1.6 COMPETITIVENESS & EXPLOITATION
Almost all projects have undertaken exploitation activities, or are planning to do so. The graph
below shows that direct contact with stakeholder is the most popular activity, both in FP7 and
Horizon 2020.
In Horizon 2020 project coordinators plan to undertake less activities overall to help exploit the
results than in FP7, as can be seen in the graph below. The reasons are unclear, as one would expect
a higher number given the focus on more market-oriented research, and the Expert Group found this
a surprising result.
Figure 9: Average number of exploitation activities targeted or achieved at the end of the project.
Source: Survey among project coordinators, September 2016, n=31 for FP7, n=20 for Horizon 2020.
Despite the slightly lower numbers on activities (inputs), project coordinators expect a slightly higher
impact on competitiveness in Horizon 2020 than in FP7, as can be seen in the graph below.
Figure 10: Assessment of impact on competitiveness targeted or achieved at the end of the project.
Source: Survey among project coordinators, September 2016, n=30 for FP7, n=20..21 for Horizon
2020.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 61
5.2 EXPECTED LONGER-TERM RESULTS FROM THE PROGRAMME
This section explains the expected long-term results based on the outputs presented in the previous
section. One of the main advantages of Horizon 2020 is that it is more focused on implementation of
research results than FP7. It is expected from this focus that more research will be put into the
market. The emphasis upstream (project preparation) is clearly on exploitation and deployment. This
should lead to outputs that matter and avoid the production of ‘vague’ scientific reports that no one
reads. Although FP7 put down the roots for later development, it has not yet yielded concrete
deliverables. Horizon 2020 is seen to be giving a clear push in stimulating innovation.
However in the responses from the stakeholders there are some concerns that the ambition for this
to be improved is clear but that the conditions for implementation may be lacking. Notably:
- A focus on project management rather than content from INEA (i.e. are the deliverables
submitted on time, do they follow the due process and are they in line with the contractual
arrangements?).
- There are many good tools and outputs from FP7 which have not been fully exploited but are
difficult to extract in the present system.
- Experienced participants sense a feeling of ‘déjà vu’ when some calls are published and feel
that these topics have already been successfully researched in previous FPs but the results
have been buried, so that newer staff members of the Commission are not aware of them
(and therefore it is unlikely that anyone else is either).
- The Horizon 2020 format is perceived as too rigid, too fragmented and too short term oriented for innovative research and development, going beyond the classical R&D.
Nonetheless, Horizon 2020 is contributing to strategic cooperation in research going beyond FP7. The
change of mind-set of the participants is perhaps one of the main achievements, by illustrating that
cooperation has added value. Private partners are brought together, and a market place is created
for tools and products that are useful for the European transport scene.
As few projects are finalised it is still unclear to what extent Horizon 2020 will solve big societal
challenges in transport such as air quality, congestion, or safety/security. Despite this at least project
coordinators are more optimistic about Horizon 2020 than on FP7 as can be seen in the results from
a survey in the graph below.
Figure 11: Percentage of projects that expect to contribute to one of the long term goals. Source:
Survey among project coordinators, September 2016, n=32 for FP7, n=22 for Horizon 2020.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 62
5.3 PROGRESS TOWARDS ATTAINING THE SPECIFIC OBJECTIVES OF SC4 -
SMART, GREEN AND INTEGRATED TRANSPORTS
The specific objective of the SC4 Smart, green and integrated transport is formulated as follows:
“Achieving a European transport system that is resource-efficient, climate- and environmentally-
friendly, safe and seamless for the benefit of all citizens, the economy and society.”
The Specific Programme is structured in four broad lines of activities aiming at:
A. Resource efficient transport that respects the environment. The aim is to minimise transport
systems’ impact on climate and the environment (including noise and air pollution) by
improving its efficiency in the use of natural resources, and by reducing its dependence on
fossil fuels.
B. Better mobility, less congestion, more safety and security. The aim is to reconcile the
growing mobility needs with improved transport fluidity, through innovative solutions for
seamless, inclusive, affordable, safe, secure and robust transport systems.
C. Global leadership for the European transport industry. The aim is to reinforce the
competitiveness and performance of European transport manufacturing industries and
related services including logistic processes and to retain areas of European leadership (e.g.
such as aeronautics).
D. Socio-economic and behavioural research and forward looking activities for policy making.
The aim is to support improved policy making which is necessary to promote innovation and
meet the challenges raised by transport and the societal needs related to it.
These activities are addressed in the 2014-201585 Work Programme by three calls for proposals:
1. Mobility for Growth
2. Green Vehicles
3. Small Business Innovation for Transport
Looking to evaluate if the objectives are translated effectively into the calls, a certain technology bias
and imbalance in weighting against the socio-economic aspects of SC4’s objectives can be observed.
The table below shows that with respect to the translation of the objectives into the number of
topics (totalling 53), around half of them (28 topics) mention an expected impact that can be related
to “Resource efficient transport that respects the environment”. The other 3 objectives (or parts of
it) are less frequently mentioned, with “Socio-economic and behavioural research and forward
looking activities for policy making” as the lowest. This was validated in several interviews, (and was
a point picked up in the ECTRI position paper as input to the new WP sent to the Commission June
2016).
The same goes for the translation of the objectives into projects (outputs). 73 of the 128 projects are
in a topic that mentions the environmental issues while only 29 are in a topic that mentions an
expected impact in transport policies. This is also confirmed in interviews where several
interviewees, especially those near the policy making process have pointed out that the connection
with EC priorities in terms of policy implementation is lagging behind in Horizon 2020. Although this 85
The 2016-2017 Work Programme could not be analysed, as the calls are still open. No results are known to
date in terms of number of projects or budgets.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 63
may not part of the Horizon 2020 programme, the process should place more emphasis on the
feedback loops to policy. This is also visible on the participant’s side: policy oriented topics are largely
oversubscribed in numbers of proposals (see further under the chapter on efficiency).
From this assessment of both the Mobility for Growth and Green Vehicles work programmes it can
be concluded that not all of the above objectives are translated evenly in the topics nor the projects.
Table 7: Number of projects in which a specific objective is mentioned as an expected impact, MG and
GV calls 2014 and 2015.
Objective Topics Projects
A Resource efficient transport that respects the environment 28 73 A1 Resource efficient 10 33
A2 Climate friendly 19 48
A3 Environmentally friendly (including noise and air pollution) 18 38
B Better mobility, less congestion, more safety and security 19 49 B1 Better mobility 11 29
B2 Less congestion 7 24
B3 More safety and security 7 19
C Global leadership for the European transport industry 22 53
D Socio-economic and behavioural research - improved policy making 16 29
TOTAL number of projects / topics
(projects or topics can be mentioned under more than 1 objective)
53 128
When distributing the budget (total grants amounting to €770 million for 128 projects distributed
across the four objectives), an even stronger emphasis on the Objective A is visible: 48.4% of the
budget goes to “Resource efficient transport that respects the environment” while only 5.9% can be
allocated to “Improved policy making”.
Budgets are split per topic into the four objectives with percentages based on the emphasis given on
each of the objectives in the topic expected results. This gives a weighted division of the budget.
Figure 12: Division of budgets by specific objectives, MG and GV calls 2014 and 2015.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 64
5.4 PROGRESS TOWARDS THE OVERALL HORIZON 2020 OBJECTIVES
The overall objective of Horizon 2020 is “Contribute to building a society and an economy based on
knowledge and innovation across the Union by leveraging additional R&D&I funding and by
contributing to attaining R&D targets, including the target of 3 % of GDP for R&D across the Union by
2020. It shall thereby support the implementation of the Europe 2020 strategy and other Union
policies, as well as the achievement and functioning of the European Research Area (ERA).”
All stakeholders and participants of the programme agree that Horizon 2020 is a unique programme
and its outcomes would have been very difficult to obtain should it not have existed.
Main expected outcomes that are mentioned in the interviews and surveys are:
• Europe-wide collaboration that could not have happened without the programme. FP7 and
Horizon 2020 have been very effective in bringing researchers and research institutes
together and sharing knowledge. In some countries it might be easier to get national funds
but the added value of the programme is to forge links with other stakeholders that would
otherwise not be possible. This is considered to be of high value.
• European cooperation leads to European rather than national access to markets, and in some
cases global.
• Results would be very difficult to obtain should the programme not exist.
5.5 LESSONS LEARNT/ AREAS FOR IMPROVEMENT
There are numerous lessons but the main lessons learnt are:
- Short-term output activities are seen as being equal or slightly lower than FP7 in terms of
KPIs86.
- Higher TRL levels are now well covered but gaps in research interests of lower levels of TRLs
should now be given more focus.
- Long term objectives of socio-economic and behavioural research and forward looking
activities for policy-making is underfunded and not given enough exposure.
86
The Key Performance Indicators which are particularly relevant for the Societal Challenges are:
• Number of publications in peer-reviewed high impact journals
• Number of patent applications and patents awarded
• Number of prototypes and testing activities
• Number of joint public-private publications
• New products, processes, and methods launched into the market
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 65
6 EFFICIENCY
6.1 INTRODUCTION
Efficiency considers the relationship between the resources used by an intervention and the changes
generated by the intervention (which may be positive or negative).
The main question considered was to establish if the effects (benefits) were achieved at a reasonable
cost.
There have been several improvements to the Horizon 2020 process compared to FP7 and the
following have been investigated as part of this interim evaluation:
- The programme’s budgets, area balance, project size and topic description
- The programme’s attractiveness to the right participants
- The processes for the evaluation of proposals and the project management
6.2 OVERVIEW OF FUNDING ALLOCATIONS
This section describes the overall budget, and the split per activity area, instrument (funding scheme)
and topic.
6.2.1 BUDGET OF THE OVERALL PROGRAMME
Up to now (September 2016), the Horizon 2020 SC4 programme has resulted in 512 projects with a
total grant of €1400 million. This includes the MG (Mobility for Growth) and GV (Green Vehicles)
calls, the SME Transport calls, as well as the calls from the JUs (SESAR, Clean Sky and Shift2Rail).
The challenge SC4 in Horizon 2020 (including the JUs) is allocated a budget of €6339 million for the
full period 2014-2020.
Compared with the FP7 budget of €4160 million over the full 7 years (2007-2013), this is a significant
increase, though the two programmes are not fully comparable as some areas have been removed or
added.
In general terms, the majority of those responding to questions87 recognised that the overall budget
is large and that there are no other European or International research programmes that are better
funded than Horizon 2020. At the same time there is a clear wish for additional budget.
The high number of applications for funding transport project shows there is a not only a need but
also a clear interest in this sector. To some extent this is unsatisfactory for those applying for funds
as despite investing time and effort in developing good proposals that may achieve a score of 14/15
points in project evaluation, they still do not get funded. This leads to disappointment and
frustration. It is recommended to find a solution how to improve dealing with high scoring projects in
the next framework programme (see further under 6.3 – Programme’s attractiveness).
87
In the survey, during the interviews and during the stakeholder hearing.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 66
6.2.2 BUDGET PER ACTIVITY AREA
Although the total budget may be considered adequate, and the general feeling is positive, a few
remarks were given during the interviews and in the survey by representatives of certain modes:
- For some areas, the Horizon 2020 budget is the same as in FP7, however elsewhere
in the RTD programmes budgets have been cancelled. For example, this is the case
for urban transport, as the STEER programme has been discontinued, meaning a
decrease in budget overall for urban topics.
- For some thematic areas, the budget is similar or even larger than in FP7, but the
share compared to the other areas has decreased.
- There is a more limited number of multimodal or crossmodal topics which is not in
line with the ambition of integration and new cross-cutting ways of working within
the Commission (a Juncker request). This is further expanded in relation to
objectives in the chapter on effectiveness.
From the data gathered, it can be observed that aviation is allocated the largest share of the budget
(34.1%). Although it received more under FP7 (23.4 Aviation + 18.7 Clean Sky + 9.3% SESAR), it
should be noted that Galileo (11.9%) was included in Transport, and the administration was also
added.
Figure 13: Budget share by activity area, including JUs.
Horizon 2020 project grants up to August
2016 (€1400 million)
Source: CORDA
FP7 Transport (including Aeronautics) budget
breakdown by activities (€4160 million)
Source: Transport Research in the 7th Framework
Programme (2007 – 2013): an overview of programme
implementation statistics
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 67
Figure 14: Grants for Horizon 2020 projects up to August 2016, per activity area, including JUs.
Source: CORDA
This also shows up in the number of projects below:
Figure 15: Number of Horizon 2020 projects up to August 2016, per activity area, including JUs.
Source: CORDA
6.2.3 BALANCE BETWEEN INSTRUMENTS AND FUNDING SCHEMES
The balance between RIA, IA, CSA and SME is generally considered good.
Some related remarks expressed by a few of the respondents during the interviews:
- Shared budgets between topics could lead to a topic having zero proposals funded
simply because the scoring of different topics is grouped and ranked into one
budget.
- The system does not encourage enough disruptive research, and favours incremental
research. This has too small a share in the programme and the impact of initiatives
like the European Institute of Innovation and Technology does not begin to compare
with output of universities like Massachusetts Institute of Technology (MIT). It is
considered that this sets European research at a disadvantage globally.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 68
Figure 16: Number of Horizon 2020 projects up to August 2016, per funding scheme and activity area,
including JUs. Source: CORDA
Figure 17: Grants for Horizon 2020 projects up to August 2016, per funding scheme and activity area,
including JUs. Source: CORDA
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 69
6.2.4 DESCRIPTION OF TOPICS
In general the Horizon 2020 topics are defined in a much broader way than in FP7. Participants in
the interviews, the survey and the stakeholder hearing have mixed views on this but there is
generally a negative bias. It is accepted that, on the positive side, wide topics favour the participation
of many applicants and allow multiple interpretations. Less prescriptive topics open the door to any
idea, which is can lead to innovations. But it is not confirmed that this actually happens. Concerns
about the evaluation of proposals are also expressed by most participants with respect to the broad
interpretation of topics.
In a survey project coordinators consider the less prescriptive topics having a minor positive impact
on the effectiveness of Horizon 2020, although the responses are mixed.
Figure 18: Assessment of the less prescriptive topics on the effectiveness of Horizon 2020 compared to
FP7 from 33 coordinators that participated both in FP7 and Horizon 2020.
Source: Survey among project coordinators, September 2016.
However, some participants in the survey and most interviewees consider this in a negative light for
the following expressed reasons:
- Topics could be more specific, with the requirements being a little bit more prescriptive and
more strategic. This has been quite noticeable compared to FP7 calls and people struggle
sometimes to understand what the Commission may be looking for – again leading to wasted
effort in project preparation. Applicants have the feeling to promise to rescue the world in
order to get their project granted.
- Although broader topics lead to a wide spectrum of proposals, it seems to be that this leads
to proposals that are too big and vague to assess the specific needs of the sectors (as
previously mentioned).
- Frustrations arise because of rejected proposals, as the evaluators do not seem to have been
able to properly assess the benefits of a project. This is a reflection on the need for a strict
number of evaluators and due to the broad and varied interpretation it is unlikely that the
selected experts can fulfil all these roles.
- Difficulty in the selection process. It is difficult to evaluate proposals that may be very
different but still responding to the same topic, thus giving room for subjectivity of the
evaluators. In addition, actually finding/sourcing experts who are able to evaluate a broad
number of interpretations of one topic is seen as being very demanding (reflected in
previous comment).
- Broader topics with less information lead to oversubscription, the success rate decreases.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 70
6.2.5 PROJECTS HAVE EQUAL SIZE
The average size of projects up till now was €4.23 million for the combined IA and RIA projects, which
is slightly lower compared to €4.47 million for the collaborative projects FP7. This is somewhat
contradictory to the more broad topic definition and the expressions of the participants in the
section above.
For CSA this is €1.80 million in Horizon 2020 compared to €1.12 million for FP7, which indicates an
increase.
Details can be found in the figures below.
Figure 19: Average funding per project by funding scheme in Horizon 2020 up to August 2016, in €,
including JUs. Source: CORDA
Figure 20: Average funding per project by activity area and funding scheme in Horizon 2020 up to
August 2016, in €, including JUs. Source: CORDA
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 71
Figure 21: Average funding per project by PPP and funding scheme in Horizon 2020 up to August
2016, in €. Source: CORDA
Figure 22: Average funding per project by research area and funding scheme, in million €. Source:
Transport Research in the 7th Framework Programme (2007 – 2013): an overview of programme
implementation statistics
6.2.6 ADDITIONAL REMARKS ON TOPIC DEFINITION
In the survey, the clarity of the goals and topics is described as ‘good’ (mark 3 out of 5), for both
Horizon 2020 as FP7 project coordinators. Respondents that participated in both programmes do
not see much difference between them, except for budget adequacy, where Horizon 2020 is
considered slightly worse.
From this In general, the quality/choice of the topics is good, and it is felt by participants that the Info
Days help to clarify detailed remarks and impressions.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 72
Figure 23: Top graph: average rating (0 very poor .. 5 excellent) according to coordinators of 32 FP7
and 21 Horizon 2020 projects. Bottom graph: assessment of Horizon 2020 compared to FP7 from 33
coordinators that participated both in FP7 and Horizon 2020. Source: Survey among project
coordinators, September 2016.
Also, project coordinators consider the multiannual Work Programme having a large positive impact
on the effectiveness of Horizon 2020. It is appreciated that the topics are known ahead, though the
other side of this is that proposal preparation now starts much earlier, leading to higher costs.
Considering this, the helpful Info Days come too late in the process.
Figure 24: Assessment of the Multiannual Work Programmes on the effectiveness of Horizon 2020
compared to FP7 from 33 coordinators that participated both in FP7 and Horizon 2020. Source:
Survey among project coordinators, September 2016.
Though the general feeling is positive, participants made the following remarks during the interviews
and in the survey:
- Topics differ too much in style and type, from very specific to very general. Continuity of
terminology would be helpful.
- There is a certain amount of duplication in the programme.
- No room for urgent topics, nor sufficient flexibility allowed for new developments. It
takes up to 2 years for many projects to go from publication of the work programme to
signed contract, which is considered very long (although it is recognised that time of
project approval to signed contract has been shortened). This makes it difficult to have a
‘close to market’ approach. To obtain good results within a technical research, a project
must be launch in a very rapid manner: in some cases the technical ideas grow old very
quickly. The two-step process is not helping to shorten this.
- Fragmentation of topics and lack of multimodal interest. Compared to FP7 it is
considered that the modal silo approach has become stronger and not weaker in Horizon
2020.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 73
6.3 PROGRAMME'S ATTRACTIVENESS
This section describes the attractiveness of the programme to the participants to the calls in terms
of:
- Participation of the right participants
- Oversubscription on some topics
- Adequacy of funding rules
- The cost of submitting a proposal
- Two-stage approach
6.3.1 ANALYSIS OF THE PARTICIPANTS
Both Horizon 2020 and FP7 have been very effective in bringing researchers and research institutes
together and sharing knowledge and this is widely accepted by the majority of the participants and
stakeholders (validated in the interviews and in the chapter in effectiveness).
Figure 25: Success rate by organisation type in FP7. Source: Transport Research in the 7th Framework
Programme (2007 – 2013): an overview of programme implementation statistics, tables 12 and 13,
page 11.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 74
Figure 26: Success rates (as % of proposals submitted, and as % of budget available) by organisation
type in Horizon 2020 for Societal Challenge 4 'Smart, green and integrated transport' up to date. All
project types (CSA, IS, RIA). Source: CORDA data, 594 projects, 1 October 2016
It was mentioned by several of the participants that the submission deadlines of Horizon 2020 were
often inconvenient (notably end of Summer, early January). It is however not clear if valuable
candidates have not taken part in proposal preparations due to poor timing.
6.3.2 OVERSUBSCRIPTION ON SOME TOPICS
The calls for proposals in FP7 over the period 2007 – 2013 attracted a total of 2656 proposals leading
to 620 grants. This represents a success rate of 23,4% in participations and 25,4% in funding.
For Horizon 2020, 512 grants have been awarded to date, with a general success rate for proposals
to make it to being awarded a grant is around 16%. This includes the SME calls, while two-stage
proposals were only counted when they make it to the second stage. It is noted, that the SME calls
are far below the other calls in Horizon 2020, with an average success rate of only 10.1% of the
proposals that make it to a grant. Leaving out the SME calls, this figure is 23.8% (number of
proposals), which is equal to the FP7 rate.
As mentioned, a 20% average success rate is perceived by the stakeholders as being low for the
effort involved of project preparation. A lack of perceived success works in two ways – on the one
hand only those that can afford to prepare proposals will do so and, on the other, it can compromise
participation and those that may be the most innovative potential partners are not able or interested
in participating, as the gains versus effort ratio is not attractive.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 75
Figure 27: Success rates for Horizon 2020 projects up to August 2016, per funding scheme and activity
area, including JUs. Note that the two-stage proposals are only counted when they make it to the
second stage. Source: CORDA
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 76
Figure 28: Success rate by research area in FP7. Source: Transport Research in the 7th Framework
Programme (2007 – 2013): an overview of programme implementation statistics
Although the figures do not really show a drop in overall success rate (when leaving out the SME
calls), all participants and stakeholders are worried about the low success rates of proposals in some
specific topics. Also there are clearly too many proposals leading to oversubscription in some topics,
which is considered a serious problem. A topic might be crucial to bring forward for European
transport research, reflected by the number of proposals submitted, but if only one project is funded
it is unlikely that this will provide the opportunity for excellence in this domain. In particular, this is
considered to be the case for topics that are more multimodal or cross-cutting, like the urban and
socio-economic calls. An extreme case mentioned is the MG-9.2-2014 topic, where 20 proposals lead
to 1 funded project. Even when it was known beforehand that the budget was limited, participants
considered this topic of great importance. Concerns were expressed that Horizon 2020 did not pick
up this message earlier.
To some extent it was felt that this is a result of the already mentioned (section 6.2.4 above)
relatively open formulation of the calls, leading to a wide spectrum of proposals. There is a
contradiction in having wide areas within topics giving the possibility to put forward a lot of different
ideas, and a high level of competition that is not connected to the amount that is available for
funding.
Secondly, the selection procedure is perceived as not being transparent enough by the project
coordinators that participated in the survey and interviews, outcomes being so uncertain; so
participants are less motivated to provide quality proposals. Unsuccessful proposals are given to
chance to ask for a redress. INEA statistics show that of the 416 received proposals, only 4 requests
for a redress were received, of which 2 were not upheld and 2 were upheld but did not lead to re-
evaluation. This shows that participants either acknowledge the result of the evaluation, or that they
do not believe the effort of going into a redress procedure would matter considering that the
available funds are probably exhausted. It is not clear to the Expert Group if one or the other is
predominant.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 77
Considering the SME calls, the low success rate is caused more by the low quality of the proposals
than strict oversubscription. SME proposals are relatively easy to submit (no other partners needed,
call open every 3 months, very broad topics) which might lead to more ‘give it a try’ types of
proposals. This is also partly visible in the data on SME proposals, and a lot of the failed proposals
are resubmitted once or even twice.
When looking at the JUs, it is clear that proposals under SESAR and Shift2Rail topics have a much
higher success rate than those under the general Horizon 2020 framework. On the other hand, Clean
Sky has a low success rate, or formulated in a positive way: a high supply of research proposals
resulting in an oversubscription.
Figure 29: Success rates for Horizon 2020 projects up to August 2016, per funding scheme and PPP.
Note that the two-stage proposals are only counted when they make it to the second stage. Source:
CORDA
A drawback of the low success rate is that some very valuable projects do not get funding. Looking at
the Mobility for Growth and Green Vehicle calls, 301 out of 510 proposals made the threshold88 with
an evaluation score of at least 10/15, but only 167 (55,5%) of them got funded with a total budget of
€862 million. The 134 remaining proposals account for a €787 million budget. This can be
considered as a latent demand for valuable research that will be lost.
The graph below shows the percentage of projects that get funded by their evaluation score.
Obviously, the high scores have a better chance to get funded than the lower scores. However, there
were proposals with a very high mark of 14/15 that did not get funding. On the other hand, some of
the fairly weak proposals with a 10/15 got funded. This is due to the fact that some topics get
oversubscribed while others do not. Also, sometimes topics are grouped into one funding budget,
causing one topic to get more funds than the other if the quality of the proposals significantly differs
88
Overall threshold is not the only indicator. Proposals do not only need to meet the overall threshold of 10/15
but also 3/5 for each of the 3 subcriteria. Proposals may be above the overall threshold but fail on an
individual criterion threshold. The data on the subcriteria was not available.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 78
between topics. This leads to certain topics getting less funds than expected and can cause another
push towards high or low success rates by topics. All this might explain the feeling a lot of
participants have when they describe the evaluation process as a ‘lottery’.
Figure 30: Success rates per evaluation score in Horizon 2020 up to August 2016 for M.G. and G.V.
calls, including JUs. Note that the two-stage proposals are only counted when they make it to the
second stage. Source: CORDA
The figures below show the two worst cases:
- Green Vehicles: 2 out of 10 proposals with 14/15, 2 out of 7 proposals with 13,5/15 and 5 out
of 13 proposals with 13/15 were not funded.
- M.G. socio-economic topics: 2 out of 2 proposals with 13,5/15 and 3 out of 5 proposals with
13/15 were not funded.
Figure 31: Success rates per evaluation score in Horizon 2020 up to August 2016 for the G.V. calls.
Note that the two-stage proposals are only counted when they make it to the second stage. Source:
CORDA
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 79
Figure 32: Success rates per evaluation score in Horizon 2020 up to August 2016 for the socio-
economic topics in the M.G. calls. Note that the two-stage proposals are only counted when they
make it to the second stage. Source: CORDA
6.3.3 ADEQUACY OF FUNDING RULES
The simplification of funding rules is overall appreciated as a good and clear step forward compared
to FP7. In particular the fixed overhead is a good idea in Horizon 2020: it leads to a lower
administrative burden and keeps out high overhead percentages some companies claimed in FP7.
However ‘simplification’ of the personnel allowances to a single rate was not appreciated, as there is
a need for different expert level involvements in projects in order to deliver excellence, which now is
difficult to include into projects.
A few other minor critical remarks were formulated by stakeholders and survey respondents:
- Equal funding is unfavourable for universities and SMEs and favourable for public bodies
(which are already funded by taxes via other means).
- Personnel cost rules (what is included and what not) are still too complicated.
- No flexibility left to increase or decrease the funding rates for some topics.
- Some large companies have relatively high overhead costs in reality, and cannot cover all
costs.
- A too large emphasis on IAs and too little focus on CSA and RIAs (or lower TRLs research).
In a survey project coordinators consider the new types of action to have a generally positive impact
on the effectiveness of Horizon 2020, although the response is quite mixed.
Figure 33: Assessment of the new types of actions (mainly IA and RIA instead of CP) on the
effectiveness of Horizon 2020 compared to FP7 from 33 coordinators that participated both in FP7
and Horizon 2020. Source: Survey among project coordinators, September 2016.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 80
6.3.4 THE COST OF SUBMITTING A PROPOSAL
The cost of submitted a proposal is recognised as an investment and is considered to be too high by
many of the project coordinators that participated in the survey and in the interviews. In particular,
looking at topics with a low success rate (below 10%), the effort does not relate positively to the
benefit of getting the grant. If the effort to be put in a proposal averages 5% of the grant amount
(which seems to be fair), for the price of 20 proposals, one can fund a project.
In the survey, the selection process is described as ‘acceptable’ to ‘good’ (marked 2 to 3 out of 5), by
both Horizon 2020 and FP7 project coordinators. Respondents that participated in both programmes
have a mixed to slightly positive feeling about the improvement in Horizon 2020. It should be noted
that this was a survey among project coordinators, thus making it biased to the ‘winners’ point of
view.
Figure 34: Top graph: average rating (0 very poor and 5 excellent) according to coordinators of 32 FP7
and 21 Horizon 2020 projects. Bottom graph: assessment of Horizon 2020 compared to FP7 from 33
coordinators that participated both in FP7 and Horizon 2020. Source: Survey among project
coordinators, September 2016.
Improvements can be made in making the system more clear to participants on the topic, leaving out
unnecessary information that is not relevant to them.
Attention should be paid to smaller companies and research institutes, who have to compete with
bigger companies who have the budgets to preparing a proposal, which is a very expensive and time
consuming activity and is not funded.
Due to the longer periods between the publication of the Work Programme and the call deadlines,
the proposal preparation phase takes longer. Partner networking and proposal writing is a
continuous process now. The drawback of this is that there is no resting period anymore and that it
increases the difficulty for newcomers to be integrated in a project.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 81
6.3.5 TWO-STAGE APPROACH
The cost of submitting a proposal is particularly high for two-stage proposals. The two-stage
approach is designed to allow a lighter application process in the first stage. In principle, a two-stage
approach seems sound. However, almost all of the participants and stakeholders in the interviews
recommended to stop using the two-stage approach or to dramatically change it, and it is not seen as
an improvement of the FP7 process. It takes as much time to put together the first of the two stages
as is needed to cover the majority of tasks for delivering the whole project – but due to the time
frame all momentum is lost and the final proposal needs to be completely revisited for the
preparation of the second stage.
This is confirmed in the survey project coordinators, who consider the two-stage approach to have a
negative impact on the effectiveness of Horizon 2020 – see graph below.
Figure 35: Assessment of the two-stage evaluation on the effectiveness of Horizon 2020 compared to
FP7 from 33 coordinators that participated both in FP7 and Horizon 2020. Source: Survey among
project coordinators, September 2016.
The stakeholders stated the following arguments against the two-stage approach, which the Expert
Group agrees with from their own experience:
1. Double work for participants as in order to be successful in stage 1, it is necessary to invest
already a lot of money and effort in creating a good proposal and a solid consortium. This makes
stage 1 already a full proposal with a high investment and an extremely low success rate.
2. It encourages a lot of small (low quality) opportunistic competition in the first round.
3. Less oversubscription and no need to have a two-stage process if the scope of the topics were
more precise and better defined.
4. Different evaluation experts in the two stages cause a lack of coherence between evaluations
(inconsistent scores for the different stages). A proposal getting low marks in stage 2 should
already have been flagged in stage 1.
5. It lengthens the time from call to grant, while one of the continuous goals in the programme is to
shorten and ease the process.
6. The success rate in stage 2 (see graph below) does not differ much from a success rate for a
single stage proposal (it is slightly higher in 2014, equal in 2015). However, there were many
more proposals in stage 1 than in stage 2. There is a massive hidden oversubscription in stage 1,
leading to large administrative costs, both on INEA’s and on the applicants’ side. It is thought
that leaving out the second stage could lead to an increase in proposals that have to go through
the full evaluation, but looking at the numbers in the one-stage topics, this would be only a small
increase and there would be significant other gains.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 82
Figure 36: Success rates for one and two stage topics in the 2014 and 2015 M.G. calls of Horizon
2020, including JUs. Note that the two-stage proposals are only counted when they make it to the
second stage. Source: CORDA
6.4 COST-BENEFIT ANALYSIS
Horizon 2020 is as a much ‘closer to the market’ programme than FP7, which is valued by the
participants and stakeholders. The take up and transfer of mature technologies is well appreciated.
As mentioned, it is too early to assess whether the resources are reasonable in light of the results
that have been or are likely to be generated. However, in terms of value for money some interesting
discussions came up around the interpretation of value.
Cost benefit looks at the ratio of cost to benefit in purely financial terms, yet many of the participants
and stakeholders clearly stated that from their perspective the main value of taking part in European
Research projects was in the collaborative exchanges, the learning and the opportunity to work on
major challenges from a multicultural and multidisciplinary perspective. This value should possibly be
more formally recognised and perhaps monetised. In addition it was felt that the cross-cutting and
independent nature of European research should also recognise the value of failure: not all projects
should be designed to succeed but to explore different routes and potential solutions. Under the
strict obligation to ‘succeed’ it is likely that innovation will be overlooked. This feedback came from
interviews and the stakeholder hearing.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 83
6.5 OTHER ISSUES RELATED TO EFFICIENCY
The new management by INEA is considered neutral to positive compared to FP7.
Three issues emerge:
- Evaluation process & selection criteria
- Length of process
- Administrative burden
6.5.1 EVALUATION PROCESS & SELECTION CRITERIA
There are some complaints by the project coordinators about the evaluation procedure in Horizon
2020, though none of the stakeholders’ experiences are seen as being worse than FP7. In general, it
is considered a fair process though some improvements are needed.
The main concern is that experts doing the evaluation are not informed enough, or have insufficient
expertise to do the job. This is partly explained by the wide interpretation of the topics, which
obviously requires a broader set of expertise to evaluate. A proposal may require an evaluator to
have scientific knowledge, practitioner experience and/or state-of-the-art business or industry
understanding. Together with gender and nationality balance, it is not easy to find experts with no
conflict of interest. As mentioned, broad topics set out in the call make it also difficult to evaluate
proposals that may be very different.
In addition to other concerns about the two-stage proposals, there are different experts in stage 1
and stage 2. These sometimes have different opinions on the proposal leading to a lack of coherence
in the evaluation and thus to a loss of trust in the process by the applicants. Obviously, the
evaluation criteria are wider in stage 2 than in stage 1: e.g. implementation is only assessed at stage
2. Also, in case of a two-stage proposal there is almost no feedback to applicants after the first stage.
In the survey, the selection process is described as ‘acceptable’ to ‘good’ (marked 2 to 3 out of 5), for
both Horizon 2020 as FP7 project coordinators. Respondents that participated in both programmes
see a slight improvement in Horizon 2020. Note that this was a survey among project coordinators,
thus making it biased to the ‘winners’ view on the picture.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 84
Figure 37: Top graph: average rating (0 very poor .. 5 excellent) according to coordinators of 32 FP7
and 21 Horizon 2020 projects. Bottom graph: assessment of Horizon 2020 compared to FP7 from 33
coordinators that participated both in FP7 and Horizon 2020. Source: Survey among project
coordinators, September 2016.
6.5.2 LENGTH OF PROCESS
When competing with research outside Europe, the shortest possible process is the best. Horizon
2020 is considered already significantly better than FP7. The time to contract has notably been
improved but the two-stage process is still seen by the participants89 as taking too long. For projects
with requested high TRLs the market opportunities might have already shifted after that long period
of time.
However, the Expert Group considers it beneficial for the applicants that the work programmes are
published two years ahead so applicants are able to prepare and know what is coming.
Areas for improvement that were stated by the stakeholders and agreed with by the Expert Group:
- Before publication of the call, the process should be speeded, up in particular when it
comes to work programme definition and publication. Information required as input in
the definition phase of the next work programme is not sufficiently available.
- The time from publication of the call text to grant agreement could be reduced with a
shorter evaluation process (which is now 5 months). Especially the time between the
delivery of evaluation expert results and the annotation of the results to the applicant.
- Either the two-stage process is abolished or changed as it now can take 24 months from
initial call to contract signing and starting research
- The non-negotiation clause, which is new in Horizon 2020, is seen to possibly save time
from the Commission’s perspective but it comes at a price: once a project is selected,
adaptation is only possible during the contract, with amendments and does not seem to
have substantially reduced the contracting time overall (from call to contract). Some
participants experience the grant preparation process as ‘hurried’ with a resulted project
start that was not optimal. Adapting the number of deliverables is seen as being
89
Both in the survey and the interviews.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 85
especially necessary. In a sample of 125 projects, in total 45 such amendments were
made: small reorientations, demonstrations that cannot be implemented, issues
discovered during contract negotiation.
In the survey, the project management is described as ‘acceptable’ to ‘good’ (marked 2 to 3 out of 5)
by both Horizon 2020 and FP7 project coordinators. Respondents that participated in both
programmes see a slight improvement in Horizon 2020, mainly on time between submitting a final
proposal and signing the grant – which is logical as there is no negotiation phase anymore.
Figure 38: Top graph: average rating (0 very poor .. 5 excellent) according to coordinators of 32 FP7
and 21 Horizon 2020 projects. Bottom graph: assessment of Horizon 2020 compared to FP7 from 33
coordinators that participated both in FP7 and Horizon 2020. Source: Survey among project
coordinators, September 2016.
6.5.3 ADMINISTRATIVE BURDEN
The management of the programme has been outsourced to the new INEA agency since the
beginning of Horizon 2020. Besides Horizon 2020, INEA also manages CEF and Marco Polo.
There has been significant effort and investment to simplify the processes, which is recognised and
appreciated by both the participants and the EC. In some cases Horizon 2020 is now considered less
bureaucratic than many national research programmes. The new participants’ portal has been
reworked but it is not considered to be entirely successful. There are still some user interface issues
as it is still quite complex and not considered to be intuitive; it requires quite some experience to
navigate it. An example is that the grant agreement preparations require to copy paste all
information manually from the proposal, while one would expect this to be automatic. The portal
can be very slow, and has unnecessary pop-up windows. Especially SMEs and smaller research
entities lack this experience and find uploading technically challenging and time consuming.
A weakness, pointed out by several project coordinators, is that the split between the management
(at INEA) and the content and programme definition (at the DGs and JUs) leads to a longer feedback
loop from on-going and finished projects to the new work programmes and towards EC policies in
general. Similarly, a scientific or strategic follow up of the project by EC project officers during the
project, complementary to the management, might turn out to be necessary to keep the loop closed.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 86
In the survey, the project management is described as ‘good’ to ‘very good’ (marked 3 to 4 out of 5),
by both Horizon 2020 as FP7 project coordinators. Respondents that participated in both
programmes see an improvement in Horizon 2020. Despite the criticism mentioned above, Horizon
2020 was considered to be better than FP7 with regard to the participant portal, the grant
preparation process and the monitor and reporting procedures.
Figure 39: Top graph: average rating (0 very poor .. 5 excellent) according to coordinators of 32 FP7
and 21 Horizon 2020 projects. Bottom graph: assessment of Horizon 2020 compared to FP7 from 33
coordinators that participated both in FP7 and Horizon 2020. Source: Survey among project
coordinators, September 2016.
6.6 LESSONS LEARNT/ AREAS FOR IMPROVEMENT
Approaches that could be considered to generate further efficiency gains:
- Topics are too broadly defined and should be made more precise.
- Change the two-stage proposal submission either by reverting to a single stage or
refining the calls so they are more precise and setting clear time frames for the
whole process
- Process from call to contract is still considered to be too long (although already
improved compared to FP7)
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 87
7 COHERENCE The Expert group has approached coherence of the Horizon 2020 Transport Societal Challenge
according to the following lines:
1. Internal coherence of Horizon 2020 Transport
- Coherence among instruments
- Coherence of actions implemented in transport under Horizon 2020 and, in
particular, with regard to PPPs
- Coherence with other Societal Challenges of Horizon 2020
- Coherence among other areas of Horizon 2020, from the ERC, EIT-KICS…
2. External coherence of Horizon 2020 Transport actions in relation to:
- Coherence with other funding mechanisms of the EU (structural funds, European
Investment Back, among others)
- Coherence with national and regional funding programmes for innovation.
7.1 INTERNAL COHERENCE
7.1.1 INTERNAL COHERENCE OF INSTRUMENTS TO IMPLEMENT HORIZON 2020
TRANSPORT
Currently, Horizon 2020 transport calls include the following list of main instruments.
Table 8: Horizon 2020 Transport instruments
General instruments Focused instruments Complementary instruments
Research and Innovation Action
(RIA)
Innovation Action (IA)
Coordination and Support
Action (CSA)
Joint Undertakings / PPP
SME Instrument(s)
Fast Track to Innovation (FTI)90
ERA-NET (Cofunded)
Inducement prize
Public Procurement for
Innovative Solutions (PPI)
Cofunded91
Provision technical services
Expert contracts
The instruments for funding for basic or general research, as well those that are more focused ones
such as JUs, PPPs, the SME Instrument or the ERA-NET Cofund92 and supporting tools related to
Public Procurement in Innovative Services (PPIS)93, tenders and expert involvement, show a cohesive
framework. None of the interviewees indicated that this was not the case. This is also the opinion of
the Expert Group. On the other side, a small number of participants in interviews and the hearing
90
https://ec.europa.eu/programmes/Horizon2020/en/Horizon 2020-section/fast-track-innovation-pilot 91
http://ec.europa.eu/research/participants/data/ref/Horizon 2020/wp/2014_2015/annexes/Horizon 2020-
wp1415-annex-e-inproc_en.pdf 92
http://ec.europa.eu/research/participants/data/ref/Horizon 2020/wp/2014_2015/annexes/Horizon 2020-
wp1415-annex-d-eranet-cofund_en.pdf 93
http://ec.europa.eu/research/participants/data/ref/Horizon 2020/other/wp/2016-2017/annexes/Horizon
2020-wp1617-annex-ga_en.pdf
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 88
referred that the reason for deciding whether an IA or a RIA instrument was used in a specific topic
was not always clear and this could therefore be improved.
With regard to the 55 project coordinators surveyed for this interim assessment, the outcome is not
so conclusive, although answers tend to be more positive than negative as the following tables show
(the high percentage of “I don’t know” responses should be noted).
Table 9: Complementarity of the different mix of available instruments (in the case of Horizon 2020,
JU/PPPs, IA, RIA, CSA, SME instrument, etc.)
0
(non-existing)
1 2 3 4 5
(at a 100% degree)
I don´t know
+ No answer
Complementarity
among instruments?
0% 4% 6% 13% 17% 0% 61%
Synergy among
instruments?
0% 7% 4% 9% 13% 2% 65%
Table 10: Overlaps and gaps among the different mix of available instruments
0
(non-existing)
1 2 3 4 5
(at a 100% degree)
I don´t know
+ No answer
Overlaps among instruments? 6% 0% 6% 0% 4% 0% 85%
Gaps among instruments? 0% 0% 2% 2% 7% 4% 85%
The following individual comments were indicated when a low scoring (between 3 and 5) was
indicated in the previous table:
- Topics in CS2 and Horizon 2020 often overlap (e.g. strong ecological impact in both areas)
- There is bias towards higher technologies with the research programmes, and less emphasis
on what may actually be needed to solve the challenges of transport especially within SC4.
This could be tackled in a low technology or policy way, but often the opportunities to do so
are missed.
- Gaps between available instruments between low TRL (1-3) to high TRL (5-6).
There is a general acknowledgement (from interviews and survey) of the advantages of the PPPs.
They are perceived by stakeholders during the interviews as part of a coherent set of instruments in
areas with clear focus and need for higher TRLs. The PPPs are in particular considered a useful
instrument for reaching higher TRLs in connection with specific technological challenges. However a
small number of stakeholders (mainly from outside the PPPs) suggested the need for greater
coordination across the various transport initiatives (i.e. how each PPP relates specifically to the
other and to the regular Horizon 2020 programme): stronger links between air and surface PPPs, for
example, and opportunities for best-practices exchange. A table with positive characteristics and
areas of improvements of JUs and PPPs is included in the European Added Value in section 8 of this
interim assessment report.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 89
The following table shows the current Joint Undertakings and contractual Public-Private Partnerships
(cPPPs) supported by the Horizon 2020 Transport budget94 or with direct transport applications:
Table 11: Chart of Joint Undertakings and contractual Public Private Partnerships fully or partially
supported by the Horizon 2020 Transport Societal Challenge
Joint Undertakings and contractual Public-
Private Partnerships (cPPPs) funded through the
Horizon 2020 transport budget
EC Contribution
(€ million)
Total Costs
(€ million)
Crowding-in
Effect
Single European Sky ATM Research (SESAR) 2020 500 1500 300%
Clean Sky 2 (CS2) 1800 4000 222%
Shift2Rail (S2R) 450 920 204%
European Green Vehicles Initiative (EGVI) 750 1500 200%
Total 3500 7920 226%
Other Joint Undertakings with direct
applications to Horizon 2020 transport
EC Contribution
(€ million)
Total Costs
(€ million)
Crowding-in
Effect
Fuel Cells and Hydrogen 2 (FCH2) 665 1300 195%
Electronic Components and Systems for
European Leadership (ECSEL)
1180 5000 424%
Total 1845 6300 341%
Without considering the budget reserved for institutional PPPs (Clean Sky 2, SESAR, Shift2Rail and
Fuel Cells and Hydrogen 2), RIA and IA account for around 75% of the budget available for calls
(including topics with the European Green Vehicles Initiative)95. More specifically, IAs represent
25.9% of the total budget while RIAs amount to 50.0% of the total work programme budget (and an
additional 0.2% is allocated to topics proposed for both IA and RIA).
This percentage largely corresponds to the one reported by the European Commission as part of its
cross-cutting KPIs reporting. Based on the EC’s data, the share of EC contribution to Innovation
Actions (IA) in signed grants represents a 29.4% of the total (for the entire Horizon 2020, this
percentage is 16.8%)96.
According to this data, the percentage of EU financial contribution focussed on Demonstration and
Piloting activities within Innovation Actions (IA) reaches 50.3% in the case of Transport while for the
entire Horizon 2020 reaches 77.8%. Although the percentage for Transport is lower than the average,
it is clear that half of the budget focussing on Demonstration and Piloting activities is still a
remarkably high percentage to this type of activities.
94
http://ec.europa.eu/programmes/horizon2020/en/area/partnerships-industry-and-member-states 95
Own analysis of work programmes conducted for this interim assessment. See annex at the end of this
document. 96
Data provided by the EC for the purposes of this interim assessment. Date of data extraction: 01/09/2016.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 90
ERA-NETs cover an important link between European Commission actions and national or regional
levels. ERA-NET actions are designed to be particularly suited for the involvement of small actors and
national, regional or local public administrations, helping to facilitate the adaptation and deployment
of innovations. However there is a gap between the objectives and ambition of ERA-NET and the
implementation that is reflected in its low take up level. This is clear when compared with other large
projects (for instance, the FP7 ERA-NET Electromobility+ pooled together some €20 million from the
participating countries and regions as well as from the EC97 and this budget for the entire ERA-NET is
comparable to the EC contribution to a single large innovation action).
It was mentioned on several occasions by stakeholders that the SME instruments including the Small
business innovation research for Transport call are seen as being successful. Respondents refer to the
SME instrument as complementary to the SME participation target in conventional Horizon 2020
calls: SMEs in the SME instrument playing the central role and considering closer to market activities
while participation in a consortium provides more opportunities for networking and becoming part
of larger value chains, for example.
As a general conclusion of this subsection, and based on document analysis and interactions with
stakeholders (interviews and hearings), the Expert Group considers that overall the structure and
goals of the instruments are seen as being coherent with the ambition of Horizon 2020 transport,
especially SC4 and the stated objectives of the Union for transport research.
7.1.2 INTERNAL COHERENCE WITH OTHER HORIZON 2020 INTERVENTION AREAS
Transport is a cross-cutting issue and there are numerous cross-references to transport research and
innovation in other Horizon 2020 intervention areas.
Furthermore, as expressed in the Energy Union paper, inter-sectorial issues requiring research and
innovation and contributing to the EU's competitiveness need to be jointly addressed in order to
identify the links, synergies and trade-offs. This is of particular importance for the production,
storage and use of alternative energies and fuels (including electricity and biofuels)98 used for
transport especially in respect to decarbonising transport and shifting it away from fossil fuel use.
The general calls of Horizon 2020 Transport include references to other calls and initiatives that are
relevant for the Transport Challenge, e.g. the calls ‘Blue Growth’, and ‘Smart Cities and
Communities’. This is intended to create synergies with other parts of Horizon 2020 such as Space,
the ‘Secure societies’ Challenge and generic research on materials and manufacturing techniques.
The latter are dealt with in the Leadership in Enabling and Industrial Technologies (LEIT) part of
Horizon 2020 under Nanotechnologies, Advanced Materials and Advanced Manufacturing and
Processing (NMP) and Factories of the Future. Recycling of materials and replacement of critical raw
materials are normally addressed by the ‘Climate action, resource efficiency and raw material’
societal challenge. A number of actions related to the production of alternative fuels for aviation or
advanced energy storage systems and interfaces between vehicles and energy recharging
infrastructures can also be found under the ‘Secure, clean and efficient energy’ challenge.
97
http://electromobility-plus.eu/wp-content/uploads/Electromobility+overview_Apr2014.pdf 98
Internal European Commission’s document that has been made available to the Expert Group.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 91
Clear indications are given in the Work Programmes of where these opportunities are and a good
example in the 2014-2015 Work Programme is the following reference: “In addition to the topics of
this call, a topic on post lithium ion batteries for electric automotive applications (NMP 17 – 2014) is
included in ‘Nanotechnologies, Advanced Materials and Advanced Manufacturing and Processing
(NMP)’ under “’Leadership in Enabling and Industrial Technologies’ (LEIT)”. The 2016-2017 Work
Programme explains, for instance, that “transport-related actions are also included in other parts of
Horizon 2020 Work Programme 2016-2017, particularly in the LEIT/NMBP call 'Nanotechnologies,
Advanced Materials, Biotechnology and Production'; LEIT/Space call 'Applications in Satellite
Navigation – Galileo'; and SC/Energy calls 'Competitive Low-Carbon Energy' and 'Smart Cities and
Communities'”. The development of batteries is a good example of collaboration within the EC
among different Societal Challenges of Horizon 2020: NMP, Energy and Transport. This internal
collaboration in the EC is performed via the Challenge Group consultation at programme level. Since
the information about this working group does not yet reach the research community, they refer to a
lack of coordination among challenges which may not entirely be the case but that there are gaps in
communication. It is therefore recommended that Horizon 2020 internal collaboration is better and
more widely communicated.
Another concrete example of opportunity for transport research and innovation is found is the
following EE-22-2016-2017: Project Development Assistance of the Energy programme99: This is
designed to help build the technical, economic and legal expertise needed for project development
with public and private actors that will lead to concrete investments.
The PDA focuses on the sectors of existing public and private buildings; street lighting;
retrofitting of existing district heating/ cooling; energy efficiency in urban transport (such as
transport fleets, the logistics chain, e-mobility, modal change and shift) in urban/suburban
agglomerations and other densely populated areas and energy efficiency in industry and
services.
In a recent survey conducted on behalf of the European Commission as part of the general Horizon
2020 Interim Assessment exercise, project coordinators were asked to assess whether their projects
were expected to have impact on the societal challenges in the next 10 years (with a ‘Yes/No’
answer). The original table included in that report is reproduced below100. Societal Challenge 4
Transport has been highlighted with a yellow background. As can be seen, respondents indicated
that transport projects will have the largest impact or contribution in the following Horizon 2020
areas: SC5 “Climate Action, Environment, Resource Efficiency and Raw Materials” (62.0%) and SC3
“Secure, clean and efficient energy” (36.5%). Around one fourth of project coordinators indicated
that their project will also impact SC1 “Health”, SC6 “Inclusive Societies” and SC7 “Secure Societies”.
Only 9.3 of Transport projects, according to this survey, would impact SC2 “Food”.
99
http://ec.europa.eu/research/participants/data/ref/Horizon 2020/wp/2016_2017/main/Horizon 2020-
wp1617-energy_en.pdf 100
PPMI. FRAMEWORK CONTRACT FOR THE PROVISION OF SERVICES TO THE COMMISSION IN THE FIELD OF
EVALUATION OF RESEARCH AND INNOVATION PROGRAMMES AND POLICIES (2012/S 144-240132). OVERVIEW
OF Horizon 2020 SURVEY RESULTS. Internal European Commission document made available for the purposes
of this interim assessment.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 92
It is also remarkable that according to this survey, transport is placed as the Societal Challenge with
smallest contribution from the Excellent Science part of Horizon 2020, with the exception of the
Space project coordinators, who mention the transport area most frequently as contributing to
excellence.
Table 12: Expected impact on societal challenges within the next 10 years
Question 15- Grid question with ‘Yes’/’No’ answer categories
Horizon 2020 section SC1 SC2 SC3 SC4 SC5 SC6 SC7
Excellent Science
Future and emerging technologies (n
= 16) 33.3 % 40 % 57.1 % 20 % 52.4 % 30 % 25 %
Research Infrastructures (n = 27) 52.2 % 52.4 % 23.8 % 18.2 % 43.5 % 40.9 % 36.4 %
Industrial leadership
NMPB (n = 96) 42.4% 29.0% 52.6% 23.2% 61.9% 18.0% 14.6%
Subtotal within NMPB: PPP projects
(n=32) 25.9% 7.0% 69.6% 19.8% 68.3% 23.4% 9.7%
Information and Communication
Technologies (n = 177) 52.0% 21.5% 32.2% 34.5% 30.0% 55.8% 38.5%
Space (n = 36) 28.2% 31.4% 33.1% 52.3% 44.0% 29.0% 50.6%
Innovation in SMEs (n = 30) 24.4% 24.3% 26.8% 19.9% 19.9% 26.0% 21.5%
Societal Challenges
SC1 (n = 106) 98.1% 9.8% 1.7% 2.1% 5.3% 35.6% 9.6%
SC2 (n = 43) 49.2% 98.6% 21.4% 4.6% 86.4% 25.8% 14.2%
SC3 (n = 124) 21.4% 19.0% 97.5% 34.2% 86.6% 29.4% 17.7%
SC4 (n = 96) 26.1% 9.3% 38.5% 96.1% 62.0% 28.9% 23.4%
SC5 (n = 71) 39.2% 57.9% 57.9% 28.5% 95.7% 34.5% 26.0%
SC6 (n = 32) 53.6% 16.5% 17.5% 20.7% 32.5% 90.2% 35.9%
SC7 (n = 31) 38.6% 33.3% 25.7% 36.2% 30.2% 53.1% 93.3%
Spreading Excellence and Widening participation + Science with and for Society + other programmes
Spreading Excellence and Widening
Participation (n = 24) 64 % 44 % 52 % 26.9 % 44 % 51.9 % 35.7 %
Science with and for Society (n = 10) 57.1 % 50 % 50 % 50 % 57.1 % 87.5 % 42.9 %
Fast Track to Innovation Pilot (n = 10) 66.7 % 33.3 % 50 % 33.3 % 66.7 % 33.3 % 0 %
Euratom (n = 3) 33.3 % 0 % 100 % 0 % 50 % 33.3 % 33.3 %
Total 46.9 % 29.4 % 41.6 % 32.8 % 50.9 % 38.6 % 27.2 %
Total number of valid responses 920 905 914 906 909 911 902
As part as the present Horizon 2020 Transport interim assessment another survey among project
coordinators was run by the EC and included the question of what other Horizon 2020 research areas
or programmes also represent an opportunity for transport. The following table shows the ranking of
areas associated with higher opportunity, complementarity or synergies (percentage indicates the
sum of percentages of respondents that indicated opportunity, complementarity or synergies) across
a number of sectors.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 93
Table 13: Ranking of areas associated with higher opportunity, complementarity or synergies with
Horizon 2020 Transport project results. Analysis of the answers to the project coordinator survey
ICT Research & Innovation 29%
Innovation 27%
Energy 26%
Environment & Climate Action 26%
Research Infrastructure 25%
International Cooperation 23%
Partnerships with Industry and Member States 23%
Key Enabling Technologies (including NMBP) 22%
SMEs 22%
Security 20%
Funding Researchers 19%
Raw Materials 13%
Bio-based Industries 12%
Agriculture & Forestry 11%
Health 10%
Space 10%
Aquatic Resources 6%
Social Sciences & Humanities 6%
Society 6%
Biotechnology 4%
Food & Healthy Diet 2%
Project coordinators were asked whether there are enough information and coordination
mechanisms within those other areas of Horizon 2020 that are complementary or synergetic for
transport. Most of the respondents agreed that there are enough information and coordination
mechanisms, as the following table shows.
Table 14: Are there enough information and coordination mechanisms within those other areas of
Horizon 2020 that are complementary or synergetic for transport? Analysis of the answers to the
project coordinator survey.
Totally agree Agree Rather agree Rather disagree Disagree Totally disagree I don´t know
4% 30% 39% 9% 13% 4%
This rather positive picture obtained from the project coordinators survey does not fully coincide
with the opinions of some interviewees, who indicated concerns about the amount or accessibility of
information regarding other funding opportunities either within other areas of Horizon 2020 or
outside Horizon 2020. One reason for this apparent discrepancy may be the different nature of
respondents (project coordinators in the survey generally agreeing but technology platform
representatives acknowledging during the interviews that they seldom deal with other areas of
Horizon 2020) and the fact that during interviews, additional comments were able to surface and
become more relevant. Another explanation, probably more important, is that while the question to
project coordinators answered a question specific about internal coherence within other areas of
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 94
Horizon 2020, interviews often mixed comments with regard to external coherence or coordination,
Anyway, as almost a quarter of the project coordinators surveyed also indicated that they either
rather or fully agreed that there are not enough information and coordination mechanisms this
indicates in the opinion of the Expert Group that, even if project coordinators tend to have a positive
opinion on this matter, opportunities for better dissemination still exist. One way of increasing
internal coherence awareness could be the periodic publication of some sort of “transport across
Horizon 2020: from ERC to the Societal Challenges and to the EIT” guide with examples and
opportunities for the transport sector.
Project coordinators responding to the Expert Group survey also generally agreed that projects will
produce interdisciplinary solutions, which cut across multiple specific objectives of Horizon 2020 (for
instance, new batteries for electric vehicles or light-weight material applications for high-speed
trains), as the following table shows.
Table 15: Will the project produce interdisciplinary solutions, which cut across multiple specific
objectives of Horizon 2020? Analysis of the answers to the project coordinator survey
Totally agree Agree Rather agree Rather disagree Disagree Totally disagree I don´t know
17% 26% 22% 13% 7% 15%
Some of the cross-cutting examples provided by the project coordinators are:
Lighter aircraft components and repair concepts that can be transferred to other transport
modes.
New antifouling coatings for marine transport, but also adoptable to static constructions (e.g.
renewable marine energy)
Expertise in propulsion and aerodynamics applicable for all other disciplines in high-speed
trains, automotive, aircraft
Application of robotics platforms to waterborne sector
Development of a technology that can be implemented in the design and production of
lightweight UAV machines and/or in the "health" monitoring of aerospace structures
Aero-servo-elastic research touching flight control, aircraft structures and aerodynamics,
which could be interesting for other industries like wind energy also.
Lightweight material applications due to innovative and composite processes for surface
transport and aerospace that are applicable to other sectors.
High efficient combustion engines that will be applied in different truck applications, but also
potential for off-road application.
New motor topology applicable to EV-s and industrial applications.
The activity will address both energy efficiency and air quality aspects.
Flexible low cost forming technology, machine learning algorithm applied to manufacturing
technologies.
When project coordinators were asked about the EC’s diversification of research and innovation
programmes (for example research infrastructures supported within the European Strategy Forum
on Research Infrastructures ESFRI, or the fundamental research supported by the European Research
Council ERC projects, Knowledge and innovation Communities launched by the European Institute for
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 95
Technology EIT…) they tended to indicate that there was little duplication and that the actions were
mostly complementary and synergetic. The results of the survey are shown in the table below.
Table 16: Results of survey on level of duplication of topics
0
(non-existing)
1 2 3 4 5
(at a 100% degree)
I don´t know
+ No answer
Duplication among actions 13% 13% 11% 7% 55%
Complementarity among
actions
11% 17% 15% 6% 52%
Synergy among actions 4% 7% 20% 15% 4% 50%
7.1.3 ENSURING THAT EVERY EURO SPENT COUNTS TWICE
The results of the analysis of the budget repartition per topic among the different lines of activities,
as shown in the following table (source: Expert Group analysis of topics) are of particular interest
here.
Table 17: Analysis of the budget repartition per topic Cross-cutting refers mostly to “global
leadership” + “resource efficiency”
Line of activity WP 2014-2015 WP 2016-2017 Total
Better mobility 231.2 26.2% 112.3 15.5% 343.5 18.9%
Cross-cutting 414.0 47.0% 514.2 46.5% 928.2 50.9%
Global leadership 55.3 6.3% 115.3 7.0% 170.7 9.4%
Resource efficiency 105.2 11.9% 124.0 12.7% 229.2 12.6%
Socio-econ. & forward looking 75.8 8.6% 72.5 18.3% 148.3 8.2%
Total general 881.5 100.0% 938.4 100.0% 1819.9 100.0%
As can be seen in the table above, half of the budget is directed into topics addressing several
general objectives, mainly global leadership and resource efficiency. In other words: leadership
through efficiency. As long as this combination is supported by strong legislation requiring products
and services to be efficient, it should support competitiveness and leadership of the European
industry (and not additional costs for guaranteeing efficiency), however the concerns over the links
between research outputs and policy that have been previously mentioned should be noted.
As the Horizon 2020 Transport Work Programme 2016-2017 explains, all new priorities were related
to one or more of the four broad lines of activities defined in the Specific Programme. One excellent
example of both competitiveness and societal objectives being coherently addressed by Transport
research and innovation in Horizon 2020 Transport is the Automated Road Transport call in Work
Programme 2016-2017:
“Automated Road Transport holds the promise to help address many of the major challenges
of today's transport system, such as user safety, energy efficiency, air quality and congestion,
and to enhance the drivers' individual comfort and convenience. At the same time, it
represents a critical testing ground for the ability of the European automotive industry to
preserve and consolidate its global leadership”.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 96
Other examples of multiple objectives are the institutional PPPs themselves. With the PPPs, the
cross-cutting nature of the players involved provides a level of multidisciplinary exchanges that
would not otherwise occur. It is more difficult for the Expert Group to dig deeper into challenges
such as the interaction between environment and transport policy with competitiveness issues of the
European industries within the PPPs. For instance, the Clean Sky Joint Undertaking and its successor
programme in Horizon 2020 Transport target primarily the reduction of the impact on the
environment through integration and validation of existing technologies up to Technology Readiness
Level 6. For that purpose, it will also include the type of work, which was carried out in Integrated
Projects under the Seventh Framework Programme.
The growing number of PPPs demonstrates that they are seen as a successful instrument. As they
include the private sector and there is significant co-funding it is considered that they are also
helping to position European Research to be more competitive, closer to the market and to help
bridge the gap between applied research, commercialisation and new business models, i.e. in other
words to be coherent with the overarching ambitions and goals of the Union. They are also seen as
serving a useful purpose in avoiding isolated actions and linking of projects. PPPs/JUs help to
generate a stronger community of research actors around one area and they cluster resources and
talent around well-defined topics. Indeed the waterborne sector is keen to create such a partnership
for their sector as they see it being able to deliver more and better continuity of research.
In the case of SESAR, a combination of optimization or air capacity and the development of new
technologies are pursued. The SESAR Joint Undertaking develops solutions for a seamless, efficient
and cost effective management of air traffic, including services of European GNSS and covers the full
range of TRL from 1 to 6. SESAR represents a clear example where both companies benefit from the
opportunity of researching and developing new generation of world-class competitive air traffic
management products and services and, simultaneously, citizens see how their travel time is reduced
and the prize of their air tickets is diminished.
Similar dual benefits can also be attributed to other transport innovations such as eCall for
automotive or the ERTMS for rail whose development was significantly supported by the European
research and innovation framework programmes. Also the European Green Vehicles Initiative will
improve at the same time competitiveness of the automotive industry, among other industries, and
improve quality of life in cities (less noise, less pollution…).
As part of this interim assessment a survey among project coordinators found strong agreement
when they were asked whether it was possible to justify that the project was going to simultaneously
tackle societal challenges (i.e. supporting mobility of the elderly while, at the same times, creating
new business opportunities) and give rise to new competitive businesses and industries. Results are
shown in the table below.
Table 18: Is the project going to simultaneously tackle societal challenges (i.e. supporting mobility of
the elderly while, at the same times, creating new business opportunities)? Analysis of the answers to
the project coordinator survey.
Totally
agree
Agree Rather
agree
Rather
disagree
Disagree Totally
disagree
I don’t know + No
answer
17% 33% 35% 2% 6% 0% 7%
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 97
The following comments were offered by those who agree to some extent:
Lowering emission of GHG, and creating new opportunities for European businesses. Spin offs
are currently undertaken.
Contribution to decreased CO2 emissions, improved marine environment, while creating
increased competitiveness of European paint industry
The high-speed aircraft development would fundamentally change the mode of operation at
airport, for business and industries and increase the time efficiency now lost during travel.
Meanwhile the use of hydrogen will open new potentials in all other areas where (stored)
energy is needed e.g. automotive...
The project was to identify future road safety research needs, thus tackling one of the
greatest societal challenges with more than 20,000 fatalities on Europe’s roads each year. At
the same time, safety innovations which will result from the proposed research have the
potential to maintain and further improve the competitiveness of the European industry.
Environmental benefits of new technologies are to be demonstrated. The use of these new
technologies will certainly open new business opportunities and moreover reduce energy use
and emission production.
With an increased level of traffic safety for older road users they will be more mobile in future
and industry partners may take their advantage of that.
New motor topologies that can open new business opportunities while reducing EU
dependency on critical materials (environmental impact).
New business opportunities for reducing congestion/mobility issues in urban areas.
Making suppliers aware of cities' needs and requirements will contribute to extending the
market for ITS solutions.
7.2 EXTERNAL COHERENCE
7.2.1 COHERENCE WITH OTHER EU FUNDING PROGRAMMES
In addition to Horizon 2020 Transport, existing and new following sources of funding can be applied
to transport research and innovation outside Horizon 2020:
Connecting Europe Facility (CEF)
European Fund for Strategic Investment (EFSI)
European Structural and Investment Funds (ESIF)
European Local ENergyAssistance (ELENA)
Coherence with other EU funding programmes is considered to be extremely important and
necessary to close the innovation divide between member states and help lagging areas. In the
COUNCIL DECISION of 3 December 2013 establishing the specific programme implementing Horizon
2020 - the Framework Programme for Research and Innovation (2014-2020)101 it is described as:
101
http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32013D0743&from=EN
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 98
F. Widening participation
The research and innovation potential of the Member States, despite some recent
convergence, remains very different, with large gaps between "innovation leaders" and
"modest innovators". Activities shall help close the research and innovation divide in Europe
by promoting synergies with the European Structural and Investment Funds (ESI Funds) and
also by specific measures to unlock excellence in low performing research, development and
innovation (RDI) regions, thereby widening participation in Horizon 2020 and contributing to
the realisation of the ERA.
The 2012 EC’s Communication “Research and innovation for Europe’s future mobility - Developing a
European transport-technology strategy” also makes reference to other EU funding programmes,
such as funding from the Trans-European Transport Network programme (Connecting Europe
Facility), Cohesion Fund and European Regional Development Fund (ERDF)102:
“Funding from the Trans-European Transport Network programme (Connecting Europe
Facility), Cohesion Fund and European Regional Development Fund (ERDF) supports market
uptake and deployment. The Programme for the Competitiveness of Enterprises and SMEs
was also specifically mentioned. The EU also fulfils an important role of coordination through
a variety of instruments and partnerships, such as European Technology Platforms and Joint
Technology Initiatives, and is a forerunner in proposing broad political commitments and
regulatory requirements. So far, in FP7, the Marie Curie Actions have awarded €43.5 million
to transport-related research, providing attractive career development opportunities to
researchers”.
It is therefore important that the Horizon 2020 transport combines, in a proper way, excellence and
leadership and at the same time helps to reduce inequalities at EU level. As a recent analysis of the
participation of European universities in FP7 highlights103: “Overall, FP7 EU support compensated to
some extent for the wide disparity in total R&D but risk of diverging trajectories: need for more
inclusive Framework Programmes”.
An analysis of synergies between Horizon 2020 Transport and the Cohesion policy 2014-2020104 was
also prepared in 2013 by an expert group from the FP7 Transport Programme Committee. The report
identified synergies between the Cohesion policy thematic priorities and issued a series of
recommendations for optimally exploiting them.
As part as this interim assessment a survey among project coordinators was run and included the
question of whether complementarities and synergies are necessary with other public support
102
Brussels, 13.9.2012.COM(2012) 501 final. COMMUNICATION FROM THE COMMISSION TO THE COUNCIL
AND THE EUROPEAN PARLIAMENT. Research and innovation for Europe's future mobility - Developing a
European transport-technology strategy.
http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2012:0501:FIN:en:PDF 103
The role and engagement of universities in the Framework Programmes. DG RTD Midday Seminar
Brussels, 28 June 2016. ANDREA CIFFOLILLI. ISMERI EUROPA. Doc made available by the EC to the Expert
Group. 104
http://bookshop.europa.eu/en/synergies-between-the-transport-component-of-Horizon-2020-and-the-
cohesion-policy-2014-2020-pbKI0214072/
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 99
initiatives such as the Connecting Europe Facility (CEF), European Structural and Investment Funds
(ESIFs), Regional Research and Innovation Strategies for Smart Specialisation (RIS3), Smart Cities
European Innovation Partnership EIP, European Investment Bank’s European Fund for Strategic
Investments (EFSI), LIFE programme. The large majority of participants in the survey indicated that
these types of complementarities and synergies were necessary, as the following table shows.
Table 19: Are complementarities and synergies with other public support initiatives necessary (CEF,
ESIFs, RIS3, EIPs, EFSI, LIFE…)? Analysis of the answers to the project coordinator survey
0 (not necessary
at all)
1 2 3 4 5
(totally necessary)
I do not know
+ No answer
Connecting Europe
Facility (CEF)
0% 0% 2% 6% 19% 19% 55%
European Structural and
Investment Funds (ESIFs)
0% 0% 4% 7% 17% 22% 50%
Research and Innovation
Strategies for Smart
Specialisation (RIS3)
0% 2% 0% 4% 17% 19% 59%
Smart Cities EIP 0% 2% 0% 6% 11% 22% 59%
EFSI 0% 2% 0% 2% 13% 15% 69%
LIFE 0% 2% 0% 2% 20% 17% 58%
According to the EIB, the transport sector has benefited from around 6% of the EFSI investments105.
This represents, in opinion of the Expert Group, a small percentage of the total, considering the large
cost of some transport projects.
Generic references in this regard, such as the sentence in the Work Programme 2014-2015
“innovation activities linked with other EU funding mechanisms such as cohesion and regional funds
should be considered”, may not be enough. Much more specific is the 2016-2017 Work Programme
that reads in one of its sections:
“In the urban mobility domain, strong attention is paid to user-driven innovation, deployment
of innovations by a topic linking with the Connecting Europe Facility of sustainable urban
mobility solutions and technical advice (topic “MG-4.3-2017: Innovative approaches for
integrating urban nodes in the TEN-T core network corridors”), and by a topic that supports
capacity building of local authorities through the ELENA (“European Local ENergy
Assistance”) facility The ELENA Facility topic is in line with the European Council's support for
the Commission's and the EIB's intention to strengthen technical assistance to projects at the
European level”.
Additionally, the infrastructure topics section in this Work Programme states that projects aiming at
a fast implementation of results, should demonstrate their readiness for timely deployment. They
could then be considered for further support under the EU complementary schemes available at the
105
www.eib.org/efsi
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 100
moment of project conclusion (e.g. follow-up of CEF). The following is an excellent example of the
use of CEF funds in order to support deployment of transport solutions106:
“The EU's TEN-T Programme will co-fund with almost €5 million a study and a pilot
deployment of 200 charging points for electric vehicles on the main French highways. The
project will contribute to the development of charging infrastructure and enable a wider use
of electric transport in Europe.”
The majority of the Transport Programme Committee members and technology platform
representatives interviewed considered that the level of information on opportunities for the
continuation of Horizon 2020 Transport projects with external sources of funding (either for further
research or for exploitation of resources) that finally reaches potential proposers is still insufficient.
Several Transport Programme Committee members reported that their countries (either at national
or at regional level) have put in place coordination mechanisms with European level initiatives,
although it is not clear whether this is the case for all European countries. However, during the
interviews requests to provide examples of projects that had been successful in combining different
funding sources for different phases did not yield any clear results. Therefore, there seems to be a
clear opportunity to explain better the whole landscape of transport research and innovation
opportunities: ERC, EIT, KETs, FIT, Public Procurement, EIB’s funds (ESIF, ELENA), EFSI, Cohesion
Funds, CEF, venture capital and so on.
The lack of information referred to by the interviewees should be put under the light of the large
amount of existing information on this issue that, for some reason, does not yet fully reach the
research community. Examples of dissemination activities are:
- EC’s website about synergies between Horizon 2020 and the European Structural and
Investment Funds107
- National or regional seminars 108,109
- Presentations during the Horizon 2020 info days…
A clear example of the need to connect Horizon 2020 Transport activities and national funding, for
instance use CEF or EIB funds, is provided in one of the interviews conducted as part of this interim
assessment: “the states or regions must perform the right investments for implementing e.g. better
connections between transport modes, otherwise nothing will happen and intermodality will not
occur”.
Links between Clean Sky and national and regional activities supported by CEF (as reported by the
JU) provide an excellent example of fostering complementarities between EU funding programmes.
Synergies between Horizon 2020 Transport and the regions’ interests (including smart
specialization RIS3) are a key element for Clean Sky.
Each region has to consider their best forces and where they want to specialize. For example
106
https://ec.europa.eu/inea/en/news-events/newsroom/200-charging-points-electric-vehicles-open-france-
eu-support 107
https://ec.europa.eu/research/regions/index.cfm?pg=synergies 108
https://era.gv.at/object/document/1297/attach/WS_ESIF_HORIZON_2020_13_Feb_2014.pdf 109
http://wire2015.eu/en/side-events/workshop-on-horizon-2020-structural-funds-synergy
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 101
there are about twenty regions that have identified aerospace as their specialization; others
are targeting transverse areas that are highly relevant for aviation such as material,
embedded software. Rather than having regions investing separately, and supporting their
individual regional initiatives, it is essential to connect to common programs, projects at
European level.
CSJU signed 11 MoUs and have another 10 in the pipeline mostly with regions but also,
according to the level where the structural funds are managed, states, (Romania, Czech
Republic and Portugal).
There are pilot projects already started with regions that are complementary to Clean Sky
and are consistent with the relevant region priorities.
Clean Sky may give input to the region for defining its programme or its call(s) to avoid
duplication of work and to perform complementary activities beyond CS
Clean Sky is also developing a kind of “seal of excellence”, like the SME instrument, for
second-ranked proposals provided that they have been positively scored and that they
represent a clear added value to the programme.
In all cases the purpose of the JU is to actively involve the beneficiary of the regional funding
with the Clean Sky network; it should not be limited to adhering to a nice “label” or “seal”. It
is a technical, practical, concrete approach.
Another very promising excellent example of action designed to promote synergies between Horizon
2020 transport and other financing instruments is the “Seal of Excellence”. As the Horizon 2020
Monitoring Report 2014 explains it110:
“The Horizon 2020 “SME instrument” has been selected for the introduction of the “Seal of
Excellence” because of its relevance to regional and national funders, as the project proposals
are mostly led by a single SME and address small-scale R&I actions close to the market with a
clear territorial impact. Regions/Member States interested in funding these types of
proposals could use European Structural and Investment Funds (ESIF) resources (in line with
ESIF priorities and in compliance with national and relevant EU rules) or their own
national/regional resources to grant funding without carrying out an additional qualitative
evaluation”.
On the other hand during the outreach held for this interim assessment, a SME highlighted that
despite having the “Seal of Excellence’ it was still very difficult to transfer this to any other funding
opportunities and there was extensive effort required again to apply for other ‘pots of money’ that in
the end meant that the SME was taking up higher and higher financial risks for less gains.
In total, a large number of stakeholders and the Expert Group think that it would be desirable that
research support instruments have the same rules so that projects that have had success, but not to
the level of funding, can more easily apply for funding from other instruments without having to start
from scratch. This concept was broadly supported.
110
http://ec.europa.eu/newsroom/Horizon2020/document.cfm?doc_id=15108
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 102
7.2.2 COHERENCE WITH OTHER PUBLIC SUPPORT INITIATIVES AT REGIONAL,
NATIONAL AND INTERNATIONAL LEVEL
As part as the interim assessment survey, project coordinators were asked whether in the project
coordinator’s country, and based on his or her experience as researcher, Horizon 2020 transport
research programme was influencing their national research programmes. The majority of
participants in the survey indicated that there was a positive influence of Horizon 2020 transport in
the national research programmes.
Table 20: Influence of Horizon 2020 Transport on national programmes
Very
negatively
Negatively No
influence
Positively Very
positively
I don’t
know
Is Horizon 2020 Transport
influencing national
research activities?
0% 2% 24% 52% 7% 15%
Two of the four Transport Programme Committee members (including one former participant in the
committee) interviewed for this interim assessment also made reference to the positive influence of
the transport components of Horizon 2020 and their national programmes. The national
representative of a small country indicated that even if there was not a dedicated transport research
programme in that country, some of the Horizon 2020 Transport priorities made its way into the
general national research programme. A representative from a large to medium sized country
indicated that Horizon 2020 Transport had contributed to align and concentrate national priorities in
transport research with those of Horizon 2020 Transport. Finally, a national representative from a
large European country referred to a long-standing cooperation with others in transport research
that could be a good example for other countries, and was helped by the underpinning of Horizon
2020 and SC4. However, some Transport NCPs stated that sometimes they feel quite unable to help
advise participants about any calls beyond Horizon 2020 Transport that could provide cross-cutting
opportunities.
In a recent survey conducted on behalf of the European Commission as part of the general Horizon
2020 Interim Assessment exercise, project coordinators were asked whether they had sought
additional or follow-up funding when developing their project and/or before being granted Horizon
2020 funding111. The following table compares the general responses for the entire Horizon 2020 and
for the Transport challenge.
Table 21: Share of beneficiaries who sought additional or follow-up funding when developing their
project and before being granted Horizon 2020 funding (by funding source).
Own funding of
project partners
Public national
/regional schemes
Other EU
programmes
Private/industrial
sources
Transport (n=96) 55.3% 35.1% 16.7% 45.4%
Horizon 2020
(n=963)
51.6% 45.5% 26.4% 39.2%
111
PPMI. Framework contract for the provision of services to the commission in the field of evaluation of
research and innovation programmes and policies (2012/s 144-240132).Overview of horizon 2020 survey
results. Internal European Commission document made available for the purposes of this interim assessment.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 103
For the whole of Horizon 2020, around half of survey respondents (52%) sought their own funding
and a substantial share (39%) also considered private/industrial sources as a potential source of
funding. In the case of transport, a slightly higher share of respondents sought their own funding
(55%) and other private and industrial sources (45%). From this information, transport is the Horizon
2020 area with highest positive responses in this regard, indicating the importance of Horizon 2020
to transport research. On the contrary, the proportion of transport respondents that sought other
public national and regional schemes or other EU programmes was considerably lower.
As the authors of this survey state, “in line with our expectations the absolute majority of the
beneficiaries expect to secure further funding for their projects in the future”. General Horizon 2020
figures and those of transport are rather similar in this case. This is shown in the next table.
Table 22: Share of beneficiaries who expect to secure additional R&D funding for their projects in the
future (by funding source)
Own funding of
project partners
Public national
/regional schemes
Other EU
programmes
Private/industrial
sources
Transport (n=96) 74.2% 74.5% 77.2% 72.4%
Horizon 2020
(n=963) 77.1% 78.1% 82.5% 72.4%
As part as this interim assessment a survey among project coordinators was run and included the
question of whether the project was using previous research from other EU-funded projects or from
other national or regional projects. The majority of participants in the survey indicated that there
was previous support to their research activities.
Table 23: Is the project using previous research from other EU-funded projects or from other national
or regional projects? Analysis of the answers to the project coordinator survey.
From other previously EU funded projects 28% From other previously EU funded projects + From previously funded national research 6% From other previously EU funded projects + From previously funded national research + From previously funded regional research + From research previously conducted at internal level by the participants
13%
From other previously EU funded projects + From previously funded national research +From research previously conducted at internal level by the participants
19%
From other previously EU funded projects + From research previously conducted at internal level by the participants
2%
From previously funded national research + From previously funded regional research + From research previously conducted at internal level by the participants
2%
From research previously conducted at internal level by the participants 9% From previously funded national research + From research previously conducted at internal level by the participants 6% From other previously EU funded projects + From previously funded regional research + From research previously conducted at internal level by the participants 2% Other 6% Not answered 9% Total general 100%
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 104
A recent analysis of the participation of the industries in FP7112 highlighted that “Framework
Programs (FPs) had had a relatively significant impact on European RPOs in several ways. In
particular, FPs have had a relevant impact on the national research policies, leading to the
harmonisation across EU MS both in terms of procedures and in terms of thematic areas of research”.
7.3 LESSONS LEARNT/AREAS FOR IMPROVEMENT
Overall the structure and goals of the instruments are seen as being coherent with the ambition of
Horizon 2020 Transport and the stated objectives of the Union for transport research. However in
some areas improvement in the implementation and how the instruments work both individually and
in complement to each other can be improved.
The Horizon 2020 Monitoring Report 2014 published in the spring of 2016 offers a general overview
of the first round of calls launched during 2014113. It covers all areas of Horizon 2020 and includes a
specific section about the Societal Challenge Transport in Annex 3. It is evident that there are
opportunities for transport researchers and innovators in many other areas of Horizon 2020 beyond
SC4 and transport. In order to obtain a general overview of those, it would be useful to assess what
role transport could play in areas or tools such as European Research Council (ERC), Future and
Emerging Technologies (FET), Marie Sklodowska-Curie Actions, European Research Infrastructures
(RI), Access to Risk Finance (ARF), Fast Track to Innovation (FTI), Space (Galileo applications), NMP,
other Societal Challenges, European Institute of Technology (EIT), SME instrument including the
Small business innovation research for Transport calls and the Seal of Excellence, Spreading
Excellence and Widening Participation (SEWP), Science with and for Society (SWAFS).
The majority of project coordinators that participated in the survey conducted for this interim
assessment indicated that generally there was a positive influence on the national research
programmes. This was corroborated by the Transport Programme Committee members and other
stakeholders who were interviewed for this interim evaluation. Based on the interviews and
exchanges with stakeholders, it is clear that while some countries or regions have put in place
coordination mechanisms with European level initiatives, this is not the case for all European
countries.
Improving the communication with NCPs on opportunities for transport so that they could provide
timely and pertinent advice to potential national participants especially in respect to cross-cutting
opportunities and funding was seen as an area for improvement.
In fact, questions have arisen in this evaluation on whether transport researchers feel they have easy
access to information on funding opportunities beyond Horizon 2020 Transport. These discussions
included some discussion on the extent to which they are interested (or able) to take these up,
especially for transport researchers applying to calls outside of those published specifically on
transport. From the stakeholder hearing, interviews and surveys it seems to be that generally it was
112
An analysis of the role and impact of industry participation in the framework programmes. Under the
"Framework Contract for the provision of services to the Commission in the field of evaluation of research and
innovation programmes and policies".DG RTD Midday Seminar. Brussels, 28 June 2016. Note: doc made
available by the EC to the Expert Group. 113
http://ec.europa.eu/newsroom/Horizon2020/document.cfm?doc_id=15108
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 105
considered not interesting for transport researchers to move out of their usual field of activity. On
one side, participants found difficult to get familiar with the specificities, networks, research agendas
and previous research belonging, for instance, to other Societal Challenges. On the other side, when
considering other programmes outside Horizon 2020, such as the CEF programme, stakeholders
mentioned different processes, financial and reporting rules that they had to make an extra effort
but the gains of doing this were not large enough to be attractive.
In spite of the important efforts made by the Commission in communicating the opportunities for
funding, there is still a lot more to do. A clear opportunity exists to explain better the landscape of
transport research, innovation and funding opportunities that could help coherence across the many
aspects of transport and the various programmes.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 106
8 EU ADDED VALUE
8.1 UNDERSTANDING ADDED VALUE
“Horizon 2020 shall maximise Union added value and impact, focusing on objectives and activities
that cannot be efficiently realised by Member States acting alone. Horizon 2020 shall play a central
role in the delivery of the Europe 2020 strategy for smart, sustainable and inclusive growth ("Europe
2020 strategy") by providing a common strategic framework for the Union's funding of excellent
research and innovation, thus acting as a vehicle for leveraging private and public investment,
creating new job opportunities and ensuring Europe's long-term sustainability, growth, economic
development, social inclusion and industrial competitiveness, as well as addressing societal
challenges across the Union.” (EU, 2013)
“The Union level funding of transport research and innovation will complement Member States'
activities by focusing on activities with a clear European added value. This means that emphasis will
be placed on priority areas that match European policy objectives where a critical mass of effort is
necessary, where Europe-wide, interoperable or multimodal integrated transport solutions can help
remove bottlenecks in the transport system, or where pooling efforts transnationally and making
better use of and effectively disseminating existing research evidence can reduce research investment
risks, pioneer common standards and shorten time to market of research results.” (EC SC, 2013)
8.2 HORIZON 2020 TRANSPORT PROGRAMME DEMONSTRATING EU
ADDED VALUE
From our outreach and analysis there is a common and enthusiastic appreciation of the EU added-
value to transport research under Horizon 2020 (although this is not unique to Horizon 2020 as it was
also the case for the previous Framework programmes). Based on interview material and survey
results among transport stakeholders (PPMI survey and experts survey), overall Horizon 2020 plays a
useful and important role in EU research, boosting the necessity and strategic coordination of
research across modes and is designed to addresses cross-cutting societal challenges (such as SC4).
The added value of the collaborative nature of Horizon 2020 is also thought to deliver more than the
sum of fragmented national programmes. The programme delivers clear contributions to the
excellence of science, the enhancement of scientific reputation and improving scientific quality
(referenced from Horizon 2020 monitoring report and mentioned by 57% PPMI survey respondents).
8.2.1 EUROPEAN INTEGRATION
As the integration of Europe is high on many agendas, it is seen as being correct and of value that
transport research should take place at European rather than only at national levels. This could not
happen with the same scope and involvement of the most relevant players without a programme
such as Horizon 2020. It is clear that one country is unlikely to be a leader in all the areas of research
required to address the major challenges Europe currently faces but, in particular, the key societal
transport challenges (such as SC4) and other cross-cutting issues. The European model works very
well in this respect.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 107
Figure 40: Percentage of answers of interim evaluation survey on the uniqueness of Horizon 2020
It is clear that the added value of knowledge gained from the programmes and projects combined
with the coordination and interactions between SMEs, research organizations, industries and
universities from different member states working together, is high. This is confirmed by nearly 80%
of the survey respondents for this interim evaluation who considered the added value in these terms
as being either very important or extremely important. This is consistent with the research
performed within FP7 where the ex-ante evaluation also reveals strong added value described in
similar terminology (TRIVALUE, 2013).
In particular this value is associated with the research and innovation that could, or would, not have
been funded at national level. The PPMI survey for Transport Societal Challenge (SC4) shows that
about half respondents (49%) indicated that the project would not have gone ahead at all without it,
while 44% indicate that the project would have gone ahead but with significant modifications. For
the projects that would have gone ahead within SC4, the data shows that the scope would have been
narrower (68%), 77% state that the timeframe would have been longer and with fewer partners
(51%), underpinning a clear added value of the SC4 Programme. Without Horizon 2020, 73% of PPMI
responders (SC4) indicated that their international relationships and networks would also have been
affected.
By involving a wide variety of players there are gains in European cross-border cooperation,
knowledge sharing, and clear synergies and cross-fertilization of ideas brought together under the
common objective. This helps to achieve research excellence at European and international levels. It
is felt that the focus on commercialising outputs tends to undervalue this collaborative aspect of
European Research, which is valued very highly by the participants (from PPMI survey, survey for this
interim evaluation and was especially mentioned in the interviews with stakeholders and stakeholder
hearing).
In addition, from the same sources, there is perceived added value gained from projects where there
is collaboration between market players who are competitors under normal circumstances. It would
be extremely unlikely for them to work together if the project was not under a European
programme, and it is also felt that this is aspect is under acknowledged by the Commission. This
allows those that do not normally share perspectives, approaches and knowledge to do so. There are
multiple benefits from this as the research community gets a better understanding of commercial
imperatives and bottlenecks; and commercial players are able to understand requirements and
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 108
positions for non-commercial ‘pure’ research to be undertaken. The input from stakeholders to this
interim evaluation indicated that this aspect is much appreciated by those taking part, especially
when the research is ‘upstream’ of commercialisation (i.e. in the pre-competitive stages). Risks and
costs, as well as knowledge, can be shared. It is also recognised (by the same players) that there are
sensitive areas when it comes to research outputs that touch on competitive aspects and ownership,
which are not always entirely comfortable between players (from interviews with stakeholders).
8.2.2 INCREASING EUROPEAN COMPETITIVENESS
In terms of impact on competitiveness and European leadership, 80% of the respondents to the
survey indicated medium to much added value in terms of improving market positions or
competitiveness. A large number (93%) of SC4 respondents to the PPMI survey indicated the
programme made a significant contribution to leadership and enabling industrial technologies to be
brought to market. Areas where Europe is perceived to be in a leadership position e.g. in aviation or
automotive, should receive continued support to keep this position as there is strong competition
from out of Europe e.g. from China, Korea or USA. However the focus of this support should be
directed towards taking these industries away from fossil fuel and helping get them onto low carbon
and more inclusive pathways.
Demonstrator projects and pilot projects undertaken by multi-players and participation of various
countries are seen as adding particularly high value. In the majority of cases it would not be possible
to conduct such trials at national level as there are not enough different players at national level nor
the funds to do so. The Horizon 2020 survey (PPMI, 2016, p. 29) records large-scale demonstration,
prototypes and testing, for new or improved commercial products and developing business models
with the highest levels of added value (more than 55% of Horizon 2020 responders).
There are also clear synergies in terms of access to larger markets and new business opportunities.
Projects implemented through national funds, as seen to have reduced competitive positions
internationally (from the same survey (PPMI SC4) 79% responded that they expected a decrease in
this aspect) as well as a decrease in access to new markets (69%). These responses can be taken as
evidence of EU added value in respect to national or regional funded programmes/projects.
Another important positive aspect of the PPPs (JUs) is the integration of several research projects
under a single programme. This has helped to promote multi-stakeholder developments and
practices, an important cooperation towards the continuation of Europe’s cutting edge technologies.
It is considered that this helps to bring stability/continuity of research and the ability to build in
flexibility.
Horizon 2020 projects also help participants to share risks, which is seen to bring competitive gains.
72% of the responders to the interim evaluation survey (question 32) indicated between medium to
much added value in risk reduction, and this was confirmed in both stakeholder hearing and
interviews. Member States may be able to do this on a national level but the Horizon 2020 scale is
significantly larger.
In terms of impact, the following table shows the contribution of expected research outputs if
implemented).
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 109
Table 24: Expected research outputs from SC4 Horizon 2020 projects: answers from survey among
project coordinators.
Contribution to Number
of
references
References /
No of
projects
Decarbonising and "greening" the transport system 39 72%
Increasing efficiency of the whole transport system 37 69%
Improving safety & security of passengers, aircraft, vehicles and vessels,
and infrastructures
22 41%
Strengthening the competitiveness of European industry 42 78%
Pioneering the Transport of the future (long term perspective) 20 37%
Enhancing and strengthening the European Research Area (ERA) 26 48%
Globalisation, external dimensions and international cooperation 17 31%
Further development and implementation of EU transport policy 25 46%
As has been mentioned, Horizon 2020 transport has been timely for preserving the competitiveness
of some subsectors. It was particularly timely for the automotive sector as many research topics
were threatened with the economic downturn (preservation of jobs and recruitment in a context of
financial crisis, EGVI, 2016). The new structures and opportunities provided by Horizon 2020 have
helped to preserve highly qualified jobs and skills in the R&D automotive value chain e.g. with the
PPP European Green Cars Initiative (EGVI, 2016). This is a clear validation of the ambition to fulfil the
Junker plan and the competitive aspect of Horizon 2020.
In terms of addressing Europe’s Societal Challenges, especially SC4, it should not be forgotten that
jobs need to be created in non-fossil fuel based industries as well and especially jobs in the
automotive industry need to be sustainable, i.e. relevant to future transport helping shift the burden
of change that this subsector must make. The negative externalities caused by the automotive
industries, especially the loss of productivity from congestion estimated at around 2% of Europe’s
GDP, as well as the societal impacts of loss of life and injury from road traffic accidents and air quality
issues, need to be balanced by the positive aspects of job retention or creation within this sector.
Well-directed research can help Europe prepare for this shift.
8.2.3 EUROPE’S ROLE IN STANDARDISATION AND HARMONISATION
Playing a role in developing international standards is an important role of the EC and its research
programme in terms of leverage of transport research, although this aspect may not be always be
visible, it is seen to be of crucial importance to the industry and for the deployment of innovative
solutions.
Interoperability, which is still considered a challenge especially in the rail sector would be even worse
without Horizon 2020 (and previous FPs). This is being delivered by improving technical standards
(TSIs), which have been worked on since 2002. While it is recognised there is some progress, there
are still considerable bottlenecks and challenges to making this a reality for both passenger and
freight services. Research has helped deliver some solutions but much of the new research will be
delivered via Shift2Rail (S2R). From our interviews Shift2Rail was heavily criticised for having a
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 110
structure that, in real terms, almost excludes the research community from being able to take part in
substantive research unless they have support from inside S2R114. This community expressed this
view quite strongly in interviews. It was also commented on that it feels excluded from being able to
help guide research due to the structure of this PPP. Interoperability is also seen as being a key topic
of interest to deliver many of Europe’s policy objectives such as the introduction of alternative fuels,
e-mobility, integrated passenger ticketing between multiple modes, constraining conventionally
fuelled vehicles in urban areas, cross border freight to mention a few.
Standardisation is also a key issue in terms of materials, components and alternative fuel standards
including electromobility (plugs and charging infrastructure) and needs to also address many societal
aspects in respect to inclusiveness, such as levels of accessibility (e.g. for citizens who are less mobile,
older, etc.). Research needs to deliver the evidence base for policy development and this often
requires managing conflicting aspects and delivering recommendations based on validated
information and data. An example includes the dangers of silent electric vehicles, seen as being a
solution to noise levels and pollution, but this lack of noise is a danger to those who are
audio/visually impaired as they can neither see nor hear them. This work would transcend national
capacities and fulfils the role of research by a supranational body such as the Union. The ART
(Autonomous Driving) call is another positive example of where European led research adds value.
Indeed without the Union promoting a choice of solutions that sets a level playing field for the
industry to respond with new products and services that are interoperable across all Member States,
it will be difficult if not impossible to achieve the inclusive, low carbon and fossil fuel free transport
future that is desired and an objective of SC4 and Horizon 2020.
8.2.4 BUILDING STRONGER EUROPE-WIDE RESEARCH COMMUNITIES AND LEVEL OF
EXPERTISE
The role of the EC in reducing gaps in research competences and knowledge across Europe is also
very relevant. On the one hand, collaboration between the wide variety of players in the research
community is seen as being rather positive as it encourages knowledge sharing as previously
mentioned. On the other hand the current trend of focussing on developing research based on
excellence is not seen as being helpful to reduce disparities. Indeed it is sometimes considered to be
counterproductive. For instance, it becomes more difficult to enter consortium bidding for proposals
if you are not a strong player, creating vicious circles where strong players are oversubscribed and no
one wants to associate themselves with weaker players (who in turn are therefore not able to learn
and develop and become strong players). This view was shared by a number of interviewees.
Therefore the Expert Group would like to highlight this as a risk rather than validated evidence of the
present situation. Comments such as “we were surprised when xyz project was funded as we did not
think it was a strong consortium” were frequent enough for the Expert Group to take note. To some
extent this underpins the opinions of many that the project evaluations are not easy to understand
114
It should be noted that despite several attempts to reach out to the rails sector the Expert Group did not get
spontaneous or enthusiastic responses to participate in this review, so these comments are based on
comments from those outside of the rail sector. However it is also the opinion of the Expert Group from their
observations that the isolation of the rail sector from other transport modes is a symptom of many potential
problems within the rail sector.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 111
(although the EG is also conscious of few requests for re-evaluation115). There are other instruments,
such as structural funds, that were mentioned as having stronger leverage in terms of reducing
disparities.
Half of the respondents in the survey said that if the research funding had been sourced from national
or regional funds, the following areas would have been affected:
Less mobility of researchers (qualitative data inputs from interviews and 53% percentage of
PPMI SC4 respondents),
Decreased ability to provide funds for PhD’s or the capability to provide training (qualitative
data interviews and 59% percentage of PPMI SC4),
Less career development of researchers (qualitative data inputs from interviews, 50%
percentage of PPMI SC4 respondents)
Diminished capacity to attract researchers and other staff (71% of PPMI SC4 respondents)
This, combined with the results from the figure below show that there is both perceived and real
added value to European funding for transport research and that this plays a strong role in creating
strong, multi-discipline Europe - wide research teams, the Expert Group is not aware of any other
region in the world that can boast this for 28 countries.
Figure 41: Percentage of answers indicating the numbers of research projects that would not have
implemented without EU funding (57% positive responses).
115
The Expert Group would like to highlight that the low figure for revaluation should not be automatically
interpreted as a high level of satisfaction with the process. It was mentioned on numerous occasions (especially
in interviews and at the stakeholder hearing) that it is not felt to be useful to complain as the budget has by
then usually been allocated and there would not be any likelihood for a project to be re-evaluated and funded.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 112
8.2.5 DELIVERING RESEARCH THAT FEEDS INTO EUROPEAN POLICY OBJECTIVES
The PPMI Horizon 2020 survey provides evidence showing a strong relation between transport and
other societal challenges such as: SC1 Health and demographics change and wellbeing (26% of PPMI
SC4 respondents); SC3 Secure, clean and efficient energy (39% of PPMI SC4 respondents), SC5
Climate action, environment, resource efficiency and raw materials (62% of PPMI SC4 respondents),
SC6 Inclusive, innovative and reflective societies (29% of PPMI SC4 respondents). One of the most
noticeable evolutions within Horizon 2020 is the aspect of multimodality or intermodality.
An example includes electrification, which may be applied through different areas, cross mobility and
handling of big data, activities within Horizon 2020 but not addressed in the previous FP. However,
based on responses and in the opinion of the Expert Group there is a need to improve coordination
between the various societal challenges. From responses gathered during this process, it is
concluded that overall the calls associated with addressing them tend to focus on the industrial
needs of particular industries and not necessarily on the transport needs. This opinion was validated
by interviews and other exchanges articulated, in some cases, as the desire to see improved
cooperation and coordination across sectors and units within the Commission. It should be noted
that this was also requested in the EU President’s letter to Commissioner Moedas mentioned earlier
in this report.
8.2.6 LINKING CALLS AND TOPICS MORE DIRECTLY WITH POLICY EXPECTATIONS
While recognising that to some extent it is difficult for projects and project officers to address what is
expected from them in terms of policy impacts, the inputs from the various sources in this evaluation
suggest that there is insufficient feedback on results and policy feedback. This was mentioned in
various ways in a number of the interviews and also in informal feedback. It should be noted from
the low score on this point from respondents to the internet survey this was not seen as a priority for
outputs.
Substantiating policy impact was not easy for this evaluation as some projects are followed up by the
EC while other projects are monitored by INEA. This is seen as a parallel effort (interviews with
stakeholders). The Expert Group could not find any clear guidance or understanding on how policy
monitoring is happening, and harvesting policy relevant data also remained unsatisfactory. In FP7,
projects were clearly disconnected from each other, but now in Horizon 2020 links are made through
“collaboration agreements”, which is seen as a positive development but it is thought that the
process in place may not be robust enough.
Many interviewees mentioned that links between policy objectives and topics are not always clear, in
particular with less-prescriptive topics, and participants also see this as a ‘grey’ area. This was also
brought up in the stakeholder hearing. The perception of those respondents and the Expert Group is
that few projects are properly evaluated post project completion.
However, the Expert Group agreed that it is too early to know about the specific project outcomes of
Horizon 2020 and give a definitive answer concerning the coordination of policies in Europe, so the
Expert Group were not able to establish any clear policy feedback loops. Nonetheless, concerning
‘contribution’ to EU transport policy objectives, 56% of respondents to the survey consider the
European added value between important to very important but from the results of the Expert
Group’s survey, fewer make the explicit connection.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 113
8.3 SUCCESS OF DIFFERENT INSTRUMENTS TO DELIVER EU ADDED VALUE
8.3.1 ERA-NET
The potential of ERA-NET and ERA-NET Plus to deliver added value mainly relies in the collaboration
and alignment of national and EU research and innovation activities (this observation is based on
interview material):
Both of them represent useful instruments to guide member state’s research priority areas
and are seen as being complementary to the rest of Horizon 2020 instruments. A good
example is the Infrastructure Innovation Programme116, with nine projects launched in 2015,
and initiative that includes pilots, and if successful, the industry will exploit results.
It would appear that the concept of these instruments is seen as being intrinsically good but
the implementation is not well thought out and overall they may not respond to market
needs as they are not easily and widely taken up. Many interviewees mentioned that in their
view they do not work well, and to date a limited amount of funds have been mobilized from
this instrument. Stakeholders find the evaluation and rules of participation barriers to
participation.
The low level of participation appears due, on the one hand, to a lack of interest from industries and
the small scale ERA-NET and, on the other, how the instrument relies mainly on richer countries co-
funding. From input to this process, ERA-NET Plus schemes are less attractive to smaller and less
wealthy member states. 92% (50 out of 54) respondents to the Expert Group survey have not
considered to apply to the ERA-NET programme and only one proposal was eligible and retained.
8.3.2 CEF
Connecting Europe Facility (CEF) delivers European added value by supporting the deployment of
promising solutions that have been validated within transport research funded activities e.g. pilot
demonstrators or prototypes. However in the Internet survey 55% of respondents answered that
they did not know if CEF was a good facility, and only 38% said that it was either very or necessary to
have. The following observations are based on interview material:
It is seen as a good instrument to ensure deployment of promising solutions identified by
research and innovation activities. SESAR connects to deployment through CEF funding and
Shift2Rail also plans to use it. Another example found concerned companies taking initiatives
based on successful research solutions (after demonstration and are able to build business
cases), such as the case of Heathrow airport. Here research was used to improve the airport
capacity in certain meteorological conditions and it was able to build a business case and
deployed the solutions from specific SESAR developments with good results (ACARE, SESAR,
2015 and two interviewees).
8.3.3 PPPS AND JUS
The role of the PPPs and JUs can boost European added value but this relies on the connection of
research and innovation to industrial strategies and access to the market, especially from the outputs
of demonstrators. These multiplayer projects usually involve Europe’s top players and best expertise,
116
Infravation, www.infravation.net
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 114
thus the resulting quality could not be achieved within one country. This section reflects insights
from consolidated interview material.
Overall, the variety of instruments is acknowledged as being important and the responses are mostly
positive. The most frequently mentioned pros and cons and recommendations taken from a variety
of sources to the Expert Group are shown in the table below.
Positive Negative
- Stability of funds (even during budgetary
cuts) allowing achievement of goals and
tangible results (such as the implementation
of roadmaps)
- Alignment and coherence between industry
and society’s interest. SESAR is put forward
as an example of the combination of
industrial interests and benefits for the
citizens.
- The market impact of PPPs is faster and
larger due to the better focus on the direct
R&D needs of industry. Industry will not
invest in products that are not marketable.
- The rate of success in calls is higher in the
PPPs. That is both due to the choice of
themes (close to market) and the more
‘cooperative’ procedure.
- PPPs/JUs generate a strong community of
research actors around one area. They
cluster resources and talent around a well-
identified topic.
- All projects are related and they are few
isolated actions. Otherwise fragmented
research is integrated, so it ends up working
in a coherent fashion
- The process for being able to launch one of
these initiatives is very long and roadmap
objectives tend to be broad, influenced by
the input from all the actors and “need” to
satisfy them all.
- If the roadmaps are not updated this can
lead to wasted efforts
- Some PPPs or JUs should be more
transparent in terms of participation, and call
preparation.
- Information to Member States do not reach
the Transport Programme Committee.
- Coherence and linking across transport
initiatives is still insufficient.
Areas of improvement that might not apply to all PPPs and JUs
- The topic should be narrow enough so that the (successful) progress of the results is able to
justify the investment. (Note from Expert Group: topics are considered to be already narrow
within some PPPs/JUs but this could possibly be improved as some JUs are more mature than
others.)
- The results to be achieved need to be in line with the available funding and ambition needs to be
tailored to what is feasible rather than what is desired.
- PPPs should have a larger leverage effect on contributions from industry and this should always
be transparent.
- Stakeholders capable of responding to the challenge should be in the lead while, at the same
time, leaving the door open to new participants, (including those that are not actual members of
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 115
the JUs). All stakeholders must be on board in an active and inclusive way. (Note from Expert
Group: Results are a little patchy as in some JUs, but success stories showed good involvement of
SMEs and new comers.)
- PPPs and JUs should concentrate on higher TRLs and more fundamental (blue sky) research
should still be tackled in the normal calls. Full coordination should exist to avoid overlapping
topics both in the PPPs/JUs and in the normal calls.
- Simultaneously, they should provide room for bottom-up proposals and approaches as well as for
longer-term visions.
- DG RTD and DG MOVE should cooperate more intensively in all the PPP and JUs and the
development of research agendas as input on research gaps is given to them rather than directly
to JUs.
- The Member States Representative Group in all PPPs should be more active in connecting
national research activities with those in Horizon 2020 (it is noted that Clean Sky is making
notable progresses in this direction).
- It is suggested to deepen synergies between PPPs and the SME Instrument.
The number of PPP/JUs initiatives is already very large and caution should be taken when considering
new ones. This should only be done after a clear gap analysis of needs. SESAR represents a particular
case where policy, benefits for the citizens and competitiveness come together. It represents the link
between industry and ATM sector and its process is seen as being excellent. It operates under a
Master Plan adopted by Member States that includes reviews every two years (content and future
deployments). SESAR also connects research results to deployment (including feedback on how
deployment is progressing). The JU supports the European Commission in their international contacts
on aviation and ATM issues and this close working is seen as delivering added value.
8.4 OTHER ISSUES RELATED TO EU ADDED VALUE
Concrete evidence that research and innovation outcomes will be exploited or built upon is part of
adding value to research investments. In Horizon 2020, the emphasis on exploitation and
deployment was more than in FP7. This is appreciated by the majority of those consulted most of the
interviewees, although some find this link difficult to determine at proposal stage, and it is also
expected that Horizon 2020 projects will lead to increased impacts, although there is no evidence of
this yet.
There are a number of examples from previous framework programmes that have been
implemented such as can the next generation of aircraft as shown in figure below. It is also clear that
these technologies and developments achieved at European level within Clean Sky, would not have
taken place without a Europe wide programme.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 116
Figure 42: Success stories – Aviation Research and Innovation (ACARE, 2016)
Under FP5, FP6 and FP7 research addressing the safe reduction of wake turbulence separation
minima has resulted in aircraft separation schemes have been initiated at two European mayor
airports (London Heathrow and Paris Charles de Gaulle). This has contributes to reduction of flight
delays (up to 50% of which are generated by adverse weather) and an increase of runway throughput
in the order of 5% without compromising safety levels for these operations. In the road transport,
examples include: Crash Test Dummies, results from this project are used by car industry to design
safer cars; Personal Rapid Transport enabled by the CityMobil project implementing 20 pods that run
automatically as illustrated in figure below and others noted in the ERTRAC publication (2016). The
European Bus System of the Future (EBSF) aimed to develop a new generation of urban bus systems
adapted to the needs of European Cities and this project made a major step forward in bringing all
the relevant actors together from industry and operations, who were at that time not used to
working together. A positive outcome has been the take up of the outcomes of this project in the
design of bus drivers’ compartments, which is now widely replicated in the buses built today.
Figure 43: Success stories from road transport (ERTRAC, 2016): THORAX – Thoracic Injury assessment
for improved vehicle safety and CityMobil PRT system was implemented by BAA (British Airport
Authorities).
The “Oceans of Tomorrow” initiative was mentioned in some interviews as it helped to coordinate
maritime research (EU, 2014) but it has since been discontinued. This was noted with
disappointment by some. Other good examples from this sector include the development of
biosensor technologies for environmental monitoring (FP7 EnviGuard ongoing) and examination of
aspects of the wind farm life cycle (LEANWIND, FP7)
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 117
Figure 44: EnviGuard and Leanwind – Oceans of Tomorrow (EU, 2014)
8.5 LESSONS LEARNT AND AREAS FOR IMPROVEMENT
There is evidence in the data collected that most of the research projects implemented could not
have been performed outside the programme. Working at European level has provided greater scope
and scale especially to many projects, making them more relevant to tackling the SC4 challenge and
create more impact, than would otherwise have occurred.
However, there is a need to improve the understanding of linking outputs to policy development and
feedback, as well as improve learning exchanges and the transfer of knowledge from project outputs
to increase impact. Current work concerning the STRIA process (under development) is appreciated
and is seen as a positive development, addressing the current perceived silo approach in some areas
of the transport programme. It is expected to address these needs (as expressed by stakeholders) for
a coordinated and strategic long term, holistic vision for transport.
Despite the generally high level of satisfaction in terms of European added value, several areas
remain for which improvements were suggested by those interviewed or surveyed.
8.5.1 INCREASED INTEGRATION
- There is little or no incentive to deliver integrated solutions beyond Horizon 2020 projects or
for PPPs to work with other transport modes to develop integrated approaches and
solutions. There is a need to rethink how this could be achieved, as the cross-fertilisation of
ideas across sectors is not sufficient and there is high potential for this to increase added
value.
- Strengthening research areas that are considered important, but that are currently less
developed, is seen as an area of attention. Comments made during interviews imply that this
may be because these topics are linked to certain interest groups or due to historical
reasons. Despite divergent views from the interviews on how to include certain topics, which
are seen as being possibly sensitive, the Expert Group would like to encourage the
Commission to reinforce taking a system wide approach, looking especially at how transport
modes can and should work together, including (inter alia) logistics, integrating with ITS
research (e.g. under C-ITS), sustainable infrastructure, and including more activities to
promote the benefit of the softer modes of transport (walking and cycling) to frame its
research agenda. In particular this refers to socio economic and behaviour areas of research.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 118
This opinion is substantiated based on responses especially from the interviews and the
discussions at the stakeholder hearing. It also refers to the comment on the vertical
approach to addressing the societal challenges mentioned earlier117.
- There appear to be few incentives to address the need for interdisciplinary research. The EC
itself is not seen to be well organised to support or enhance interdisciplinary research –
leading to the question of how integration can be achieved without this.
- It is seen that Europe needs to play a very strong role in facing global challenges such as
increased automation and standardisation (examples include electric mobility, alternative
fuels, emissions testing standards and ITS). The added value of Europe acting as one entity in
such areas carry weight in the global market and applying research outputs should put
Europe in the lead, but presently this pathway is not secure.
8.5.2 FORMAL EVALUATION AND FOLLOW-UP OF PROJECTS
- From the feedback, the Expert Group concludes that the evaluation of finalised projects
needs further attention, especially since there is little incentive to take any outputs beyond
the final project presentation. Several interviewees expressed frustration that the
Commission/Project Officer did not attend the final conference or presentation of final
results. In addition, there were numerous comments about the duplication of research
topics that had been treated in previous framework programmes. The proper value of these
projects was not seen to be fully harvested (diminished added value), as project results are
being recorded in ways that do not allow anyone who had not worked on the project to fully
understand the potential of the output and the present (listed as a deliverable in databases
with no reference to outputs).
- A loss of domain expertise and a focus on project management was cited frequently as being
a real threat. There is a significant reduction in the content review from project officers and
the Commission is seen as being resource stretched. This results in a focus on process rather
than content, examples of this include prioritising administrative aspects rather than project
R&D content.
- It is felt that the TRIP118 portal is focussed rather on quantity than quality, despite certain
improvements and which to some extent is understandable as those managing the portal are
not responsible for the quality of the documentation but rather that deliverables are
uploaded. However it is difficult to access meaningful information on a project via this site
once it has finished. If one knows the name of the project, searches can work well, but if for
instance a local authority might be interested to know what is available on urban road pricing
there are not enough parameters that would allow to extract meaningful research that could
be used for policy or programme development. In other words too many ‘hits’ are presented
and it would be very time consuming to wade through this information to find relevant
examples.
117
See also section on internal coherence. 118
The TRIP portal is the Transport Research Innovation Portal formerly known as the Transport Research
Knowledge Centre (TRKC) - to collect, structure, analyses and disseminate the results of EU supported transport
research and research financed nationally I the European Research Area (ERA) as well as selected global
research programmes (www.transport-research.info)
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 119
8.5.3 FUTURE RESEARCH DIRECTIONS ORGANIZING FOR INNOVATION
- Gaps are also noted in the follow-up of on implementation of research and innovations
(Interview material from stakeholders representatives from aviation, road and maritime).
These are most apparent in the follow-up from demonstrator and pilot projects to full-scale
applications. It is felt by the Expert Group that this is due (partly) to the lack of support for
project coordinators and participants to take successes to market. Many good products are
not able to become mainstream, sometimes expressed as ‘overcoming the valley of death’,
as they cannot access the necessary funding levels easily for furthering research. The effort
to try to get further funding is beyond available resources. This is seen as a bottleneck for
innovation, as it is thought (and confirmed in interviews) that there are many small projects
and demonstrators that could be taken further or that might be relevant to other sectors (an
example is an environmental friendly seat based on cane sugar fibre). Solutions to take
forward high potential outputs could be better considered within this programme.
- Some subsectors feel that they are lagging behind in investment and attention (based on
interview responses). For example, it was mentioned that the maritime sector the EC
currently has a limited role, less than desirable. This could (and should) be much larger and a
better cooperation with River Committees and IMO could engender this. The relation
between different functions of maritime are not sufficiently considered e.g. fishery, seismic
activities, LNG as new fuel. Based on interview material and Expert Group analysis: a PPP in
the maritime sector could possibly present an opportunity to integrate policy research in a
broad programme, overcoming the silos that naturally exist in the sector – the willingness
and interest of the sector for this was reinforced at the stakeholder hearing.
- Overall remarks were made by enough interviewees to be noted by the Expert Group that
current research appears to have a too narrow a focus. There is a definite bias towards
encouraging technological easily quantified solutions to the detriment of socio economic and
behaviour research (linked to the KPIs of patents and registered outputs). It is felt that this
misses opportunities and creates research gaps in areas of multimodality, linking modes
together and activity areas within a transport journey (stations were mentioned in this
respect but it is not thought that this is the only example), in other words having a more
focussed systems approach.
8.5.4 DISSEMINATION AND EXPLOITATION ASPECTS
The growing attention for dissemination is overall seen as positive. The survey of Horizon 2020
projects indicated that without Horizon 2020 funding, participation in scientific conferences,
seminars or workshops and other outreach activities would have been significantly reduced (63% of
participants to PPMI SC4 Survey). Specific activities targeting policy makers (58% of participants of
PPMI SC4 Survey) would have been less and fewer publications in referred journals and books (56%
of participants to PPMI SC4 Survey).
At the end of 2015 the EC conducted a survey among FP7 Transport project participants with the aim
of analysing the exploitation of transport R&I outcomes119. In the introductory section it is explained
119
Exploitation of transport R&I results: report on a survey of FP7 participants. June 2015. EUROPEAN
COMMISSION, DIRECTORATE-GENERAL FOR RESEARCH & INNOVATION. Directorate H – Transport H.1 –
Strategy. Note: internal EC doc.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 120
that an overall strategy for the dissemination and exploitation of project results is currently
developed for Horizon 2020 with the aim to:
- Increasing the availability of project outputs (results/data)
- Describing a process for the feed-back of results into policy making
- Supporting the exploitation of Horizon 2020 research results at project level
The aim of the FP7 Transport Results Survey was to identify project results at (individual) participant
level and related needs for their dissemination exploitation. In this document the results of the FP7
transport survey are presented and some conclusions drawn, some of which can be considered to be
of interest to the dissemination of Horizon 2020 projects (noting that the majority of them are not
completed at this time).
Dissemination should be understood differently from communication. While the first is directly
towards peers and researches, communication (a two way process) should address more actively the
policy makers and the general public. Most projects still rely on traditional methods of dissemination
– newsletter, conferences, and a website. These tools do not always reach the desired target
audience today as people (especially decision makers and those in charge of policy development or
implementation at national and local levels) are inundated with such requests by email. The lack of
knowledge about good projects beyond those involved directly or indirectly with the projects is seen
as being a result of this. Therefore, in terms of added value, dissemination actions with impact (for
both FP7 and Horizon 2020 so far) are seen as being insufficient even within the research
community.
There are some good examples of dissemination such as the Clean Sky Forum, the Green Aviation
Conference, projects represented in aeronautics conferences, and most especially the Transport
Research Arena (TRA), which is used as a key opportunity for dissemination (www.traconference.eu).
SESAR and ETRAC have also showcased results from projects together with a catalogue of solutions.
Such catalogues open a possibility to the European industry and service providers in providing
solutions outside Europe and for examples to be easily sourced and referenced.
There have also been recent publications Research Theme Analysis Reports (e.g. on Urban Mobility)
based on input from the TRIP portal. These are looked at positively as extracting information from
this portal is not easy. The search fields are quite limited and adding more layers of search or having
a more performing search engine use could help people find the information that they want more
quickly. Although the cost of printing is always high people still have a mild preference for paper
copies of reference documentation but electronic versions should also be easy to find on the
Internet. As these are spread across many websites, this is not always the case.
It is suggested that the EC should ensure that proper dissemination partners are included in the
projects, as this is not seen to be a priority of partners (besides academic papers and publications). In
addition researchers are not communication specialists and they are comfortable using channels of
communication for research purposes, which may not be the best ones for dissemination for
exploitation purposes. Events where prototypes of already available or if research outcomes can be
demonstrated that they might benefit from a more formal EU communication service where experts
are available, even if this facility only provided guidance. The EC has an effective communications
machine that could be very useful to provide such support to researchers. .
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 121
From the Internet-based survey conducted by the expert group, suitable actions to support the
dissemination and exploitation of transport results include:
- Brokerage events including the EU information days and those hosted at major transport
events, such as the TRA conference120, commercial conferences and other transport relevant
events offer FP7 and future Horizon 2020 project participants (and owners of IPRs/results)
the opportunity to establish business contacts with potential users, buyers, and funders of
downstream development.
- Ad-hoc initiatives address specific needs, in particular those of SMEs. The 'Transport SMEs
Innovation Day' provide SMEs the opportunity to be briefed on financing possibilities e.g.
EASME, RSSF etc., and to have possible meetings with venture capitalists/risk financers and
this is welcomed.
- Continuous monitoring and analysis of transport research projects and related results (FP7
& Horizon 2020 projects) via surveys on research outcomes at individual project level and
also the analysis at aggregated level collected into achievements reports also help to get the
word out.
- Clustering activities (as they occur at TRA and AERODAYS) were mentioned on several
occasions as being underutilized. These events are seen as being ideal forums for the
dissemination of Horizon 2020 results as they cluster several research and innovation
initiatives as well as allowing researchers to meet and exchange ideas (often leading to
innovative solutions being developed). The TRA is seen especially as being a very useful
event for the many areas of transport research and it is useful for dissemination beyond the
research community that the proceedings are made publically available via Elsevier and
Science Direct121. However the audience is often the research communities.
Suggestions for improvement include:
- Level of dissemination is still artificial, limited and not creative. It is not easy for researchers
to become communication specialists or for them to fully grasp expectations and the EC
could provide help in this area from its own communication specialists.
- Dissemination should focus on a broader community of researchers, consultants, city
authorities, transport operators and other decision makers (peer-to-peer). However,
dissemination towards the general public is not seen as being very relevant for Horizon 2020
projects.
- Too many projects use traditional means such as websites, conferences and newsletters,
considered innovative ten years ago. Today people are inundated with requests to attend
meetings/workshops and conferences and without budget to travel it is difficult to attract
the target audiences mentioned above. The use of social media is not widely used and could
be an efficient and effective channel for dissemination. New ways and channels can be
encouraged.
120
www.traconference.eu 121
www.sciencedirect.com/science/journal/23521465/14
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 122
- The use of questionnaires and Internet surveys is also not seen to be as effective as it was.
This was confirmed by the experience of the Expert Group who would have liked to have
more responses to their survey but stakeholders are frequently requested to take part in
surveys and their time for such activities is limited.
- For effective dissemination of outputs, there is a need to balance a holistic vision on the one
hand, emphasising integral approaches to knowledge and, on the other, translating this into
accessible, transparent and comprehensible language for wider audiences. It is not felt that
this is optimal yet.
Overall Horizon 2020 is seen as adding a lot of value to both transport research and to achieving
European Union objectives. The collaborative approach to investigating transport and societal
challenges and the funding opportunities available with Horizon 2020 are seen as being both ‘first in
class’ in Europe but also globally, as the Expert Group is not aware of any other research programme
of this size and scope that can be compared to Horizon 2020.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 123
9 SUCCESS STORIES FROM PREVIOUS FRAMEWORK
PROGRAMMES This has been covered in the text with a number of examples that can be developed further
Please provide three examples of success stories based on an assessment of FP7 projects whose
outcomes/results have had a significant added value and/or impact on the society and/or the
economy and which have outstanding EU added value using the same criteria as in the previous
section: effectiveness, efficiency and synergy.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 124
10 LESSONS LEARNT / CONCLUSIONS Overall from the information gathered by the Expert group there is quite broad support for Horizon
2020 from all stakeholders, including industry (private sector), member state governments, civil
society and the research community. Recognising that transport is unique in terms of its societal
value, economic and environmental impacts, and the need for coordination of research in this field,
there is definite support for a new Framework Programme post Horizon 2020.
Looking more closely at the five criteria set out by the Commission to frame this work, and
considering the Societal Challenge 4 as requested, the Expert Group puts forward the following that
can be considered as lessons learnt and suggestions for improvements in a new round of research
activities for transport.
The Expert Group has structured this chapter to show what in their opinion is working relatively well;
what is not and how this might be addressed. These observations are validated by the various
interactions with the stakeholders (interviews, survey results, data provided, the stakeholder hearing
and the opinions of the Expert Group based on their experience and the information reviewed as
part of this exercise). The recommendations and conclusions set out at the end of the chapter refer
to Horizon 2020 and the SC4 challenge in general unless specified otherwise.
10.1 RELEVANCE
Generally transport research under Horizon 2020 is seen as being extremely relevant for European
policy, environmental sustainability, society at large, competitiveness and boosting the European
economy. It becomes more and more relevant to international challenges such as security, reducing
carbon emissions and the fight against climate change. Its relevance during the current economic
crisis is also clear and this was especially noted for the automotive sector. It is also important in
terms of making funding for research available and in the way it has helped to identify research
priorities and to focus available resources at national level.
Horizon 2020 was mostly based from the policy perspective on the Transport White Paper and as
long as that is still pursued and relevant, so is Horizon 2020. From the technology point of view,
Horizon 2020 Transport is largely based on the contributions of the European Technology Platforms,
among other actors. There are clear links between Horizon 2020 and President Junker’s priorities and
the general objectives of Horizon 2020 transport can be clearly mapped against Juncker’s top
priorities122.
The Expert Group agrees that there has been progress compared to other framework programmes
and the Horizon 2020 research agenda is helping to achieve a Resource Efficient Europe in terms of
technology, energy transition and to some extent the transformation of transport systems. There is
definite support for the EC’s efforts to direct research towards concrete results, deployment and thus
to increasing the impact on solving the transport societal challenge(s) and there is full support for
how the programme is promoting excellence in science.
122
http://ec.europa.eu/priorities/index_en
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 125
The process of defining research themes and work programmes is sound and well developed.
Relevance of the work programmes is notably assured by reflecting new developments in the
definition of new calls: a relevant example being the ART call in WP 2016-2017 which put vehicle
automation on the research agenda in parallel to innovative developments in the industry.
Some stakeholders are more involved in Horizon 2020 compared to FP7.Examples include end users
and cities. This is seen as being very positive because it facilitates innovation. Large players are
already fully involved and smaller players (especially those who have experience of FP7 or Horizon
2020) have opportunities to take part. Direct contact between the private sector and end users is
also of seen as being of vital importance to the success of the programme, and there has been
notable progress to involve them in Horizon 2020 compared to FP7.
PPPs and ETPs also fulfil positive roles in achieving the Horizon 2020 goals by bringing together
different interests through their multi-actor composition (industry, research providers, governments,
urban interests, etc.). Their role in defining and agreeing medium and long-term visions, strategies
and roadmaps is also seen as being highly relevant.
The general goals of Horizon 2020 are clear for stakeholders and there is not much doubt that
Horizon 2020 as a whole is pursuing the right objectives overall.
Areas for improvement
In general, and based on interviews, hearings and our own expert opinion, it can be stated that the
challenge of Horizon 2020 Transport to address the broad number of important issues and to deliver
excellence and impactful research and development outcomes under one programme is large.
The Expert Group identified a number of areas where improvements can be considered. These focus
primarily on improving communication between the European Commission (high level) objectives
and the concrete project expectations.
1. Reconciling different objectives
While Horizon 2020 is considered to be on the right track to realise its objectives, there is discussion
if Horizon 2020’s accent on impact and deployment (direct economic objectives) sometimes
overshadows the objectives of fundamental research. New topics need to be able to be brought into
calls quicker and innovation should be valued higher.
Although the overarching goals of the programme are clear, it is more difficult for stakeholders to
establish clear links between the high-level policy objectives and particular project objectives.
International cooperation in Horizon 2020 is seen as being more restricted but more strategic, when
compared to FP7, yet the relevance of SC4 in particular is seen as being highly relevant
internationally. ‘Open to the World’ and international cooperation activities are already included in
the work programmes but some countries are more important strategically for transport than others
(e.g. USA). International cooperation, on the other side, is a strategy issue with always increasingly
importance.
It is recognised that the free flow of information is essential within projects, but this is not yet always
seen as being optimal. Too much openness reinforces doubts about conflict of interests. This was
especially noted in relation to international exchanges, as EU and national industrial interests and
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 126
competitiveness on technology-oriented research in transport has a direct relevance on competition
within the European market and outside (for instance, new Asian companies entering the rail sector).
Perhaps the main instrument for realising open innovation is through networks of participants both
in general calls and in PPPs while, at the same time, protecting confidentiality.
A number of interviewees and other stakeholders questioned whether the societal challenges in
broad transport work programmes cover too many aspects to be properly treated under the current
programme. They suggested that it might be better to concentrate (or at least focus extra efforts) on
where the biggest gains can be obtained in terms of achieving the general objectives and added
value. Some respondents referred to the option to focus on market failures (those areas that actors
are not eager to tackle but that are vital for the society and the achievement of the policy goals).
2 Improve and clarify the links between overarching objectives and research calls
The process to develop the work programmes is mature and efforts are continuously being made to
strike the right balance between competitiveness and policy (although this is not yet fully
convergent. Stakeholders do not always find it easy to link the high level objectives to the specific
calls. It is felt that although this may be clear within the EC, but the intended participants are not
aware of these internally agreed objectives and the links upstream could be made clearer. The new
STRIA could fill this gap by producing roadmaps and to more precisely make the links between
research and design pathways to achieve the overarching objectives.
For example, researchers do not always get a clear message of what is expected from them in some
specific topics, in particular those that are less prescriptive, in respect to policy linkages. How the
objectives regarding Open Science, Openness to the world etc. are linked back to the decision making
process is not clear. It is suggested that progress towards objectives should be measured at
programme level and not only at project level. In other words, to explain how project X reduction in
pedestrian fatalities in the elderly age group adds up to the reduction of project Y in the risk of
severe head injuries of cyclists and to the improvement in frontal crash avoidance in rural roads of
project Z in order to reach the general goal of reduction fatalities in the EU by 50% in the next
decade. A simple but effective system could be developed with inputs from stakeholders on how this
might be achieved.
3 Complementarity and continuity of research calls
The overall opinion of the Expert Group is that individually projects yield relevant results: “as
generally there are no bad projects”. Several experienced participants (who may have been involved
with earlier framework programmes) are concerned that they see calls coming up that resemble
work done in previous projects reinforcing incremental rather than innovative research outputs.
More emphasis should be put on robust and simple structures and tools to maximise the use of
previous research work (a catalogue of research results from projects and a how they are been used
and evolved by later projects, for example).
Stakeholders also felt that the sum of the projects (per call and in total) does not yet yield a
sufficiently integrated approach to provide the answer to the particular societal challenges of SC4. It
is suggested that the project selection process is not focussed enough to form consistent packages of
projects that are complementary and this could be improved. The Expert Group supports this line of
thinking.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 127
New emerging issues (as already mentioned by the TAG, in the Scoping Paper and brought up during
the public consultation) need to be introduced earlier in the preparation of the next round of calls.
This was also mentioned in the White Transport Review 2016. In addition some issues identified by
the above sources are still receiving limited attention in the work programmes (softer modes of
transport were mentioned).
4 Preparation of work programmes and call texts
The majority of stakeholders understand (and appreciate) the process for consultation and input into
the work programmes but they are not always satisfied with the traceability of the contributions in
the final proposal texts. This point is also argued by some members of the Transport Programme
Committee. The impression is that the advice given by the different advisory bodies and stakeholders
are put in a “black box” from which emerges a work programme and calls, and it is only when this is
published that they may see (or not) if their suggestions have been taken up. As the text is published
it is too late to modify this in any way. More explanation on how and why certain choices were made
would be highly welcomed and would improve relationships.
5 Enhanced work programme content flexibility
Many stakeholders keep insisting on the rapid emergence of new research areas and that currently it
may take several years until these topics find their way into the calls. This was mentioned in the TAG
report in 2014 and in its follow-up report submitted to the European Commission (May 2016). This
group also stressed the need to continue with a disruptive rather than incremental approach to
research in order to respond rapidly to shifting transport paradigms with new revolutionary
technologies, business environment and mobility patterns. These views were widely supported by
other stakeholders taking part in this interim evaluation and the Expert Group thinks that this directly
influences relevance as results may arrive too late for market developments. There is a need to find a
better balance to address research topics, which were not initially foreseen.
This could be guaranteed by the introduction of “quick-reaction” small or focused calls via
amendments of WPs or open topics. Transport Programme Committee members should be open to
reserve a small budget for this sort of calls, utilizing express written procedures (a mechanism
frequently used in between scheduled Transport Programme Committee meetings) to expedite the
content approval or by directly delegating to the EC the possibility to define calls of limited scope
without the need of them being included in the biennial work programmes.
6 International cooperation
Caution not to open up research results too much, in view of competition, was frequently mentioned
as an issue. Precompetitive research in coordination with international partners in areas such as
automation is welcomed, but there is a fine line between early precompetitive research and when
such actions enter the competitive zone.
However, there is room for further improvement in:
• Increasing the focus of international cooperation on removing international regulatory obstacles
in large markets for instance with US or Asia.
• Enhancing the deployment of European solutions internationally in countries like China and USA
as currently dissemination seems to be mostly limited to the EU area.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 128
Of particular international relevance is the link between transport research and climate where
European transport research has excellent credentials and also in connection with road safety.
10.2 EFFECTIVENESS
The effectiveness chapter looked into the intended and unintended effects on the programme and
the extent that the Societal Challenge SC4 caused the observed changes or effects. As previously
mentioned these observations are based on the premise of similarities between FP7 and Horizon
2020 project outputs.
When measuring effectiveness in terms of market readiness of research results, there is a positive
development as the output of research shows higher TRL levels especially in relation to the outputs
of research from PPPs.
There are well-defined ambition levels for increasing the effectiveness of results, which are
appreciated by participants. It is also felt that this is an important area to keep in focus especially if
research is to help keep Europe competitive and to protect, maintain and create jobs. It is therefore
clear that the expected impact of the research (proposals) is high and there is nothing to indicate
that this will not be the case.
The Expert Group also found that Horizon 2020 is effective in helping many countries and industries
in the transport sector to overcome the economic down turn. Three areas were mentioned
consistently:
• Ensuring that transport research as such was being addressed by funding important research
topics
• Helping to maintain and create highly skilled job opportunities where Europe is in a leading
position
• Helping countries to increase the effectiveness of member states national and regional research
systems (for instance, through being able to set priorities more effectively against European
research opportunities). This is of particular interest with lower income member states.
Areas for improvement
1 Transition from FP7 to Horizon 2020
Effectiveness is partly defined by communication about the research process. The transition from FP7
to HORIZON 2020 may have been underestimated. The changes are much bigger than they appeared
and the estimated 18 months may have been too little to adapt to the new way of working.
Communication on these changes as well as on the broader relationships between the EC objectives
and the work programmes and project selection should have taken a higher priority.
2 Technology Readiness Levels
Although Horizon 2020 is definitely on track to deliver higher end TRLs, there is some concern about
the relatively smaller attention to research with lower TRL outcomes. Effectiveness should not only
to be measured in terms of quantitative results (e.g. % reduction of CO2). Creating and maintaining
an innovative research ‘ecosystem’ should also be acknowledged as an effectiveness objective of
Horizon 2020 (especially as that is a generally recognised added value of EU intervention). The accent
on quantitative targets can lead to pseudo accurateness. The value of learning (as mentioned in the
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 129
chapter as ‘the right to fail’) as such seems to be underrated compared to FP7. This presents a
concern that elements intrinsic to delivering excellence in research are either being, or are at risk of
being lost in the implementation of this new system.
There is a need to improve the transition to deployment beyond traditional dissemination and
exploitation activities and this is seen as one of the big challenges in Horizon 2020. Horizon 2020
funded research should not only project transport futures, but also seek to influence them in various
ways. More effort should be put in market readiness through diffusion and involving targeted clients.
From this investigation there is some concern about the relative attention to research with lower TRL
outcomes, which may provide valuable transitions in the short and medium term.
3 Links between socio-economic and behavioural research
The fourth objective of the programme “Socio-economic and behavioural research and forward
looking activities for policy making” appears to be under developed. Long term objectives on socio-
economic and behavioural research and forward looking activities for policy making is underfunded
and not given enough exposure.
10.3 EFFICIENCY
Out of the five criteria, efficiency evoked much discussion among stakeholders and there are
diverging views. Overall it is agreed that Horizon 2020 is clearly being managed and run more
efficiently than its predecessor. There are a number of improvements that have been implemented
and which are generally appreciated by stakeholders, they are seen to have increased the efficient
delivery of Horizon 2020 projects. For example, INEA is seen to be doing its job in quite an efficient
and effective manner. Time between project approval and grant has decreased and the management
of the projects has been standardised.
Although this evaluation did not evaluate the PPPs, they were considered where they touched on the
delivery of the research objectives of SC4. Overall the PPPs are considered by the participants to be a
very efficient (and effective) way to allocate funds to projects and they increase focus on a variety of
otherwise fragmented research topics.
Areas for improvement
1. Rules and procedures
Although the improvements compared to FP7 are recognized it is still felt that the rules governing
calls are too many and complicated. This is passable for specialists and large institutes but for those
only occasionally participating or for small companies, it constitutes a major barrier.
2. Expectations and terminology of calls
The openness (or broadness as it is often called) of the call topics is not generally considered an
improvement. Although this approach attracts a larger number of interested parties and invites more
responses than a more focussed text, often the smaller, more specific research areas are lost in final
proposals. It is widely felt that as Horizon 2020 call texts are so broad, encompassing many different
possible research proposals, it is also excessively difficult to evaluate and compare proposals
properly.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 130
This also leads to oversubscription on some topics. Here, there is a risk that excellent researchers
become discouraged by high oversubscription rates and low possibility of achieving project funding.
Confidence in the evaluators is certainly not unanimous. Many experienced and knowledgeable
evaluators are not allowed because of the conflict of interest clause which is handled very
stringently, leaving the evaluation in the hands of what are perceived as less competent evaluators.
Feedback from INEA into the policy directorates is not yet standardised. There are few possibilities
for INEA to challenge unclear topic descriptions or make suggestions in respect to how they might be
evaluated in the calls pre-publication and the feedback from the selection process should be
improved so participants understand more clearly what will be evaluated.
3. Two stage process
There are definitely negative feelings about the two-stage approach in its present form. Although the
intention of simplification is appreciated, in order to develop a good proposal the effort required for
these types of proposals is seen as being almost the double. Some would therefore want to go back
to single stage, others would prefer the first stage to be simplified considerably, e.g. by reducing the
number of proposing consortia to a minimum at a much earlier stage.
The Expert Group suggests that some improvements can be put in place in the near future and there
is no need to wait for the next framework programme.
4. Non-negotiation
The role of negotiations about project proposals has dramatically changed since FP7. Some consider
this an improvement as this diminishes the influence of the EC policy officials in the proposal phase.
Others deem this a clear deterioration because of diminished flexibility when submitting what is
really needed. It would be helpful to revisit the whole decision making chain from objective to
project to identify opportunities to improve the project and to make necessary changes or
modification identified during proposal evaluation or negotiation phase.
10.4 COHERENCE
10.4.1 INTERNAL COHERENCE
The coordination and mix of different actions of Horizon 2020 Transport (CSA, RIA, IA, conferences,
etc.) should provide a coherent approach to address the objectives of SC4 and Horizon 2020, and
especially create synergies between research fields and topics. The internal coherence of the
programme in terms of available instruments has noticeably improved and there is a general support
of the current mix of instruments. In particular, the new SME instrument gets very positive feedback
and the variety of PPPs are acknowledged to be important.
Several other challenges faced by the Union are relevant for transport and represent an opportunity
for the sector. In particular the link with the Energy Societal Challenge is evident. There are also clear
links between Transport and several KET.
On the whole, the structure, goals and instruments of Horizon 2020 are seen as being internally
coherent and coherent with the ambition and objectives of the Union.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 131
Areas for improvement
1. Coherence of instruments and types of actions
Although smaller than in FP7, the number of instruments is still quite large, and caution should be
taken when considering new ones. This should continuously be kept in mind, with a view to
simplification. Several actors refer to difficulties to understand the difference between the different
instruments, such as a RIA and an IA. Information and definitions should be more precise and
clearer. New instruments should only be acceptable after a clear gap analysis and under the principle
of “one new comes in, two old go out”.
Industry participation in some CSAs is still seen as being insufficient, particularly when they are used
to monitor progress, prepare new technologies or roadmaps.
The public procurement of innovative solutions is still underexploited, in particular the results of
large-scale pilots. Some instruments are either underused or unanswered within Horizon 2020
transport. For example the take up of ERA-NET is not yet widespread and while researchers are
rather interested in CSAs, it seems that the level of interest of (national or regional) public
administrations is not yet equivalent.
2. Relationships between PPPs and other SC4 instruments
Despite the general appreciation of PPPs, it is considered that some are working better than others.
It is suggested to increase coordination among PPPs and JUs by integrating their work programmes
more closely, and learning from those that are now quite mature. Care to ensure that there is the
opportunity for the research community to take part in research with the JTUs is also key (rail and
the Shift2Rail PPP was mentioned here in particular as the research community considers itself
excluded from influencing research topics, calls and to some extend even responding to the limited
number of calls that are made outside of the PPP).
The Expert Group would like to recommend to increase efforts to ensure that modal ‘research silos’
are not created and that the value of an integrated approach, in respect to responding to the
objectives of SC4, and to making Horizon 2020 coherent with transport being smart, green and
integrated, is promoted more forcefully.
3. Combined funding opportunities
Existing links between transport and other societal challenges are not always translated into
combined funding opportunities from the different challenges and instruments associated with
them. The need for more joint calls has been proposed on several occasions (for instance in the
maritime domain combining fishing and transport issues) but this is not yet widespread enough. It is
also thought that deepening combined funding opportunities would also help respond more fully to
the encouragement for ‘new ways of working’ suggested by the Juncker Commission (Open
Innovation, Open Science and the Presidents letter to the Commissioner).
In spite the existence of internal coordination mechanisms within the EC, such as the Challenge Task
Forces, more coordination inside the EC is frequently mentioned. In particular, it is recognized that
efforts have been made within the EC services to institute cross-cutting/transversal themes and
related coordination structures which are successful in a number of fields, notably the relation
between SC4 and SC5, it is felt that this could go further.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 132
Increased transversal coordination is suggested to be encouraged within advisory groups and ETPs,
since feedback from stakeholders suggests that they are also thinking “vertically” within this
transport challenge.
It is suggested that FP9 increasingly gathers efforts from all DGs into key societal challenges. Once
these overarching objectives are defined, contributions from the various DGs could be combined into
a really coordinated research and innovation programme.
4. Role of advisory groups
It is felt that some advisory groups are oriented towards direct interests and specific research areas,
and less towards the usefulness of the sum of the total of research outputs, supporting a silo
approach. Technology platforms are an example of this in some cases. It is suggested that they could
produce roadmaps that address more than one societal challenge and this could reinforce coherence
between the various research topics affecting transport (for instance, it is difficult for ACARE SRIA to
influence other work programmes outside transport).
The Transport Programme Committee could provide support to participants in terms of facilitating
information on national and regional opportunities in the transport research and innovation arena.
5. Communication as an instrument for coherence
Although the Participant’s Portal represents a notable improvement, it is still difficult for many
participants to identify all opportunity for transport innovation. Despite efforts to simplify access it
was still considered by many stakeholders to be too complicated – differing terminologies and
generally there is still progress to be made in simplifying and clarifying processes and procedures.
Researchers still mention that they find it difficult to identify and take advantage of all opportunities
(outside Pillar III). Improving the clarity of information on opportunities for transport in the ERC, FET,
KETs etc. would be welcomed by many. More information was also requested regarding new
instruments such as the Open Research Data Pilot of the Fast Track to Innovation (FTI) calls.
There are difficulties in properly communicating the number and scope of programmes to the
researchers. Those that are used to participating in European projects are seen as having an
advantage, over new comers. It is suggested that it would be useful to have a ‘map’ of programmes
to help interested parties “find their way” or navigate to those programmes that respond best to
their interests. It would also be welcomed if the work programmes would be more specific about
opportunities in other areas of Horizon 2020. A dedicated section in the work programme is
suggested to centralize information on other internal opportunities for funding within Horizon 2020.
Some NCPs stated that sometimes they feel unprepared for changes to systems and that they do not
receive updated and regular communications from the Commission to help advise them about any
calls that could provide cross-cutting opportunities. The Expert Group felt that this could easily be
addressed by the Commission.
10.4.2 EXTERNAL COHERENCE
The landscape of related programmes to SC4 is clearly very large and having more opportunities is
seen as being positive, as long as there is coordination and overlapping activities are avoided.
Structural Funds play a key role to help more actors and regions become part of the Horizon 2020
top-league (particularly through areas such as research infrastructures and research
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 133
capabilities).Cohesion Funds are also seen as being important for European regions and the same can
be said for RIS3.A Spanish region for instance, has successfully utilized Cohesion Funds in connection
with one of the Clean Sky calls and SESAR represents a good example when it comes to using CEF
funds for the deployment of new technologies.
Horizon 2020 is clearly guiding national priorities, in particular in smaller countries and even in those
where there is not a specific transport research and innovation programme. Clean Sky and Shift2Rail
are good examples of this. New cooperative arrangements such as MOUs between PPPs and member
states are emerging and expectations are high that they will also accelerate research take up and fill
any gaps that are not presently covered by Horizon 2020.The MoU between Clean Sky and several
European countries or regions in the Czech Republic, Spain is one example.
Member States also use European Technology Platforms roadmaps and agendas (such as, for
instance, the aeronautics SRIA) as guidance for their programmes. In this regard, there is certain level
of coordination and also coherence in objectives.
Areas for improvement
1 Coherence with other EU funding instruments
Structural Funds and Cohesion Funds help to bring in more actors and regions, however, the
coordination of Horizon 2020 research with such funding sources in the design stage (and later) is
perceived to be complicated because of the different rules. In fact, it was not possible to identify
projects that explicitly present at the beginning how they plan to combine different funding sources
until they reach the market or the solutions are deployed or reach the society. There are still legal
barriers among different DGs or EC initiatives that hinder the combination of funds (for instance
funds from DG Regio and funds from CEF).
Although sometimes awareness and knowledge of external partners is limited, respondents show a
keen interest in the involvement of external parties, especially in finding other financing sources to
assure continuity of implementation of solutions after the completion of research. External partners
mentioned were financial institutions, industrial partners as well as governments. On the other hand,
research partners are slightly wary of too large such involvement at an early stage as this may
influence the independence of research and introduce too many political elements into the research
proposals and compromise the research itself.
Generally it is extremely difficult for participants (in particular for industry participants) to be experts
in the requirements of all opportunities in all related areas: Horizon 2020, Structural Funds, EIB,
especially as there are different financial rules in the different funding programmes. Organizations
and companies currently need to have dedicated experts in all the instruments in order to take
advantage of the opportunities.
2 Coherence with regional or national programmes
There is still a need for better communication and better coordination concerning national and
Horizon 2020 research agendas and outcomes. The relation with regional funds is still considered
weak. The respondents to the Expert Survey were generally not aware of any structural effort to
involve regional funds in the transport programme and they frequently referred to the complexity
implied in different programmes having different financing rules.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 134
Especially in view of the supported and increasing coordination with national programmes, there is
still a need for better communication and better coordination. The Transport Programme Committee
might provide support to participants in terms of facilitating information on national and regional
opportunities in the transport research and innovation arena
It is thought that more seconded national experts working in the EC, could enrich both sides (EC and
nations and regions).
10.5 ADDED VALUE
This criterion addresses the question of added-value resulting from the EU intervention(s), compared
to what could be achieved by member states at national and/or regional level.
Out of the five criteria, EU added value consistently scored highly and creating and maintaining an
innovative research “ecosystem” is seen as being important. While recognising that the respondents
to interviews, surveys, hearing etc. are all more or less involved professionally with EU research,
which could create a bias in favour of the existence of an EU research programme, it was also noted
that these respondents were sufficiently critical of the programme to provide valuable input. It is
therefore safe to say that this bias does not stand in the way of the conclusions that the added value
of EU funded research is broadly recognised as being very good.
The most frequently mentioned positive reflections in terms of added value are:
• Without EU funding many projects would not have gone ahead (or not in the same scope);
• Increased competitiveness, access to larger markets and new business opportunities;
• Contributions to excellence of science, enhancement of scientific reputation and improving
scientific quality
Horizon 2020 (as well as FP7) is seen as being highly instrumental in creating a strong and connected
European research community, based on open exchange of knowledge and researchers. This is
essential in view of the increasing complexity of the research landscape, new research methods and
the requirement to involve interdisciplinary top-researchers to address Europe’s societal challenges,
especially SC4.
PPPs and JUs are acknowledge as positive instruments in terms of market impact as they address
industry needs and help align research to them. In particular the sharing of costs and risks in the
development of new technologies, as well as, testing and validating multi-player demonstrator and
pilot exercises.
Setting the basis for standardisation (as a follow up of research) is seen as being an important
outcome of European added value. There is also a deep appreciation on the dissemination of results,
the open science approach and opportunities that the EC offers notably conferences like Transport
Research Arena and AERODAYS.
Areas for improvement
The following issues require more attention to improve on the (perceived) added value of the
programme.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 135
1. Increased emphasis on implementation, deployment and cross fertilization of existing research
There is still not enough research and innovation being implemented from FP7 or (to date) Horizon
2020. This includes follow-up from pilots to full scale deployment. Participants expect more support
from the Commission to help take results to the market and to feed results of (small) projects and
demonstrators into other sectors. Even from the Commission side, interest sometimes seems to
wane after project completion.
The cross fertilisation through sharing research findings between transport modes and even other
societal challenges would also add more value. The programme relies too much on participants
themselves looking for external opportunities to exploit project results. Some useful lessons can be
learned from SESAR, on the use of complementary instruments like CEF to support deployment of
successful outcomes.
Possible solutions identified from interview material and Expert Group knowledge include:
• Making more resources available from the Commission to assist following up research and
innovation content and increase exploitation.
• Organising earlier commitment from clients.
2. Increase coverage and equality in research
The programme should encourage participation of new players and/or improve coordination with
other instruments as means to enhancing the impact of the Horizon 2020 Programme and reduce
disparities. Addressing the disparities that exist in transport research has to be in balance with
scientific excellence, gender and skills.
All modes should feel that they are equally covered – it has already been mentioned that softer
modes need more attention. Currently the EC plays a limited role in the maritime sector and it is felt
that this could and should be much larger, with a focus on improving cooperation key players and
bringing multimodality into play more strongly.
3. Future research directions organizing for innovation
Stakeholders identified some potentially fruitful areas where they would like to see action taken,
such as more cooperation and where possible system integration of transport modes (a systems
approach), logistics, ITS, infrastructures, the softer modes of transport (that are much less
organized). In spite of the broader call topics, there seems to be little incentive for interdisciplinary
research. Areas where this is needed include the new forms of digitalisation and automation,
interconnectivity, cyber-security, drones, cloud applications, the interface between technology and
social acceptability, demographic changes such as the recent explosion of migrants into Europe,
changes in mobility behaviours, the sharing economy and unanticipated consequences of new forms
of transport were all mentioned by stakeholders.
4. Delivering research and innovation results that feed European policy
There is a need to improve information to projects concerning expectations in terms of impact.
Better guidance and monitoring mechanisms are required to ensure harvesting projects relevant
data into policy development.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 136
5. Increased involvement with transport relevant players
Increasing the involvement with civil society was also remarked on in previous evaluations and to
some extent this has improved but not yet to the extent that is desirable. There are still many
(possibly too many) traditional players, few representatives of the ‘softer modes’, especially walking,
passenger bodies and civil society organisations that may be only indirectly associated with transport
but are key players in societal issues including SC4.
6. Dissemination
The efforts for increased dissemination via the Horizon 2020 Portal and dissemination through
conferences such as bi-annual Transport Research Arena are well appreciated.
However, there seems to be a need for a clear definition of what dissemination means for projects,
what the programme expectations are and what the criteria are to evaluate the effectiveness of
dissemination.
Improving the added value of project results by better dissemination (especially beyond those
directly involved in the project) beyond the research community and also outside of Europe is seen
as being useful. To some extent this is also associated with exploitation mentioned earlier.
7. Improving the added value of some instruments
Some of the instruments are not easy to access. For example, many respondents were aware of the
ERA-NET facility as a means of cooperating on a European scale and although it is believed to have
many advantages in terms of added value, as it is clear that the motivation to use it lies with the
member states. The drawback of ERA-NET is that it is necessary to have initial funding from a number
of partners before the EC can top up the budget (and then lays down the rules). INFRAVATION was
mentioned frequently as a good example of how ERA-NET may work, with a specific accent on
innovation in the transport infrastructure field. At the same time it is felt that the rules governing
ERA-NET might be revisited to widen the possibility of participation.
10.6 CONCLUSIONS AND RECOMMENDATIONS
This report is clearly structured around the five main criteria that the Commission requested the
Expert Group to consider – namely: Relevance; Coherence; Effectiveness; Efficiency and Added
Value. The Expert Group also chose to look at how Horizon 2020 was responding to certain inward
looking aspects chosen from key high level Commission objectives such as to what extent Horizon
2020 was helping to encourage new collaborative ways of working and cooperation across portfolios
(and linking to the sections looking at coherence). In addition, it touched on considering if the
objectives of the initiatives/ programmes and research topics were clear, to what extent it could
show an increase in results based outcomes (with policy feedbacks) and how modes and sectors
were increasingly integrated in respect to the SC4 challenge.
It was also necessary to make links with a number of key outward facing aspects that would help to
underpin the relevance of the leadership positioning of European research in terms of Europe’s
responses to job creation and competiveness; climate resilience and low carbon economy; the
societal challenges of inclusion and equity (including safe and seamless transport); and the ambition
of the Commission for Open Science; Open Innovation and being Open to the World.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 137
Overall the Expert Group can wholeheartedly say that there is enthusiastic and broad support for EU
research and in particular Horizon 2020, among all stakeholders, from industry to member state
governments and the research world. Its role in advancing cutting edge transport research is
undeniable. There is also definitely support for a new FP.
Horizon 2020 is seen as being of great relevance for EU policy, environmental sustainability, society
and help keep Europe’s economy competitive. It is also seen to have growing relevance in emerging
areas of societal interest such as security, the fight against climate change
The following recommendations are based on the perceptions of stakeholders involved in the
Horizon 2020 programme, the Expert Group findings and expert judgement.
1. Shorten timelines to be able to adapt to change
The programme’s flexibility should be increased to accommodate the sudden emergence of
important topics, at the same time increasing opportunities to link portfolios and funding
opportunities. It is recommended to consider the feasibility of focused “emergency calls” in between
the publication of biennial Work Programmes or, as an alternative, the introduction of a permanent
topic on “emerging issues” in all calls.
2. To consider the focus of the programme
The Commission should consider if focussing the programme on a smaller number of really pressing
transport challenges that can only be tackled on a European level and have the greatest EU added
value has more advantages than disadvantages. At the moment funds are too widely spread over too
many issues for this to take place. Although it is recognised that it is necessary to achieve a balance
between the many different smaller and larger stakeholders, the different modes, industry, policy
and other interests, the current balance of players tends to yield a compromise that is suboptimal for
all parties and is difficult to explain to a critical general public. Increased attention to this within and
between EU institutions and stakeholders would be welcomed.
3. Overcome current bias on technology within the Societal Challenge 4
There is a perceived technology bias to the allocation of funding and social-economic aspects of
transport are seen as being underfunded. There is a focus on higher levels of technology overlooking
possible low cost and accessible solutions that could help bridge a number of gaps and help in the
transition phase especially towards low carbon transport and changing behaviour. The Expert Group
encourages the Commission to further close this gap.
4. Improve the balance between short term and long-term objectives
Especially in respect to socio-economic and behavioural research and forward looking activities for
policy making are seen as being underfunded and this should be addressed in upcoming calls and
programmes. The EC could look 5-10 years ahead and propose research themes that support the
necessary developments (back-casting), rather than the present focus of addressing incremental
research.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 138
5. Increase the opportunities for innovation
The Commission should reconsider the balance between incremental research and visionary or
radical evolution. In respect to the choice of topics for work programmes and calls the advisory and
stakeholder bodies by their input seem to consolidate the traditional choice of topics rather than to
be champions of breakthrough innovation. The current decision making process does not specifically
encourage innovations.
6. Review the progress indicators and what is valued.
Progress towards objectives should be measured not only at project level, or at transport model
level, but also at programme level. For instance, the impact of aviation depends also from the surface
transport part of the trip. In addition consider the value of exchange and the creation of robust
multinational research networks and the knowledge gained from cooperation as a progress indicator,
as this is presently undervalued.
7. Improve the clarity and precision of calls
Improving the transparency of the decision making process and giving clearer indications of how calls
will be evaluated would be welcomed and could also address the low rate of success and over
subscription problems currently encountered. New research policies as intended with Horizon 2020
or priorities such as “Open innovation” must also be better explained and understood by Transport
research participants and there must be enough time to adapt to these changes. It is still worth the
investment in communication for the rest of the Horizon 2020 work programme.
8. Revise and simplify some structures and processes
The following areas would benefit from streamlining and simplification of processes:
• Simplify (access for participants to) existing instruments and actions (rather than creating new
ones) such as ERA-NET
• Revisit the two-stage proposal process taking into account the key problematic areas
• European Technology Platforms should produce roadmaps that include specific targets to be
achieved and ways to control the progress towards them
• Consider turning TPs into Innovation rather than Technology Platforms.
• Consider putting more emphasis in the evaluation criteria on activities that would support the
market introduction and deployment of research findings, and especially to reinforce the links
with policy. This could be achieved by the increased weighting of impact evaluation criteria:
impact on policy, impact on competitiveness, impact of sustainability, impact on economy and
society at large.
9. Improve knowledge sharing especially in the following areas:
• Knowledge of the results of previous projects
• Knowledge about funding opportunities from other sources and improve links with other funding
instruments
• Improve dissemination of results (including tools) outside of the research community
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 139
10. More attention to a system-wide innovative approach
In order to achieve SC4, transport needs to be addressed as a system rather than as a sum of modal
solutions, with a stronger accent on innovation. Possible areas of action include:
• Consider transforming the “European transport corridors” financed by the CEF into “European
transport and innovation corridors” where the development, testing, demonstration and
deployment of transport innovation would acquire a more important role.
• Reinforce the links between EU level Transport research and innovation and national and
regional action.
• Support regional transport research cooperation efforts in particular in areas with less tradition
such as the South of Europe or Eastern Europe and using the inspiration of other regions or areas
such as the France-Germany transport research joint programme, Nordic cooperation…
• Develop clearer links between transport research and innovation and skills for jobs (for instance,
in the new market of electric vehicles and battery manufacturing and recycling).
• Consider preparing a report on “transport research and innovation opportunities across all
Horizon 2020 areas”, as a learning exercise and base for further dissemination actions.
11. Retain high levels of knowledge and professionalism
It is recommended that note is taken of the risks of under resourcing the management of the
programme on the Commission side in terms of expertise (including the outsourced executive role of
INEA) and taking care that there is a good balance of project management (necessary for the efficient
running of the programme) with practical knowledge and policy experience in place to provide
content support to projects.
CONCLUSION
In conclusion the Expert Group can see a high level of potential within this challenge of Horizon 2020.
There are a number of noticeable improvements in Horizon 2020 compared to FP7 and efforts to
follow through on previous evaluation recommendations by the Commission are clear.
Despite this there are still a number of key areas especially in terms of linking research to job
creation, policy development and taking a more holistic approach to find solutions to this Societal
Challenge that can further improve both Horizon 2020 and future Framework Programmes. The
important economic contribution the transport sector brings in terms of direct and indirect European
employment opportunities are widely recognised. However the negative externalities such as loss of
productivity (in the region of 2% of GDP) due to congestion, and the social burden of poor air quality
and traffic accidents cannot be overlooked as these need to decline more swiftly than current trends
indicate.
Europe must not only face major societal challenges as outlined in SC4 but use its research
programmes to help to create a new paradigm for transport, based on evidence of what works. This
is critical for its transport systems and will help it shift away from the current reliance on fossil fuel,
build new skills and ensure that it is ready for major socio-demographic changes.
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 140
LITERATURE Analysis of the Role and Impact of Industry Participation in the Framework Programmes, ppt, EC 2016
Analysis of the Role and Impact of Research Performing Organisations in the Framework
Programmes, ppt, EC 2016
Aviation 15 years of Research and Innovation – Success stories; Advisory Council for Aviation
Research and Innovation in Europe (ACARE), 2016
Clean Sky 1st Interim Evaluation, December 2010
Clean Sky 2 Impact Assessment, final report Expert group, EC September 2012
Commitment and Coherence, ex post evaluation transport research FP7, EC November 2015
CORDA Horizon 2020 Dataset xls, EC 28-6-2016
Data Dictionary, Horizon 2020 CORDA standard definitions, EC 22-April 2016
Dataset on impact criterion for projects Horizon 2020, EC August 2016-08-23
Diverse documents relating to Horizon 2020, Corda etc.
ERTRAC, 12 Success Stories, ERTRAC April 2016
European Green Vehicles Initiative Impact Assessment, EC 2016
Executive Summary Of The Impact Assessment Accompanying the Communication from the
Commission 'Horizon 2020 - The Framework Programme for Research and Innovation', EC, 2011
First Horizon 2020 Monitoring Report 2014, EC 2016
First Orientations towards Scoping Paper Transport Horizon 2020 WP 2018-2020, EC 2016
From collaborative research projects to market deployment, 12 success stories; European Road
Transport Research Advisory Council (ERTRAC), 2016
Horizon 2020 Indicators, EC 2015
Horizon 2020 Work Programme 2014 – 2015 Smart, green and integrated transport
Horizon 2020 Work Programme 2016 – 2017 Smart, green and integrated transport
Impact Assessment Accompanying the Communication from the Commission 'Horizon 2020 - The
Framework Programme for Research and Innovation' (EC, Commission staff working paper, Brussels
30 November 2011)
Impact Assessment of the European Green Cars Initiative, EGVI, 2016
Interim Evaluation Horizon 2020, Template for the thematic contributions, EC 2016
Intervention logic Horizon 2020 evaluation, and detailed version, EC July 2016
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 141
Interview guidelines, EC internal doc
Kick off Meeting, ppt, EC 26-6-2016
Open Innovation, Science and World, EC 2016
PPMI Survey results on Added Value, EC September 2016
Preliminary Data Horizon 2020 Transport, xls, EC June 2016
Regulation (EU) No 1291/2013 of the European Parliament and of the council of 11 December 2013
establishing Horizon 2020 - the Framework Programme for Research and Innovation (2014-2020) and
repealing Decision No European Union (2013), 1982/2006/EC
Report Support Group Horizon 2020 evaluation, EC internal July 2016
Second Interim Evaluation of the Clean Sky Joint Undertaking, EC, November 2013
Strategic Research and Innovation Agenda, Introductory paragraph, EC 2016
Survey model for Horizon 2020 NMBP projects, EC internal doc
Terms of reference interim Evaluation Transport Horizon 2020, EC June 2016
The Contribution of Framework Programmes to Major Innovations, ppt, JIIP June 2016
The Ocean of Tomorrow Projects (2010-2013) Joint Research Forces to meet Challenges in Ocean
Management; European Union (EU), 2014
The Role and Engagement of Universities in the Framework Programmes, ppt, ISMERI EUROPA, June
2016
Transport advisory group reports: Response to 7 questions, EC June 2014; Summary advice on WP
2016-17; European Transport decarbonisation, EC December 2015, Final Transport Advisory Group
report, EC May 2016
Transport in FP7, Facts and Figures, EC, version 26-2-2014
Transport Research and Innovation Achievements Report , draft, EC March 2016
TriValue Cooperation Programme, Deliverable 4.2
TriValue, Ex post evaluation of Transport Research and Innovation in the FP7 “Cooperation
Programme”
White Paper, Roadmap to a Single European Transport Area – Towards a competitive and resource
efficient transport system /* COM/2011/0144 final
WP 2018-2020, Summary of replies to public consultation, EC
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 142
ANNEX: SURVEY QUESTIONS FOR PROJECT
COORDINATORS Project data
Although, as explained, results will be presented in an aggregated manner (not at project level), we
ask you to provide the following data so we can get a better understanding of your answers when we
process the information.
What is your project acronym?
Project number:
Start date (month, year):
End date (month, year):
Transport sector addressed:
- Aviation
- Rail
- Road
- Waterborne
- Infrastructure
- Intelligent Transport Systems
- Logistics
- Urban transport
- Intermodal or multimodal
- Other, please specify
Programme objective addressed:
- For Horizon 2020 projects (ref. https://ec.europa.eu/programmes/horizon2020/en/h2020-
section/smart-green-and-integrated-transport):
o Resource efficient transport that respects the environment
o Better mobility, less congestion, more safety and security
o Global leadership for the European transport industry
o Socio-economic and behavioural research and forward looking activities for policy
making
o Other, please specify
- For FP7 projects:
o Aeronautics projects
The greening of air transport
Increasing time efficiency
Ensuring customer satisfaction and safety
Improving cost efficiency
Protection of aircraft and passengers
Pioneering the air transport of the future
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 143
Other, please specify
o Surface projects
The greening of surface transport
Encouraging modal shift and decongesting transport corridors (co-modality)
Ensuring sustainable urban mobility
Improving safety and security
Strengthening competitiveness
Other, please specify
In your opinion is your project more policy oriented or more technology oriented?
- Technology oriented
- Policy oriented
- Both
- I don’t know
Type of action:
- For Horizon 2020 projects:
o Innovation Action (IA)
o Research and Innovation Action (RIA)
o Coordination and Support Action (CSA)
o Other, please specify (Fast Track to Innovation -FTI, SME Instrument, Inducement
Prize, ERA-NET Cofund…)
o Other, please specify
- For FP7 projects
o Collaboration Project (CP) – Level 0 (aeronautics only)
o Collaboration Project (CP)
o Collaborative Project – Focused Project (CP-FP)
o Collaboration Project – Integrated Project (CP-IP)
o Coordination and Support Action – Coordination Action (CSA-CA)
o Coordination and Support Action – Support Action (CSA-SA)
o Other, please specify (ERA-NET, ERA-NET+…)
Project website (if already available):
Relevance of the Horizon 2020 Transport activities (Note: if you are an FP7 coordinator, please skip
this entire section on “Relevance”)
1. Would you say that the overarching Horizon 2020 Transport programme objectives, beyond
those of your topic, are clear enough? Note: if you are an FP7 coordinator, please skip this
question
Totally agree Agree Rather agree Rather disagree Disagree Totally disagree I don’t know
2. Do you think that the overarching Horizon 2020 Transport programme objectives take into
account the latest scientific, socio-economic, political or any other nature developments in
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 144
the field of transport research and innovation at the national/European and international
level? Note: if you are an FP7 coordinator, please skip this question
0 (not taken into
account)
1 2 3 4 5 (fully taken into
account)
I don’t
know
Scientific
Socio-economic
Political
Other emerging
needs
Other emerging needs (please comment briefly):
If you have selected a value between 0 and 3 in any of the items, please briefly explain:
3. To what extent does the Horizon 2020 Transport programme address stakeholders’ needs in
terms of tools used? Note: if you are an FP7 coordinator, please skip this question
0 (not
covered)
1 2 3 4 5 (fully
covered
I don’t
know
Degree to which the stakeholders’ needs
are covered
If you have selected a value between 0 and 3 in any of the items, please briefly explain:
4. To what extent does the Horizon 2020 Transport programme address stakeholders’ needs in
terms of research areas covered? Note: if you are an FP7 coordinator, please skip this
question
0 (not
covered)
1 2 3 4 5 (fully
covered
I don’t
know
Degree to which the stakeholders’ needs
are covered
If you have selected a value between 0 and 3 in any of the items, please briefly explain:
Internal coherence (among Horizon 2020 activities)
5. To your knowledge, what other Horizon 2020 areas or programmes also represent an
opportunity for transport research, innovation and implementation? If you are not an FP7
coordinator, please skip this question
Opportunity Complementary
area
Synergetic
area
Overlapping
area
Duplicated
area
Agriculture &
Forestry
Aquatic Resources
Bio-based Industries
Biotechnology
Energy
Environment &
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 145
Climate Action
Food & Healthy Diet
Funding Researchers
Health
ICT Research &
Innovation
Innovation
International
Cooperation
Key Enabling
Technologies
Partnerships with
Industry and Member
States
Raw Materials
Research
Infrastructures
Security
SMEs
Social Sciences &
Humanities
Society
Space
Other: please specify
6. Would you say that there are enough information and coordination mechanisms within
those other areas of Horizon 2020 that are complementary or synergetic for transport? If you
are an FP7 coordinator, please skip this question
Totally agree Agree Rather agree Rather disagree Disagree Totally disagree I don´t know
7. Please comment on the coherence of the different mix of available instruments (in the case
of Horizon 2020, JU/PPPs, IA, RIA, CSA, SME instrument, etc.; in the case of FP7, JU/PP, CP,
CSA, Competitiveness and Innovation Programme CIP for SMEs, etc.)?
0 (non-
existing)
1 2 3 4 5 (at a 100%
degree)
I don’t
know
Complementarity among
instruments?
Synergy among instruments?
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 146
If you have indicated above a low mark (0 to 3), please briefly explain the reasons:
0 (non-existing) 1 2 3 4 5 (at a 100% degree) I don’t know
Overlaps among instruments?
Gaps among instruments?
If you have indicated above a high mark (3 to 5), please briefly explain the reasons:
8. Is your project going to produce interdisciplinary solutions which cut across multiple specific
objectives of Horizon 2020 (for instance, new batteries for electric vehicles, or lightweight
material applications for high-speed trains)?
Totally agree Agree Rather agree Rather disagree Disagree Totally disagree
Please briefly explain:
9. Would you be able to justify that your project is going to simultaneously tackle societal
challenges (i.e. supporting mobility of the elderly while, at the same times, creating new
business opportunities) and give rise to new competitive businesses and industries?
Totally agree Agree Rather agree Rather disagree Disagree Totally disagree I don’t know
Please briefly explain:
10. From a broader perspective, what is your opinion about the EC’s diversification of research
and innovation programmes (for example research infrastructures supported within
European Strategy Forum on Research Infrastructures ESFRI, or the fundamental research
supported by the European Research Council ERC projects, Knowledge and innovation
Communities launched by the European Institute for Technology EIT…)?
0 (non-existing) 1 2 3 4 5 (at a 100% degree)
Duplication among actions
Complementarity among actions
Synergy among actions
If you have indicated that there are duplications or you have given a low mark (0 to 3) for
complementarity and synergy, please briefly explain the reasons:
External coherence (still within the EU but beyond Horizon 2020 activities)
11. In your country, and based on your experience as researcher, would you say that the Horizon
2020 Transport research programme is influencing your national research programmes?
Very
negatively
Negatively No
influence
Positively Very
positively
I don’t
know
Is Horizon 2020
influencing national
research activities?
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 147
If you have indicated that Horizon 2020 is having a negative influence on your national programme,
please briefly explain
12. Do you think that complementarities and synergies are necessary with other public support
initiatives such as the Connecting Europe Facility (CEF), European Structural and
Investment Funds (ESIFs), Regional Research and Innovation Strategies for Smart
Specialisation (RIS3), Smart Cities European Innovation Partnership EIP, European
Investment Bank’s European Fund for Strategic Investments (EFSI), LIFE programme?
0 (not
necessary at
all)
1 2 3 4 5 (totally
necessary)
I do not
know
Connecting Europe Facility (CEF)
European Structural and Investment
Funds (ESIFs)
Research and Innovation Strategies for
Smart Specialisation (RIS3)
Smart Cities EIP
EFSI
LIFE
If you have selected a low mark (0 to 3) in any of the items, please briefly explain:
13. Is your project using previous research from other EU-funded projects or from other national
or regional projects:
- From other previously EU funded projects
- From previously funded national research
- From previously funded regional research
- From research previously conducted at internal level by the participants
- Other, please comment
14. If you project is based on previously funded research (see previous question), do you think
that the previous funding research programmes (either EU, national, regional or internal)
represent a coherent scenario for research and innovation?
0 (not coherent at all) 1 2 3 4 5 (fully coherent)
Coherence among research programmes
Effectiveness
15. How many of the following human resource related activities were undertaken during or
after the project?
Number already
achieved to date
Number targeted or
projected at the end of the
project
I don´t
know
Number of training events organised
Average number of participants to 1
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 148
training event
Total number of PhD Thesis and Master
Thesis supported by the project
Total number of personnel involved in
the project implementation - males
Total number of personnel involved in
the project implementation - females
16. Please indicate what impact your FP7 or Horizon 2020 participation has had (or is expected
to have) for you or your partners in the area of human capital.
Strong
increase
Medium
increase
Low
increase
No effect I don´t
know
Gender equality
Mobility of own researchers
Ability to hire researchers
across the EU
17. How many of the following outcomes were achieved or will be achieved?
Number
already
achieved to
date
Number targeted or
projected at the end
of the project
I don´t
know
Publications in peer reviewed journals or books
Publications at conferences
Patent applications
Prototypes (i.e. new materials, physical parts,
machinery, facility infrastructure)
Pilot scale applications / demonstration
activities
Testing activities -- validations & verifications
Software, tools, methods, databases, models,
system architecture applications that are not
marketed (i.e. internal tools, etc.)
New norms and standards
New products introduced to the market: new
materials, physical parts, machinery, facility
infrastructure
New services introduced to the market
(includes ICT and non-ICT services)
New technologies
PhD Thesis or MSc Thesis within the project
Assessments (impact assessments, consultancy
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 149
studies)
Handbooks and guidelines
Policy recommendations
18. Please identify the Technological Readiness Level (TRL) of the outcome / results of your
project.
TRL level I don´t know
Before the project
Already achieved
Targeted at the end of the project
Not applicable
Note: Horizon 2020 TRL definition can be found in
https://ec.europa.eu/research/participants/data/ref/h2020/wp/2014_2015/annexes/h2020-
wp1415-annex-g-trl_en.pdf
19. Please indicate what level of impact your Horizon 2020 participation has had (or is expected
to have at the end of the project) for you or your partners on research, development and
innovation capabilities.
Strong
increase
Medium
increase
Low
increase
No
effect
I don´t
know
Understanding / knowledge in existing areas
Understanding / knowledge in new areas
Access to physical R&D infrastructure (labs,
technology, …)
Access to non-physical R&D infrastructure
(organisation, coordination, …)
Scientific or technological collaboration with
other institutes/businesses/universities
Access to international networks and
collaborations
20. Please indicate what level of impact your Horizon 2020 participation has had (or is expected
to have) for you or your partners on competitiveness.
Strong
increase
Medium
increase
Low
increase
No
effect
I don´t
know
Access to known markets and
business opportunities
Opening up of new markets and
business opportunities
Competitive position nationally
Competitive position internationally
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 150
21. Were any of the following exploitation activities undertaken during the project? Please tick
the corresponding boxes.
Targeted at the end
of the project
To
date
I don´t
know
Development of business plan and/or market analysis
Direct contacts with stakeholders
Contact with potential clients
Contacts with private investors (Venture Capitalists,
Business Angels...)
Agreements with private investors
Technology transfers
Creation of companies (spin off/start ups) as result of the
project
Creation of association / non-commercial legal entity(ies)
to maintain and further develop project network
Other, please briefly explain
22. Did the project help you to obtain or leverage additional investment or capital from other
sources or from other financial institutions for activities not covered by the European
Commission’s grant (for instance, continuation of the project, additional market activities)?
- Yes
o If yes, please indicate source of additional funding: ______
o What covered that additional funding: ____________________
- No
- I don´t know
23. Assuming that the project research outputs are implemented, do you expect your research
outputs to contribute to(please select as many answer as you wish):
- Decarbonising and "greening" the transport system
- Increasing efficiency of the whole transport system
- Improving safety & security of passengers, aircraft, vehicles and vessels, and
infrastructures
- Strengthening the competitiveness of European industry
- Pioneering the Transport of the future (long term perspective)
- Enhancing and strengthening the European Research Area (ERA)
- Globalisation, external dimensions and international cooperation
- Further development and implementation of EU transport policy
- I don´t know
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 151
Efficiency
24. Please provide ratings as regards the following Horizon 2020 processes for your project:
0
(very
poor)
1
(poor)
2
(acceptable)
3
(good)
4 (very
good)
5
(excellent)
I do
not
know
Effort required to find new
calls
Effort required to prepare
and submit a proposal
Effort required to
participate as a newcomer
Clarity of general goals of
the Transport Programme
Clarity of the rules for
participation (funding,
administrative,
reporting…)
Clarity of the calls for
proposals
Adequacy of the topic
budget to the topic
description
Clarity of the evaluation
criteria and processes
The time taken to evaluate
and select proposals
The transparency of the
selection process
The quality of the
selection process
The feedback given in
relation to the final
rejection of proposals
The time between opening
of the call and deadline for
proposal submission
The time from proposal
submission to signature of
contracts and launch of
projects
The grant preparation
process
The procedures for project
monitoring and reporting
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 152
The role of the experts
taking part in the periodic
and final evaluations
The time taken to issue
payments to grant holders
Support by the project
officer (technical content
of the project)
Support by the project
officer (administrative,
flexibility…)
The Participants Portal
Networking events
supported or organised by
the EC
If you have indicated any low rating (0, 1, 2 or 3) in any of the items above, please briefly explain:
25. Only if you have participated in both FP7 and Horizon 2020: which of the following
procedures considering the whole project life cycle did you experience as an improvement?
If you are just an FP7 coordinator or a Horizon 2020 coordinator, please skip this question
FP7 was better
than Horizon
2020
No change from
FP7 to Horizon
2020
Horizon 2020
is better than
FP7
I do not
know
Effort required to find new calls
Effort required to prepare and
submit a proposal
Effort required to participate as a
newcomer
Clarity of general goals of the
Transport Programme
Clarity of the rules for participation
(funding, administrative,
reporting…)
Clarity of the calls for proposals
Adequacy of the topic budget to the
topic description
Clarity of the evaluation criteria and
processes
Effort required to participate as a
newcomer
The time taken to evaluate and
select proposals
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 153
The transparency of the selection
process
The quality of the selection process
The feedback given in relation to
the final rejection of proposals
The time between opening of the
call and deadline for proposal
submission
The time from proposal submission
to signature of contracts and launch
of projects
The grant preparation process
The procedures for project
monitoring and reporting
The role of the experts taking part
in the periodic and final evaluations
The time taken to issue payments
to grant holders
Support by the project officer
(technical content of the project)
Support by the project officer
(administrative, flexibility…)
The Participants Portal
Networking events supported or
organised by the EC
Please add any remarks or explanations we might wish:
26. Only if you have participated in both FP7 and Horizon 2020, please indicate whether the
following changes from FP7 to Horizon 2020 have had any impact in the effectiveness of the
Horizon 2020 Transport research programme:
Major
negative
effect
Minor
negative
effect
No
effect
Minor
positive
effect
Major
positive
effect
I don´t
know
Multiannual Work
Programmes
Less prescriptive topics
New types of actions
(mainly IA and RIA
instead of CP)
Two-stage evaluation
If you have indicated “negative effect” in any of the items above, please briefly explain:
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 154
EU added value
27. What were the main reasons to apply for EC funding for your project?
Not
important
at all
Slightly
important
Important Very
important
Extremely
important
I
don´t
know
Contribution to EU
transport policy
objectives
Funding opportunity
Cooperation with other
countries/institutes
Access to complementary
expertise
Maximise exposure of
results
Understanding the needs
and characteristics of a
wider market
Overcoming specific
barriers to the market
uptake of research results
Capitalising on previous
EU projects experience
and networks
Other
If other, please briefly explain:
28. In your opinion or based on your experience, would a similar project have taken place at all
without the EC funding?
- Yes
- No
- I do not know
29. In particular, does the Horizon 2020 Transport programme offer participants unique
opportunities to do work that could not be done through other programmes (for instance,
because of the inter-regional collaboration)?
- Yes
- No
- I do not know
30. In your opinion, would you say that the Horizon 2020 Transport programme addresses policy
objectives (such as resources efficient, safe, environmentally friendly transport…) which are
not covered by other European or international initiatives?
Totally agree Agree Rather agree Rather disagree Disagree Totally disagree I don’t know
If you have disagreed, please briefly explain:
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 155
31. To what extent does the fact that this is an EU research project influence the impact of the
project?
0 (no
impact)
1 2 3 4 5 (maximum
impact)
I do not
know
Influence on the impact of the
project
Degree of synergies
32. Attributable to the EC funding, what kind of additionality effects do you expect in your
project, e.g.:
Added value/ size Little
added
value
Medium
added value
Much
added
value
I don´t
know
Funding being leveraged in addition to the EC
contribution(input added value)
Faster progress (added value by acceleration)
Work done on a larger scale(scale
additionality)
Risk reduction
Improved market position or
competitiveness
Improving scientific quality
Other
If you indicated “little added value”, please explain your answers briefly:
33. Attributable to EC funding, are there any conditions that hamper creation of impact of your
project, e.g.:
Hampering factor Please provide brief explanations Not applicable
Compromises on scope and/or method
Less proficient partners from other countries
Other factors, please explain briefly
34. Did you consider applying for funding under the ERA-NET programme?
Please explain briefly
This is an ERA-NET project - what are the advantages?
Yes, but this was rejected
No, not considered
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 156
Lessons learned
35. Please include any final comments or suggestions for improvement of the Horizon
2020Transportresearch programme(2,000 characters maximum)
Contact info (optional)
Do you agree that we follow-up / contact you?
- Yes
- No
If so, please indicate your contact name:
- Contact e-mail:
- Contact phone number:
THANK YOU VERY MUCH FOR YOUR PARTICIPATION!!!
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 157
ANNEX: INTERVIEW QUESTIONS FOR STAKEHOLDERS Introduction
As part of the general interim evaluation of the Horizon 2020 research programme, the European
Commission has asked a group of external experts to evaluate the transport paragraph of the
Programme. This expert group of four has decided to hold ca. 25 interviews among stakeholders,
policy makers, national representatives and project coordinators. The findings from these interviews
will be aggregated and not traceable to the individual interviewees, thus assuring confidentiality.
Together with results from surveys, document and data search they will form the basis of this
evaluation.
The results of this evaluation will be published by the European Commission as part of the overall
Horizon 2020 interim evaluation.
General background
1. Date of interview …../…../2016
2. Name of interviewee (NB.: interview answers will be rendered anonymously) ……………
3. Affiliation (please specify name)
0 Research Provider ……………..
0 PPP ……………..
0 Joint Undertaking ……………..
0 Advisory Committee ………….
0 Technology Platform …………
0 Industry …………..
4. What is the specific responsibility of your organisation in relation to Horizon 2020 and/or
FP7: ………………
5. What is your own specific responsibility with regard to FP7 and/or Horizon 2020
Relevance
6. Please describe in your view the overall objective of the Horizon 2020 SC4 programme. What
needs and priorities is it designed to address? To what extent have (FP7 and) Horizon 2020
work programmes contributed to EC priorities (notably: New Commission (Juncker) priorities;
Open Innovation, Open Science, Open to the World; White paper Transport policy objectives;
European Research Area and Innovation Union flagship initiative)? Do you think today that
the current transport programme objectives are still relevant: have you run some kind of gap
analysis?
7. To what extent did objectives evolve from FP7 to Horizon 2020? How was that
communicated? To what extent do the original (FP7 and) Horizon 2020 objectives still
correspond to the needs and developments within the EU (e.g. socio-economic, political ,
scientific, technological needs and developments)?
8. Does the Horizon 2020 SC4 programme take into account the latest developments in the
field of transport research and innovation at the national/European and international level?
Did circumstances change since the outset of Horizon 2020? How do/did Horizon 2020 work
programmes address these changes?
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 158
9. How have the research themes been initially defined in FP7 and Horizon 2020? Are you
aware of how are these themes translated into Work Programmes, research calls, project
selection?
10. Looking at Horizon 2020 and FP7, are there any gaps or overlaps that influence the relevance
of the programmes? What elements in particular ensure the relevance of the programme?
Synergies?
11. Which mechanisms are in place to keep ensuring relevance of the transport programme? To
what extent does the programme address stakeholder needs in terms of tools and research
areas covered?
12. In case of PPP or Joint Undertaking:
Please describe to what extent your PPP or TP proposed dissemination and outreach
activities that will effectively reach citizens
Please explain why your PPP or research agenda is relevant to European Objectives: (list
different objectives)
Please briefly explain what is the added value of PPP or JU as an instrument.
Coherence (internal)
13. Please comment on the coherence of the different mix of available actions (CSA, RIA, IA,
conferences, etc)? Does this mix provide a coherent , coordinated approach to address the
objectives (overlap; complementary; synergy; gaps)? How is coordination ensured between
the different initiatives?
14. Please provide illustrations and highlight the explicit actions undertaken to foster coherence
and complementarity.
15. What other work programmes (and their funding mechanisms) exist within Horizon 2020 and
how far do these funds support transport research and implementation? (e.g. Energy,
Climate action, security)
16. What is your opinion about the EC’s diversification of research and innovation programmes
(for example research infrastructures within ESFRI, ERC projects, Knowledge and innovation
Communities launched by the EIT…)?
17. Can you provide examples of projects or programmes that score positively in terms of
coherence (interdisciplinary solutions, supporting cross-cutting objectives or purposes (e.g.
societal challenges and business support))
18. In the case of PPPs or JU’s, how would you justify this particular instrument as part of the mix
of instruments?
Coherence (external)
19. In your opinion does the Horizon 2020 SC4 programme fill a policy gap that is not covered by
other European or international initiatives in the field of transport?
20. How are your research agenda’s influencing the EU FP7 and Horizon 2020 programmes.
21. In particular does the Horizon 2020 SC4 programme offer participants unique opportunities
to do work that could not be done through other regional/national/EU programmes. Was
there a marked difference between FP7 and Horizon 2020?
22. What are the links between these programmes and the Horizon 2020 SC4 programme? Do
you see a need for ensuring coherence and synergies between the Horizon 2020 SC4
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 159
programme and European/international programmes serving similar objectives? How do you
think this can be achieved.
23. Are you aware of possible complementarities and synergies with other public support
initiatives at regional, national and international level in the Transport arena? E.g.
Connecting Europe Facility (CEF), Regional Development Fund, Structural and Investment
Fund, Regional Research and Innovation Strategies for Smart Specialisation (RIS3), European
Fund for Strategic Investments (EFSI), LIFE programme? To what extent are they
complementary? Do they act in synergy? Are there any gaps or overlaps? How is it ensured
that these programmes are coherent and complementary with EC actions (explicit actions
undertaken to ensure them)? Please indicate areas where further synergies and
complementarity should be sought.
Effectiveness
24. Overall, what are the main expectations from the projects i.e., the outcomes that the
projects should guarantee to deliver at the end of their lifetime?
25. In your opinion, will/has the delivery of the results together lead to the achievement of the
programme’s objectives (resource efficiency respecting environment; better mobility, more
safety/security; global leadership for Eur. transport industry; socio-economic and policy
supporting research)? Is this measurable and verifiable? In particular, and in your opinion, do
the current and expected achievements of the programme allow to deliver on the SC4
objectives (resource efficient/respect environment; better mobility; global leadership T-
industry; socio-economic policy)?
26. Do you think the project application and selection processes are effective? Are they more
effective than in FP7? In your opinion how could they be improved to maximise the benefits
of the programme? Can you assess the potential benefits of the Horizon 2020 SC4
programme participation for the participants (public and private). Are the participants the
ones that the programme targeted?
27. What do you think will be the main long-term result of the programme? (top 3) If the
programme would not have been implemented, how could this result have been achieved
otherwise?
28. To what extent does the programme communication/valorisation strategy allow identifying,
capitalising upon and (possibly) transferring good practices/results to stakeholders?
29. Are there any elements of the programme that are more or less effective than others, and, if
so what lessons can be drawn from this?
Efficiency
30. How is the Horizon 2020 SC4 programme managed in terms of procedures, administration,
contractual issues etc.? How does Horizon 2020 compare to FP7 in this respect. Do you see
any possible improvements in the way the Horizon 2020 SC4 programme is run?
31. Do you consider that the benefits achieved through the Horizon 2020 SC4 programme have
been/will be gained at a reasonable cost? Is there a difference between the different types of
activities of the programme? Is it possible to compare cost and benefits of the Horizon 2020
SC4 programme with FP7 or with other still existing similar programmes?
32. Do you think the foreseen budgets in the programme are sufficient? What is your opinion on
the allocation of budgets to the different objectives and topics?
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 160
Added Value
33. What role does the EC play in coordinating transport research policies in Europe (e.g. reduce
disparities, raise standards, create synergies). Please give examples of added value in this
respect? Has that role evolved from FP7 to Horizon 2020?
34. What is the added value of research outcomes in FP7 and Horizon 2020 that could not have
been achieved at National, level (e.g. publications in high impact journals; patents;
prototypes; public-private publications; cleaner products with lower environmental impact;
more efficient process (e.g. including minimize waste, reuse or recycle); new/alternative
technologies, services, products, knowledge; widening participation)? Please specify and give
examples.
35. What is the added value of the programme in terms of:
- Multiplier effect
- Larger scale of research
- Complementarity in researcher expertise
- Less risk
- Better quality through more expertise
- Trusted and objective process
- Mobility of researchers
- Global actor objective
- Other
36. To what extent does ERA-NET(plus) under FP7 or Horizon 2020 stimulate programmes with
EU added value: please indicate what added value under which programmes, give examples.
Does the EC stimulate ERA-NET? What criteria relevant for added value apply for EU to
participate in ERA-NET(+). How is the combination of ERA-NET and ERA-NET+ in Horizon 2020
working out?
37. What is your opinion about the way projects are taking up their responsibility for EU wide
dissemination? Good examples? How will other EU countries or institutes profit of that
dissemination?
38. Do you have any evidence so far that the research and innovation outcomes will be exploited
or built upon, e.g. with funding from other sources, after the end of funding period of the
projects (sustainability), (added value, valley of death)?
39. Is there any evidence yet that FP7 has had a leverage effect in terms of impact (e.g. speeding
up innovation, market introduction of new technology, additional financing)? In how far is
Horizon 2020 fitted with conditions to achieve a (larger?) leverage effect? Please give
examples. In your opinion, what further actions are needed to maximise the impact of the
Horizon 2020 SC4 programme?
40. Do you have concrete examples of Horizon 2020 Transport contributing to Open Innovation,
Open Science and Open to the world (e.g. improving R&I capacities of participants, increased
transparency, better communication)?
41. To your opinion, what are the current strengths and weaknesses of the Horizon 2020 SC4
programme? What would you modify in the way the Horizon 2020 SC4 programme is
organised and implemented to attain a higher EU added value? Any disadvantages of
participating in EU projects (having to compromise on scope and/or method; bureaucracy,
slow process; IPR issues; less proficient partners from other countries; language issues)?
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 161
42. Lessons learned
Please provide three examples of success stories based on an assessment of FP7 projects
whose outcomes/results have had a significant added value and/or impact on the society
and/or the economy and which have outstanding EU added value based on their
effectiveness, efficiency and synergy. Are the instruments IA and RIA better than previous
ones?
43. Do you have ONE success story regarding Horizon 2020 Transport contribution to each of
the following areas:
- Area 1 Resource Efficient transport that respects the environment
- Area 2 Better mobility, less cohesion, more safety (and security)
- Area 3 Global leadership for the European Transport Industry
- Area 4 Socio-Economic and Behavioural Research and Forward Looking activities for
policy making.
44. Final question: do you have any comments, observations, opinions, etc., whether on content
or process of this interview or the Horizon 2020 interim evaluation in general that we have
not touched upon and that you would like to share with us?
Supporting report for the Interim Evaluation of Horizon 2020 – SC4 Smart, green and integrated transport 162
This report has been prepared as part of an interim evaluation of the Societal Challenge 4
(Smart, Green and Integrated Transport) for the European Commission and not for
general publication. The information comes from a number of sources including an
Internet based questionnaire, interviews with key stakeholders, a stakeholder hearing
and the collecting of data (some of which were provided by the Commission). Comments
from these sources are confidential and should not be able to be traced back to their
owners or their organisations. Stakeholder views were only considered when they were
also shared by the Expert Group and validated by other evidence to support this opinion.
It is indicated in the text when this is not the case.
Studies and reports