Embed Size (px)
Citation preview
1
NATIONAL REPORT ON THE
IMPLEMENTATION AND IMPACT OF
THE ERASMUS+ PROGRAMME -
CROATIA
This report is prepared for the Ministry of Science and Education, and the
Ministry for Demography, Family, Youth and Social Policy
Editors:
Branko Ančić, Ph.D., Institute for Social Research in Zagreb ([email protected])
Marija Brajdić Vuković, Ph.D., Centre for Croatian Studies, University of Zagreb
The National Report is based on research reports prepared by: Branko Ančić, Ph.D. (Institut for Social
Research in Zagreb), Marija Brajdić Vuković, Ph.D. (Centre for Croatian Studies, University of
Zagreb), Natalija Lukić Buković (Agency for Mobility and EU Programmes, Department for
Coordination and Quality Assurance of the Erasmus+ programme)
Advisory Board: Ksenija Rukavina (Ministry of Science and Education), Tajana Krstonošić (Ministry
for Demography, Family, Youth and Social Policy), Annamaria Vuga (Ministry for Demography,
Family, Youth and Social Policy), Antonija Gladović (NA Director, Agency for Mobility and EU
Programmes), Filip Gašparović (Head of Department for Coordination and Quality Assurance of the
Erasmus+ programme, Agency for Mobility and EU Programmes).
25/05/2017
Ref. Ares(2017)3347562 - 04/07/2017
2
Table of Contents
Acronyms ................................................................................................................................................ 3
Introduction ............................................................................................................................................. 4
Evaluation results .................................................................................................................................. 11
Effectiveness ...................................................................................................................................... 11
Efficiency ........................................................................................................................................... 18
Relevance .......................................................................................................................................... 22
Internal and external coherence and complementarity ..................................................................... 25
European added value and sustainability ......................................................................................... 26
Conclusions ........................................................................................................................................... 28
Appendix I: Evaluation Questions ......................................................................................................... 3
Appendix II: General and specific objectives of the ERASMUS+ Programme .................................. 33
Appendix III: Analysis of survey results ............................................................................................. 36
Appendix IV: E+ National evaluation - Qualitative research report .................................................... 67
Appendix V: Combined Presentation of Empirical Findings: Lifelong Learning Programme, Youth in
Action and Erasmus+ ............................................................................................................................ 79
Appendix VI: Questionnaire [in Croatian] ......................................................................................... 117
Appendix VII: Interview Protocol [in Croatian] ................................................................................ 132
3
Acronyms
LLP Lifelong Learning Programme
YiA Youth in Action Programme
LDV Leonardo da Vinci (sectoral programme within LLP)
ERA Erasmus (sectoral programme within LLP)
GRU Grundtvig (sectoral programme within LLP)
COM Comenius (sectoral programme within LLP)
KA1 Key Action 1: Learning Mobility of Individuals
KA2 Key Action 2: Cooperation for innovation and the exchange of good practices
KA3 Key Action 3: Support for policy reform
YOU Youth field in ERASMUS+
SE School education field in ERASMUS+
VET Vocational education and training field in ERASMUS+
HE Higher education field in ERASMUS+
ADU Adult education field in ERASMUS+
EVS European Voluntary Service
NA HR National Agency - Agency for Mobility and EU Programmes
NGO Non-govermental organisation
CSOs Civil society organisations
HEI Higher education institution
STEM Science, technology, engineering and mathematics
CEEPUS Central European Exchange Programme for University Studies
ECTS European Credit Transfer and Accumulation System
E&T Education and Training
ESF European Social Fund
IRO International relations office
NAU National Authority
RAY Research-based Analysis and Monitoring of Erasmus+: Youth in Action – RAY
4
Introduction
In 2014, with the introduction of ERASMUS+, it was expected that the new approach would promote
closer links between programme and policy objectives, more synergies and interaction between
formal, informal and non-formal learning, and more cross-sectoral partnerships with the world of
work. In terms of structure, ERASMUS+ introduced a streamlined, simpler architecture as well as a
stronger focus on EU added value than predecessor programmes, an increase of the budget, and
additional funding from external action instruments to support the international dimension of higher
education. In terms of policy objectives, the programme is linked to EU strategic documents (Europe
2020, Education and Training 2020, The renewed framework for European cooperation in the youth
field 2010-2018, The European dimension in Sport / The EU Work Plan on Sport), while additional
specific objectives are set separately for the Education and Training and the Youth field.
After the three years of implementation, it is still too early to asses the impact of the programme, so
the national mid-term report aims to provide a national perspective, assess the degree to which the
programme was successful in realizing its objectives, assess the strengths and weaknesses in
programme implementation, and recommend a way forward for a post-2020 programme.
The report is prepared for the national authorities in charge of strategic guidance of the programme at
the national level: the Ministry of Science and Education and the Ministry for Demography, Family,
Youth and Social Policy of the Republic of Croatia. The conclusions and findings presented in this
report are based upon the evidence gathered, research reports written by external evaluators, and
monitoring data prepared by the internal NA evaluator. To answer the evaluation questions,
triangulation of both qualitative and quantitative methods is used, as is analysis of secondary sources.
The survey was implemented among project managers and legal representatives who submitted their
applications within the LLP (2009-20131), YiA (2009-2013) or ERASMUS+ (2014-2020). An in-
depth presentation of survey results is available in Appendix III. The findings from the interviews
conducted of key respondents (i.e. representatives of relevant stakeholder groups) are available in
Appendix IV. A combined presentation of empirical findings from programme monitoring data,
Participant Reports and Research-based Analysis and Monitoring of Erasmus+: Youth in Action (RAY
Standard Survey - national results) is available in Appendix V.
In this report, key findings generated from multiple research sources are presented according to five
evaluation criteria2:
1) Effectiveness, which considers the extent to which the objectives of the ERASMUS+ programme
are achieved, as well as the extent to which various actions have succeeded in terms of achieving or
progressing towards ERASMUS+ objectives;
2) Efficiency, which considers the relationship between the resources used and the results/changes
generated by an intervention;
3) Relevance, which looks at the relationship between societal needs/problems and programme
objectives;
4) EU added value, which looks at changes that are the result of EU intervention alone;
1 Croatia participated in the European Commission's Lifelong Learning Programme and Youth in Action Programme as of
2009. 2 Not all evaluation questions are answered in this report. For example, based on research results, the extent to which the
realisation of specific objectives at the national level contributed to realisation of the ERASMUS+ general objectives could
not be concluded, since different factors which were not investigated in the research could also contribute to the realisation of
these broader programme objectives.
5
5) Coherence, which considers the extent to which an intervention does not contradict other
interventions with similar objectives. The analysis of coherence also involves looking at how well or
how poorly different actions work together.
Executive summary
The purpose of this report is to present the perspectives of project managers, end-users, and
representatives of key stakeholders in ERASMUS+ and its predecessor programmes on the
effectiveness, efficiency, relevance, coherence and complementarity, and added value of ERASMUS+
at the national level.
Programme effectiveness was operationalised in such a way that respondents assessed the overall
visibility of ERASMUS+, evaluated the contributions of Key Actions in realization of ERASMUS+
specific objectives, and evaluated accomplishments in realization of ERASMUS+ specific objectives.
On average, survey respondents estimate the visibility of information about ERASMUS+ as good. The
analysis shows that those who were beneficiaries of ERASMUS+ and its predecessor programmes, as
well as those who were engaged only in ERASMUS+, perceive the overall visibility of ERASMUS+
to a greater extent than others. However, analysis indicates a need to increase the overall visibility of
ERASMUS+ among applicants who did not receive grants (i.e. non-beneficiaries) and among those
who did not participate in the programme. Therefore, in the future the programme should aspire
towards wider-visibility outreach in order to inform a broader interested audience about its results and
good practices. There are also differences between action fields, since survey respondents from the
VET express a higher level of satisfaction with ERASMUS+ visibility and promotion on a national
and regional level, and with the results that the programme brings, as compared to respondents from
other fields. Concerning the Key Actions, KA1 and KA2 are recognized as having contributed to the
realization of ERASMUS+ specific objectives, while this is not the case with KA3, which could be
due to the fact that it is not known to a wider circle of applicants. However, even among its
beneficiaries, KA3 has a lower level of recognition in contribution to the realization of ERASMUS+
specific objectives. In general, the realisation of specific objectives that apply on an individual level
(i.e. skills development) is far more recognized among end-users, project managers and key
informants then the realisation of general and specific objectives that are aimed at producing
systematic (policy) changes.
Concerning the perspective of Youth sector respondents, results indicate that the realization of all
specific objectives is not seen in the same manner. Respondents recognize that ERASMUS+ enables
the realization of all of its specific objectives, but it seems that from the perspective of interviewees
(i.e. relevant actors and youth professionals), the programme lags behind in terms of strengthening
links between the Youth field and the labour market, since project managers recognize the
programme’s contribution to these objectives less than they do thatn to intercultural dialogue, the
improvement of key competences and skills of youth, and social inclusion. This comes as no surprise,
since programme activities in the YOU are not directly focused on youth employment. However, they
do enable participants to develop competences through informal learning by project participation,
volunteering and mobility, thus indirectly enhancing their position in the labour market. In any future
generation of the programme, the link between specific objectives and programme activities should be
examined closely, so that all objectives can be realized through programme activities.
Programme efficiency was operationalised by exploring overall perceptions of the efficiency of the
ERASMUS+ programme in general, as well as in relation to its predecessor programmes, through the
6
perception of the adequacy of IT tools, through the perception of obstacles to participation in
ERASMUS+, and through an assessment of administrative, financial and human capacities.
Respondents with experience in ERASMUS+ and its predecessor programmes recognize the
improvements that the programme ushered in by improving opportunities for cross-sectoral
cooperation, simplifying the financial management of projects, adjusting grants to the needs of users,
and standardizing administration in a way that made project management easier. However, it should be
noted that respondents recognize to a lesser extent that application procedures are clear and
customized to users, that evaluation criteria and implementation are transparent, and that regulations
for project financial assets management are clear. Since similar findings emerged in the LLP
evaluation study (Milanović-Litre, Puljiz and Gašparović, 2016),3 the future programme should
consider further simplification and adaptation of applications, financial procedures and regulations.
From the perspective of the YOU sector, respondents recognize to a lesser extent that grants are
adjusted to the real needs of their organizations, and that evaluation criteria and implementation are
transparent. Concerning differences between actions fields, it is indicated that survey respondents from
the HE recognize the efficiency of the programme in terms of administrative regulation
standardization and project financial management simplification to a greater extent than do
respondents from the SE. These findings should be analysed within the context of perceived obstacles
to participation and the assessment of administrative, financial and human capacities for
implementation of projects within ERASMUS+. In previous research (LLP Evaluation Study), survey
respondents and representatives of key stakeholder groups recognized challenges to implementation
such as a lack of administrative, human and financial resources, the discouragement of employees
through administrative formalities, excessive workloads, and the non-recognition or lack of
valorisation of participation in projects for the purpose of career advancement. Even though the overall
budget available to Croatia is seen as sufficient by the interviewed stakeholder representatives, half of
the surveyed project managers (50%) consider the financial resources available for pre-financing the
implementation of project activities insufficient, since beneficiaries receive grants in two or more
installments, meaning they need to have sufficient funds to finance activities before the full grant
amount is paid. In Croatia, this seems to be challenging to organisations with limited financial
capacities. Moreover, those who claim to have fewer organizational capacities in terms of accounting
staff expertise (required for financial monitoring) perceive ERASMUS+ regulations (i.e. accounting
rules, expenditures, payment terms) to be less clear. There are differences among sector fields, as those
from the YOU or the ADU perceive fewer obstacles and estimate that they have greater project
implementation capacities within the ERASMUS+ programme. To overcome these inequalities, any
future programme should try to adjust and design its rules and procedures in such a way as to
recognize and accept the differences among sectors, since competing conditions and potential are not
the same for all interested users. One way to tackle these challenges would be to introduce a ‘fast
track’ for newcomers and small applicants, i.e. a simplified procedure that is based on the same rules
and regulations, but that entails a smaller administrative burden and involves smaller grant amounts. In
the LLP, there were more opportunities for less experienced applicants to take part in smaller projects
that also won smaller grants in line with the capacities of those institutions.
Concerning the adequacy of IT tools, Mobility Tool+, URF, the Valor-dissemination platform and
Online Linguistic Support-OLS are perceived as being fully functional, with clear guidance in their
usage for programme beneficiaries. However, constant changes to IT tools in the initial years caused
3 Milanović-Litre, I. Puljiz, I., Gašparović, F., ed. (2016). Towards Internationalisation of Education – Participation of the
Republic of Croatia in the Lifelong Learning Programme, 1st ed. [online] Zagreb: Agency for Mobility and EU Programmes.
Available at: http://www.mobilnost.hr/cms_files/2016/12/1481199381_k-internacionalizaciji-obrazovanja-web.pdf [Accessed
23 Feb. 2017].
7
implementation problems and delays for the NA rather than for beneficiaries.
Programme relevance is explored by assessing the extent to which the specific ERASMUS+ objectives
are relevant to users in their sector. In both the YOU and E&T sectors, all specific objectives are
recognized as relevant to respondents, which is important in regard to success attracting and reaching
various target groups. Programme attractiveness at the national level is indicated by the high demand
rate (the number of applications received is much higher than the number of awarded projects).
However, in terms of reaching the target groups, with an increase in project complexity and a high
demand for funding, it became challenging to reach and maintain the interest of applicants having a
lower level of administrative and human capacities, coming from less developed regions (often with
less access to information and overall project experience) or being smaller, grassroots CSO
organisations. Moreover, there is a need to make the programme more relevant and attractive to
specific target groups outside the education field, such as enterprises as well as local and regional
authorities whose involvement is crucial for strengthening links with the labour market. Even though
there is some progress in terms of expanding access to disadvantaged groups (especially in the HE),
there is still a need for more dynamic outreach towards these groups across different sectors (i.e. the
VET).
In terms of complementarity with other programmes, in the HE field, Croatian HEIs participate in
programmes of academic mobility, such as bilateral scholarship programmes and the Central European
Exchange Programme for University Studies (CEEPUS). These programmes were implemented prior
to 2009, and continued to coexist with Erasmus, but the budget available through CEEPUS amounts to
just 3% of the ERASMUS+ funding available for Higher Education, and total funding for bilateral
scholarships amounts to 2.85% of the ERASMUS+ budget in the HE field. Therefore, ERASMUS+
had a significant benefit to HEIs; since it is the largest source of funding for international mobilities, it
has contributed to a significant increase in the scope of participation. In 2018, Croatian HEI’s will
have the opportunity to participate in a mobility scheme funded through the European Social Fund.
This mobility scheme will be open to students and staff from STEM fields (i.e. science, technology,
engineering and mathematics). Since participation among STEM students in comparison to that among
social science and humanities students is lower in KA1, multiple schemes targeting the same group
and offering different financial benefits could overlap, resulting in lower demand for Erasmus grants
or ESF grants, depending on the conditions.
Considering the lack of other funding sources, the added value of EU funding through the LLP and
ERASMUS+ has considerable significance for international cooperation and internationalisation
across sectoral fields. There are, however, institutional obstacles that may inhibit further growth.
A majority of survey respondents in the E&T agrees that the programme is realising the following
objectives: promoting participation in democratic life in Europe, raising awareness of an emergent
European lifelong learning area, enhancing the international dimension of education and training, and
establishing cooperation among EU institutions in the VET and HE.4 In addition, it increases the
attractiveness of European HEIs, improves the teaching and learning of languages, and promotes broad
linguistic diversity as well as intercultural awareness in the European Union.
The integrated framework of ERASMUS+ did bring significant changes, but the core values and
objectives of the programme remained similar; therefore the LLP Evaluation Study, with its
institutional-level focus, still provides relevant insight into the programme’s added value for
4 Based on the analysis of survey results (Appendix III).
8
educational institutions. In the HE field, according to the LLP Evaluation Study (Brajdić Vuković,
Klasnić and Baketa, 2016), Croatian universities had already established international cooperation
prior to joining the Erasmus programme in 2009. International cooperation was mostly organized
through bilateral agreements as well as Fulbright and Tempus Individual Mobility Grants, and through
the Central European Exchange Programme for University Studies (CEEPUS). Meanwhile, applied
universities and polytechnics had significantly less experience in internationalisation prior to joining
Erasmus. Because of their participation in Erasmus, HEIs widened partnerships with foreign HEIs and
established relevant procedures (especially credit recognition procedures), increased their visibility
and attractiveness, and introduced courses in foreign languages. Erasmus also triggered an increase in
mobilities among students as well as teaching and non-teaching staff. According to the monitoring
data analysis, since 2014 there is a significant increase in incoming mobilities (in:out ratio of students:
1:1.24), and it is expected that the number of incoming and outgoing students will balance out in the
following years. Even though complementary programmes such as bilateral scholarships and CEEPUS
continued to coexist with Erasmus, their budgets are much smaller. Therefore, it is not surprising that
Erasmus, as the largest source of funding for international mobilities on the national level, has
significant value for the HE sector.
Unlike the HE, other sectoral fields in education don’t have comparable programmes on the regional
or inter-regional level to support international mobility projects. Research conducted among project
participants and nonparticipants in LLP beneficiary organisations (Ančić and Klasnić, 2016)5 shows
that employees of beneficiary organisations recognize the impact of projects on educational
institutions, with the strongest impacts being on the personal development of participants, the
promotion of the European dimension in education, and an increased willingness/motivation of
employees to participate in professional development activities. The weakest impacts of projects on
the institutional level were recognized in the development of international and cross-sectoral
partnerships and the development of specific skills, knowledge and language competences. Similarly
as in the HE, across other educational sectors, there are difficulties and challenges in implementation
that can limit the further development of internationalisation and the scope of impact.
Methodology
The preparation of the methodology was guided by the evaluation criteria and questions defined by the
European Commission. We recognized the need, firstly, to adopt a retrospective focus in order to
identify results achieved and challenges faced during past implementation, and secondly, to include a
forward-looking component that explored the extent to which the current programme is in line with
expectations and objectives. Moreover, the analysis distinguishes, where appropriate, between the
Youth and Education & Training fields and the key actions. Evaluation criteria and questions appear in
Appendix I. Therefore, the choice of research methods and analysis is organised in line with the
overarching evaluation criteria: effectiveness, efficiency, relevance, internal and external coherence,
and complementarity. For the preparation of the national report, a detailed evaluation matrix was
developed in order to structure the evaluation questions, the related criteria, sub-questions, and the list
of participants.
A team of two external evaluation consultants/researchers with substantial evaluation expertise and
experience in the field of education carried-out the research and prepared corresponding activity
5 Ančić, B., Klasnić, K., 2016. Evaluation of the Comenius, Grundtvig and Leonardo Da Vinci Sectoral Programmes. In:
Milanović-Litre, I. Puljiz, I., Gašparović, F., ed. Towards Internationalisation of Education – Participation of the Republic of
Croatia in the Lifelong Learning Programme, Zagreb: Agency of Mobility and EU Programmes, pp. 60-150.
9
reports. The aim was to assess programme achievements and challenges, with a focus on (intended)
outcomes, by identifying what works as well as all relevant reasons, situations and circumstances. In
addition, an internal evaluator from the Department for Coordination and Quality Assurance of the
Erasmus+ programme prepared the report by synthesizing pre-existing evidence and monitoring data.
The methodology employed a mixed-method approach, and three separate research-analytical
activities were conducted in order to collect qualitative and quantitative data:
1) An online survey was conducted among project managers and legal representatives of applicant
organisations. The applicants from the LLP and YiA were included in order to allow a comparison
between ERASMUS+ and the previous generation of the programme. Programme beneficiaries and
non-beneficiaries (i.e. applicants who did not pass the selection process) alike were asked to respond
to the questionnaire. Survey results and a more detailed description of respondent structure appear in
Appendix III.
The dataset of beneficiaries and non-beneficiaries consisted of 4118 persons who are legal
representatives and/or project managers. The questionnaire was completed in full by 967 respondents,
which is 23,48% of the entire surveyed population.The surveyed sample indicated a similar regional
distribution, thus confirming the sample’s representation of the researched population.
Structure of the surveyed sample
% N
Structure of
beneficiaries and non-
beneficiaries
Beneficiaries of Erasmus+ and/or its predecessor
programmes
79,2 766
Non-beneficiaries of Erasmus+ and/or its predecessor
programmes
20,8 201
Total 100 967
Respondent´s role Project manager 68,6 663
Legal representative of organisation 19,2 186
Project manager/Legal representative of organisation 12,2 118
Total 100 967
Structure of
beneficiaries in regard
to Erasmus+ and its
predecessor
programmes
Participated in the LLP 62,2 475
Participated in YiA 18,4 141
Participated in Erasmus+ 73,6 564
Participated in other international cooperation
programmes (i.e., Tempus, Erasmus Mundus)
5,1 39
Structure of non-
beneficiaries in regard
to Erasmus+ and its
predecessor
programmes[1]
Submitted application for the LLP 45,3 91
Submitted application for YiA 13,4 27
Submitted application for Erasmus+ 55,2 111
Submitted application for other international cooperation
programmes (i.e., Tempus, Erasmus Mundus)
8,5 17
Type of organisation
which applied for
financial support under
Erasmus+ and/or its
predecessor
programmes
State administration organisation 3,0 29
Public institution 67,8 656
Non-governmental organisation 19,9 192
For-profit organisation 2,5 24
Informal youth group 0,7 7
Other 6,1 59
Total 100 967
10
Gender Male 22,2 215
Female 77,8 752
Total 100 967
M SD
Age 59,2 10,01
[1] The total number of non-beneficiaries that participated in Erasmus+ and/or its predecessor programmes is 201, but the
same respondent could have participated in more than one programme.
2) Semi-structured interviews (face-to-face and via Skype) involved representatives of stakeholder
groups: a) the implementation body (i.e. the national agency); b) key decision makers on the national
level (i.e. national authorities tasked with strategic management of the programme) as well as other
relevant decision-making bodies in the education field (i.e. relevant education agencies); and c)
relevant civil society organisations representing programme beneficiary groups (e.g. Youth, Higher
Education etc.). Within these stakeholder groups, key informants (i.e. individuals with specialist
knowledge about programme and policy processes at the national level) were identified and
approached with interview requests. In sum, we talked to 12 respondents, of whom 6 were women and
6 were men, and of whom all have had a wealth of experience related to the LLP and YiA, and now to
Erasmus+. Interview analysis results and detailed methodological explanations appear in Appendix
IV.
3) A combination of existing evidence was employed, including a review of the LLP Evaluation
Study, the Research-based Analysis and Monitoring of Erasmus+: Youth in Action (RAY Standard
Survey - national results), and a secondary analysis of Participant Reports. Research results and a
detailed explanation of the methodology appear in Appendix V.
The evaluation was organized as a two-stage process. First, the data were collected simultaneously
using three separate data collection methods (survey, interviews, secondary data collection). In the
second stage, based on the three different data sources, key findings relevant to each of the evaluation
criteria were identified. Overlapping and contradictory data were compared for different evaluation
criteria. Key findings emphasized across multiple data sources were combined and presented in the
National report on implementation and impact of ERASMUS+.
Methodological triangulation (i.e. the use of survey, the interviews and an analysis of secondary
sources) as well as triangulation of data sources (i.e. representatives of key stakeholders, legal
representatives and project managers in applicant organisations, and end users i.e. participants) was
used as a means of verifying the findings of different methods. The presumption was that, if data from
two or more sources converged on the same information, and if multiple sources corroborated in
support of the same conclusion, then there was more confidence in the conclusions drawn.
11
Evaluation results
Effectiveness6
Contribution to realisation of the ERASMUS+ specific objectives
Irrespective of their sectoral fields, respondents share a common opinion that the implementation of
programme activities has contributed to the realisation of the ERASMUS+ specific objectives. This is
especially so in terms of meeting specific objectives that are aimed at the individual level (i.e. those
related to personal development and future career prospects), while contributions to policy objectives
are recognized rather more sporadically.
As presented in Appendix III (survey results), from the perspective of project managers as well as
legal representatives from funded and non-funded projects alike, the ERASMUS+ programme
contributes to some extent to the realisation of all of the specifically listed objectives vis-a-vis the
Education and Training sector. The vast majority (i.e. 90%) of respondents from Youth as well as
Education and Training stated that the programme enables the improvement of key competencies and
skills in general, the improvement of language instruction and learning, the promotion of broad
linguistic diversity and intercultural awareness in the European Union, a contribution to social
cohesion, the enhancement of the international dimension of Education and Training, and the
emergence of a European lifelong learning area.
Contributions to specific objectives focused on the Higher Education field are recognized less
frequently. Although over 50% of respondents recognize the realisation of Higher Education field
objectives, almost 40% do not know and cannot estimate if the programme is contributing to their
achievment. If we control for differences between respondents with only LLP and YiA experience and
those with only ERASMUS+ programme experience, perceptions differ with respect to several
specific objectives. As compared to LLP and YiA respondents, ERASMUS+ respondents recognize to
a greater extent their programme’s impact on the improvement of key competencies and skills with
regard to labour market needs, cooperation between the world of education/training and the world of
work, and the internationalization of Education and Training.
Youth
The relevant stakeholders (i.e. NAUs, the NA, CSOs, and relevant agencies) in interviews recognize
the influence of ERASMUS+ on the development of skills and acquisition of new knowledge, and
6 In this chapter, answers to the following evaluation questions are provided: To what extent have ERASMUS+ and its
predecessor programmes contributed to the realisation of the ERASMUS+ specific objectives in your country? Are there
differences across fields? Please provide, where relevant, your assessment for each of the specific objectives, and provide
evidence and examples where possible.
To what extent have ERASMUS+ actions influenced policy developments in the domains of education and training, youth
and sport in your country? Which actions were most effective in doing so? What are the determining factors in making these
programme actions more effective? Is the size of the budget appropriate and proportionate to what ERASMUS+ sets out to
achieve? Is the distribution of funds across programme fields and actions appropriate in relation to their level of effectiveness
and utility?
What challenges and difficulties do you encounter while implementing the various ERASMUS+ actions? What changes
would need to be introduced in ERASMUS+ or its successor programme in order to remedy these? To what extent are the
approaches and tools that are used for disseminating and capitalising on the results of ERASMUS+ and its predecessor
programmes in your country effective?
12
emphasize that the programme has helped young people in terms of both their active involvement and
their acquisition of skills that are relevant to the labour market. Although ERASMUS+ contributes to
the realisation of all of its specific objectives, it seems that, from the perspective of project managers
and youth organisation representatives, it is less sucessful at connecting the Youth field with the
labour market than it is at its other proclaimed objectives. Between 28% and 40% of respondents do
not know and cannot estimate if the programme has contributed to the promotion of youth
participation in the labour market, the strengthening of links between the YOU and the labour market,
the complementing of policy reforms at the local, regional and national levels, and the development of
knowledge and evidence-based youth policy. It should however be mentioned that programme
activities within Youth are not geared towards strenghtening the connection between youth
organisations and enterprises, and they do not directly link young people to the labour market (in
contrast to the VET, whereby pupils spend a traineeship period abroad).
So it seems that the effectiveness of ERASMUS+ and its predecessor programmes in terms of youth
policy development and reforms as well as youth and the labour market is perceived to a lesser extent
among the wider public. In addition, if we control for the difference in perception between those who
are engaged in the YOU (i.e. those who have, either individually or through their organisations,
participated in projects connected with Youth) and those who are not, the only statistically significant
difference concerns two specific objectives. Those actively engaged in the YOU recognize the
contribution to the promotion of participation of youth in the labour market and to the strengthening of
links between the YOU and the labour market to a lesser extent. Other specific objectives, such as
intercultural dialogue, the improvement of key competencies and skills of youth, the enhancement of
the international dimension of youth activities, the increase of learning mobility opportunities for
young people, and social inclusion, have the highest level of recognition, since over 80% of
respondents recognize the role of ERASMUS+ in enabling the realisation of these objectives.
Additional evidence with respect to the YOU is available in the Research-based Analysis and
Monitoring of Erasmus+: Youth in Action Survey (national results). According to Gregurović (2017),
project leaders assessed that they had developed communication skills, foreign language skills and the
ability to cooperate in teams. It appears that project managers and end-users (i.e. participants)
recognize the relavance of the programme on an individual level, especially in terms of improvements
to key competences, but do not perceive a direct link to the labour market, even though
those competences are necessary for successful entry into the labour market. In the post-2020
programme, it is important to match specific and general objectives with the content of programme
activities that should contribute to the realisation of these objectives.
Education and Training
From the perspective of survey respondents, ERASMUS+ contributes to the realisation of all E&T
objectives. This conclusion is further supported by the shared opinion of the interviewed stakeholders’
representatives, who recognized the ERASMUS+ programme as an instrument for internationalisation.
Stakeholders further elaborate that ERASMUS+ is the only programme in Croatia that enables
mobility in the context of the European and international dimension of education. Individual mobility
enables the acquisition of new educational experiences, the development of foreign languages, and
new cultural experiences, adding to overall personal development. Interviewees further recognize that
mobility activities and the supporting administration mechanisms required to implement mobility
triggered organisational development within higher education institutions and other education
institutions. Croatian education curricula are viewed as outdated, so programme activities that are
focused on transversal skills and new knowledge acquisition usher important innovation into schools
13
that participate in the programme. Such schools become more aware of the European Development
Plan and rethink their long-term goals and strategies, introducing a new perspective into the Croatian
educational sector. Pre-primary education institutions are focusing more on ways in which they can
accommodate needs of vulnerable groups and increase awareness among children, thanks to
participation in ERASMUS+ projects.
According to the survey results, there is a difference between those who are experienced only with the
predecessor programmes and those who are experienced only with ERASMUS+; perceptions differ in
regard to several specific objectives. Respondents with ERASMUS+ experience assess that the
programme enables improvements to key competencies and skills with regard to labour market needs,
cooperation between the world of education/training and the world of work, a greater international
dimension in education and training, and an increase in the attractiveness of European HEIs to a
greater extent. Still, two specific objectives are less visible: the establishment of cooperation between
EU institutions in the HE and an increase in the attractiveness of European HEIs. Although over 50%
of respondents recognize the realisation of those two objectives, almost 40% do not know and cannot
estimate if the programme is enabling their realisation.
Visibility and promotion of the programme
General perspectives drawn from survey results indicate that ERASMUS+ visibility and dissemination
of information in terms of best practices/results is good. Respondents generally agree that
dissemination and usage of information about ERASMUS+ is satisfactory. If we control for
differences between beneficiaries of ERASMUS+ as well as its predecessor programmes and other
respondents, non-parametric testing indicates that the beneficiaries are, of course, more informed.
But the perception of the visibility of ERASMUS+ differs among action fields. Overall programme
visibility on the national level is recognized to a lesser extent among those respondents who are
primarily engaged in the ADU and the SE, while those who are primarily engaged in the VET are
more satisfied with how visible ERASMUS+ is on the national level. Similarly, the regional
distribution of information and the promotional activities of the NA are more widely recognized
among those who are engaged in the VET. The visibility of programme results in the areas of work in
which respondents are active is greater among those who are engaged in the VET and the SE. It seems
that VET respondents express greater satisfaction with programme visibility and promotion on the
national and regional levels, as well as with programme results.
Respondents were also asked in an open-ended format to write their ideas on the dissemination and
usage of relevant information. A recurring response relates to the need for better regional distribution
of information and promotion of good practices, both of which stem from ERASMUS+. Significant
regional differences in terms of the numbers of organizations that have applied for funding or the
numbers of proposed projects should be strongly taken into consideration in the context of
ERASMUS+ implementation. In Croatia, the highest share of programme applicants comes from four
major cities, and from the counties where these cities are situated. This regional distribution of
participating organisations indicates that applicants and beneficiary organisations from urban areas are
highly overrepresented, since the counties in question all have high concentrations of urban
population, high numbers of registered NGOs and educational institutions (at all levels), and higher
shares of population that have completed higher education and are at lower risk of poverty. In contrast,
counties with lower concentrations of urban population, higher risks of poverty and higher rates of
unemployment participate less in the programme. The findings of the Research-based Analysis and
Monitoring of Erasmus+: Youth in Action Survey (national results) confirm that the YOU is
14
characterized by a prevalence of participants from urban areas, a low share (10%) of participants from
rural areas, and slightly more representation in Youth Exchange projects (Gregurović, 2017.). There is
also a shared view among the interviewed stakeholders that the YOU sector has lost its visibility in the
ERASMUS+ programme, since with the emphasis on key actions rather than sectoral fields, some of
the content related to the structure of activities is lost in comparison to to the more distinctive Youth in
Action programme.
Improvement of key competences and skills – end users’ perspective
The perception among the end-users available from the Participants' Report (EU Survey) reflects the
previously stated opinions of stakeholders and survey respondents, suggesting an overall perception
that the influence of mobility on the enhancement of one’s abilities is positive, although there are
differences across sectoral fields/mobility types. There is a significant difference between two groups,
HE and VET learners, and the results indicate that VET learners have a more positive perception of
skill enhancement than HE students do. However, among HE students, those who have received a
disadvantaged background-related top-up grant have a more positive perception of skill enhancement,
while there is no statistical difference between students who spend traineeships abroad and VET
learners who were engaged in practical work. Results indicate that participants who have taken part in
traineeships perceive a higher degree of skills enhancement.
In the YOU field, the participants and group leaders who took part in ERASMUS+ youth activities
both expressed a high degree of agreement with statements about the development of skills/abilities
that are relevant for lifelong learning. In addition, the highest degree of agreement was expressed
about cultural knowledge, communication in foreign languages and cooperation in teams. The
enhancement of teamwork was also highly recognized as an outcome of project participation among
project leaders and project participants.
Teaching and learning languages and intercultural awareness
The percentage of ERASMUS+ participants in long-term mobility (E&T) or voluntary services (YOU)
declaring that they have enhanced their language skills is high among Croatian participants across all
sectoral fields. The proportion of learners from the VET who stated that they have improved their
language skills was significantly higher than that of students from the HE or volunteers from the YOU
field, while there was no statistically significant difference between the HE and YOU field
participants.
Active social and political participation
In the Participants’ Report, learners were asked whether, after having taken part in mobility, they
intended to participate more actively in the social and political life of their community. A majority of
HE students (60,3%), VET learners (80,4%) and EVS volunteers (75,5%) declared that they did intend
to participate more actively after their periods of time abroad. However, there is a statistically
significant difference between HE students on the one hand and VET learners and EVS volunteers on
the other, with the former declaring their intent to participate more actively in democratic life to a
lesser degree.
In the Youth field, the RAY Standard Survey (national report) effect of ERASMUS+ Youth projects
on the involvement of participants in active citizenship and social participation has been assessed on a
frequency scale with three response options. Respondents were asked about their behaviour before and
15
after the project. The results show that a significant share of participants recognized no change in pre-
versus post-project behaviour in this respect. There are only two areas: a) appreciation of cultural
diversity (64%) and b) interest in involvement in and development of youth policy (50%) in which
half of the participants recognized a change after returning from a mobility period abroad.
Enhancement of employability and career chances
According to the results, HE students have a more positive perception of their employment
opportunities than do VET learners. Among staff participants, school staff have a more positive view
of career opportunities enhancement than do YOU staff. There was no statistically significant
difference between other groups.
Policy developments
From the perspective of the interviewed stakeholders, in terms of contributions to youth policy, there
is a view that, although there are visible results in KA1 and KA2, these are often limited to the micro -
local level, and there is a lack of an integrated broader impact, especially in the policy domain. The
interviewees recognized the same issues in the E&T, emphasizing the need to think about projects in
the context of long-term goals and the policy cycle, and not just as micro-level, short term initiatives.
They also recognized a need for more synergy and joint effort to address national objectives, as well as
the current lack of strategizing and continuity.
Even though all three Key Actions are recognised as having contributed to the realisation of Erasmus+
specific objectives, the KA3 objective of providing support for policy reforms is not perceived as a
successful mechanism in comparison with the first two actions. This perception is the same regardless
of whether respondents are engaged within the YOU, the E&T, or any action field for that matter.
It has been recognized that in the YiA and LLP there was the possibility to set national objectives in
ERASMUS+ that changed in favour of common EU goals, which brought wider possibilities while
potentially reducing focus on nationally relevant objectives. There is a view among some stakeholders
that questions as to the wider impact of ERASMUS+ should be addressed nationally. This could be
done through one working group that would consist primarily of experts related to youth research and
policy, and another for educational and higher educational policies, which would be tasked with
reflecting on ERASMUS+ objectives in relation to other programmes and initiatives, and with a view
towards the wider social and political context. The recommendations of these working groups could
be utilized by decision makers to increase synergy among different goals, and to produce a wider
impact at the national level. Some of the NAU interviewees noticed that centralized actions, such as
KA3 - Initiatives for policy innovation - European policy experimentation, could be utilized to
develop new policies. The involvement of national authorities in such international centralized
projects, which demand cooperation between relevant authorities and other actors, can have a wide
impact on the formation of future policies on the national level.
Adequacy of the ERASMUS+ budget
Two patterns appear to be present in terms of Erasmus+ funding and financial management: firstly, a
mismatch between supply and demand, and secondly, differences between sectoral fields in terms of
average participant cost and average project grant. The discrepancy between available funding and the
demand for such is reflected in low success rates across all fields, and especially in KA2, as well as in
the high share of quality applications that receive no grants due to the lack of funds. The difference in
16
the average cost between actions is significant. For example, the mobility costs of students and staff in
the HE are a few times higher than the cost per participant in the YOU. The distribution of funds
between the E&T and the YOU is almost the same as it is between the LLP and YiA, although during
the initial years of Erasmus+, demand for funding increased particularly in the YOU, and our results
indicate that respondents from this sector perceive having more capacities to implement projects
successfully.
Challenges and difficulties in implementation
Even though at this point there is no evidence regarding the impact of ERASMUS+ on participating
institutions, there is an in-depth evaluation of Croatian educational institutions that took part in the
LLP in 2016. This evaluation is still a relevant source of information, since its findings indicate
challenges that the participating organisations faced on an institutional level as well as in regards to
participation in and implementation of international projects.
Some of the key findings in the LLP Evaluation Study (Milanović-Litre, Puljiz and Gašparović, 2016)
are related to institutional obstacles to the participation of adult education providers, primary and
secondary schools as well as pre-primary institutions in programme activities. Institutional obstacles
identified by the respondents include: a) limited financial resources for project pre-financing (paid to
the beneficiary prior to the first and second instalments), b) a high administrative burden that
discourages staff from applying for programme funding, c) a perception that participation in the
programme demands lots of financial, human and administrative resources that are not available
within institutions, d) insufficient foreign language skills among staff and pupils, e) a lack of formal
recognition of teacher training/professional development activities carried out abroad (as part of the
projects) as well as an insufficient level of information regarding project participation possibilities.
The statistical analysis conducted within the LLP Evaluation Study confirmed that the more
institutional obstacles are recognized at an institution, the lower the impact of project participation;
since obstacles are negatively correlated with indexes of progress on an institutional level. The authors
further recognized four intercorrelated groups of obstacles: a) administrative obstacles; b) a lack of
information about the programme; c) a lack of interest and motivation on the part of staff; and d) a
lack of institutional knowledge to successfully implement projects. (Ančić and Klasnić, 2016).
During the LLP period and after 2014, HEIs also faced challenges in Erasmus implementation.
Findings suggest that there is a difference among HEIs in terms of human and financial capacities
invested in internationalization. Some HEIs decide to advance other strategic objectives and tend to
move resources from international cooperation to other areas, resulting in a lack of human capacities
to implement the programme at a quality level and discontinuity in progress. IRO employees and
ECTS coordinators identify the following obstacles to implementation: an insufficient number of
Erasmus grants (i.e. high demand versus low funding available), insufficient administrative human
capacities at HEIs (mostly the case at major national universities), a low number of courses available
to foreign students that are offered in foreign languages, a lack of formal recognition of teacher
training/teaching assignments abroad, and a lack of support for students of lower socioeconomic status
who are not able to spend a mobility period abroad. Some HEIs signed bilateral agreements that were
rarely if ever realized (in practice only 20% of bilateral agreements were realized).
Moreover, an insufficient number of courses offered in foreign languages can prove challenging for
inbound students in Croatia. Erasmus coordinators/ECTS coordinators emphasize that teaching staff
are often not interested in conducting courses in foreign languages, since no form of valorisation is in
place for teaching in a foreign language. Similarly, a lack of valorisation of short-term teaching staff
17
mobilities (i.e. lasting less than three months) on the institutional level is perceived as an obstacle to
wider participation of teaching staff at HEIs (Brajdić Vuković, Klasnić and Baketa 2016: 203).7
We asked the survey respondents about challenges and obstacles recognized in the LLP in order to
determine if the same obstacles persist among ERASMUS+ applicants. In the Education and Training
field, similarly as with the predecessor programme, the main obstacles recognized by at least 50% of
respondents concern a lack of administrative, human and financial resources, the discouragement of
employees via administrative formalities, excessive workloads and the non-recognition of
participation in projects for the purpose of career advancement. Concerning financial resources, the
lack of capacities mainly reflects challenges in pre-financing for project implementation activities,
since over 50% of respondents declare this as an insufficient capacity. A similar finding relates to the
time available to employees for project participation. In regard to administrative and human resources,
respondents recognize insufficient capacities in accounting staff expertise required for the financial
monitoring of EU projects, the knowledge and skills of employees in regard to the administrative
monitoring of projects (i.e. the preparation of written reports, project proposals and accompanying
documentation) and practical experience in project management to a greater extent. It is important to
note that statistical analysis shows that those respondents who assess their organizations to have lesser
capacities also assess ERASMUS+ as less efficient. In the Youth field, respondents seem to encounter
fewer challenges to implementation than do respondents from the Education and Training field. They
also assess to have more knowledge and skills among employees concerning the administrative
monitoring of projects, practical experience with project management, managerial skills, intra-
organizational cooperation and communication, well-developed partnerships with institutions from
abroad, and accounting staff expertise required for the financial monitoring of EU projects.
Points for improvement
● Ways to achieve a more systematic, policy-level impact, rather than a micro (i.e. local) -level
effect of the programme should be thought through. A strategic approach is needed, coupled
with a bottom-up perspective that is guided by societal needs. The possibility of having
national objectives in addition to common European objectives can contribute to a stronger
policy-level impact in member states. Cooperation with policy experts could be organized at
the national level in order to reflect programme goals in a wider social and political context,
and to increase the possibility of synergy between programme objectives and national policies
or other complementary programmes.
● The introduction of longer projects or thematic programmes could increase the continuity and
sustainability of programme results.
● The relationship between the size and number of grants may require some consideration.
Downsizing at least some of the grants may broaden the participant base. At the same time,
the possibility of assuring adequate funding for high-level projects aimed at systemic-level
impacts should not be omitted either.
● An increase in funding for the Youth sector is needed, as it currently has a budget that is at
odds with its substantial number of applicants and participants. As KA2 and KA3 aim for
systematic impacts, appropriate funding is needed in order to achieve such purposes.
7 Brajdić Vuković, M., Klasnić, K., Baketa, N., 2016. Evaluation of Erasmus Sectoral Programme. In: Milanović-Litre, I.
Puljiz, I., Gašparović, F., ed. Towards Internationalisation of Education – Participation of the Republic of Croatia in the
Lifelong Learning Programme. Zagreb: Agency for Mobility and EU Programmes, pp.159-226.
18
Efficiency8
Integration of programmes in ERASMUS+
If we compare the efficiency of ERASMUS+ with its predecessor programmes, at least from the
perspective of users (i.e. survey respondents) with the experience to assess it, it is mostly recognized
that ERASMUS+ brought improvements. These are recognized in terms of improving opportunities
for cross-sectoral cooperation, simplifying financial management, adjusting grants to the needs of
users, and standardizing administration, which made it easier to manage projects. This is especially so
for cross-sectoral cooperation and the simplification of project financial management. Insofar as
ERASMUS+ efficiency in general is being evaluated, a majority of respondents agree that the
programme is efficient because project implementation timelines are in accordance with the real
abilities of users, and because time granted for reporting on project results is adequate. A somewhat
lesser majority of respondents consider application procedures to be clear and user-friendly, evaluation
criteria and implementation to be transparent, and regulations regarding the management of projects’
financial assets (i.e. accounting rules, expenditures, and payment terms) to be clear.
Respondents engaged in the YOU have different views about the efficiency of an integrated
programme, since they recognize that grants are adjusted to the real needs of user organizations to a
lesser extent. Additionally, when commenting on the efficiency of ERASMUS+ in general, they are
less inclined to agree that evaluation criteria and implementation are transparent.
Among interviewees, opinions related to ERASMUS+ as an integrated programme are mixed.
Although the programme is more comprehensible and coherent in terms of administration, there is a
common view that the YOU sector has lost visibility within the ERASMUS+ programme. With an
emphasis on key actions rather than sectors, some of the content is lost (as perceived by respondents)
in comparison to the Youth in Action programme. Youth is perceived as a heterogenous group, e.g.,
needs and life goals are different for 16 year-olds than they are for 27 year-olds, so a programme
framework that requires a highly professional approach keeps youth initiatives and less experienced
NGOs away from ERASMUS+. Therefore, the spread and the impact of ERASMUS+ is uneven across
the YOU sector, leaving out a lot of potential users of different ages and experience levels.
Interviewees have suggested that the next generation of programme should consider earmarking funds
for short-term or at least less professional initiatives, and construing projects in such a way that highly
professional formal support is not required in order to apply.
The “older generation” of programs is perceived to be much more complicated, with an
overabundance of different activities being too complicated for citizens to understand and for
implementing bodies to implement and monitor. Now, with only three key actions, it has become
much easier to explain, implement and monitor ERASMUS+ projects. Financial monitoring has also
become easier, although from the perspective of respondents from the NA, recurring changes remain
among the most problematic challenges for all involved in the implementation process. A common
opinion is that this change to ERASMUS+ was significant and far-reaching in terms of programme
architecture and implementation. However, those changes are difficult and should not be made again
8 To what extent is the system of cooperation and division of tasks between the Commission, the Executive Agency, National
Agencies, the European Investment Fund, National Authorities, Independent Audit Bodies, and the ERASMUS+ Committee
efficient and well-functioning from the point of view of your country? What are areas of possible improvement or
simplification in the implementation of ERASMUS+ or a successor programme? To what extent is the level of human and
financial resources that is available for the implementation of the programme in your country adequate? What steps did you
take to optimise the efficiency of the resources deployed for ERASMUS+ implementation in your country?
19
unless necessary. As implementing bodies and users need time to understand and adjust to changes,
there is a common opinion that any future programme should not involve a lot of changes. This
especially concerns IT tools, whose diminished functionality at the beginning of the implementation
period caused serious problems for users and the NA alike, thus creating lots of tension and
frustrations.
The administrative burden is perceived as still being quite heavy, especially for smaller schools and
other smaller organisations in general, because they lack sufficient experience and capacity to
reasonably cope with the administration. Some respondents from the non-governmental sector hold the
opinion that CSOs have “lost out” in the new generation of the programme, because the project
application and selection process has become much more difficult. In addition, they believe that they
face much harsher competition concerning KA2 and KA3 activities, which is especially difficult for
smaller higher education institutions as well as smaller CSOs in both the YOU and E&T fields. They
perceive that larger governing bodies, such as agencies and larger universities, are in a much better
position to access KA2 and KA3 projects, and view them as unfair competition.
Youth
If we analyse perceptions of the efficiency of ERASMUS+ among respondents engaged in the YOU
and compare them with all others, some significant differences emerge. In terms of the efficiency of
the integrated ERASMUS+ programme, respondents engaged in the YOU recognize to a lesser extent
that grants are adjusted to the real needs of user organizations. In addition, as for the efficiency of
ERASMUS+ in general, they agree that evaluation criteria and implementation are transparent to a
lesser extent. They also perceive themselves as having more sufficient human and administrative
capacities to implement projects then respondents from other fields.
In the interviews, one frequently mentioned problem within the YOU sector is that partnerships are not
truly achieving cooperation on projects – that they are partnerships in name only. Respondents believe
that this should be improved through additional support in terms of requirements within the
programme.
Education and Training
Concerning perceptions of the efficiency of the integrated ERASMUS+ programme, survey findings
reveal the difference between respondents engaged in the HE and those engaged in the SE.
Respondents in the HE recognized improvements in the standardization of administrative regulations
and the simplification of financial management more often than those engaged in the SE.
In terms of the adequacy of human capacities, the recognition of obstacles to participation in
ERASMUS+ seem to differ across E&T fields in several respects, and mostly in such a way that
respondents from the ADU encounter fewer obstacles. For instance, those in the ADU experience
fewer obstacles than those in the SE in terms of insufficient interest among employees to participate in
projects, insufficient knowledge of foreign languages and a lack of administrative staff knowledge for
project implementation. In comparison to those engaged in the VET, the HE and the SE, those
engaged in the ADU perceive to have more sufficient capacities in terms of time available to
employees for project participation.
Cooperation and division of tasks
20
In terms of cooperation and the division of tasks between different bodies, there has been agreement
among interviewed stakeholders’ representatives that this division works very well, and that all of the
actors perform their duties regarding implementation according to agreement. Ministries, agencies and
users from the non-governmental sector are very satisfied with the work of the NA, and all of the
comments were predominantly positive. Comments were directed towards advertising and
implementation efforts as well as programme monitoring. The only problem mentioned was related to
weaknesses in the evaluation system – in particular the need for some of the evaluators to improve
their evaluation skills. This is evident in their comments on project proposals, which are perceived as
insufficiently expert. It should also be noted that some respondents hold that the NA could be more
independent in terms of decision making. They have recognized two main problems. One is related to
difficulties in the NA’s internal functioning during times of political transition after elections, when
the NA awaits necessary signatures from the NAU, which makes the everyday functioning of the NA
much more difficult. The second problem, identified by respondents from non-governmental
organizations, is related to the opinion that the NA has much more insight into and understanding of
the programme than any other actor, and that it should therefore have much more independence and
political influence when it comes to ERASMUS+-related decision making on the national level. The
NAUs are very satisfied with the NA, but also believe that the NA is independent enough, and that it is
impossible for such an ‘implementing body’ to be more independent. The NA would like to stay
within the same framework of independence as it relates to implementation and to its own role as a
partner and interlocutor in policy decision making. One of the significant and highly emphasized
problems is a general shortage of staff at NAUs and the NA, which makes programme implementation
unnecessarily difficult and overly reliant on the enthusiasm of existing staff. There is a common
opinion among respondents from the other relevant governmental agencies, ministries, and the NA
that there should be more communication between all actors (especially between relevant government
agencies in the field of education), within the relevant ministries (in terms of their different divisions
that are in charge of ERASMUS+) and, lastly, between ministries and agencies in the field of
education. In order to really achieve better synergy in programme planning and implementation, there
should be more connectivity, a more strategic division of tasks, and more communication and
planning, both horizontally and vertically. This would benefit all, because obviously all respondents
are aware of it. But more staff would be needed before any of the above are even possible.
IT tools
In order to assess the adequacy of IT tools, respondents with experience in ERASMUS+ and its
predecessor programmes were asked about Mobility Tool+, URF, the VALOR-dissemination platform
and OLS- Online Linguistic Support. Although there are small differences across the distributions of
answers, none are statistically significant. Respondents think that IT tools such as Mobility Tool+,
URF, the Valor-dissemination platform and Online Linguistic Support-OLS are adjusted to the IT
knowledge of users, are fully functional, and include clear instructions for users. The adequacy of IT
tools is being assessed in the same manner, regardless of the sector in which respondents are engaged.
Adequacy of financial resources for implementation
ERASMUS+ brought expectations of simplified financial management. Next to the harmonisation of
rules and regulations across sectors, one key approach in this respect has been the introduction of the
unit cost system and flat rates. It was expected that the unit cost system would make financial
management easier for the NAs and beneficiaries. More harmonized rules across sectors and actions
were adopted in ERASMUS+, although the budget distribution continues to reflect a more sectoral
21
division, as in the predecessor programmes. The distribution of funds between the E&T and the YOU
is almost the same as in the LLP and YiA, with 80% of the funds being allocated to the E&T and 20%
of the overall country budget allocated to the YOU. However, during the initial years of ERASMUS+,
the demand for funding in Croatia increased, particularly in the YOU. As shown in Appendix VI, the
demand for funding, as indicated by the number of proposed projects, is significantly higher in the
YOU than in other sectoral fields, and the number of applications reaching the minimum quality
threshold is almost two times higher than the number of grants awarded across the three Key Actions
and across all fields. As presented in Appendix VI, a closer review of quality applications shows that,
across all actions, the number of quality applications exceeds the number of awarded projects. This is
particularly the case with the YOU (KA1), where 60% of submitted applications passed the quality
threshold, but only 26% secured grant funding, indicating that insufficient funds, rather than poor
application quality, tend to drive application rejections. A discrepancy between available funds and
requested funding is also particularly evident in the YOU, with success rates lower than those in other
sectoral fields (8,52% for KA105; 7,69% for KA205).
As previously mentioned, on the institutional level, applicants face challenges in the pre-financing of
project activities. A similar finding relates to the time available for employees to participate in
projects. In regard to administrative and human resources, survey respondents recognize to a greater
extent insufficient capacities in terms of accounting staff expertise required for EU project financial
monitoring, the knowledge and skills of employees in regard to the administrative monitoring of
project (i.e. the preparation of written reports, the project proposal and accompanying documentation)
and practical experience with project management. Problems with capacities in terms of human and
financial resources are more often mentioned in the E&T than in the YOU, and are highlighted as
maybe the most important problems related to implementation. Interviewees from the NAUs as well as
the non-governmental and non-profit sectors are also recognizing these problems.
Even if experts who can successfully design high quality projects are available, there is a problem with
the administrative part of the project that has to be delegated to administrative staff. Such staff are
usually lacking not only in terms of necessary knowledge and skills related to international projects,
but also – and mostly – in terms of the number of people that can be burdened with the the additional
workload. There is a common opinion that ERASMUS+ project implementations should include an
institutional mechanism that can support those who are willing to do project work, whereas currently ,
such a mechanism is largely absent, so every single task, including administrative ones, becomes the
additional responsibility of the individuals who applied for the project. Moreover, those individuals
are not additionally rewarded for handling international projects. There are lot of teachers, professors
and researchers who are enthusiastic enough to be willing to apply for ERASMUS+ projects despite
the lack of support, but enthusiasm is not an inexhaustible source. In order for ERASMUS+ to be
more successful, national policies that would facilitate international project applications should be
implemented, together with institutional strategies related to internationalisation and, therefore,
strategies related to the management of administrative responsibilities in the context of international
projects.
One problem detected within the HE sector is related to centralized activities where project proposals
are quite rare, although grants in such cases are perceived as potentially the most beneficial for
institutions. Respondents express the opinion that a lack of such projects might be related to the
absence of international connections and partnerships that would facilitate such project ideas and
proposals, and it is noted that these are issues with which HE institutions should be most concerned.
Human and financial resources are seen as problems in the YOU sector, in terms of the
22
aforementioned problem regarding the professional potential of youth NGOs on the one hand and the
administrative needs of projects on the other. The evaluation study examined the financial and human
resources of HEIs under the predecessor programme, and revealed similar challenges in terms of
human/financial/administrative capacities.
Points for improvements
● Further simplification and adaptation of application and financial procedures are needed, as
are the regulation and integration of IT tools. The Commission is encouraged to improve and
integrate existing IT tools, rather than develop new ones. Simplifying project applications and
data entry would be particularly beneficial.
● An increase to the budget for KA2 is needed, as demand for funding exceeds available grant
monies considerably, resulting in low grant award rates.
● Due to programme complexity, participation among non-formal youth groups is rather scarce.
Consideration should be given to re-instituting the national youth initiatives that were
available in the YiA.
● A more flexible budget may be more responsive to ever-changing project demand. For
instance, allowing transfers between key actions and/or between sectors may foster both
efficiency and effectiveness in smaller countries with a limited number of experienced
applicants.
● Applicants with different levels of financial, human and administrative capacities should have
equal access to the programme. Currently, access is limited not only by financial constraints,
but also by operational and technical aspects. One way to tackle these challenges would be to
introduce a separate track for newcomers and smaller applicants, i.e. a simplified fast-track
procedure based on the same rules and regulations, but with a reduced administrative burden.
Relevance9
The relevance of ERASMUS+ and its success in attracting and reaching various target groups also
depends on the relevance of its proclaimed objectives. That is why, in the survey, respondents were
asked to assess, based on the experience of their respective sectors, whether the proclaimed
ERASMUS+ objectives were sector-relevant. Interviews with relevant stakeholders and survey results
provide more insight into the needs of beneficiaries, applicants and relevant stakeholders, while
monitoring data can give insight into the participation of target audiences and the demand for funding.
Perceived relevance of objectives in regards to the needs of target groups
Youth
All respondents, regardless of whether they have participated in ERASMUS+, gave their view on the
relevance of the ERASMUS+ specific objectives important to the YOU. Objectives such as promoting
participation in the labour market, strengthening links between the YOU and the labour market, and
developing knowledge- and evidence-based youth policy lag behind in relevance compared to other
ERASMUS+ objectives. Respondents are seen as being more concerned with local issues related to a
lack of initiative on the part of local governments to recognize youth as an important population, to
9 To what extent are the needs of different stakeholders and sectors addressed by the ERASMUS+ objectives? How
successful is the programme in attracting and reaching target audiences and groups within different fields of the programme's
scope? Is the ERASMUS+ programme well known to the education and training, youth and sport communities? In case some
target groups are not sufficiently reached, what factors are limiting their access and what actions could be taken to remedy
this?
23
further help in defining policy goals and implementing ERASMUS+ programme results.
Education and Training
As for most of the specific objectives in the E&T, a majority of respondents recognize them as being
sector-relevant except for two goals, which are specific to the HE. Given the field-specific focus of
these goals, the difference is understandable. It is interesting to note the differences between those
respondents who only have experience with ERASMUS+ and those who only engaged in its
predecessor programme. Those with experience in “old programmes” exclusively recognize to a lesser
degree objectives such as the improvement of key competencies and skills with regard to labour
market needs, the improvement of quality, excellence in innovation and internationalization on the
level of education institutions, the improvement of the teaching and learning of languages, and the
promotion of broad linguistic diversity and intercultural awareness in the European Union. Not all of
the specific objectives proclaimed by the ERASMUS+ programme are targeted at all action fields
within the E&T, hence the differences in assessing the relevance of the objectives. For instance,
objectives like improving key competencies and skills with regard to labour market needs and
strengthening cooperation between the world of education/training and the world of work are
recognized among all respondent in the E&T, although to a greater extent among those in the VET.
The biggest differences can be seen with objectives such as establishing cooperation of EU institutions
in the HE and increasing the attractiveness of European HEIs, since these are mostly recognized
among those engaged in the HE.
Participation of target groups
The Erasmus+ Programme Guide recognizes individuals – students, trainees, apprentices, pupils, adult
learners, young people, volunteers, professors, teachers, trainers, youth workers, professionals from
organisations active in the fields of Education and Training as well as Youth – as target groups of the
programme. Aside from individuals, participating organisations (i.e., mostly organisations from the
field of education) are recognized as relevant programme actors, including groups of at least four
young people who are active in youth work, although not necessarily in the context of youth
organisations (also referred to as informal groups of young people). Within these general target
populations, in order to promote equity and inclusion, the programme also targets specific groups that
face challenges to programme participation. These three groups consist of participants with: a)
disadvantaged backgrounds, b) fewer opportunities and c) special needs. Within the YOU, an
Inclusion and Diversity Strategy has been designed as a common framework to support the
participation and inclusion of young people with fewer opportunities.
In order to determine how successful the programme is in reaching target audiences and groups within
different fields, it is important to understand the scope of participation. The high level of demand for
funding indicates that there is a need for international projects and mobility opportunities in the
Croatian national context. This is particularly the case among staff from primary, pre-primary and
secondary education institutions in which teaching and non-teaching staff often lack opportunities for
professional development or lifelong learning opportunities due to the limited financial resources that
are available to their institutions. The number of participating primary and secondary educational
institutions dropped in ERASMUS+, due to a lower success rate and an overall lower number of
projects. Around 20% of all educational institutions (in pre-primary, primary, and secondary
education) in Croatia have taken part in the programme since 2009, leaving lots of space for widening
participation. There is also an urban-rural divide recognized in beneficiary distribution. The regional
24
distribution of participating organisations and applicants reveals that organisations from more
developed regions/counties (i.e. those with a higher level of urban population, a higher share of the
population that has completed higher education, and a lower poverty rate) are overrepresented among
beneficiaries. Data indicate that there is still a need for information provision and capacity building in
order to widen participation among organisations from rural and remote areas, thus achieving more
balanced participation at the national level. Contrasting this situation is participation among HEIs in
the programme, which is extremely high (46 out of 49 registered HEIs take part in KA1).
Some other groups, including enterprises and local as well as regional authorities, have not
significantly raised their level of participation in ERASMUS+, for even though there has been an
increase in the number of applications received from these organisations, they often do not pass the
thresholds for funding. Once again there is a need for capacity-building and tailor-made information
campaigns for the benefit of these specific groups.
In the YiA, a majority of applicant and beneficiary organisations were NGOs, and there was good
participation among youth organisations as well. Since 2014, there has been an increase in the number
of NGOs among participating organisations, while some of the traditional beneficiaries in the LLP,
such as primary schools, are participating less in the new programme. In the YOU, there is still a
prevalence of NGOs, while there is very modest participation among smaller, youth-led and volunteer-
based youth organisations.
In the last three years there has been an increase in the number of newcomers, but a still-significant
share (25%) of participating organisations are recurrent beneficiaries. ERASMUS+ offers more
opportunities for different organisations to submit applications across sectoral fields (especially in
KA2), although limited funding and more demanding projects seem to discourage less experienced,
capacity-lacking organisations from applying or successfully passing the selection process.
In terms of participation among specific target groups (i.e. those with fewer opportunities, special
needs and disadvantage backgrounds), there is still a need to widen access through more dynamic and
targeted outreach at the institutional and national levels. The introduction of top-up grants for
disadvantaged background students in the HE has broadened mobility participation among students
from a lower socio-economic background, and is generally regarded as a positive step towards more
inclusion and equity. Meanwhile, the share of participants with fewer opportunities in the VET and the
SE is relatively low, considering that in the VET there is a high concentration of students of lower
socio-economic status. The introduction of top-up grants, based on national criteria, could be a useful
measure to support participation among pupils from disadvantaged backgrounds. In conclusion, there
is still a need for targeted capacity-building and tailor-made information campaigns in order to secure
more diverse beneficiaries and applicants.
Points for improvement
● There is a need to make the programme more relevant and attractive to specific target groups
outside the education field, such as enterprises as well as local and regional authorities, whose
involvement is crucial for strengthening links with the labour market. Even though there has
been progress in terms of widening access to disadvantaged groups (especially in the HE)
there is still a need for more dynamic outreach towards these groups across different sectors
(i.e. the VET).
● The programme should be further promoted and advocated outside the primary target groups
(i.e. potential beneficiaries) so that local, regional and national governing bodies can get
familiar with the possible impacts of the programme. Broader advocacy could trigger
25
necessary changes in attitudes, support policy adjustments, facilitate implementation and steer
programme impacts.
● Stronger partnerships should be built between the NAs, leading youth CSOs and academia
with a view towards educating grassroots youth CSOs/initiatives on key features of youth
policies, thereby fostering best-practice examples.
● The positive practice of top-up grants should be built upon for disadvantaged students outside
of the HE sector (and particularly in the VET sector).
● Access to centralized activities should be made more transparent, and their visibility should be
increased.
Internal and external coherence and complementarity10
Respondents from the YOU sector have frequently mentioned their perception that the YOU has
suffered in terms of visibility under ERASMUS+, and that although a kind of synergy has been
achieved in ERASMUS+, this could be further thought through within the new programme. There are
no other mentioned problems or obstacles related to the coherence of the programme; respondents are
mostly satisfied. What is frequently mentioned, however, is the challenge of adjusting ERASMUS+
and ESF goals. Respondents from agencies, ministries and CSO sectors all see a potential problem in
those two funds targeting the same users in the future, which should be avoided by more careful
thinking through of the goals of both ERASMUS+ and the ESF. There should be complementarity
rather than competition.
In the HE, Erasmus mobilities had a significant added value for the internationalisation of HEIs, since
the programme is the largest source of funding for international mobilities on the national level, and it
has increased the scope of participation as compared to the previous period. The Erasmus programme
also triggered an increase in incoming mobilities among students as well as teaching and non-teaching
staff into Croatia. Complementary programmes, such as bilateral scholarships and the CEEPUS,
continued to coexist with Erasmus. Although on an annual basis, the budget available through
CEEPUS amounts to just 3% of the ERASMUS+ funding available for Higher Education, and total
funding from bilateral scholarships amounts to 2.85% of ERASMUS+ HE funding, these programs
complement each other, since bilateral scholarships give students and staff opportunities to spend
mobility periods outside of European countries. This overlaps with opportunities available within
ERASMUS+ from 2015, under the KA1 (KA107) International Credit Mobility activity, but due to
limited funding and differences in activity content, these programmes are rather complementary. In
contrast to the HE, other sectoral fields in education do not have comparable programmes on the
regional or inter-regional level to support international mobility projects.
10
To what extent are the various actions that have been brought together under ERASMUS+ coherent? Can you identify any
existing or potential synergies between the actions within ERASMUS+? Can you identify any tensions, inconsistencies or
overlaps between actions within ERASMUS+? To what extent does ERASMUS+ complement other national and
international programmes available in your country? Can you identify any tensions, inconsistencies or overlaps with other
programmes?
26
European added value and sustainability11
Under these criteria, the definition of EU added value is examined, alongside changes that are
reasonably attributable to EU intervention.
According to the Erasmus evaluation (Brajdić Vuković, Klasnić and Baketa, 2016), Croatian
universities had already established international cooperation prior to joining the Erasmus programme
in 2009. International cooperation was mostly organized within bilateral agreements, Fulbright and
Tempus Individual Mobility Grants, and the Central European Exchange Programme for University
Studies (CEEPUS). Meanwhile, universities of applied science and polytechnics had significantly less
experience in internationalisation prior to joining Erasmus. Still, regardless of the type of HEI,
research participants recognized widening partnerships with foreign HEIs, the establishment of
procedures (especially recognition procedures), an increase in the visibility and attractiveness of
institutions, and the introduction of courses in foreign languages since participation in Erasmus
started. Erasmus also triggered an increase in mobility among students as well as teaching and non-
teaching staff. According to the monitoring data analysis, since 2014 there has been a significant
increase in incoming mobilities, and it is expected that the number of incoming student and staff
mobilities will balance out in the coming years. Even though complementary programmes, such as
bilateral scholarships and the CEEPUS, continue to coexist, there is still a high demand for
ERASMUS+ funding in the HE. Therefore, it is not surprising that the programme represents a
significant added value to HEIs, since it is the largest source of funding for international mobilities.
Still, with the increase in outbound and inbound mobilites, institutional obstacles that could limit
further growth are recognized as well. Some of the institutional obstacles include: a lack of recognition
or valorisation of short-term mobilities on the part of teaching staff, a lack of strategic management of
Erasmus funds (there is a discrepancy between institutional targets and available funding, and some
HEIs lack internationalisation strategies altogether), difficulties in the social and academic integration
of incoming students, a lack of dynamic outreach and information provision towards special groups of
students (i.e. those of lower socio-economic status or with special needs), and a lack of
experience/confidence speaking in foreign languages.
Schools, adult education providers and youth organizations participate in various EU-funded projects
that involve some aspects of international cooperation, but data on their cooperation in projects not
related to ERASMUS+ were not available for analysis and comparison. Research conducted among
project participants and nonparticipants in LLP-beneficiary organisations (Ančić and Klasnić, 2016)
provides an overview of the added value that LLP projects had within educational institutions (pre-
primary, primary and secondary education). The integrated framework of ERASMUS+ did bring
about significant changes, but the core values and objectives of the programme remained similar, so
the LLP Evaluation Study that focused on the institutional level still provides relevant insight into
programme value for educational institutions. For example, employees of beneficiary organisations
recognize that project implementations did have an impact on educational institutions, with the
strongest impacts being on the personal development of participants, the introduction of a European
dimension in education, and the willingness/motivation of employees to participate in professional
development activities. The weakest impact of projects on the institutional level was recognized on the
development of international and cross-sectoral partnerships, and on the development of specific
skills, knowledge and language competences.
11
To what extent have ERASMUS+ and its predecessor programmes resulted in impacts that are above and beyond those that
would have resulted from similar actions initiated merely at the regional or national levels in your country? Could the
programme make effective use of higher budgets?
27
Interviewed stakeholder representatives differ in opinion as to how much ERASMUS+ has influenced
the YOU and E&T sectors. While respondents from the YOU believe that there has been very slow
progress, with impacts mostly observed on the individual and micro levels,, in the E&T, especially the
HE, the notion is that significant internationalisation and europeanisation has been achieved on the
institutional and individual levels, while it is perceived to have not yet sufficienty permeated the
policy level.
Some respondents are quite sure that an increase of the ERASMUS+ budget in the coming years will
be followed by increased absorption, at least in terms of mobility projects, because those projects still
have a very high rejection rate (around 70%). In addition, unlike in other EU countries, in Croatia
ERASMUS+ is the largest source of funding for mobility actions. However, some respondents are
quite sceptical regarding future absorption, especially when it comes to primary schools and even
secondary schools and their projects. Here again, respondents perceive obstacles to schools’ inclusion
in more projects, such as a lack of adequate rewards for staff who are engaged in Erasmus+ projects.
Often it is mentioned that beneficiaries are being held responsible for large sums of money, and at the
same time, they are not rewarded for their efforts in any way. This is also mentioned as a challenge to
further absorption of additional funding, because it is hard to believe that those potential beneficiaries
– those schools that are not participating, and are possibly afraid of such unrewarded responsibility –
will hold a different opinion in the future, all else held constant. Respondents from governmental
agencies have also frequently mentioned the problem of a lack of information about schools that do
not participate in the programme, i.e. on their reasons for opting out. If we do not know what makes a
school opt out of the programme, it is difficult to understand what can be done to change that. Related
to the HE, what is frequently noted is a lack of capacities and knowledge related to international
projects, especially concerning smaller higher education institutions.
Generally, when talking about greater funds and their absorption, the NA holds that this will present a
challenge in the future, arguing that with an increase of funded projects, there will come an increase in
projects of different and perhaps lower quality. However, respondents from the NA remarked that
projects of lower quality are more closely monitored and often receive greater support from NA staff.
Consequently, these projects can become very successful, although the amount of time and energy that
the NA staff must invest in such projects is substantial. Therefore, the NA sees this aspect as
challenging, and wonders how to sustain the quality level of projects while improving the absorption
of an expected funds increase.
Points for improvement
An increase in funding should be accompanied by the introduction of capacity building
activities at the national level, in order to maintain the current demand for and quality of
projects on the one hand, and to secure growth on the other.
28
Conclusions
The evidence collected and presented in this report provides a national perspective on the current
achievements of the ERASMUS+ programme with respect to initial objectives and expectations. It
should be considered that at this point, in the middle of the programme cycle, it is still not possible to
evaluate the overall impact of the programme, especially on the societal level. However, the findings
of the mid-term review can provide information about progress towards realising objectives,
challenges that occurred, and forward-looking recommendations for the new generation of the
programme (post 2020).
Evidence collected from multiple sources with multiple methods yielded similar and complementary
results, indicating the validity of the conclusions. There is agreement among research participants (i.e.
survey respondents and interviewees) that programme activities do contribute to the realisation of
programme objectives. In general, progress in realizing objectives is mostly identified with respect to
specific objectives addressing personal development (i.e. development of skills and acquiring new
knowledge, especially in the areas of culture and foreign languages). In terms of objectives related to
the institutional level, the LLP Evaluation study as well as the mid-term review results indicate that, in
Croatia, ERASMUS+ and its predecessor programmes triggered changes in beneficiary organisations,
ranging from boosting motivation and willingness to take part in international projects, to increasing
overall knowledge of project management and creating partnerships with other European
organisations. In Higher Education, the programme was seen as an instrument of internationalization,
since it helped HEIs increase the scope of international students as compared to the pre-Erasmus
period, and triggered changes in managing and thinking about internationalisation. Still, at this level,
most of the obstacles to further growth and development of programme activities can be summed up as
a lack of human and administrative capacities, legal obstacles, and a lack of interest or motivation to
participate among targeted populations. Insufficient financial, administrative and human capacities are
relevant factors in the underrepresentation of certain institutional actors among programme
beneficiaries (e.g. small CSO organisations, pre-primary and primary schools – especially from the
less developed counties in Croatia). In terms of policy level objectives, results indicate that project
managers/legal representatives recognize to a lesser extent the realisation of these objectives, similarly
as did the interviewed key informants. Therefore, it is not surprising that the contribution of mobility
activities in KA1 is recognized more among respondents than the policy oriented KA3.
In terms of efficiency, there is a general opinion that the integration of programmes brought more
advantages than difficulties, although NA respondents pointed out initial challenges related to changes
in IT tools. Still, in the YOU field, there is a common perception that with integration, they lost
recognition, and some activity content (such as local and national initiatives) is not available. At the
same time, the demand for funding is especially high in the YOU field, but fewer projects receive a
high amount of funding even as the portion of the budget allocated to the YOU did not change
significantly in comparison to that allocated to the YiA. This situation favours more experienced,
professional non-governmental organisations, while making it hard for newcomers and small CSOs to
take part in projects. Contrasting the situation in the YOU field, in Education and Training the budget
size is considered appropriate and proportionate. As previously mentioned, differences in levels of
human, administrative and financial capacities among various target groups and across counties limits
the efficiency of the Programme in the national context. The division of roles among the NA, EC, and
NAUs is mostly perceived as good and functional, although some of the interviewees advocate for a
more independent NA that has a relevant role in policy-making. In order to increase programme
outreach and achieve more synergy with national goals, cooperation with academia, CSOs and policy
29
making experts in form of an advisory group is recommended.
The ERASMUS+ objectives are perceived as being in line with the needs of applicants (both
beneficiaries and non-beneficiaries), but there is a difference between the E&T and YOU fields.
Objectives such as promoting participation in the labour market, strengthening links between the YOU
and the labour market, and developing knowledge- and evidence-based youth policy are seen as less
relevant compared to other ERASMUS+ objectives. In the E&T sector, all of the objectives are
identified as relevant within the sector. Some more specific objectives, related to the HE, are relevant
mostly to the beneficiaries and less so to others. As for attracting and reaching different target
audiences, there is a need to widen access in order to include more newcomers, enterprises and local
and regional authorities, and to enable more participation on the part of institutions from
underdeveloped counties. Participation among pre-primary and primary schools also decreased, since
they lack capacities to compete with more experienced organisations for limited funding. In general,
small organisations with lower levels of human, financial and administrative capacities would benefit
from a higher number of projects entailing smaller individual grant amounts. Currently, especially in
KA2, high demand stresses the limited funding available, allowing just a relative handful of awarded
projects per field, per year. In this situation, schools and small CSOs are competing for funding with
organisations boasting more project implementation experience. Participation among disadvantaged
groups is good, especially in the HE ever since the introduction of top-up grants, although more
dynamic outreach campaigns could increase participation among these groups in the SE and VET
fields alike.
Complementarity with other programmes is seen as a relevant to the HE field, where there are multiple
coexisting programmes that provide funding for mobility actions. Since the content of activities
available under CEEPUS and bilateral scholarships in terms of duration, type of mobility and targeted
audiences is not the same, and since ERASMUS+ is still the largest source of funding, these
programmes are seen as complementary rather than competitive in the national context. However,
there is concern that ESF-supported mobility schemes targeting STEM students and staff will affect
participation among this group in ERASMUS+ if better financial benefits become available through
the planned ESF funding.
In the Croatian national context, ERASMUS+ has a significant benefit for all fields of education and
training, is perceived as an instrument for promoting and strengthening the international dimension in
education, and is the largest source of funding for academic mobility. Still, multiple challenges in
terms of the human, financial and administrative capacities of institutions could inhibit further growth
or limit access to funding for those with a lower level of capacities. Therefore, the increase in budget
needs to be paired with more opportunities for organisations of varying capacity levels to take part in
projects. In order to achieve sustainability and secure long-term impacts from programme activities,
there is a need for more strategic approaches and synergy at the institutional and national levels alike.
Long-term impacts that are in line with programme objectives can be achieved only with a forward-
looking strategic approach together with a bottom-up perspective that is guided by societal needs.
30
Appendix I: Evaluation Questions
Effectiveness
(1) To what extent have ERASMUS+ and its predecessor programmes contributed to the
realisation of the ERASMUS+ specific objectives in your country? Are there differences
across fields? Please provide, where relevant, your assessment for each of the specific
objectives and provide evidence and examples where possible.
(2) To what extent has the progress on the realisation of the specific objectives contributed
to the realisation of the ERASMUS+ general objectives in your country?
(3) To what extent have ERASMUS+ actions influenced policy developments in the
domains of education and training, youth and sport in your country? Which actions were
most effective in doing so? Are there marked differences between different fields?
(4) What specific approaches (such as co-financing, promotion or others) have you taken in
order to try to enhance the effects of ERASMUS+ in your country? To what extent have
these approaches been effective? Can any particular points for improvement be
identified?
(5) Do you consider that certain actions of the programme are more effective than others?
Are there differences across fields? What are the determining factors for making these
actions of the programme more effective?
(6) To what extent has the integration of several programmes into ERASMUS+ made the
programme more effective in your country? Do you see scope for changes to the
structure of ERASMUS+ or its successor programme that could increase effectiveness?
(7) Is the size of budget appropriate and proportionate to what ERASMUS+ is set out to
achieve? Is the distribution of funds across the programme’s fields and actions
appropriate in relation to their level of effectiveness and utility?
(8) What challenges and difficulties do you encounter while implementing the various
actions of ERASMUS+? What changes would need to be introduced in ERASMUS+ or
its successor programme to remedy these?
(9) To what extent are the approaches and tools that are used for disseminating and
exploiting the results of ERASMUS+ and its predecessor programmes in your country
effective? Where can you see the possibilities for improvements?
31
Efficiency
(10) To what extent is the system of cooperation and division of tasks between the
Commission, Executive Agency, National Agencies, European Investment Fund,
National Authorities, Independent Audit Bodies, and ERASMUS+ Committee efficient
and well-functioning from the point of view of your country? What are the areas for
possible improvement or simplification in the implementation of ERASMUS+ or a
successor programme?
(11) To what extent has the integration of several programmes into ERASMUS+ resulted in
efficiency gains or losses for the implementation of the programme in your country, both
at the level of the National Agency/ies and on the beneficiaries' and participants' level?
Do you see scope for changes to the structure of ERASMUS+ or its successor
programme that could increase efficiency?
(12) Do you consider that the implementation of certain actions of the programme is more
efficient than others? Are there differences across fields? What good practices of these
more efficient actions of the programme could be transferred to others?
(13) To what extent has the system of simplified grants resulted in a reduction of the
administrative burden for National Agencies and programme beneficiaries and
participants? Are there differences across actions or fields? What elements of the
programme could be changed to further reduce the administrative burden, without
unduly compromising its results and impact?
(14) To what extent are the IT tools provided by the Commission adequate for the efficient
management and implementation of the programme in your country? Do they answer
your needs? Give specific examples where they can be improved. Is the set of IT tools
appropriate or should it cover more/less elements of the programme implementation?
(15) To what extent is the level of human and financial resources that is available for the
implementation of the programme in your country adequate? What steps did you take to
optimise the efficiency of the resources deployed for the ERASMUS+ implementation
in your country?
Relevance
(16) To what extent do the ERASMUS+ objectives continue to address the needs or
problems they are meant to solve? Are these needs or problems (still) relevant in the
context of your country? Have the needs or problems evolved in such a way that the
objectives of ERASMUS+ or its successor programme need to be adjusted?
32
(17) To what extent are needs of different stakeholders and sectors addressed by the
ERASMUS+ objectives? How successful is the programme in attracting and reaching
target audiences and groups within different fields of the programme's scope? Is the
ERASMUS+ programme well known to the education and training, youth and sport
communities? In case some target groups are not sufficiently reached, what factors are
limiting their access and what actions could be taken to remedy this?
Internal and external coherence and complementarity
(18) To what extent are the various actions that have been brought together in ERASMUS+
coherent? Can you identify any existing or potential synergies between actions within
ERASMUS+? Can you identify any tensions, inconsistencies or overlaps between
actions within ERASMUS+
(19) To what extent does ERASMUS+ complement other national and international
programmes available in your country? Can you identify any tensions, inconsistencies or
overlaps with other programmes?
European added value and sustainability
(20) To what extent ERASMUS+ and its predecessor programmes produce effects that are
additional to the effects that would have resulted from similar actions initiated only at
regional or national levels in your country? What possibilities do you see to adjust
ERASMUS+ or its successor programme in order to increase its European value added?
(21) To what extent ERASMUS+ will be able to absorb in an effective way the sharp increase
in the budget that is foreseen in the coming years up to 2020 in your country? Could the
programme use even higher budgets in an effective way? Do you see challenges to
effectively use more money for particular actions or fields of the programme?
33
Appendix II: General and specific objectives of the
ERASMUS+ Programme
Source: COM (2011) 788: Regulation (EU) No 1288/2013 of the European Parliament and of the
Council of 11 December 2013 establishing 'ERASMUS+': the Union programme for education,
training, youth and sport and repealing Decisions No 1719/2006/EC, No 1720/2006/EC and No
1298/2008/EC Text with EEA relevance.
General objectives (Art. 4)
(a) The objectives of the Europe 2020 strategy, including the headline education target;
(b) the objectives of the strategic framework for European cooperation in education and training
('ET 2020'), including the corresponding benchmarks;
(c) the sustainable development of partner countries in the field of higher education;
(d) the overall objectives of the renewed framework for European cooperation in the youth field
(2010-2018);
(e) the objective of developing the European dimension in sport, in particular grassroots sport, in
line with the Union work plan for sport; and
(f) the promotion of European values in accordance with Article 2 of the Treaty on European
Union.
Specific objectives Youth (Art. 11) Sport Education and Training
In line with the general objective of the Programme as specified in Article 4, in particular the
objectives of the renewed framework for European cooperation in the youth field (2010–2018),
the Programme shall pursue the following specific objectives:
To improve the level of key
competences and skills of young
people, including those with fewer
opportunities, as well as to promote
participation in democratic life in
Europe and the labour market, active
citizenship, intercultural dialogue,
social inclusion and solidarity, in
particular through increased learning
mobility opportunities for young
people, those active in youth work or
youth organisations and youth
leaders, and through strengthened
links between the youth field and the
labour market
To tackle cross-
border threats to
the integrity of
sport, such as
doping, match-
fixing and
violence, as well
as all kinds of
intolerance and
discrimination;
To improve the level of key
competences and skills, with particular
regard to their relevance for the labour
market and their contribution to a
cohesive society, in particular through
increased opportunities for learning
mobility and through strengthened
cooperation between the world of
education and training and the world
of work;
34
To foster quality improvements in
youth work, in particular through
enhanced cooperation between
organisations in the youth field
and/or other stakeholders
To promote and
support good
governance in
sport and dual
careers of
athletes;
To foster quality improvements,
innovation excellence and
internationalisation at the level of
education and training institutions, in
particular through enhanced
transnational cooperation between
education and training providers and
other stakeholders;
To complement policy reforms at
local, regional and national level and
to support the development of
knowledge and evidence-based youth
policy as well as the recognition of
non-formal and informal learning, in
particular through enhanced policy
cooperation, better use of Union
transparency and recognition tools
and the dissemination of good
practices
To promote
voluntary
activities in
sport, together
with social
inclusion, equal
opportunities
and awareness
of the
importance of
health-
enhancing
physical activity
through
increased
participation in,
and equal access
to, sport for all.
To promote the emergence and raise
awareness of a European lifelong
learning area designed to complement
policy reforms at national level and to
support the modernisation of education
and training systems, in particular
through enhanced policy cooperation,
better use of Union transparency and
recognition tools and the dissemination
of good practices;
To enhance the international
dimension of youth activities and the
role of youth workers and
organisations as support structures
for young people in complementarity
with the Union's external action, in
particular through the promotion of
mobility and cooperation between the
Union and partner-country
stakeholders and international
organisations and through targeted
capacity-building in partner countries
to enhance the international dimension
of education and training, in particular
through cooperation between Union
and partner-country institutions in the
field of VET and in higher education,
by increasing the attractiveness of
European higher education institutions
and supporting the Union's external
action, including its development
objectives, through the promotion of
mobility and cooperation between the
Union and partner-country higher
education institutions and targeted
capacity-building in partner countries;
35
To improve the teaching and learning
of languages and to promote the
Union's broad linguistic diversity and
intercultural awareness;
To promote excellence in teaching and
research activities in European
integration through the Jean Monnet
activities worldwide, as referred to in
Article 10.
36
Appendix III: Analysis of survey results
Prepared by: Branko Ančić, Ph.D., Institute for Social Research in Zagreb ([email protected])
Contents
Executive summary ............................................................................................................................... 38
Methodology ......................................................................................................................................... 40
Survey results: ....................................................................................................................................... 43
I. Effectiveness ...................................................................................................................................... 43
a) Youth ............................................................................................................................................. 46
b) Education and Training ................................................................................................................. 47
II. Efficiency .......................................................................................................................................... 49
a) Youth ............................................................................................................................................. 53
b) Education and Training ................................................................................................................. 55
III. Relevance ........................................................................................................................................ 57
a) Youth ............................................................................................................................................. 57
b) Education and Training ................................................................................................................. 59
IV. Internal and external coherence and complementarity .................................................................... 62
V. European added value and sustainability .......................................................................................... 63
Conclusions and suggestions for improvements to Erasmus+ and for a future programme ................. 64
Literature ............................................................................................................................................... 66
37
Acronyms
AMEUP Agency for Mobility and EU Programmes
LLP Lifelong Learning Programme
YiA Youth in Action
KA1 Key Action 1: Learning Mobility of Individuals
KA2 Key Action 2: Cooperation for innovation and the exchange of good practices
KA3 Key Action 3: Support for policy reform
E&T Education and training field in Erasmus+
YOU Youth field in Erasmus+
SCH School education field in Erasmus+
VET Vocational education and training field in Erasmus+
HE Higher education field in Erasmus+
ADU Adult education field in Erasmus+
HEI Higher education institution
38
Executive summary
The goal of this study was to explore the opinions of the participants and representatives of user
organisations within the Erasmus+ programme and its predecessor programmes on the effectiveness,
efficiency, relevance, coherence and complementarity as well as added value of Erasmus+ at the
national level. By means of a questionnaire survey and an analysis of the data gathered therefrom, the
aim was to assess the implementation and impact of the Erasmus+ programme, including its
advantages, disadvantages and examples of good practice. It follows that the elements of Erasmus+
realisation that were explored in the survey were designed to showcase the extent to which
respondents considered the programme’s realisation to be a success.
Programme effectiveness was operationalised in such a way that respondents assessed the overall
visibility of Erasmus+, evaluated the contribution of the Key Actions in the realisation of Erasmus+
specific objectives , and evaluated the realisation of those objectives. First of all, respondents on
average think that the dissemination and usage of information about Erasmus+ is good. The analysis
shows that those who were beneficiaries of Erasmus+ and its predecessor programmes, as well as
those who were otherwise engaged in Erasmus+, perceive the overall visibility of Erasmus+ to a
greater extent. This indicates a need to increase the overall visibility of Erasmus+ among those who
were not granted financial support (i.e., non-beneficiaries) and among those who did not participate in
the programme. Concerning the extent to which Erasmus+ is effective in realising its specific
objectives in the youth sector, survey results indicate that the realisation of all specific objectives is
not viewed in a uniform manner. Although Erasmus+ contributes to the realisation of all of its specific
objectives, it seems that, from the perspective of various relevant actors and youth professionals, the
programme falls short in terms of connecting youth with the labour market. Since Erasmus+ puts an
emphasis on strengthening ties between the youth field and the labour market as well as on the
promotion of youth participation in the labour market, and since those objectives are recognised to a
lesser extent than, for example, intercultural dialogue, the improvement of key competencies and skills
among youth, or social inclusion, it seems that any future programme should put more effort into the
realisation of objectives regarding youth and the labour market.
Programme efficiency was operationalised by exploring perceptions of the efficiency of Erasmus+ and
its predecessor programmes in general on the one hand, and of the adequacy of IT tools, the obstacles
to Erasmus+ participation, and administrative, financial and human capacities on the other.
Respondents with experience in both Erasmus+ and its predecessor programmes recognise the
improvements that Erasmus+ made by expanding opportunities for cross-sectoral cooperation,
simplifying the financial management of projects, adjusting grants to the needs of users, and
standardising administrative processes which made it easier to manage projects. However, it should be
noted that respondents recognise to a lesser extent that application procedures are clear and user-
specific, that the evaluation criteria and implementation are transparent, and that the project’s financial
asset management regulations are clear. Any future programme should consider further simplifying
and adapting procedures and regulations for the benefit of users. From the perspective of the YOU,
respondents recognise to a lesser extent that grants are adjusted to the real needs of their organisations,
and that the evaluation criteria and implementation are transparent. Concerning the differences among
action fields, it is indicated that respondents from higher education recognise the efficiency of
Erasmus+ in terms of administrative regulations standardisation and project financial management
simplification to a greater extent than do those from primary and secondary school education. These
findings were analysed in the context of perceived obstacles to participation in Erasmus+ and an
assessment of the administrative, financial and human capacities for project implementation under
39
Erasmus+. Obstacles recognised the most as those impacting the ability of people and their institutions
to participate in the Erasmus+ have to do with a lack of administrative, human and financial resources,
the discouragement of employees with administrative formalities, excessive workloads, and a lack of
recognition for participation in projects for the purpose of career advancement. In addition, over 50 %
of respondents report financial capacities (i.e., resources) that are insufficient to pre-finance project
activity implementation, while those who report limited organisational capacity in the form of
accounting staff expertise required for financial monitoring also perceive less clarity in Erasmus+
regulations that apply to project financial asset management (i.e., accounting rules, expenditures,
payment terms). Any future programme should better account for differences in the absorption
potential of users in terms of their administrative, human and financial capacities and resources. In
other words, any future programme should try to adjust and design its rules and procedures in such a
way as to recognise and accept the differences among sectors and among action fields, since
competitive conditions and potential are not the same for all interested users. Meanwhile, respondents
perceive IT tools such as Mobility Tool+, URF, Valor-dissemination platform and Online Linguistic
Support-OLS to be fully functional, and that guidance in their usage is clear.
Programme relevance is explored by assessing the extent to which Erasmus+ specific objectives are
relevant to users in their respective sectors. In both sectors, the YOU and the E&T, respondents
recognise all specific objectives as relevant, which is important when it comes to successfully
attracting and reaching various target groups. A distinction should be made about objectives that
tackle challenges and issues related to policy and the labour market. For instance, concerning specific
objectives within the YOU, just over 45% of respondents are unable to estimate or assess the
relevance of specific objectives – such as complementary policy reforms at the local, regional and
national level, or the development of a knowledge- and evidence-based youth policy – to their sector.
A similar situation characterises the objective of strengthening links between the YOU and the labour
market. In the E&T, objectives such as improving key competencies and skills with regard to labour
market needs and strengthening cooperation between academia and the wider economy are recognised
to a greater extent among those in the vocational education and training field than among those in the
other three fields. Therefore, any future programme should put more effort into developing more
effective mechanisms that create conditions of more intense cooperation with the wider economy.
Internal and external coherence and complementarity cannot be effectively measured based on a
survey, so the analysis of respondents’ answers is limited. The only finding is in regard to
differentiating between those who are experienced in Key Action 1 (Learning Mobility of Individuals)
and those who are experienced in Key Action 2 (Cooperation for Innovation and Exchange of Good
Practices) in such a way that the former find Erasmus+ application procedures and regulations that
apply to project financial asset management (i.e., accounting rules, expenditures, payment terms) clear
to a greater extent than do the latter.
European added value and sustainability likewise cannot be effectively measured based on a survey
like this. However, respondents do recognise the realisation of specific objectives concerning
European added value. A majority of respondents agree that Erasmus+ promotes participation in
democratic life in Europe as well as the emergence and increasing awareness of a European lifelong
learning area, enhances the international dimension of education and training, establishes cooperation
among EU institutions in the VET and the HE, increases the attractiveness of European HEIs,
improves the teaching and learning of languages, and promotes broad linguistic diversity and
intercultural awareness in the European Union.
40
Methodology
The goal of the study was to explore the opinions of the participants and representatives of user
organisations within Erasmus+ and its predecessor programmes on the effectiveness, efficiency,
relevance, coherence and complementarity as well as added value of Erasmus+ at the national level.
By means of a questionnaire survey and an analysis of the data gathered therefrom, the aim was to
assess the implementation and impact of the Erasmus+ programme, including its advantages,
disadvantages and examples of good practice. Since the assessment of the social impact is a systematic
analysis of permanent and/or significant changes in peoples’ lives – be they positive or negative,
intended or unintended, and which are the result of certain actions (or of a sequence of actions) – the
social impact of Erasmus+ can be estimated through subjective perception. Therefore, by surveying
Erasmus+ users, the questionnaire explored the extent of programme outcomes and efficiency. During
the period from 19 December 2016 until 15 January 2017, the survey was conducted online in the
form of a Computer Assisted Self-Interview (CASI). The survey was administered via the Lime
Survey tool. A request to participate in the survey was sent to all beneficiaries and non-beneficiaries
who submitted a project application for LLP/YiA activities which, in modified/similar form, continued
to be implemented under Erasmus+ and among all Erasmus+ applicants. The survey was sent to a list
of legal representatives and/or project managers listed in the register of applicants, which is
administered by the AMEUP. The register contained contact information of project managers and/or
legal representatives who submitted project applications as far back as 2010. Prior to sending the
survey by email, several modification were made: applicants from activities that were discontinued
under Erasmus+ programme (footnote 1) were removed from the list, duplicate email addresses were
removed from the list, and generic email addresses were replaced by personal addresses. Since the
survey was intended to be sent to personal email addresses, at the end of November 2016 a contact
form was sent to all contacts who had previously provided generic email addresses, asking them for
their personal email addresses. The dataset was updated to include personal email addresses from the
contact form, and other generic email addresses were deleted from the final survey mailing list.
However, some problems could not be avoided. Some personal email addresses from the register were
outdated and no longer in use, and in some cases more than one email address was connected with a
single person.
The dataset of beneficiaries and non-beneficiaries consisted of 4118 persons who are legal
representatives and/or project managers. The questionnaire was completed in full by 967 respondents,
which is 23,48 % of the entire surveyed population.
Table 1. Structure of the surveyed sample
% N
Structure of
beneficiaries and
non-beneficiaries
Beneficiaries of Erasmus+ and/or its predecessor programmes 79,2 766
Non-beneficiaries of Erasmus+ and/or its predecessor
programmes 20,8 201
Total 100 967
Respondent´s role
Project manager 68,6 663
Legal representative of organisation 19,2 186
Project manager/Legal representative of organisation 12,2 118
Total 100 967
Structure of
beneficiaries in
regard to Erasmus+
and its predecessor
Participated in the LLP 62,2 475
Participated in YiA 18,4 141
Participated in Erasmus+ 73,6 564
Participated in other international cooperation programmes 5,1 39
41
programmes (i.e., Tempus, Erasmus Mundus)
Structure of non-
beneficiaries in
regard to Erasmus+
and its predecessor
programmes12
Submitted application for the LLP 45,3 91
Submitted application for YiA 13,4 27
Submitted application for Erasmus+ 55,2 111
Submitted application for other international cooperation
programmes (i.e., Tempus, Erasmus Mundus) 8,5 17
Type of organisation
which applied for
financial support
under Erasmus+
and/or its
predecessor
programmes
State administration organisation 3,0 29
Public institution 67,8 656
Non-governmental organisation 19,9 192
For-profit organisation 2,5 24
Informal youth group 0,7 7
Other 6,1 59
Total 100 967
Sex Male 22,2 215
Female 77,8 752
Total 100 967
M SD
Age 59,2 10,01
Table 2. Comparison of surveyed sample alongside the number of organisations that
submitted the application and the number of applied projects – regional distribution
County
Number of organisations
that submitted the
application (applicant or
coordinator)*
%
Number of
applied
projects*
%
% of
respondents in
the survey
Bjelovar-Bilogora 19 1,6 94 1,7 1,9
Brod-Posavina 45 3,8 236 4,3 4,4
Dubrovnik-Neretva 22 1,9 73 1,3 2,5
City of Zagreb 378 32,1 1984 35,9 25,0
Istria 60 5,1 304 5,5 6,1
Karlovac 33 2,8 164 3,0 2,8
Koprivnica Križevci 28 2,4 109 2,0 3,2
Krapina-Zagorje 34 2,9 98 1,8 2,8
Lika-Senj 9 0,8 47 0,9 0,7
Međimurje 28 2,4 132 2,4 4,0
Osijek-Baranja 75 6,4 362 6,6 6,5
Požega-Slavonia 16 1,4 83 1,5 2,4
Primorje-Gorski Kotar 87 7,4 379 6,9 8,0
Sisak-Moslavina 36 3,1 131 2,4 2,6
Split-Dalmatia 106 9,0 513 9,3 7,9
Šibenik-Knin 23 2,0 111 2,0 2,1
Varaždin 49 4,2 214 3,9 5,0
Virovitičko-podravska 20 1,7 49 0,9 1,2
Vukovar-Srijem 31 2,6 138 2,5 3,1
Zadar 26 2,2 116 2,1 2,2
12
The total number of non-beneficiaries that participated in Erasmus+ and/or its predecessor programmes is 201,
but the same respondent could have participated in more than one programme.
42
Zagreb 54 4,6 186 3,4 5,7
Total 2026 100 6888 100 100 *Source: Database of organisations (Agency for Mobility and EU Programmes)
Based on the data presented in Table 2. The surveyed sample indicated a similar regional distribution,
thus indicating the sample’s representation of the researched population.
43
Evaluation results
I. Effectiveness
For the purpose of exploring the effectiveness of Erasmus+ and its predecessor programmes from the
perspective of the actors (i.e. beneficiaries or applicants), several questions were raised by the survey
results. Respondents were asked to assess the overall visibility of Erasmus+ on a national level as well
as the promotion of information and good practices regarding Erasmus+. Further on, respondents were
asked about the extent to which Erasmus+ contributes to the realisation of specific objectives
regarding the YOU and the E&T. Response distributions are presented in the subsections on the YOU
and the E&T. Based on their own experiences, those respondents who participated in Erasmus+ were
asked to assess the extent to which the structure of the programme, through the Key Actions,
contributed to the realisation of Erasmus+ specific objectives.
All respondents were asked to assess their level of satisfaction with the general visibility of Erasmus+
in Croatia, and to express their own ideas about the dissemination and promotion of information
regarding Erasmus+.
Table 3. Perception of visibility of Erasmus+ (all respondents)
PO
OR
SU
FF
ICIE
NT
GO
OD
VE
RY
GO
OD
EX
CE
LL
EN
T
I C
AN
NO
T
ES
TIM
AT
E
% M SD
Overall visibility of Erasmus+ results on the
national level 11,1 11,7 31,2 27,3 9,9 8,8 3,15 1,152
Promotion of good practice by the Agency for
Mobility and EU Programmes 9,1 11,2 29,0 29,5 15,0 6,3 3,32 1,167
The regional distribution of information and
promotional activities by the Agency for
Mobility and EU Programmes
13,5 14,5 30,3 19,5 9,1 13,0 2,96 1,197
The availability of promotional materials with
examples of good practice 14,8 15,6 29,3 21,0 11,8 7,5 2,99 1,244
Visibility of the results of the programme in
the respondent’s area of work 13,0 12,5 29,3 26,6 10,8 7,9 3,10 1,203
Based on the distribution of percentages of answers, the general perspective indicates that Erasmus+
visibility and dissemination of information concerning best practices/results are good. All respondents,
on average, think that the dissemination and usage of information about Erasmus+ is good. If we test
for differences in perspective among those who were beneficiaries of Erasmus+ and its predecessor
programmes on the one hand and those who were not on the other, then non-parametric testing
indicates that beneficiaries report greater visibility and good practices/results of Erasmus+.
Additionally, if we test for differences among those who were engaged only in predecessor
programmes and those who were engaged in Erasmus+, a similar pattern can be observed, since those
engaged in Erasmus+ assess its visibility and its good practices/results more favourably. Respondents
were also asked in an open answer format to write their ideas on the dissemination and usage of
information about Erasmus+. One of these ideas – which was repeated – relates to the regional
distribution of information and promotion of good practices stemming from Erasmus+. If we consider
44
the significant regional differences in the number of organisations that have applied for funding, and if
we consider the same with respect to the number of applied projects (see Table 2), it seems that
regional differences should be seriously taken into consideration in the context of Erasmus+
implementation. Diverse perspectives between action fields also indicate significant differences.
Graph 1. Perceptions of Erasmus+ visibility (difference among action fields)
45
Perceptions of Erasmus+ visibility differ among action fields (Graph 1.). The overall visibility of
Erasmus+ on the national level is recognised to a lesser extent among those actors which are primarily
engaged in the ADU and the SCH, while those who are primarily engaged in the VET are more
satisfied with Erasmus+ visibility on the national level. Similarly, the regional distribution of the
AMEUP’s informational and promotional activities is recognised to a greater extent among those who
are engaged in the VET. Programme results in the areas of work in which respondents are active are
more visible among those who are engaged in the VET and the SCH. So it seems that VET actors
express greater satisfaction with the visibility and promotion of Erasmus+ on the national and regional
levels, as well as with the results that the programme brings.
Based on the information available to them, respondents who participated in Erasmus+ were asked to
assess whether Key Actions contribute to the realisation of programme´s specific objectives.
Table 4. Evaluation of the contribution of Key Actions in realising the specific objectives of
Erasmus+ (respondents with Erasmus+ experience)
TO A
LESSER
EXTENT
MODERATELY TO A GREAT
EXTENT
I CANNOT
ESTIMATE
%
Key Action 1 – Mobility of
Individuals 2,5 11,0 72,9 13,7
Key Action 2 – Cooperation for
Innovation and Exchange of Good
Practices
1,2 10,6 67,9 20,2
Key Action 3 – Support for Policy
Reform 8,5 14,4 15,1 62,1
Table 5. Evaluation of the contribution of Key Actions in realising the specific objectives of
Erasmus+ (respondents with experience within each of the Key Actions of Erasmus+)
TO A
LESSER
EXTENT
MODERATELY TO A GREAT
EXTENT
I CANNOT
ESTIMATE
%
Key Action 1 – Mobility of
Individuals 1,0 8,1 86,7 4,2
Key Action 2 – Cooperation for
Innovation and Exchange of Good
Practices
1,3 9,9 85,7 3,1
Key Action 3 – Support for Policy
Reform 5,9 29,4 47,1 17,6
From the perspective of those with Erasmus+ experience, KA1 and KA2 are regarded as actions which
contribute the most to the realisation of the programme´s specific objectives. Conversely, the potential
contribution of KA3 to the realisation of specific objectives is not recognised, since a majority of
respondents (62,1 %) were unable to evaluate the extent to which this action contributed to the
realisation of specific objectives. If we take a look at the distribution of answers from those
respondents with experience in each of the Key Actions, a similar pattern is visible. KA1 and KA2 are
recognised in their contributions to the realisation of Erasmus+ specific objectives, while KA3 lags
significantly behind. Although respondents who have participated in KA3 recognise its contribution to
the realisation of specific objectives, they do so to a lesser extent than those who have participated in
the first two activities. Even though all three Key Actions are recognised as having contributed to the
46
realisation of Erasmus+ specific objectives, the KA3 objective of providing support for policy reforms
is not perceived as a successful mechanism in comparison with the first two actions. This perception is
the same regardless of whether respondents are engaged within the YOU, the E&T, or any action field
for that matter.
a) Youth
All respondents were asked to assess, based on their experience and familiarity with Erasmus+, the
extent to which Erasmus+ enabled the realisation of specific YOU objectives.
Table 6. Realisation of Erasmus+ goals – YOU (all respondents)
DO
ES
NO
T E
NA
BL
E
NE
ITH
ER
EN
AB
LE
S
NO
R D
OE
S N
OT
EN
AB
LE
E
NA
BL
ES
I D
O N
OT
KN
OW
/ I
CA
NN
OT
ES
TIM
AT
E
%
Improving key competencies and skills among youth ( "key competencies" =
the basic set of knowledge, skills and attitudes that all individuals need for
personal fulfilment and development, active citizenship, social inclusion and
employment)
0,1 2,4 85,1 12,4
Including young people with fewer opportunities or who encounter obstacles
that prevent an active approach to education, training and work 0,9 8,2 70,6 20,3
Promoting participation in democratic life in Europe 1,8 9,5 68,6 20,2
Promoting participation in the labour market 2,6 12,8 56,6 28,0
Active citizenship 0,8 7,4 74,1 17,6
Intercultural dialogue 0,1 2,7 85,6 11,6
Social inclusion 0,3 5,2 80,4 14,2
Achieving solidarity 0,9 9,0 70,7 19,3
Increasing learning mobility opportunities for young people, those active in
youth work or youth organisations, and youth leaders 0,3 3,4 82,3 14,0
Strengthening links between the youth field and the labour market 2,6 12,2 53,9 31,3
Enhancing cooperation between organisations in the youth field and/or other
stakeholders 0,9 5,1 73,2 20,8
Complementing policy reforms at the local, regional and national levels 4,1 17,1 41,5 37,3
Developing knowledge- and evidence-based youth policies 1,8 12,0 45,5 40,7
Recognising non-formal and informal learning 2,8 11,0 63,9 22,3
Enhancing the international dimension of youth activities 0,5 3,1 83,2 13,1
Strengthening the role of socio-pedagogical staff and organisations as
support structures for young people 1,0 8,2 66,4 24,4
The Erasmus+ contribution to the realisation of the above-listed objectives is noticeable, since all
respondents, regardless of whether they were Erasmus+ beneficiaries, recognised an Erasmus+
contribution to those objectives. In addition, it was statistically tested if there was a difference in
perspective between those who had participated only in predecessor programmes and those who had
participated in Erasmus+, and nonparametric tests showed no difference in perception of the Erasmus+
contribution to the specific objectives listed above.
Several of the above-listed specific goals seem to be recognised as those that Erasmus+ enables the
47
strongest realisation of. For instance, intercultural dialogue; improving key competencies and skills of
youth; enhancing the international dimension of youth activities; increasing learning mobility
opportunities for young people, those active in youth work or youth organisations, and youth leaders;
and social inclusion have the highest level of recognition as those specific objectives that Erasmus+
enables as a programme, since over 80 % of respondents recognise the role of Erasmus+ in their
realisation. A discrepancy between objectives can be seen in regard to those specific objectives whose
realisation respondents are to a greater extent unable to estimate the Erasmus+ contribution to.
Between 28 % and 40 % of respondents do not know and cannot estimate if Erasmus+ has contributed
to the realisation of promoting the participation of youth in the labour market, of strengthening links
between the YOU and the labour market, of complementing policy reforms at the local, regional and
national levels, and of developing knowledge- and evidence-based youth policies. So it seems that the
effectiveness of Erasmus+ with respect to youth policy development and reforms as well as youth and
the labour market is perceived to extent lesser extent among the wider public as it relates to Erasmus+
and its predecessor programmes. In addition, if we test the difference in perception between those who
have engaged in the YOU (i.e., those who have individually or through their organisations participated
in projects connected with youth) and those who have not, the only statistically significant difference
concerns two specific objectives. Those engaged in the YOU recognise to a lesser extent an Erasmus+
contribution to the promotion of youth participation in the labour market and to the strengthening of
links between the YOU and the labour market. Although Erasmus+ contributes to the realisation of all
of its specific objectives, it seems that from the perspective of various relevant actors and youth
professionals, the programme falls short in connecting youth with labour market as compared to its
other stated objectives.
b) Education and Training
All respondents were asked to assess, based on their experience and familiarity with Erasmus+, the
extent to which Erasmus+ enables the realisation of specific objectives concerning the E&T.
Table 7. Realisation of Erasmus+ goals – E&T (all respondents)
DO
ES
NO
T E
NA
BL
E
NE
ITH
ER
EN
AB
LE
S
NO
R D
OE
S N
OT
EN
AB
LE
E
NA
BL
ES
I D
O N
OT
KN
OW
/ I
CA
NN
OT
ES
TIM
AT
E
%
Improving key competencies and skills in general ( "key competencies" =
the basic set of knowledge, skills and attitudes that all individuals need for
personal fulfilment and development, active citizenship, social inclusion and
employment)
0,4 3,1 91,1 5,4
Improving key competencies and skills with regard to labour market needs 0,8 12,6 72,7 13,9
Contributing to social cohesion 0,3 4,7 88,8 6,2
Creating opportunities for mobility for the purpose of formal/non-
formal/informal education 0,4 2,0 93,0 4,7
Strengthening cooperation between academia and the wider economy 1,4 17,6 63,6 17,4
Fostering improvement of quality, excellence in innovation and
internationalisation within academic institutions 0,5 7,2 80,2 12,0
48
Promoting the emergence and raising awareness of a European lifelong
learning area 0,4 5,8 84,3 9,5
Enhancing the international dimensions of education and training 0,5 3,8 88,6 7,0
Establishing cooperation between EU institutions in the field of vocational
education and training 0,9 4,4 68,5 26,2
Establishing cooperation between EU institutions in the field of higher
education 1,1 4,7 54,9 39,3
Increasing the attractiveness of European higher education institutions 1,1 7,8 51,4 39,7
Improving the teaching and learning of languages and promoting broad
linguistic diversity and intercultural awareness in the European Union 0,2 2,8 90,4 6,6
From the perspective of all survey respondents, Erasmus+ contributes to the realisation of all of the
above-listed objectives in regard to the E&T. It specifically enables the realisation of improving key
competencies and skills in general, improving the teaching and learning of languages and promoting
broad linguistic diversity and intercultural awareness in the European Union, contributing to social
cohesion, enhancing the international dimensions of education and training, and promoting the
emergence and raising awareness of a European lifelong learning area, at least from the perspective of
around 90 % of respondents. Establishing cooperation between EU institutions in the HE and
increasing the attractiveness of European HEIs are specific objectives to which Erasmus+ seems to
make a smaller contribution. Although over 50 % of respondents recognise the realisation of those two
objectives, almost 40 % do not know and cannot estimate if the programme is enabling their
realisation. If we test the difference between those with experience in Erasmus+ predecessor
programmes exclusively and those with experience in Erasmus+ exclusively, perceptions differ in
regard to several specific objectives. Respondents with Erasmus+ experience assess to a greater extent
that the programme enables the realisation of improving key competencies and skills with regard to
labour market needs, strengthening cooperation between academia and the wider economy, enhancing
the international dimensions of education and training, and increasing the attractiveness of European
HEIs.
49
II. Efficiency
Exploring the efficiency of Erasmus+ consisted of several aspects. The first aspect takes into
consideration the overall perception of the efficiency of Erasmus+ in general, but also that of its
predecessor programmes. The second aspect refers to the adequacy of IT tools such as Mobility Tool+,
URF, the Valor-dissemination platform and Online Linguistic Support-OLS, which were introduced in
order to assist with project management. The third aspect relates to the assessment of obstacles to
participation in Erasmus+. The final aspect explores the assessment of administrative, financial and
human capacities for the implementation of projects under Erasmus+.
In order to explore the efficiency of Erasmus+ in comparison with its predecessor programmes,
respondents with experience in both Erasmus+ and its predecessor programmes were asked to assess
the extent to which the integration of former programmes, such as the LLP and YiA, into Erasmus+
made the programme more efficient. In addition, those respondents with Erasmus+ experience were
asked to assess the programme’s efficiency per se.
Table 8. Perceptions of efficiency of integrated Erasmus+ (respondents having experience in
both Erasmus+ and its predecessor programmes)
TO A
LESSER
EXTENT
MODERATELY
TO A
BIGGER
EXTENT
I CANNOT
ESTIMATE
%
The standardisation of administrative
regulations and documents made it
easier to manage projects
13,0 26,6 37,6 22,9
It has simplified the financial
management of projects 12,5 22,4 41,5 23,5
Grants are to a larger extent adjusted
to the real needs of user
organisations
16,0 27,9 33,6 22,4
Erasmus+ improved opportunities for
cross-sectoral cooperation 5,7 19,6 42,4 32,3
Table 9. Perceptions of efficiency of Erasmus+ (respondents having Erasmus+ experience)
DO NOT
AGREE
NEITHER
AGREE NOR
DISAGREE
AGREE I CANNOT
ESTIMATE
%
Application procedures are clear and
customised for users 6,7 24,8 65,2 3,2
Evaluation criteria and implementation are
transparent 10,1 25,2 58,2 6,6
Regulations that apply to project financial
asset management (i.e., accounting rules,
expenditures, payment terms) are clear
10,8 28,7 57,6 2,8
The timeframe for project implementation
is commensurate with the real abilities of
users
1,8 11,2 84,9 2,1
The timeframe for reporting on project
results is adequate 2,7 8,2 84,4 4,8
If we compare the efficiency of Erasmus+ with its predecessor programmes, at least from the
perspective of users with the experience necessary to make such assessments, most of them recognise
50
the improvements that Erasmus+ brought. Improvements were recognised in terms of improving
opportunities for cross-sectoral cooperation, simplifying the financial management of projects,
adjusting grants to the needs of users, and standardising administration so as to ease project
management. This is especially so for cross-sectoral cooperation and simplifying the financial
management of projects. As for the evaluation of Erasmus+ efficiency in general, the majority of
respondents agree that the programme is efficient because the timeframe for project implementation is
commensurate with the real abilities of users and because the timeframe for reporting on project
results is adequate. A lesser majority of respondents consider application procedures to be clear and
customised for users, evaluation criteria and implementation transparent, and regulations that apply to
project financial asset management (i.e., accounting rules, expenditures, payment terms) clear.
In order to assess the adequacy of IT tools, respondents with experience in both Erasmus+ and its
predecessor programmes were asked about Mobility Tool+, URF, the VALOR-dissemination platform
and OLS- Online Linguistic Support.
Table 10. Assessment of the adequacy of IT tools (respondents having experience in both
Erasmus+ and its predecessor programmes)
MOBILITY
TOOL+ URF
VALOR-
DISSEMINATIO
N PLATFORM
OLS-ONLINE
LINGUISTIC
SUPPORT
Yes No Yes No Yes No Yes No
%
The IT tool is
adjusted to the IT
knowledge of users
91,7 8,3 88,9 11,1 87,5 12,5 90,8 9,2
The IT tool is fully
functional 79,7 20,3 82,3 15,2 83,6 16,4 86,7 13,3
There is clear
guidance for using
the IT tool
85,4 14,6 84,8 15,2 87,8 12,2 88,8 11,2
Although there are small differences between the distributions of answers, there is no statistically
significant difference. Respondents report that IT tools such as Mobility Tool+, URF, the Valor-
dissemination platform and Online Linguistic Support-OLS are adjusted to the IT knowledge of users,
are fully functional, and that there is clear guidance for using them. The adequacy of IT tools is
assessed in the same manner, regardless of the sector in which respondents are engaged.
Furthermore, the efficacy of Erasmus+ should be recognised within the context of obstacles that could
stand in the way of the programme´s realisation. Therefore, all respondents, regardless of whether they
have participated in Erasmus+ and its predecessor programmes, were asked to assess the extent to
which they encountered obstacles to participation in Erasmus+.
51
Table 11. Encountering obstacles to participation in Erasmus+ (all respondents)
NO
T A
T A
LL
MO
ST
LY
NO
MO
DE
RA
TE
LY
MO
ST
LY
YE
S
YE
S
CO
MP
LE
TE
LY
I C
AN
NO
T
ES
TIM
AT
E
% M SD
Project participation requires too many
administrative, human and financial resources 5,4 14,5 26,1 29,5 21,9 2,7 3,49 1,154
Excessive workload 7,8 12,8 22,3 32,0 20,9 4,2 3,47 1,200
Administrative formalities related to project
participation discourage employees from
engaging in mobility
6,7 16,4 21,6 26,6 24,1 4,6 3,47 1,234
Lack of funding within the institution to cover
costs related to project participation 10,1 14,9 21,0 23,6 26,2 4,2 3,43 1,319
Non-recognition of project participation for
the purpose of career advancement 12,4 14,3 19,0 21,6 25,5 7,1 3,36 1,376
Insufficient interest of employees in project
participation 13,7 15,9 27,8 26,1 12,7 3,8 3,09 1,236
Lack of knowledge among administrative staff
for project implementation 14,6 18,9 24,2 21,2 17,6 3,5 3,09 1,320
Insufficient knowledge of foreign languages 14,1 21,1 30,3 24,0 8,7 1,9 2,92 1,175
Lack of evaluation of work on international
projects within one’s own institution 21,7 19,2 16,5 19,4 16,8 6,3 2,90 1,431
Lack of information among employees about
possibilities for going on mobility 18,4 21,3 23,9 22,9 10,0 3,6 2,84 1,271
Difficulty finding replacements for employees
who would like to go on mobility 30,3 24,4 16,0 14,4 8,5 6,4 2,43 1,321
Lack of support from partner institutions
abroad 19,6 31,4 24,0 9,8 3,2 11,9 2,38 1,060
Due to the specificities of our programs, it is
very hard to find partner institutions with
similar programs
25,7 30,8 23,8 9,1 4,9 5,7 2,33 1,127
Inability to obtain permission to go on
mobility 40,8 22,6 17,0 9,0 5,9 4,7 2,12 1,236
The main obstacles recognised as those influencing participation in Erasmus+ (around or over 50 % of
respondents recognise them) concern a lack of administrative, human and financial resources, the
discouragement of employees via administrative formalities, excessive workloads and the non-
recognition of project participation for the purpose of career advancement.
Since a lack of administrative, human and financial resources is recognised as the most prevalent
obstacle, it is instructive to explore it in more depth. All respondents were asked to assess the
(in)sufficiency of capacities in terms of administrative, financial and human resources for project
implementation within Erasmus+.
52
Table 12. Assessment of administrative, financial and human capacities for implementation
of projects within Erasmus+ (all respondents)
INS
UF
FIC
IEN
T
CA
PA
CIT
IES
SU
FF
ICIE
NT
CA
PA
CIT
IES
I D
O N
OT
KN
OW
/ I
CA
NN
OT
ES
TIM
AT
E
%
Financial resources available for pre-financing of project activities
implementation 56,4 32,6 11,1
Time available for employees to participate in projects 53,9 37,6 8,5
Expertise of accounting staff required for the financial monitoring of EU projects 39,2 47,8 13,0
Knowledge and skills of employees in regard to the administrative monitoring of
projects (i.e., the preparation of written reports, project proposals and
accompanying documentation)
36,0 57,7 6,3
Practical experience with project management 35,9 58,5 5,6
Existing partnerships with foreign institutions 31,2 61,0 7,8
Knowledge of foreign languages in spoken and written form 20,9 74,7 4,4
Cooperation and communication within the organisation 19,0 76,1 4,9
Management skills of managers within the organisation 15,8 74,6 9,6
Usage of information and communication tools 14,1 81,1 4,9
Concerning financial resources, the lack of capacities mainly reflects challenges related to pre-
financing of project activities implementation, since over 50 % of respondents declare this an
insufficient capacity. Findings are similar regarding the time available for employees to participate in
projects. In regard to administrative and human resources, respondents recognise to a greater extent
insufficient capacities in the expertise of accounting staff required for the financial monitoring of EU
projects, the knowledge and skills of employees in regard to the administrative monitoring of projects
(i.e., the preparation of written reports, project proposals and accompanying documentation) and
practical experience with project management. In addition, when we compare the assessment of
capacities concerning the expertise of accounting staff required for financial monitoring and the
perception of the efficiency of Erasmus+ in terms of the extent to which regulations that apply to
project financial asset management (i.e., accounting rules, expenditures, payment terms) are clear, then
it is observable that those respondents who assess their own organisations to have lower capacities
likewise perceive Erasmus+ to be less efficient..
The aspects analysed above will be separately analysed specifically with respect to the YOU and the
E&T.
53
a) Youth
If we analyse perceptions of Erasmus+ efficiency among respondents who are engaged in the YOU
and compare these with the perceptions of all others, some significant differences are observable.
Table 13. Perceptions of integrated Erasmus+ programme efficiency (respondents from the
YOU who are experienced with Erasmus+ and its predecessor programmes)
TO A
LESSER
EXTENT
MODERATELY TO A GREAT
EXTENT
I CANNOT
ESTIMATE
%
Grants are to a greater extent
adjusted to the real needs of user
organisations
21,6 32,4 26,1 19,8
Table 14. Perceptions of Erasmus+ programme efficiency (respondents from the YOU with
Erasmus+ experience)
DO NOT
AGREE
NEITHER AGREE
NOR DISAGREE AGREE
I CANNOT
ESTIMATE
%
Evaluation criteria and
implementation are transparent 17,9 31,3 47,0 3,7
In terms of integrated Erasmus+ programme efficiency, respondents engaged in the YOU recognise to
a lesser extent that grants are adjusted to the real needs of user organisations. Additionally, with
respect to the efficiency of Erasmus+ in general, the same respondents agree to a lesser extent that the
evaluation criteria and implementation are transparent.
Concerning obstacles to participation in Erasmus+ within the YOU, respondents seem to encounter
fewer issues in terms of several elements.
Table 15. Encountering obstacles to participation in Erasmus+ (respondents from the YOU)
NO
T A
T A
LL
MO
ST
LY
NO
MO
DE
RA
TE
LY
MO
ST
LY
YE
S
YE
S
CO
MP
LE
TE
LY
I C
AN
NO
T
ES
TIM
AT
E
% M SD
Project participation requires too many
administrative, human and financial resources 7,5 20,7 25,9 25,9 16,7 3,4 3,34 1,279
Non-recognition of project participation for
the purpose of career advancement 17,2 19,0 23,0 14,4 15,5 10,9 3,25 1,606
Administrative formalities related to project
participation discourage employees from
engaging in mobility
11,5 24,1 21,8 21,8 14,4 6,3 3,22 1,423
A lack of knowledge among administrative
staff to implement projects 17,8 23,0 25,3 16,1 13,8 4,0 2,97 1,424
A lack of evaluation of international project
work within the institution 34,5 16,1 12,1 17,8 9,8 9,8 2,82 1,724
Insufficient interest of employees in project
participation 20,1 27,0 24,7 14,9 8,0 5,2 2,79 1,407
54
Insufficient knowledge of foreign languages 28,2 27,0 25,3 9,8 7,5 2,3 2,48 1,320
Due to the specificities of our programs, it is
very hard to find partner institutions with
similar programs
32,8 30,5 19,5 7,5 3,4 6,3 2,37 1,420
In comparison to those who are not engaged in the YOU, those who are assess fewer obstacles such as
insufficient interest of employees in project participation, insufficient knowledge of foreign languages,
non-recognition of project participation for the purpose of career advancement, a lack of evaluation of
international project work within the institution, and a lack of knowledge among administrative staff
to implement projects. They also think to a lesser extent that project participation requires too many
administrative, human and financial resources, that administrative formalities related to project
participation discourage employees from engaging in mobility, and that due to the specificities of their
programs, it is hard to find partner institutions with similar programs.
Table 16. Assessment of administrative, financial and human capacities for Erasmus+ project
implementation (respondents from the YOU)
INS
UF
FIC
IEN
T
CA
PA
CIT
IES
SU
FF
ICIE
NT
CA
PA
CIT
IES
I D
O N
OT
KN
OW
/ I
CA
NN
OT
ES
TIM
AT
E
%
Cooperation and communication within the organisation 8,6 88,5 2,9
Management skills of managers within the organisation 10,3 84,5 5,2
Practical experience with project management 20,1 78,7 1,1
Existing partnerships with foreign institutions 18,4 77,0 4,6
Knowledge and skills of employees in regard to the administrative monitoring of
projects (i.e., the preparation of written reports, project proposals and
accompanying documentation)
30,5 68,4 1,1
Expertise of accounting staff required for the financial monitoring of EU projects 28,2 60,9 10,9
Respondents report more sufficient capacities in areas such as the knowledge and skills of employees
in regard to the administrative monitoring of projects (i.e., the preparation of written reports, project
proposals and accompanying documentation), the practical experience of project management; the
management skills of managers within the organisation, cooperation and communication within the
organisation, existing partnerships with foreign institutions, and the expertise of accounting staff
required for the financial monitoring of EU projects.
55
b) Education and Training
The differences across sectoral fields were also analysed.
Table 17. Perceptions of integrated Erasmus+ programme efficiency (respondents across
action fields in the E&T who have experience with Erasmus+ and its predecessor
programmes)
TO A
LESSER
EXTENT
MODERATELY
TO A
GREATER
EXTENT
I CANNOT
ESTIMATE
%
The standardisation of
administrative regulations and
documents made it easier to
manage projects
HE 7,5 18,9 40,6 33,0
VET 10,7 25,0 42,9 21,4
ADU 12,0 32,0 36,0 20,0
SCH 21,1 28,4 25,7 24,8
It has simplified the financial
management of projects
HE 7,5 17,0 41,5 34,0
VET 21,4 3,6 50,0 25,0
ADU 8,0 32,0 44,0 16,0
SCH 17,4 26,6 27,5 28,4
In regard to perceptions of integrated Erasmus+ programme efficiency, non-parametric testing
indicates differences between those respondents engaged in the HE and those engaged in SCH. The
standardisation of administrative regulations and documents, which made it easier to manage projects,
and the simplification of the financial management of projects due to the integration of previous
programmes into Erasmus+ were recognised to a greater extent by those respondents engaged in the
HE than by those engaged in SCH.
Table 18. Encountering obstacles to participation in Erasmus+ (respondents across action
fields in the E&T)
NO
T A
T A
LL
MO
ST
LY
NO
MO
DE
RA
TE
LY
MO
ST
LY
YE
S
YE
S
CO
MP
LE
TE
LY
I C
AN
NO
T
ES
TIM
AT
E
% M SD
Non-recognition of project
participation for the purpose of career
advancement
HE 9,9 15,3 19,8 27,5 21,4 6,1 3,53 1,399
VET 10,2 8,5 23,7 28,8 22,0 6,8 3,64 1,374
ADU 22,1 22,1 14,7 17,6 13,2 10,3 3,09 1,664
SCH 12,5 11,3 17,5 20,8 33,8 4,2 3,65 1,462
A lack of knowledge among
administrative staff to implement
projects
HE 11,5 26,7 29,8 23,7 7,6 0,8 3,06 1,385
VET 8,5 22,0 39,0 27,1 3,4 2,81 1,210
ADU 35,3 25,0 25,0 4,4 8,8 1,5 2,56 1,376
SCH 8,8 15,8 33,3 30,0 11,3 0,8 3,30 1,351
A lack of evaluation of international
project work within the institution
HE 17,6 20,6 13,7 20,6 19,8 7,6 3,27 1,594
VET 20,3 16,9 23,7 22,0 15,3 1,7 3,00 1,414
ADU 35,3 22,1 19,1 10,3 10,3 2,9 2,47 1,471
SCH 18,8 16,3 20,4 20,4 19,2 5,0 3,20 1,518
Insufficient interest of employees in
project participation
HE 13,7 15,3 29,0 30,5 9,2 2,3 3,13 1,255
VET 20,3 8,5 32,2 27,1 10,2 1,7 3,03 1,326
ADU 30,9 20,6 19,1 11,8 10,3 7,4 2,72 1,610
56
SCH 9,6 13,8 30,8 26,7 16,7 2,5 3,35 1,251
Insufficient knowledge of foreign
languages
HE 11,5 26,7 29,8 23,7 7,6 0,8 2,92 1,157
VET 8,5 22,0 39,0 27,1 3,4 2,95 ,990
ADU 35,3 25,0 25,0 4,4 8,8 1,5 2,31 1,319
SCH 8,8 15,8 33,3 30,0 11,3 0,8 3,22 1,137
The recognition of obstacles to participation in Erasmus+ seems to differ across fields of the E&T in
several respects, but mostly in such a way that respondents from the ADU encounter fewer obstacles.
For instance, non-parametric testing indicates that those in the ADU experience fewer obstacles than
those in the SCH in terms of insufficient interest of employees in project participation, insufficient
knowledge of foreign languages and a lack of knowledge among administrative staff to implement
projects. In addition, those in the ADU experience fewer obstacles to participation in Erasmus+ in
terms of non-recognition of project participation for the purpose of career advancement and a lack of
evaluation of international project work within the institution than those working in higher education
and school education.
Similar differences can be seen in the assessment of administrative, financial and human capacities for
Erasmus+ project implementation.
Table 19. Assessment of administrative, financial and human capacities for Erasmus+ project
implementation (respondents across action fields in the E&T)
INS
UF
FIC
IEN
T
CA
PA
CIT
IES
SU
FF
ICIE
NT
CA
PA
CIT
IES
I D
O N
OT
KN
OW
/ I
CA
NN
OT
ES
TIM
AT
E
%
Time available for employees to participate in projects
HE 61,1 33,6 5,3
VET 62,7 25,4 11,9
ADU 35,3 58,8 5,9
SCH 52,5 37,5 10,0
Practical experience with project management
HE 38,2 56,5 5,3
VET 35,6 61,0 3,4
ADU 19,1 76,5 4,4
SCH 39,2 55,4 5,4
Existing partnerships with foreign institutions
HE 18,3 73,3 8,4
VET 23,7 67,8 8,5
ADU 23,5 67,6 8,8
SCH 34,2 58,3 7,5
Those engaged in the ADU recognise more capacities than those in the SCH in terms of existing
partnerships with foreign institutions. They recognise more sufficient capacities in terms of practical
experience with project management than do those engaged in the HE and those engaged in the SCH.
In comparison to those engaged in the VET, the HE and the SCH, those engaged in the ADU claim to
have more sufficient capacities in terms of time available for employees to participate in projects.
57
III. Relevance
The relevance of Erasmus+ and its successfulness in attracting and reaching various target groups
depends on the relevance of its proclaimed objectives. This is why we have asked all respondents to
assess, based on their experiences in the sectors in which they are engaged, whether the proclaimed
Erasmus+ objectives are relevant for their sector. For each of the Erasmus+ specific objectives,
respondents made this assessment. The results are presented in particular for the YOU and the E&T.
a) Youth
All respondents, regardless of whether they have participated in Erasmus+, gave their views on the
relevance of specific Erasmus+ objectives that are important to the YOU.
Table 20. Assessment of the relevance of Erasmus+ specific objectives (all respondents)
OBJECTIVE IS
RELEVANT
FOR MY
SECTOR
OBJECTIVE IS
NOT RELEVANT
FOR MY
SECTOR
I DO NOT
KNOW / I
CANNOT
ESTIMATE
Improving key competencies and skills
among youth ( "key competencies" = the
basic set of knowledge, skills and attitudes
that all individuals need for personal
fulfilment and development, active
citizenship, social inclusion and employment)
84,8 7,8 7,4
Including young people with fewer
opportunities or who encounter obstacles that
prevent an active approach to education,
training and work
74,3 13,3 12,4
Promoting participation in democratic life in
Europe 69,6 16,8 13,7
Promoting participation in the labour market 60,9 21,1 18,0
Active citizenship 75,3 12,8 11,9
Intercultural dialogue 84,6 8,1 7,3
Social inclusion 84,3 8,0 7,8
Achieving solidarity 78,1 11,1 10,9
Increasing learning mobility opportunities for
young people, those active in youth work or
youth organisations, and youth leaders
78,1 12,9 9,0
Strengthening links between the youth field
and the labour market 59,4 20,9 19,8
Enhancing cooperation between organisations
in the youth field and/or other stakeholders 71,5 14,3 14,3
Complementing policy reforms at the local,
regional and national levels 52,1 20,5 27,4
Developing knowledge- and evidence-based
youth policies 52,7 20,5 26,8
Recognising non-formal and informal
learning 78,1 9,5 12,4
Enhancing the international dimension of
youth activities 76,4 12,8 10,8
Strengthening the role of socio-pedagogical
staff and organisations as support structures 71,8 12,3 15,9
58
for young people
Overall, each of the above-listed objectives seems to be relevant for the majority of respondents, but
some differences are significant. Some objectives seem to be more widely recognised as relevant for
almost all respondents. Around or over 80 % of respondents recognise improving key competencies
and skills among youth, intercultural dialogue, social inclusion, recognising non-formal and informal
learning, and increasing learning mobility opportunities for young people, those active in youth work
or youth organisations, and youth leaders as objectives which are relevant for their sector. Objectives
with a significantly lower lever of recognition as relevant for respondents’ sectors, although still
relevant, are those concerning policy and the labour market. Just over 45 % of respondents are unable
to estimate or assess specific objectives, such as complementing policy reforms at the local, regional
and national levels and developing knowledge- and evidence-based youth policies, as relevant for their
sector. Responses are similar regarding the specific objective of strengthening links between the YOU
and the labour market. If we test the difference between programme beneficiaries and those who were
merely applicants, then there is no significant difference. The only significant difference is that those
engaged in the YOU recognise most objectives as relevant for their sector to a greater extent than do
those who are not engaged in the YOU. An assessment of the relevance of objectives such as
intercultural dialogue, complementing policy reforms at the local, regional and national levels,
recognising non-formal and informal learning, and strengthening the role of socio-pedagogical staff
and organisations as support structures for young people reveals responses distributed in the same
manner regardless of whether respondents are engaged in the YOU.
Table 21. Assessment of the relevance of Erasmus+ specific objectives (respondents from
the YOU)
OBJECTIVE IS
RELEVANT
FOR MY
SECTOR
OBJECTIVE IS
NOT RELEVANT
FOR MY
SECTOR
I DO NOT
KNOW / I
CANNOT
ESTIMATE
Improving key competencies and skills
among youth ( "key competencies" = the
basic set of knowledge, skills and attitudes
that all individuals need for personal
fulfilment and development, active
citizenship, social inclusion and employment)
94,3 2,9 2,9
Including young people with fewer
opportunities or who encounter obstacles that
prevent an active approach to education,
training and work
86,8 8,6 4,6
Promoting participation in democratic life in
Europe 82,2 12,1 5,7
Promoting participation in the labour market 68,4 14,9 16,7
Active citizenship 92,0 5,7 2,3
Social inclusion 94,3 4,0 1,7
Achieving solidarity 90,8 5,7 3,4
Increasing learning mobility opportunities for
young people, those active in youth work or
youth organisations, and youth leaders
87,9 9,2 2,9
Strengthening links between the youth field
and the labour market 68,4 12,6 19,0
Enhancing cooperation between organisations
in the youth field and/or other stakeholders 86,8 7,5 5,7
59
Developing knowledge- and evidence-based
youth policies 68,4 15,5 16,1
Enhancing the international dimension of
youth activities 91,4 6,3 2,3
Although youth sector respondents assess the relevance of specific objective more widely, the pattern
is similar. Objectives such as promoting participation in the labour market, strengthening links
between the YOU and the labour market, and developing knowledge- and evidence-based youth
policies lag behind in comparison to other Erasmus+ objectives.
b) Education and Training
All respondents, regardless of whether they have participated in Erasmus+, gave their views on the
relevance of Erasmus+ specific objectives that are important to the E&T.
Table 22. Assessment of the relevance of Erasmus+ specific objectives (respondents from
the E&T)
OBJECTIVE IS
RELEVANT
FOR MY
SECTOR
OBJECTIVE IS
NOT
RELEVANT FOR
MY SECTOR
I DO NOT
KNOW / I
CANNOT
ESTIMATE
Improving key competencies and skills among
youth ( "key competencies" = the basic set of
knowledge, skills and attitudes that all
individuals need for personal fulfilment and
development, active citizenship, social
inclusion and employment)
90,7 4,8 4,6
Improving key competencies and skills with
regard to labour market needs 70,7 16,1 13,1
Contributing to social cohesion 83,2 7,4 9,3
Creating mobility opportunities for the purpose
of formal/non-formal/informal education 90,4 4,9 4,8
Strengthening cooperation between academia
and the wider economy 69,0 16,9 14,2
Fostering quality improvement, excellence in
innovation and internationalisation on the level
of educational and training institutions
77,5 12,5 10,0
Promoting the emergence and raising
awareness of a European lifelong learning area 84,0 7,9 8,2
Enhancing the international dimension of
education and training 85,3 8,0 6,7
Establishing cooperation among EU
institutions in the field of vocational education
and training
54,8 30,6 14,6
Establishing cooperation among EU
institutions in the field of higher education 34,7 43,8 24,1
Increasing the attractiveness of European
higher education institutions 35,0 44,3 20,8
Improving the teaching and learning of
languages, and promoting broad linguistic
diversity and intercultural awareness in the
European Union
83,8 10,1 6,1
60
For most of the specific objectives in the E&T, a majority of respondents recognise them as being
relevant for their sector, except for two which apply to the HE. Since those two objectives are
specifically tailored to those engaged in the HE, the difference is understandable. Therefore, it is more
instructive to consider how respondents engaged in the E&T assess the relevance of Erasmus+ specific
objectives, and what the differences are across action fields. However, it is interesting to first briefly
note the difference among those respondents with only Erasmus+ programme experience and those
who engaged only in its predecessor programmes. The latter recognise to a slightly lesser extent the
objectives of improving key competencies and skills with regard to labour market needs, improving
quality, excellence in innovation and internationalisation on the level of education institutions, and
improving the teaching and learning of languages and promoting broad linguistic diversity and
intercultural awareness in the European Union. It would be instructive to explore in future research
endeavours the reasoning of those experienced only in the “old programmes” for not engaging in the
Erasmus+ programme.
Table 23. Assessment of the relevance of Erasmus+ specific objectives (respondents across
action fields in the E&T)
OBJECTIVE IS
RELEVANT
FOR MY
SECTOR
OBJECTIVE IS
NOT
RELEVANT
FOR MY
SECTOR
Pearson
Chi-
Square
Sig.
%
Improving key competencies and skills
with regard to labour market needs
HE 87,3 12,7
14,892 ,002 VET 94,4 5,6
ADU 72,1 27,9
SCH 76,9 23,1
Strengthening cooperation between
academia and the wider economy
HE 88,3 11,7
23,252 ,000 VET 94,7 5,3
ADU 77,6 22,4
SCH 71,0 29,0
Fostering quality improvement,
excellence in innovation and
internationalisation on the level of
educational and training institutions
HE 89,0 11,0
16,617 ,001 VET 87,7 12,3
ADU 79,3 20,7
SCH 95,6 4,4
Establishing cooperation among EU
institutions in the field of vocational
education and training
HE 72,6 27,4
19,325 ,000 VET 84,2 15,8
ADU 66,1 33,9
SCH 56,0 44,0
Establishing cooperation among EU
institutions in the field of higher
education
HE 71,1 28,9
36,600 ,000 VET 40,8 59,2
ADU 50,0 50,0
SCH 35,7 64,3
Increasing the attractiveness of European
higher education institutions
HE 73,3 26,7
42,063 ,000 VET 34,8 65,2
ADU 35,8 64,2
SCH 39,0 61,0
Improving the teaching and learning of
languages, and promoting broad
linguistic diversity and intercultural
awareness in the European Union
HE 91,3 8,7
30,210 ,000 VET 85,2 14,8
ADU 76,6 23,4
SCH 97,0 3,0
Enhancing the international dimension of HE 92,2 7,8 n.s.
61
education and training VET 89,5 10,5
ADU 95,2 4,8
SCH 97,4 2,6
Promoting the emergence and raising
awareness of a European lifelong
learning area
HE 91,8 8,2
n.s. VET 92,7 7,3
ADU 95,4 4,6
SCH 96,1 3,9
Contributing to social cohesion
HE 89,5 10,5
n.s. VET 87,5 12,5
ADU 93,8 6,3
SCH 95,6 4,4
Creating mobility opportunities for the
purpose of formal/non-formal/informal
education
HE 93,0 7,0
n.s. VET 94,8 5,2
ADU 97,0 3,0
SCH 97,8 2,2
Improving key competencies and skills
in general ( "key competencies" = the
basic set of knowledge, skills and
attitudes that all individuals need for
personal fulfilment and development,
active citizenship, social inclusion and
employment)
HE 95,4 4,6
n.s.
VET 98,3 1,7
ADU 92,3 7,7
SCH 97,9 2,1
Not all specific Erasmus+ programme objectives are targeted for all action fields within the E&T, so
differences in assessing the relevance of objectives exist. For instance, objectives such as improving
key competencies and skills with regard to labour market needs and strengthening cooperation
between academia and the wider economy are recognised among all respondents in the E&T, but more
so among those in the VET. The biggest differences can be seen with objectives such as establishing
cooperation among EU institutions in the HE and increasing the attractiveness of European HEIs,
since these are mostly recognised among those engaged in the HE.
62
IV. Internal and external coherence and complementarity
In regard to internal and external coherence and complementarity, survey analysis does not allow us to
provide deeper insights which could give us perspective on potential synergies between actions within
Erasmus+. The only analytical possibility given a survey like this is to explore the differences in
perception among respondents who each have experience in one of the Key Actions of Erasmus+.
Non-parametric testing shows us that the only statistically significant difference can be found in regard
to perceptions of Erasmus+ programme efficiency. For instance, those respondents who have
experience in KA1 think that Erasmus+ application procedures and regulations for project financial
asset management (i.e., accounting rules, expenditure, payment terms) are clear to a greater extent than
do those respondents who have experience in KA2.
63
V. European added value and sustainability
Concerning European added value and sustainability, the future development and implementation of
the Erasmus+ programme will surely rely on various elements. A survey analysis of the type
conducted here is also limited in providing answers which could help us forecast the potential of
individuals and institutions which would benefit from the programme. The only use of such a survey is
to present in a “brainstorming” manner the open answers of respondents in regard to their ideas on the
future development of Erasmus+. A random selection of these answers is presented in the table
immediately below.
Table 24. Ideas on the future objectives of Erasmus+ (all respondents)
A continuity of efforts from the previous programming period. Focusing on further streamlining and
optimising the administration of procedures. Strengthening support for the triple helix paradigm,
linking the education system, public administration and industry.
Promotion of healthy lifestyles, social entrepreneurship and free media.
Intercultural dialogue, with an emphasis on understanding global migration.
I consider it necessary to invest more in those young people who really have significantly fewer
opportunities than most to participate in social processes. They are reluctant to apply for project
participation because they believe that, even given support, they will not have sufficient funds to
cover travel expenses or prepare for mobility. I think that these are groups that need a different form
of financing.
Promoting a culture of peace.
Encourage employers to get more involved and receive students on mobility for the purpose of
gaining professional experience.
Ecology and sustainable development of communities.
The focus should be on mitigating radicalism in any form, and on strengthening cooperation and
partnerships.
Provide subsidies for education in the field of green construction.
If we take a look once more at the responses from tables 6 and 7, then it is obvious that respondents
recognise the European dimension, since a majority of respondents recognise that Erasmus+ promotes
participation in democratic life in Europe, promotes the emergence and raises awareness of a European
lifelong learning area, enhances the international dimension of education and training, establishes
cooperation among EU institutions in the VET and the HE, increases the attractiveness of European
HEIs, improves the teaching and learning of languages, promotes broad linguistic diversity and
intercultural awareness in the European Union.
64
Conclusions and suggestions for improvements to Erasmus+ and for a
future programme
The intent of the analysis of data in this report was to present research on the perspectives of
participants and representatives of user organisations within the Erasmus+ programme and its
predecessor programmes as to the effectiveness, efficiency, relevance, coherence and
complementarity, and added value of Erasmus + at the national level. Therefore, each of the explored
elements of Erasmus+ realisation was conceptualised in order to present the extent to which
respondents assessed the programme’s realisation.
Programme effectiveness was operationalised in such a way that respondents assessed the overall
visibility of Erasmus+, evaluated the contribution of the Key Actions in the realisation of Erasmus+
specific objectives, and evaluated the realisation of those objectives. First of all, respondents on
average think that the dissemination and usage of information about Erasmus+ is good. The analysis
shows that those who were beneficiaries of Erasmus+ and its predecessor programmes, as well as
those who were otherwise engaged in Erasmus+, perceive the overall visibility of Erasmus+ to a
greater extent. This indicates a need to increase the overall visibility of Erasmus+ among those who
were not granted financial support (i.e., non-beneficiaries) and among those who did not participate in
the programme. Therefore, it could be recommended that any future programme should strive for
wider visibility in order to inform a wider interested audience about its results and good practices. An
additional argument could be drawn from the differences among the action fields, since VET actors
expressed greater satisfaction with the visibility and promotion of Erasmus+ on the national and
regional level as well as with programme results. Concerning the Key Actions, KA1 and KA2 are
recognised as having contributed to the realisation of Erasmus+ specific objectives, with the exception
of the KA3 possibly resulting from the fact that it was not open to a wider circle of applicants.
However, when we take a look at the distribution of answers from those respondents that have
experiences within all of the Key Actions, the KA3 has a lower level of recognition in terms of its
contribution to the realisation of Erasmus+ specific objectives. Concerning the extent to which
Erasmus+ is effective in the realisation of its specific objectives in the youth sector, the perspectives of
respondents indicate that the realisation of all specific objectives is not seen in the same manner.
Although Erasmus+ contributes to the realisation of all of its specific objectives, it seems that, from
the perspective of various relevant actors and youth professionals, the programme falls short in
connecting youth with the labour market. Since Erasmus+ puts an emphasis on strengthening links
between the YOU and the labour market as well as promoting the participation of youth in the labour
market, and these objectives are recognised to a lesser extent than objectives such as intercultural
dialogue, improving key competencies and skills of youth, and social inclusion, it seems that any
future programme should put more effort into realising objectives having to do with youth and the
labour market.
Programme efficiency was operationalised by exploring perceptions of the efficiency of Erasmus+ and
its predecessor programmes in general on the one hand, and of the adequacy of IT tools, the obstacles
to Erasmus+ participation, and administrative, financial and human capacities on the other.
Respondents with experience in Erasmus+ and its predecessor programmes recognise the
improvements that Erasmus+ made by expanding opportunities for cross-sectoral cooperation,
simplifying the financial management of projects, adjusting grants to the needs of users, and
standardising administrative processes that made it easier to manage projects. However, it should be
noted that respondents recognise to a lesser extent that application procedures are clear and user-
65
specific, that the evaluation criteria and implementation are transparent, and that the project’s financial
asset management regulations are clear. Since a similar finding was noted in the evaluation study of
LLP implementation in Croatia (Ančić and Klasnić, 2016), any future programme should consider
further simplifying and adapting application and financial procedures as well as regulations. From
the perspective of the YOU, respondents recognise to a lesser extent that grants are adjusted to the real
needs of their organisations, and that the evaluation criteria and implementation are transparent.
Concerning the differences among action fields, it is indicated that respondents from the HE recognise
the efficiency of Erasmus+ in terms of administrative regulations standardisation and project financial
management simplification to a greater extent than do those from the SCH. These findings should be
analysed in the context of perceived obstacles to participation in Erasmus+ and an assessment of the
administrative, financial and human capacities for project implementation under Erasmus+. Obstacles
recognised the most as those impacting the ability of people and their institutions to participate in the
Erasmus+ have to do with a lack of administrative, human and financial resources, the discouragement
of employees with administrative formalities, excessive workloads, and a lack of recognition for
participation in projects for the purpose of career advancement. In addition, over 50 % of respondents
report financial capacities/resources that are insufficient to pre-finance project activity
implementation, while those who report limited organisational capacity in the form of accounting staff
expertise required for financial monitoring also perceive less clarity in Erasmus+ regulations that
apply to project financial asset management (i.e., accounting rules, expenditures, payment terms).
Since there are differences among sector fields and action fields – for example, participants from the
YOU or the ADU perceive fewer obstacles and greater capacities for project implementation within
Erasmus+ –, any future programme should be more aware of the differences in the potential of users to
absorb administrative, human and financial capacities and resources. In other words, any future
programme should try to adjust and design its rules and procedures in such a way as to recognise and
accept the differences among sectors and among action fields since competitive conditions and
potential are not the same for all interested users. Meanwhile, respondents perceive IT tools such as
Mobility Tool+, URF, Valor-dissemination platform and Online Linguistic Support-OLS to be fully
functional, and that guidance in their usage is clear.
Programme relevance is explored by assessing the extent to which Erasmus+ specific objectives are
relevant to users in their respective sectors. In both sectors, the YOU and the E&T, respondents
recognise all specific objectives as relevant, which is important when it comes to successfully
attracting and reaching various target groups. A distinction should be made about objectives that
tackle challenges and issues related to policy and the labour market. For instance, concerning specific
objectives within the YOU, just over 45% of respondents are unable to estimate or assess the
relevance of specific objectives – such as complementary policy reforms at the local, regional and
national level, or the development of a knowledge- and evidence-based youth policy – to their sector.
A similar situation characterises the objective of strengthening links between the YOU and the labour
market. In the E&T, objectives such as improving key competencies and skills with regard to labour
market needs and strengthening cooperation between academia and the wider economy are recognised
to a greater extent among those in the VET than among those in the other three fields. Therefore, any
future programme should put more effort into developing more effective mechanisms that create
conditions of more intense cooperation with the wider economy.
Internal and external coherence and complementarity cannot be effectively measured based on a
survey, so the analysis of respondents’ answers is limited. The only finding is in regard to
differentiating between those who are experienced in KA1 and those who are experienced in KA2 in
such a way that the former find Erasmus+ application procedures and regulations that apply to project
66
financial asset management (i.e., accounting rules, expenditures, payment terms) clear to a greater
extent than do the latter.
European added value and sustainability likewise cannot be effectively measured based on a survey
like this. However, respondents do recognise the realisation of specific objectives concerning
European added value. A majority of respondents agree that Erasmus+ promotes participation in
democratic life in Europe as well as the emergence and increasing awareness of a European lifelong
learning area, enhances the international dimension of education and training, establishes cooperation
among EU institutions in the VET and the HE, increases the attractiveness of European HEIs,
improves the teaching and learning of languages, and promotes broad linguistic diversity and
intercultural awareness in the European Union.
Literature
Ančić, B., Klasnić, K., 2016. Evaluation of the Comenius, Grundtvig and Leonardo Da Vinci Sectoral
Programmes. In: Milanović-Litre, I. Puljiz, I., Gašparović, F., ed. Towards
Internationalisation of Education – Participation of the Republic of Croatia in the Lifelong
Learning Programme, Zagreb: Agency of Mobility and EU Programmes, pp. 60-150.
67
Appendix IV. E+ National evaluation - Qualitative
research report
Prepared by: doc. dr. sc. Marija Brajdić Vuković, University of Zagreb
Contents
Introduction 68
Methodology 69
Evaluation results: 70
I. Effectiveness 70
II. Efficiency 72
III. Relevance 72
IV. Internal and external coherence and complementarity 74
V. European added value and sustainability 75
Conclusions and suggestions for improvements to E+ and for a future programme 77
68
Introduction
This report is based on qualitative research which took the form of semi-structured interviews that
were conducted with respondents from a member sample of three different populations perceived to
have the broadest outlook and perspective related to the E+ programme and its effectiveness,
efficiency and relevance in Croatia. Specifically, these respondents were three representatives of the
NA as the main and formal E+ implementing organization on the national level, and four
representatives of government organizations.
Semi-structured interviews with the stakeholders about E+ were, from the researcher perspective,
generally very pleasant, positive and informative. All of the respondents were very interested in the
topic and obviously engaged enough to have the perspective not just of those involved in
implementation but also of those very interested in making the most of it for users, institutions
involved in projects and the country in general. The interview protocol, with topics and questions
based on the evaluation questions set by the Commission, was sent in advance to the respondents so
that they had a chance to prepare the necessary information and reconsider their views on the
implementation of E+. However, the interview was guided by spontaneous conversation with
respondents about the E+ programme, its implementation in Croatia, their experiences with the
programme, perceived problems, opportunities for advancement and ideas related to the future of the
programme. Rather than sticking firmly to the interview protocol, the researcher conducted interviews
in a spontaneous conversational style, trying to explore the subject as widely and as deeply as possible.
Because the researcher discovered very early on that respondents from different sectors have very
different views with respect to the E+ programme, she proceeded with the interviews by trying to
present the observations of previous respondents, and to elicit comments – or even better, reflections.
This approach functioned perfectly, yielding more in-depth opinions, experiences and data on the
subject.
For the purpose of this report, gathered data and the thematic analysis (which was conducted with the
help of MaxQDA software) was framed around key questions and topics related to the effectiveness,
efficiency, relevance, internal and external coherence, and European added value of the E+
programme. Occasional topical differences emerged in the points of view not just of different
stakeholders but also with respect to Youth as well as to Education and Training; when appropriate,
these are described and explained.
69
Methodology
Research participants were selected with the help of an NA official, but their names are known only to
the researcher and to the NA official. Government representatives included respondents from two
Ministries that oversee implementation of the E+ programme in Croatia (the Ministry of Science and
Education and the Ministry for Demography, Youth, and Social Policy), and from the two
governmental agencies that are in charge of monitoring sectors (education, higher education) that are
covered by the E+ programme. An additional five respondents are from the civil non-profit sector –
NGOs that are active in the fields of youth, education and higher education in a way that gives them
expertise on programme implementation because they at once advocate and oversee policies related to
the Youth, Education and Higher Education sectors. In sum, 12 respondents – 6 women and 6 men –
participated, all of whom have had a wealth of experience related to both the LLP and YiA, and now
to E+.
In conducting the interviews, the highest ethical standards were maintained, with all of the participants
being contacted directly and informed about the research in advance via an informed consent form and
an interview protocol. Respondents acknowledged that, although their positions in the system would
make it almost impossible to guarantee their anonymity in the research results, their identities would
not be exposed in any way in the final report or the data, i.e., their experiences and opinions on the
programme and its implementation in Croatia would be part of a group report, immersed in group
experiences and opinions.
Interviews were conducted mostly outside of the workplace, in a casual atmosphere, and lasted on
average an hour and a half. Interviews were transcribed verbatim, and were analyzed with the help of
MaxQDA software. Coding was emergent, but thematic analysis was arranged based on predefined
evaluation topics related to efficiency, efficacy, relevance, internal-external coherence and
complementarity, and European added value and sustainability. Because of the intention, purpose and
structure of the document, the data analysis presented in this report is not substantiated with
respondents’ verbatim statements, as in standard qualitative research data presentations, but instead is
presented as a summary of findings related to the evaluation questions.
70
Evaluation results
I. Effectiveness
There is a shared opinion among the respondents, irrespective of the sector they are coming from, that
the E+ programme as it relates to youth has been very successful in its contribution to the realization
of the E+ specific and general objectives. On the individual level, our respondents recognized impacts
related to the development of skills and the acquisition of knowledge and experience that are helpful to
individuals and institutions alike. It is emphasized that the programme has helped young people in
their need for active involvement as well as with the acquisition of skills needed for the labor market.
Often mentioned are also activities that relate to European volunteer service and the education of
various personnel working with youth as having an important influence on both an institutional and
individual level. Institutional partnerships are seen as a valuable source of social and cognitive capital.
Respondents from non-governmental monitoring organizations frequently talk about the positive
influence of E+ on awareness of the importance of the E+ goals for personal, institutional and social
development in general. According to respondents, the impact of E+ has been the least evident at the
policy level, which can be improved in the future. Meanwhile, the budget that is available for E+
implementation in the Youth sector is perceived to be sufficient, and the absorption of funds is
perceived to be good.
Views related to E+ as an integrated successor programme are mixed; different opinions are
expressed. Although the programme is found to be more comprehensible and coherent in terms of
administration, there is a shared view that the Youth sector has lost visibility in the E+ programme.
With an emphasis on key activities rather than sectors, some of the content is lost (as perceived by
respondents) when compared to the Youth in Action programme. This observation usually
accompanies an opinion expressed by respondents from different sectors concerning youth in general,
starting with the perception that youth as a population is a non-coherent group, e.g., the needs and
goals of 16 year-olds differ from those of 27 year-olds, such that a programme framework requiring a
highly professional approach keeps youth initiatives and less experienced NGOs away from E+. It
follows that the reach as well as the impact of E+ is uneven across the Youth sector, effectively
excluding many potential users of different ages and experience levels. Respondents have suggested
that the next generation of the programme should consider designating funds for short-term and less
professional initiatives, and that projects should not be construed in a way such that one needs highly
professional formal support and institutional infrastructure in order to apply.
Related to the Education and Training sector, there is a shared opinion that E+ in Croatia is recognized
primarily as an instrument for internationalization; this could almost be its alternative name, as some
respondents suggest. It seems that this internationalizing aspect of E+ is widely recognized by
students, teachers, professors and all other citizens. The NA considers this aspect of E+ a ‘main brand’
in Croatia, especially so because it is the main such mobility programme in the country, particularly in
terms of financing. Respondents from the NAU recognize that the European and international
dimensions of the programme are the most important and influential in the Higher Education sector.
But all respondents, irrespective of the population they represent in the sample, are primarily focused
on the contribution of the programme to mobility (mostly individual mobility, on all educational
levels). Individual mobility has influenced users in terms of educational experience, foreign language
acquisition and exposure to different cultures and norms. All of this contributes to personal
development and the development of social skills. To respondents it is also obvious that personal
mobility – and all of the administration that is required to support mobility – has contributed to the
71
organizational development of universities, schools and kindergartens. Because of the fact that the
Croatian education curriculum is not very modern, and that teachers and professors in their everyday
work usually do not pay special attention to students’ social skills, respondents that are engaged in
some type of monitoring of primary and high school education recognize that E+, with its accent on
transversal skills, attitudes and knowledge, has brought important benefits into Croatian schools.
Through their work on E+ projects, our schools have become aware of the need for and advantages
conferred by such skills in order for students’ education to be complete. Additionally, in relation to the
European development plan, it is emphasized that schools are now rethinking their long term
development, goals and future strategies, and following up on the latter two. Moreover, they do it in
the context of the EU. Projects are also planned not only in terms of their specific goals, but also in
terms of wider institutional strategies and long term project sustainability. This presents a completely
new kind of institutional thinking in our educational sector, which respondents consider extremely
important. For kindergartens, the most noted impact is related to children with disabilities or special
needs, who are now recognized as a vulnerable group of children. Thanks to the implementation of E+
projects, kindergartens are now more focused on those children and their needs, and on ways in which
to respond to those needs and expand awareness among other children as to their more vulnerable
kindergarten friends.
Regarding E+ as a successor programme, respondents note that there are fewer rules and documents,
but also that there is greater coherence between the Youth and the Education and Training sectors. The
LLP was much more complicated, with over 50 activities, and was perceived as such – as not clear
enough for the average citizen. It seems that E+ has brought a fresh and clear outlook, especially to
new beneficiaries. It is also much easier for the implementing agencies to explain and advertise the
programme, as they now disseminate information about one rather than four different sectoral
programmes. It is much more user-friendly, the model of financing is much more flexible (flat rate
financing), and it makes financing available for flexible management – a change that has greatly
helped users in terms of project management and that respondents have stressed the benefits of.
Additionally, the lack of the need for co-financing is seen as an important and helpful change,
especially for beneficiaries from governmental or publicly financed organisations that do not have
funds allocated for international projects. Such funding was previously accounted for by presenting the
working hours of institutional staff as co-financing, which was frowned upon as a solution to the
problem for various reasons. Respondents – especially those working for the NAU – also note that
there are new possibilities related to so-called policy experiments, which they support and believe
could help a lot in the development of new policies. When the NAU is involved in such a project,
according to respondents, it can have a broad impact on national policies in the future, which is ideal
given that policies are developed in an international context, based on the work of different ministries
and higher education institutions.
72
II. Efficiency
When talking about efficiency, because of the nature of the questions, the Youth and the Education
and Training sectors will be mentioned individually only if there are some distinctive results that are
related to a particular sector; otherwise the discussion applies generally to both sectors.
In terms of cooperation and the division of tasks between different bodies, there has been agreement
that this division works very well and that all of the actors perform their duties regarding
implementation according to a plan. Ministries, agencies and users from the non-governmental sector
are very satisfied with the work of the NA; all of the comments were completely positive, concluding
that the NA is probably the best agency in terms of the quality of work performed in Croatia. These
comments were directed towards the advertising, implementation and monitoring of the programme.
Respondents who have been in contact with programme users or who were users themselves noted
only one shortcoming of the programme, on the evaluations side: that the evaluators of project
proposals under Erasmus+ sometimes created the impression that they were insufficiently competent.
This can be observed in evaluators’ comments on project proposals, in which a lack of expertise is
evident. Related to the OCD opinion, it should also be noted that some respondents from the non-
governmental sector believe that the NA is not independent enough in its decision making. They
recognize two main problems. One is related to difficulties in the NA’s internal functioning during
post-election transitions, while sometimes-considerable delays in obtaining necessary signatures from
the ministries make everyday agency functioning much more difficult. The other is related to the
opinion that the NA has much more insight into and understanding of the programme than any other
actor responsible for the implementation, and that therefore it should have much more political
influence when it comes to decision making vis-à-vis the E+. The ministries are very satisfied with the
agency, but hold the opinion that the NA is independent enough, and that it is impossible for such an
‘implementing body’ to be more independent, while the NA also holds that it should stay within the
same framework of independence vis-à-vis implementation, and as a partner and interlocutor in policy
decision making. One of the most significant and commonly emphasized problems is a general
shortage of staff in the NAUs and the NA, making programme implementation needlessly difficult and
heavily reliant on the enthusiasm of the existing staff. There is a shared opinion among respondents
from other relevant government agencies, ministries and the NA that there should be more
communication between actors – especially between government agencies in the field of education as
well as within the relevant ministries – in terms of the different divisions in charge of E+ and, in the
end, between ministries and agencies in the field of education. In order to truly achieve better synergy
in programme planning and implementation, there should be more horizontal and vertical connectivity
and strategy in terms of the division of tasks, as well as more communication and planning. This
would benefit all, as all respondents acknowledge. But more staff are needed as a precondition for all
of the above to even be possible.
Older programme generations are perceived to be much more complicated, with over 50 different
activities that were relatively obscure to citizens and relatively burdensome for implementing bodies
to implement and monitor. Now, with only three key activities, it has become much easier to explain,
implement and monitor E+ projects. It is also easier regarding financial monitoring, although when
talking to respondents from the NA, the most important insight gained is that changes have presented
the biggest challenge to all involved in the process of implementation. A common opinion is that this
change to E+ was significant and wide in terms of programme architecture and implementation.
Nonetheless, such changes entail difficulties of their own, and shouldn’t be made again unless
necessary. Because implementing bodies and users need time to understand and adjust to these
73
changes, there is a shared opinion that any future programme should not change a lot. This concerns
IT tools in particular, which caused a lot of tension and frustration for users and the NA alike at the
outset of E+ implementation.
Respondents also believe that the programme is now much less unwieldy for users. For instance,
whereas previously there were three different programme guides, now there is only one; meanwhile
there is greater synergy between youth and education, and citizens recognize unequivocally that E+ is
a programme for education, mobility and projects. However, the administrative burden is still
perceived as being quite heavy, especially for smaller schools and institutions in general, where
inexperience and a lack of capacity can make administration overwhelming.
Some respondents from the non-governmental sector hold the opinion that OCDs have ‘lost’ in the
sense that project applications and selections have become much more difficult in the context of the
new generation of the programme. Additionally they believe that competition is much fiercer with
respect to K2 and K3 activities – a reality that is especially difficult for smaller higher education
institutions and OCDs in the Youth and the Education and Training sectors alike. They perceive that
larger governing bodies, such as agencies and larger universities, are in a much better position to win
K2 and K3 projects; they see it as unfair competition.
One frequently mentioned problem within the Youth sector is that partnerships are not truly partnering
on projects, i.e., that they are partnerships in name only. Respondents believe that this is one area
which should be improved by additional support in terms of programme requirements.
Together with the view regarding the introduction of changes to the programme, which was previously
mentioned and attributed to respondents from the Youth and the Education and Training sectors alike,
and related to the need to fund fewer professional initiatives while initiating expert groups to help
make long-term decisions about the goals of funded international projects, there is an often-expressed
view that the NA should have flexibility in funding related to other, non-mobility projects. In further
talks with other respondents, this view is mainly supported and even greeted with approval by the
NA’s financial division, which has expressed the view that such would be manageable as long as
funding allocation procedures become simpler.
Problems with capacities in terms of human and financial resources are often perceived in the sector of
Education and Training, possibly comprising the most important problems related to E+ programme
implementation. This holds as much among respondents working at the NA as it does for those from
the NAU and NGOs. While we have experts that can successfully design high quality projects, the
administrative responsibilities for these must be delegated to staff who often lack not only
international project-related knowledge and skills, but also a headcount sufficient to reasonably
shoulder the additional workload. There is a common opinion that, for the implementation of any E+
project, there has to be an institutional mechanism that can support those who are willing to carry out
the project; such an institutional mechanism is mostly absent, so every single task, including the
administrative workload, becomes the responsibility of the same individuals that have applied for the
project. Furthermore, those individuals receive no recognition or recompense for their international
project-related efforts beyond what they already receive for their usual duties (i.e., teaching and, in
HEs, research), so they become seen as eccentrics, or as enthusiasts with questionable motives. There
are a lot of teachers, professors and researchers who are enthusiastic enough to apply for E+ projects
despite the lack of support, but enthusiasm is not an inexhaustible source. Blame for this goes to a lack
of policies that would make it easier for individuals to apply for projects, and to a lack of institutional
strategies related to internationalisation – which includes a lack of administrative support for
74
international projects. Respondents note international experiences, such as one from the University of
Dresden, which has developed a special administrative unit dealing exclusively with international
projects, which itself has very quickly expanded and even become financially self-sustainable. It is
suggested that Croatian HE institutions should learn more from international experiences, and should
adopt and implement ideas that can aid the future absorption of Erasmus programme funds. Yet
another opinion from the respondents was that Croatian institutions face a challenge in the form of the
very novelty of thinking in terms of projects, which is not taught in our educational system on any
level and presents difficulties for our citizens.
Another observed problem within the HE sector is related to centralized activities from which project
proposals are quite rare; such projects could, by virtue of the reach of their goals and purposes, be of
great benefit to HE institutions. Respondents believe that the lack of such projects could be related to
the lack of international connections and partnerships that would facilitate such project ideas and
proposals, and it is noted that these are the issues with which HE institutions should be most
concerned.
Human and financial resources are also seen as a problem in the Youth sector, in terms of the
aforementioned problems related to the professional potential of youth NGOs on one hand, and of the
administrative needs of the projects on the other.
III. Relevance
With respect to the Youth sector, respondents from the NAU express an opinion that “youth” as such
is not a coherent group, since youth at age 16 and at age 27 have different needs and goals in life, and
therefore a frame that is too professional keeps initiatives and less experienced NGOs away from
Erasmus+. It follows that there is a potential lag in the relevance of goals, because this aspect of Youth
is not thought through. Meanwhile, all other respondents express more concern about local issues that
relate to a lack of initiative on the part of local governments to recognize youth as an important
population, which could further help in the definition of goals in terms of E+ programme policies and
project implementation. This opinion is somewhat common among respondents from the Education
and Training sector. Except for one goal that aims to lower the dropout rate, which they do not
consider nationally relevant, these respondents view goals as relevant for Croatia, although they
recognize a lot of problems and obstacles to achieving them. There is a view that Croatia, as a small
and not particularly wealthy country, does not have the supporting infrastructure or political will – or
is not developed enough politically – to achieve a true synergy of projects that can result in the
achievement of planned goals. However, respondents have noted that institutions are taking positive
steps, displaying much more responsiveness, and taking more strategic and goal-oriented approaches,
which could indicate a good start towards a more positive future in terms of goals.
More specifically, regarding the impacts of the E+ programme – and especially its contribution to the
development of youth policies – respondents believe that, although there are visible results and the
programme is effective in terms of achieving its goals, especially where K1 and K2 activities are
concerned, the results are too often limited to the micro - local level, and there is no broader,
integrated impact, especially in the domain of policies. It has been recognized that in the LLP there
was the possibility of setting national goals, while in E+ this has changed in favor of common EU
goals; this change has ushered in a wider set of goals and possibilities, but has also reduced the
possibility of focusing on national priorities. This has brought dispersion and a lack of joint efforts. It
is suggested, especially by the respondents coming from NGOs and governmental agencies, that the
75
question of the wider impacts of the E+ programme is a topic that should be thought through
nationally. This could be done through a working group, consisting primarily of experts related to
youth research and youth policies, that would be in charge of reflecting on the goals of the E+
programme in relation to other programmes and initiatives, in terms of the wider social and political
context as well as the possibility to synthesize different goals in order to form stronger and wider
impacts within both the sector and society as whole. This group of experts could navigate E+ in terms
of project mutuality, with the goal of joint- as well as long-term broader impacts. Respondents from
the NA have found this idea interesting and thought that it would be helpful to have such an advisory
committee to reflect on projects in a much broader context. Another common opinion on how to make
implementation more effective concerns broader advocacy related to the E+ programme and its
impacts. Some respondents, mainly from OCDs, hold the opinion that those who share responsibility
for programme implementation nationally should increasingly advocate for the programme outside of
the primary group of users, to would-be beneficiaries such as local and national governing bodies that
mostly obstruct rather than support the wider impacts of the programme. That is, these respondents’
opinion is that broader advocacy of the programme could influence necessary changes in attitudes and
support as well as policy adjustments by local and national governing bodies toward E+ projects,
which would facilitate the implementation of E+ projects and strengthen the impact of their results.
Related to the impacts of the programme in terms of its goals in the Education and Training sector,
there has been a lot of discussion with participants regarding the wider implementation of conclusions
or recommendations resulting from any given project. Problems related to local and national politics
that lack continuity in strategy, goals, purpose, and decision-making seem to be the greatest obstacle to
E+ projects reaching the policy level in terms of influence. Similarly as in the Youth sector, in the HE
and EDU sectors what is frequently noted is the need to consider goals as policy-cycle, long-term
efforts instead of short-term objectives. To all respondents who were engaged in the discussion about
the HE and EDU sectors, it was obvious that the impacts fall mostly on the micro-level, and that there
is a lack of synthesis and joint effort in terms of reaching national goals, a lack of continuity in terms
of topics, and a lack of wider strategizing. Some respondents propose longer projects – thematic
programmes, for instance, that would consist of different projects but be applied by a single user – as a
solution to these types of problems.
IV. Internal and external coherence and complementarity
Respondents from the Youth sector have frequently mentioned that it seems to them that Youth
visibility has suffered under E+, and that although a kind of synergy has been achieved, this could
benefit from further consideration under the new programme. Respondents are mostly satisfied, noting
no problems or obstacles related to programme coherence. What they do frequently mention is the
problem – or rather the challenge – of adjusting E+ and ESF goals. Respondents from agencies,
ministries and OCDs all see a potential competitive problem between these two funds for the same
users in the future, which should be avoided by a more careful consideration of the goals of both E+
and the ESF. There should be mutuality rather than competition.
V. European added value and sustainability
There is a difference in opinion on how much of an impact E+ has had in terms of added value in the
Youth and the Education and Training sectors. While Youth-sector respondents believe that progress
has been very slow, impacts are to be found mostly on the individual- and micro-levels; in the
Education and Training sector as well as the HE sector in particular, the notion is that significant
76
internationalisation and Europeanisation has obtained, both on the individual and institutional level.
Nonetheless, these impacts are perceived as insufficiently dispersed.
Some respondents are quite sure that the increase in the E+ budget in the coming years will be
followed by increased absorption, at least in terms of mobility projects, because those projects still
have a very high rejection rate (i.e., around 70%). Additionally, unlike in other EU member countries,
in Croatia E+ is the only programme that finances mobility in such a way, so there is no competition
for E+ in terms of other mobility programmes. However, some respondents are quite sceptical
regarding future absorption, especially when it comes to primary schools and even to high schools and
their projects. Here again we note an opinion regarding obstacles to schools’ inclusion in more
projects as well as the lack of adequate rewards for staff who are engaged in E+ projects. Often it is
remarked that participants are held responsible for large sums of money, yet at the same time they are
not rewarded for their effort in any way. This is considered very challenging to further absorption of
additional funding because it is hard to believe that those potential beneficiaries, i.e., those schools
that are not participating and that may be apprehensive about the prospect of unrewarded work, will
form different opinions in the future, all else held constant. This is mentioned as input rather than as
scepticism, in order to resolve these questions nationally and legally, as soon as possible, in order to
have a different starting point when the next generation of the Erasmus programme unfolds.
Respondents from government agencies have also frequently mentioned the problem constituted by a
lack of information on schools that do not participate in the programme, and on their reasons for
opting out. If we do not know what makes a school opt out of the programme, it is difficult to
understand what can be done to change that. Regarding higher education, frequently noted is a lack of
capacities and knowledge related to international projects, especially concerning smaller higher
education institutions.
Generally, regarding projects, the NA holds that there will be a challenge in the future: when greater
funds and their absorption are at issue, respondents believe that any preference for high-quality
projects exclusively will have to be scaled back. This is simply because an increase in funded projects
will necessarily entail an increase in the number of projects of different – and often imperfect –
quality. It is meanwhile interesting to note that, according to respondents from the NA, lower-quality
projects are more closely monitored and more attentively supported by the Agency, which may result
in such projects becoming very successful. However, the amount of time and energy that the Agency
must invest in such projects is considerable, and it would be nearly impossible for the NA to provide
such support to a growing number of low-quality projects. The NA considers the perceived need to
sustain the level of quality projects amidst increases in expected funding – and the need to absorb said
funding – a most challenging aspect of the future.
77
Conclusions and suggestions for improvements to E+ and for a future
programme
As can be concluded from the present analysis, respondents that have a broad outlook on the
programme talked mostly about and were mostly satisfied with its implementation in Croatia, and with
its efficiency, efficacy and relevance. They were somewhat less concerned with its internal coherence
and European added value. However, there are particular points of improvement that were emphasized
and proposed for the next generation of the programme, and those are:
the implementation of funds related to short-term and less professional initiatives, i.e., projects
that are not construed in a way such that one needs highly professional formal support and
institutional infrastructure in order to apply
a more flexible budget may be more responsive to ever-changing project demand; allowing
transfers between key actions and/or between sectors may foster both efficiency and
effectiveness in smaller countries with a limited number of experienced applicants, and would
be manageable given a simplification of funds allocation procedures
the provision of additional support in terms of requirements within the programme, as Youth
sector partnerships are partnerships in name only, taking the form of such without displaying
any real substance
Important are also proposed improvements related to implementation that should be considered on the
national policy level:
there should be more communication between all actors, and especially between agencies
themselves as well as within the ministries in terms of the different divisions that manage E+,
in order to truly achieve better synergy in the planning and implementation of the programme;
there should also be more horizontal and vertical connectivity and strategy in terms of the
division of tasks, as well as more communication and planning
a general shortage of staff within NAUs, the NA and NGOs makes implementation of the
programme much more difficult than it should be
a lack of adequate rewards for staff that are engaged in E+ projects challenges the further
absorption of additional funding because it is hard to believe that those potential beneficiaries,
i.e., those schools that are not participating and that may be apprehensive about the prospect of
unrewarded work, will form different opinions in the future, all else held constant
the question of the wider impacts of the E+ programme should be addressed nationally
through a working group, consisting primarily of experts related to youth research, youth
policies and educational as well as higher educational policies, that would be tasked with
reflecting on the goals of the E+ programme in relation to other programmes and initiatives,
both on their own and in terms of the wider social and political context
those who share responsibility for programme implementation nationally should increasingly
advocate for the programme outside of the primary group of users, to would-be beneficiaries
such as local and national governing bodies that mostly obstruct rather than support the wider
impacts of the programme; this could influence necessary changes in attitudes and support as
78
well as policy adjustments by local and national governing bodies toward E+ projects, which
would facilitate the implementation of E+ projects and strengthen the impact of their results
centralized activities from which project proposals originate are quite rare, which is related to
a lack of international connections and partnerships that would facilitate such project ideas and
proposals; HE institutions should be very concerned about these matters, as nationally there is
much room for improvement
79
Appendix V: Combined Presentation of Empirical
Findings: Lifelong Learning Programme, Youth in Action
and Erasmus+
Prepared by: Natalija Lukić Buković, Senior Expert Advisor in the Department for
Coordination and Quality Assurance of the Erasmus+ programme at the Agency for Mobility
and EU Programmes ([email protected])
Contents
1. Executive summary ........................................................................................................................... 81
2. Methodology ..................................................................................................................................... 82
3. Overview of programme monitoring data ......................................................................................... 83
4. Effectiveness ..................................................................................................................................... 99
5. Efficiency ........................................................................................................................................ 109
6. Relevance ........................................................................................................................................ 110
7. European added value and sustainability ........................................................................................ 112
8. Conclusions ..................................................................................................................................... 114
9. Literature ......................................................................................................................................... 116
80
Acronyms
LLP Lifelong Learning Programme
E+ Erasmus+ programme
LDV Leonardo da Vinci (sectoral programme within LLP)
ERA Erasmus (sectoral programme within LLP)
GRU Grundtvig (sectoral programme within LLP)
COM Comenius (sectoral programme within LLP)
KA1 Key Action 1: Learning Mobility of Individuals
KA2 Key Action 2: Cooperation for innovation and the exchange of good practices
KA3 Key Action 3: Support for policy reform
YOU Youth field in Erasmus+
SCH School education field in Erasmus+
VET Vocational education and training field in Erasmus+
HE Higher education field in Erasmus+
ADU Adult education field in Erasmus+
NGO Nongovermental organisation
Y-(TRANS)NAT Transnational activities in Youth field
Y- EXCH Youth Exchanges
81
1. Executive summary
This report includes a review as well as a combined presentation of empirical findings with regard to
the Lifelong Learning Programme and the Youth in Action programme, as well as the initial years of
Erasmus+, in order to inform the mid-term review process on the national level. The evidence
presented in this report, as well as in those of external experts, should guide the conclusions in the
National Report about how the situation has evolved since the programme began, how the programme
was implemented, and what has become of various stakeholders. It should also provide answers about
the progress made towards achieving programme objectives and the extent to which these address
(current) needs and problems.
The general and specific objectives of Erasmus+, as defined in the Regulation on establishing
Erasmus+ (European Commission/European Council, 2013), focus on the common EU objectives of
improving the learning mobility of individuals in the EU, developing joint master’s degree programs
across institutions, organising large-scale European voluntary service events, cooperating with a view
towards innovation, and exchanging good practices. In addition, the programme aims to develop
strategic partnerships in the fields of Education and Training as well as Youth, and to develop
knowledge alliances and fund structured dialogues between decision makers in the Youth field.
A review of empirical findings shows that there are several highly-pertinent patterns/trends in the
programme’s implementation in Croatia. Firstly, the integrated framework brought together previous
users from the LLP and the YiA programme, resulting in stronger competition for limited funding and
a change in the structure of the participating organisations (i.e., an increase in participation on the part
of NGO’s and a decrease in participation on the part of educational institutions – particularly among
pre-primary and primary schools as well as youth organisations13
). Secondly, the findings revealed a
discrepancy between high demand and low success rates, especially within the YOU and across all
sectoral fields in KA2. Thirdly, project managers in educational organisations (i.e., from pre-primary
up through higher education) as well as project leaders and end-users in the YOU see mainly positive
impacts from programme participation on individuals’ skills, abilities and knowledge. However, the
most relevant impact of mobility on Education and Training as well as Youth participants is the
enhancement of cultural knowledge14
and communication in foreign languages. Further expected
impacts, such as stronger involvement on the part of participants in social and political life, are less
recognized among participants. Programme objectives such as the “improvement of key competencies
and skills among youth” seem too broad and generic; it is more likely that participants, based on the
duration and content of their mobility periods, will to some degree gain new information, contacts and
perspective on other cultures than they will enhance the whole range of “key competencies.” Fourthly,
the results of the LLP Evaluation Study conducted in Croatia from 2014 till 2015 indicate that, at the
institutional level, there is a set of obstacles that can limit the impact of project participation or even
de-motivate potential participants. Some of the most pertinent findings suggest that, for instance, a
lack of institutional valorisation and recognition of staff mobility periods, a lack of support from
institutional decision makers, a widespread perception that projects demand a high degree of
administrative, human and financial capacity, or a lack of foreign language knowledge, can deter
individuals and organisations from applying for grants (Milanović-Litre, Puljiz and Gašparović, 2016).
Nonetheless, project managers across LLP sectoral programmes (i.e., LDV, COM and GRU)
recognized the positive impact of programme participation on the following areas: willingness to
13
Youth organisations are generally understood to be youth-led, non-profit, voluntary, and participatory non-governmental
associations.
82
participate in professional development activities, motivation among school staff to introduce
innovations in teaching, project management skills, and willingness of school principals to support
project participation and promote the European dimension in education. In the higher education field,
the implementation of ERA contributed to an increase in the number of outbound and inbound
mobilities, triggered a development of institutional and administrative capacities (especially the
development of formal and informal procedures at institutions, as well as networking) in international
cooperation.
2. Methodology
Several methods were utilized in order to prepare a review of existing empirical findings that were
relevant in the context of mid-term evaluation questions:
1) A secondary analysis of EU Surveys (Participants' Reports – Croatian participants)
A secondary analysis by definition entails the utilisation of existing data, collected for the purposes of
a prior study, in order to pursue a research interest which is distinct from that of the original work. In
order to collect evidence that was relevant for answering evaluation questions, an additional sub-set
analysis and an additional in-depth analysis of the EU Survey – Participants' Reports were conducted.
The pre-existing datasets were extracted from Mobility Tool+ with a special focus on participants'
perspectives on the development of key competencies, social participation, and perceived project
relevance for future employability. All participants in KA1, upon finishing their mobility periods
abroad, are required to submit the EU Survey – MT+ Participant Report. The EU Survey results
collected from Croatian participants (i.e., learners and staff) who submitted reports by January 9, 2017
have been analysed in order to obtain insight into end-users' (i.e., participants') perspectives on any
enhancement of skills, language skills, future prospects for employability and general satisfaction with
project participation. In the answer descriptions, standard measures of descriptive statistics have been
used for quantitative variables: frequency, percentage, mean, and median. Due to unequal sample sizes
and an abnormal distribution of quantitative variables during statistical testing, non-parametric tests
were used (i.e., Kruskall-Wallis, Chi-square test, Mann-Whitney U).
2) A review of existing research studies
A review in this context is a summary of information that is already available, both published and
unpublished, which can provide additional input that is relevant to evaluation questions. This review
provides descriptive summaries of the included empirical studies. In Croatia, since 2009, four studies
have been conducted which focus on the impact that participation in the LLP and Erasmus+ – YOU
programmes had on the beneficiary organisations and participants (i.e., end-users). The objective of
this review is to summarize the current results of the Erasmus+ programme and its predecessor, the
LLP, in Croatia, and to provide a critical review (i.e., an assessment) of previously published
information. The review process has been organised into four stages: defining relevant evaluation
topics/questions; evaluating data (i.e., screening available studies and selecting those chapters that
make a significant contribution to an understanding of mid-term evaluation questions); summarising
findings; and conclusions and thematic analysis (based on the evaluation questions).
83
3) An analysis of monitoring data
In order to provide an overview of national programme implementation, monitoring data (i.e.,
administrative data on programme implementation) and national statistics were reviewed and
exploited so as to describe the socio-demographic and organisational structures of programme
beneficiaries, budgets and project demands.
3. Overview of programme monitoring data
Croatia started its implementation of the LLP and the YiA programme in 2009. A limited scope of
activities were implemented prior to signing a Memorandum of understanding that enabled Croatia to
participate fully in the LLP and the YiA programme as of January 2011. Considering the limited
funding that was available for international cooperation programmes at the national level prior to
2009, membership in EU programmes in the Education and Training as well as Youth fields opened a
new chapter in the internationalisation of Croatian educational institutions and new possibilities for
citizens to engage in international projects. In this section, programme implementation in the national
context is presented, which is based on available monitoring data that indicate the adequacy of
available funding and the scope of participation among target groups as well as specific disadvantaged
groups.
3.1. Adequacy of funding
Figure 1: Available EU funding for Croatia: year-on-year growth rate
LLP YiA E&T YOU
2009/2010 2.85 4.64
2010/2011 1.64 1.89
2011/2012 0.48 0.24
2012/2013 0.18 0.57
2013/2014
2014/2015 0.21 -0.02
2015/2016 0.03 0
In this section, the adequacy of available funding is examined by a) comparing the total funding
demand represented within high-quality applications against the available budget, and b) looking at the
cost of programme actions. As shown in Figure 2, the total funding available to Croatia in 2009, which
was the first year of YiA and LLP implementation, amounted to € 572.418,00. Over subsequent years
the amount of available EU funding increased steadily. As presented in Figure 1, the initial year of
programme implementation saw sharp funding increases of 2.85-fold for the LLP and 4.64-fold for the
YiA. However, year-on-year increases in available funding were reduced to 0.18-fold for the LLP and
0.57-fold for the YiA at the end of the LLP. In 2014, the first year of Erasmus+, allocations to Croatia
increased by 20% for the E&T and by 17% for the YOU as compared to the final year of predecessor
programmes, although year-on-year growth was lower for Erasmus+ funding than it was during the
LLP/YiA period. After three years of Erasmus+, the total available funding allocated to Croatia
amounts to € 44.510.635,98, which is 52% higher than the total budget for both YiA and LLP between
2009-2013. A higher growth rate of the available budget is expected in the 2017-2020 period.
According to the Erasmus+ Programme Annual Report for 2014 (European Commission, 2015)
84
Erasmus+ will allocate a cumulative 40% budget increase for the 2014-2020 financial period. An
increase of the programme budget from 2017 till 2020 will be reflected in the country allocations, with
sharper increases in available funds starting in 2017.
85
Figure 2: Available EU funding (LLP, YiA and Erasmus+)
Source: Yearly Reports and Business Objects – EP012
2009 2010 2011 2012 2013 2014 2015 2016
LLP/E+ - Education and Training- available budget 494.958,00 € 1.904.605,00 5.028.000,00 7.421.000,00 8.730.751,00 10.455.437,00 12.623.217,24 12.953.425,00
YIA /E+ - Youth - available budget 77.460,00 € 436.849,00 € 1.261.155,00 1.561.593,00 2.445.082,00 2.863.461,00 2.812.525,00 2.802.570,74
494.958,00 €
1.904.605,00 €
5.028.000,00 €
7.421.000,00 €
8.730.751,00 €
10.455.437,00 €
12.623.217,24 € 12.953.425,00 €
77.460,00 € 436.849,00 €
1.261.155,00 € 1.561.593,00 €
2.445.082,00 € 2.863.461,00 € 2.812.525,00 €
2.802.570,74 €
86
Figure 3: Contracted amount (LLP, YiA and Erasmus+)
Source: Business Objects - EP012 and Yearly Reports for LLP and YiA
2009 2010 2011 2012 2013 2014 2015 2016
LLP/E+ - Education and Training - contracted amount 480.934,82 1.900.523,85 4.903.283,54 7.304.908,34 8.735.468,52 10.443.972,48 12.496.556,13 12.740.032,42
YIA/E+ Youth - contracted amount 77.460,00 437.716,85 1.182.651,92 1.525.033,76 2.163.921,00 2.702.091,42 2.782.723,65 2.795.495,97
480.934,82
1.900.523,85
4.903.283,54
7.304.908,34
8.735.468,52
10.443.972,48
12.496.556,13
12.740.032,42
77.460,00 437.716,85
1.182.651,92 1.525.033,76
2.163.921,00 2.702.091,42 2.782.723,65
2.795.495,97
87
3.2. Success rate of high-quality applications
With Erasmus+ came expectations of simplified financial management. Aside from the harmonisation
of rules and regulations across sectors, one key aspect of this simplification has been the introduction
of the unit cost system, including flat rates. It was expected that the unit cost system would make
financial management easier for the NAs and the beneficiaries. Further harmonized rules across
sectors and actions were adopted in Erasmus+, yet the budget distribution continues to reflect greater
sectoral divisions, such as those that existed during the predecessor programme period. The
distribution of funds between the E&T and the YOU is almost the same as was that between the LLP
and the YiA, with 80% of the funds being allocated to the E&T and 20% of the overall national
programme budget being allocated to the YOU. However, during the initial years of Erasmus+, the
demand for funding in Croatia increased, particularly within the YOU. As shown in Figure 4, the
demand for funding as indicated by the number of projects submitted is significantly higher within the
YOU than within other sectoral fields, while the number of applications achieving the minimum
quality threshold is almost two times higher than the number of grants awarded across three Key
Actions and all fields (Figure 4). As presented in Figure 4, a closer review of high-quality
applications15
shows that, across all actions, the number of high-quality applications exceeds the
number of projects awarded. This is particularly the case within the YOU (KA1), where 60% of
applications submitted achieved the quality threshold but only 26% secured grant funding, indicating
that insufficient funds rather than poor application quality tends to explain application rejections. A
discrepancy between available and requested funds is also particularly evident within the YOU, which
displays lower success rates than do other sectoral fields (8,52% for KA105, 7,69% for KA205). A
combination of strong demand and insufficient funding can hinder participation and de-motivate
potential applicants (Ferencz, Mitić, Wächter, 2016).
15 Applications that achieved the minimum quality threshold.
88
Figure 4: Minimum application quality threshold achievement and funding in Erasmus+ (2014
– 2016)
Action
APPLICATIONS GRANT AMOUNT in APPLICATIONS (in EUR)
Received Awarded
Achieved
minimum
quality
threshold
Received Awarded Achieved minimum
quality threshold
KA1
KA101 353 71 20,11% 182 51,56% 6.213.080,00 1.244.065,00 20,02% 3.565.400,00 57,39%
KA102 325 103 31,69% 185 56,92% 22.451.714,00 7.040.314,00 31,36% 13.777.088,00 61,36%
KA104 60 27 45,00% 49 81,67% 1.011.335,00 291.491,00 28,82% 825.261,00 81,60%
KA105 1,185 338 28,52% 721 60,84% 23.639.026,90 6.197.427,00 26,22% 14.116.871,56 59,72%
KA116 6 2 33,33%
673.728,00 179.811,00 26,69%
KA1
(Excluding
HE)
1,929 541 28,05% 1,137 58,94% 53.988.883,90 14.953.108,00 27,70% 32.284.620,56 59,80%
KA2
KA200 13 3 23,08% 5 38,46% 4.050.041,00 708.051,00 17,48% 1.577.465,00 38,95%
KA201 72 11 15,28% 42 58,33% 15.183.176,75 1.618.875,00 10,66% 9.287.451,75 61,17%
KA202 50 10 20,00% 21 42,00% 11.831.046,00 2.159.416,00 18,25% 5.881.382,00 49,71%
KA203 24 5 20,83% 15 62,50% 6.524.089,00 1.076.661,00 16,50% 3.926.739,00 60,19%
KA204 42 7 16,67% 30 71,43% 8.459.041,00 1.123.884,00 13,29% 6.157.102,00 72,79%
KA205 130 10 7,69% 50 38,46% 19.135.349,09 1.475.201,00 7,71% 7.025.690,40 36,72%
KA219 52 13 25,00% 23 44,23% 7.038.813,25 1.656.057,00 23,53% 2.946.062,00 41,85%
KA2 383 59 15,40% 186 48,56% 72.221.556,09 9.818.145,00 13,59% 36.801.892,15 50,96%
KA3 KA347 67 22 32,84% 38 56,72% 1.199.190,08 361.136,00 30,11% 683.011,00 56,96%
KA3 67 22 32,84% 38 56,72% 1.199.190,08 361.136,00 30,11% 683.011,00 56,96%
Source: Business Objects - General Statistics
89
Figure 5: Comparison of requested amounts in high-quality applications with awarded grants per programme activity in KA1
Source: Business Objects – General Statistics
KA101 KA102 KA104 KA105 KA116
Requested in Application 6.213.080,00 22.451.714,00 1.011.335,00 23.639.026,90 673.728,00
Awarded Amount 1.244.065,00 7.040.314,00 291.491,00 6.197.427,00 179.811,00
Requested amount in good quality applications 3.565.400,00 13.777.088,00 825.261,00 14.116.871,56
3.565.400,00
13.777.088,00
825.261,00
14.116.871,56
0,00
2.000.000,00
4.000.000,00
6.000.000,00
8.000.000,00
10.000.000,00
12.000.000,00
14.000.000,00
16.000.000,00
0,00
5.000.000,00
10.000.000,00
15.000.000,00
20.000.000,00
25.000.000,00
90
3.3. Average funding and unit costs
Despite the fact that an in-depth analysis of cost effectiveness is needed in order to determine the
adequacy of funding within each sectoral field or action, average funding and unit costs will be used in
the following section to describe “most expensive” and “least expensive” actions in the Croatian
context.
Figure 6: Number of participants and average funding per participant - Call 2014 - 2016
Action
Field
Action
Type
Activities Participants in Contracted Projects
Grant Amount per
Activity Type
(EUR)
Total #
With
Special
Needs
With Fewer
Opportunities
Accompanying
persons
Average
funding per
Participant
(EUR)
SCH Subtotal -
KA1 SCH 1.020.315,00 598 1 . . 1.706,21
VET Subtotal -
KA1 VET 6.027.885,00 3.798 24 66 388 1.587,12
HE Subtotal -
KA1 HE 15.206.888,80 7.292 8 . . 2.085,42
ADU Subtotal -
KA1 ADU 237.471,00 160
. . 1.484,19
YOU Subtotal -
KA1 YOU 6.094.710,00 8.449 62 3.540 . 721,35
GRAND TOTAL 28.587.269,80 20.297 95 3.606 388 1.408,45
Source: KA1 Overview table – Business Objects
The greatest share (53%) of the total contracted amount was allocated to the HE, which is reflected in
the high number of participants (7.292) as well as in the highest average funding per participant (in
EUR). The HE is followed by the YOU, which accounted for 21% of the contracted amount in KA1,
but which provided much lower funding per participant (721 EUR) given a higher number of
mobilities (8.449). The VET accounted for 21% of the total KA1 budget and included 3.798
participants, each of whom received an average funding amount of 1.587,12 EUR.
In the HE, a majority of participants (i.e., learners and staff alike) undertake mobilities in countries
with high or medium-high living costs. Meanwhile, as of 2014, around 15% of outbound students have
received top-up grants based on their relatively low socioeconomic status. These factors, together with
unit costs, contribute to higher average costs per participant.
Two patterns appear to be present when it comes to Erasmus+ funding and financial management:
firstly, a mismatch between supply and demand, and secondly, differences between sectoral fields
when it comes to the average cost per participant and the average project grant amount. The
discrepancy between the supply of and the demand for available funding is reflected in low success
rates across all fields (and especially in KA2), as well as in the large share of high-quality applications
that are not funded due to a lack of resources. Differences in the average cost per action vary
significantly. For example, the mobility of students and staff in the HE is a few times higher as
compared to the cost per participant within the YOU.
91
3.4. Participation of target groups in the Erasmus+ programme
The Erasmus+ Programme Guide recognizes individuals – students, trainees, apprentices, pupils, adult
learners, young people, volunteers, professors, teachers, trainers, youth workers, professionals from
organisations active in the fields of Education and Training as well as Youth – as target groups of the
Programme. Apart from individuals, participating organisations (i.e., mostly organisations from the
field of education) are recognized as relevant Programme actors, including groups of at least four
young people who are active in youth work, although not necessarily in the context of youth
organisations (also referred to as informal groups of young people). Within the general target
populations, in order to promote equity and inclusion, the Programme also targets specific groups that
face challenges to Programme participation. These three groups consist of participants with: a)
disadvantaged backgrounds16
, b) fewer opportunities17
and c) special needs.18
Within the YOU, an
Inclusion and Diversity Strategy has been designed as a common framework to support the
participation and inclusion of young people with fewer opportunities.
Figure 7: Staff and learners mobility 2009-2013/14
Programs:
LLP and
YiA
Year
Higher Education VET School
Education
Staff
Adult
Education
Staff
Youth
TOTAL Students Staff Learners Staff
young
people
youth
workers EVS
2009 235 49 14 13
79 22 4
2010 545 126 87 45 92 56 473 174 12
2011 882 194 460 278 972 255 1.202 231 59
2012 1.124 291 763 459 1.441 261 1.699 315 62
2013 1.403 396 1.000 263 960 75 2.859 625 119
sub-total 4.189 1.056 2.324 1.058 3.465 647 6.312 1.367 256 20.418
Erasmus+ -
Key Action 1 Year
Higher Education VET School
Education
Staff
Adult
Education
Staff
Youth
TOTAL
Students Staff Learners Staff young
people
youth
workers EVS
2014 1669 556 953 226 209 85 870 197 669
2015 1702 600 2982 371 170 39 432 84 417
sub-total 3371 1156 3935 597 379 124 1302 281 1086 12.231
Source: Yearly Reports for LLP/YiA, Mobility Dashboard for Erasmus+
16 The category of students from disadvantaged backgrounds (i.e., other than those with special needs) is defined by national
authorities together with the national agencies responsible for the implementation of Erasmus+. In Croatia, students of low
socioeconomic status are eligible for an additional top-up if they apply for mobility under KA1 within the HE. 17 Persons facing obstacles that prevent them from effectively accessing Education and Training as well as Youth work
opportunities due to disabilities, educational difficulties, economic obstacles, cultural or geographic obstacles, social
obstacles, or health issues. 18 Participants with special needs are considered to be people with mental (i.e., intellectual, cognitive, learning), physical,
sensory or other disabilities.
92
Figure 8. Mobility of student, learners, young people in the LLP, YiA and Erasmus+
Source: Yearly Reports for LLP/YiA, Mobility Dashboard for Erasmus+
2009 2010 2011 2012 2013 2014 2015
HE students 235 545 882 1.124 1.403 1669 1702
VET learners 14 87 460 763 1.000 953 2982
YOU - young people 79 473 1.202 1.699 2.859 2465 1860
0
500
1000
1500
2000
2500
3000
3500
93
Figure 9. Mobility of staff in the LLP, YiA and Erasmus+
Source: Yearly Reports for LLP/YiA, Mobility Dashboard for Erasmus
2009 2010 2011 2012 2013 2014 2015
HE - staff 49 126 194 291 396 556 600
VET staff 13 22 157 266 263 226 371
School Education Staff 92 112 127 132 209 170
Adult Education Staff 44 29 37 42 85 39
YOU - youth workers 22 174 231 315 625 735 465
0
100
200
300
400
500
600
700
800
94
As shown in Figure 8 and Figure 9, the overall numbers of mobile individuals increased steadily
during the years of the LLP and YiA. Even though the number of contracted projects dropped under
Erasmus+, more individual participants took part in projects. In the first two years of Erasmus+ (i.e.,
2014 and 2015), the scope of participants was broader than that in the LLP/YiA. The overall number
of mobile participants in Erasmus+ (excluding those participating in the partnerships) reached 10.148,
while during the years of the LLP and YiA this number was 8.192. An increase in the budget and
changes in the structure of programme activities contributed to this widening participation. However,
there was a difference in the scope of participants across sectoral fields. In the HE and especially the
VET, the scope of participants increased under Erasmus+, and this was most evident in the increase of
participating staff and learners/students in the initial year of Erasmus+ as compared to the final year
of the LLP. The introduction of an international dimension (KA107) in Higher Education contributed
to this growth. Meanwhile, a downward trend was observed in the ADU and the YOU, where the
number of participants was much lower than in the GRU, and the number of youth workers and young
people did not continue to increase in 2015 at the same pace as it had in previous years (see Figures 8
and 9).
Even though the number of participants increased, in the context of the wider picture it should be
noted that outreach on the national level is still relatively weak: participants consist of just a small
share of the targeted populations. In the HE, the number of outbound mobile students accounted for
0,97% of participants, while the number of inbound students accounted for 0,90%19
of students
enrolled in Croatia during the 2014-15 academic year. In that same academic year, the number of
outbound teaching staff from HEIs accounted for 4%20
of the teaching staff (full-time equivalent)
employed at Croatian HEIs. In the VET, around 2%21
of secondary education staff were involved in
mobility activities, while 1,9% of learners enrolled in secondary schools in Croatia took part in
mobility in 2015. The RAY Standard Survey (national report) (Gregurović, 2017) shows that among
Erasmus+ YOU project leaders, 10,9% are unemployed, and among the Youth participants, 11,7% are
unemployed. The scope of the programme in terms of reaching the target groups is still limited by the
budget, and by its often being insufficiently inclusive for groups that could benefit the most from
participation in programme activities.
3.5. Degree of participation of disadvantaged groups
According to the Programme Guide, the Erasmus+ Programme aims at “promoting equity and
inclusion by facilitating access to participants with disadvantaged backgrounds and fewer
opportunities compared to their peers whenever disadvantage limits or prevents participation in
transnational activities. Targeting young people with fewer opportunities, notably through project
funding for organisations working in this area” (EC, 2016, p. 9). Between 2007 and 2013, work on
social inclusion in the context of the YiA was supported by the Inclusion Strategy, launched in 2007.
The revised Inclusion and Diversity Strategy was launched in 2014 in order to support Erasmus+
implementation. As underlined in the legal base: “when implementing the Programme, inter alia as
19 The source of information about the total number of enrolled students in Croatia is the Croatian Bureau of Statistics,
(2015). Students enrolled in professional and university study, winter seminar, 2014/2015 academic year. [online] Zagreb:
Croatian Bureau of Statistics. Available at: http://www.dzs.hr/Hrv_Eng/publication/2015/08-01-07_01_2015.htm [Accessed
19 Feb. 2017]. 20 The source of information on the number of staff at higher education institutions in Croatia is the Croatian Bureau of
Statistics, (2016). Higher Education, 2015. Zagreb: Croatian Bureau of Statistics. 21 The source of information on the numbers of enrolled pupils and staff employed at secondary schools is the Croatian
Bureau of Statistics report Upper Secondary Schools – end of 2014/15 school year and beginning of 2015/16 school year.
95
regards the selection of participants […], the Commission and the Member States shall ensure that
particular efforts are made to promote social inclusion and the participation of people with special
needs or with fewer opportunities” (European Commission/European Council, 2013).
Since 2014, national authorities, together with the national agencies responsible for the
implementation of Erasmus+, could decide (on an opt-in/opt-out basis, depending on the support
already provided at the national level) if HEIs in each respective country should top-up individual
support from the EU grant for students from disadvantaged backgrounds with an additional payout of
between EUR 100-200 per month. The Croatian national authority decided to grant this top-up for
students from disadvantaged backgrounds, which resulted in 15,9% of outbound students in Higher
Education being classified as having disadvantaged backgrounds, while the rate of special needs
participants remained at the same level as in the LLP, at 0,1% of outbound students. The share of
special needs participants was low across all sectoral fields: VET (2%), SCH (0,2%), YOU (1,1%).
The third special target group, participants with fewer opportunities, was well represented in the YOU
(32%)22
but much less so in the VET (2,4%).
While Erasmus+ represents a step forward in widening the participation of disadvantage background
students in the HE, the participation rates of students with fewer opportunities and those with special
needs in the first three years suggests that outreach towards these groups is still not in line with the
strong emphasis that the programme objectives put on equity and inclusion. Some of the reasons for
low participation among special target audiences are addressed in the LLP Evaluation Study.
Reaching disadvantaged groups is considered difficult, since there is a lack of active outreach to these
groups on an institutional level, and they face obstacles to mobility due to certain conditions (e.g.,
health, economic, cultural etc.). A majority of the HEIs participating in the programme do not have a
systematic approach to reaching special needs students and students from disadvantaged backgrounds,
even though they are aware that students of lower socioeconomic status are less likely to take part in
mobility (Baketa, Brajdić Vuković and Klasnić, 2016). In the LLP Evaluation Study, respondents
from the COM, LDV and GRU sectoral programmes estimated that projects had a limited influence
on work with people having disabilities and fewer opportunities (Ančić and Klasnić, 2016).23
3.6. Changes in the type of applicants
Apart from individuals, organisations are among the key actors in the programme. Erasmus+ opened
up more opportunities for a wide variety of organisations to participate across sectors, with the aim of
diversification and more cross-sectoral cooperation. Considering the programme’s change in
structure, changes in the distribution of participating organisations do not come as a surprise. In
Erasmus+, secondary schools have the highest degree of participation among beneficiary
organisations (172) followed by non-governmental organisations (134) and primary schools (128).
When it comes to applicants, the majority come from the ranks of the non-governmental sector. There
is also a significant share of repeating beneficiaries; as shown in Figure 10, there are 204 LLP
organisations that also took part in Erasmus+. At the same time, there has also been an increase in
newcomers: 233 organisations submitted their first applications under the Erasmus+. A smaller share
of organisations (42) submitted applications under both the LLP and E+, but were unsuccessful in
getting a grant.
22 Source of data: Mobility Dashboard as of 19.02.2017. 23 Respondents were asked to estimate the degree to which participation in the LLP contributed to the social inclusion of
learners with special needs or fewer opportunities, and stronger support for learners from disadvantaged backgrounds.
96
Among repeating beneficiary organisations that took part in both the LLP and Erasmus+, secondary
schools made up the largest share (90), followed by primary schools (39) and HEIs (21). Among
newcomer organisations, most represented were NGOs (106), followed by secondary school
organisations (42), HEIs (26) and primary school institutions (23). A majority of organisations
submitted their applications within a single sectoral field, with the greatest demand evident in the
SCH and the YOU. However, a certain number of organisations were cross-field applicants,
submitting their applications in several different fields over the years. The highest rates of overlap
between beneficiaries were between the VET and the ADU on the one hand, and between the VET
and the SCH on the other.
Figure 10. Number of applicant organisations submitting applications across different fields
Sectoral field AE SE VET HE YOUTH
AE 112 12 27 9 22
SE
421 80 11 17
VET
132 9 23
HE
55 6
YOUTH
257
Source: LLP and E+ Organisations database, Mobius+
As presented in Figure 11, there is a steady year-on-year increase in the number of participating
organisations due to the increase in available funding and more organisations learning about
programme participation possibilities. However, with the transition to Erasmus+, there are some
changes in the structure of participating organisations. The number of secondary schools dropped
from 73 to 60 in 2016, while the most downward trend is present among primary schools (from 75 in
2013 to 22 in 2014). There is also an increase in the participation of HEIs over the years, and the
upward trend continued from 2014 up through 2016. Some possible explanations for the lower level
of participation could be found in the changed structure of programme activities and an overall
reduction in the number of contracted projects. The downward trend could also be the result of a lack
of capacities to compete for more demanding projects (particularly among primary schools). In
contrast, participation among NGOs increased significantly, partially due to the merger of the
Education and Training as well as Youth sectors under the same umbrella programme. The
participation of NGOs increased in all sectoral fields, since within Erasmus+, NGOs are eligible
applicants/partners in other fields such as the VET and the ADU. Some other target groups
participated less in Erasmus+ than in its predecessor programmes. For example, private businesses
and adult education providers are less represented among the participating organisations in Erasmus+
than in the previous generation of the programme, although an increasing number of these
organisations have applied.
97
Figure 11: Participation of organisations in LLP and E+
Source: Mobius organisation database
3.7. Degree of participation of public bodies, NGOs and enterprises compared to the predecessor
programmes
Educational institutions (from different fields of education) have the highest degree of participation in
Erasmus+. This is in line with the programme objectives, mostly tackling the issues relevant in
education. However, since the objective of Erasmus+ is to promote and facilitate cross-sectoral
cooperation, participation on the part of different types of organisations is crucial for meeting these
expectations, although cross-sectoral cooperation remains difficult to track, since the indicators as to
whether the programme is truly cross-sectoral are unclear. In 2014, applicants could submit
applications under the KA200 if their projects were mostly oriented towards horizontal, cross-sectoral
cooperation. However, only a few applications were submitted and selected under this activity. The
participation of public bodies, NGOs and enterprises can also indicate the diversification of
programme users, although participation among enterprises and public bodies remains low. Some of
the answers given in the interviews with representatives from these groups indicate that an increased
administrative burden and a lack of human capacities, as well as a prioritisation of organisational
goals that differ from programme goals, tend to influence interest in applying for funding under
Erasmus+ (Lukić, 2015). Overall, programme beneficiaries remain mostly educational institutions,
with increased participation on the part of NGOs.
2009 2010 2011 2013 2014 2015 2016
Local and regional authorities 1 2 2 4 1 3 1
Civil society organisations 11 16 20 19 72 74 67
Elementary schools 22 37 53 75 22 31 39
Private bodies (companies) 6 11 23 17 11 2 5
Secondary schools (general and
vocational programmes)31 46 76 73 67 64 60
Adult education providers 10 13 11 12 5 7 6
Pre-primary education 3 3 5 15 5 1 4
Higher education institutions 10 17 19 26 37 41 45
Scientific institutions 2 2 3 1 0 4
0
10
20
30
40
50
60
70
80
98
3.8. Regional distribution: rural-urban divide
The highest share of programme applicants comes from four major cities, and from the counties
where these cities are situated. A majority of applicant organisations are from the city of Zagreb
(1178), followed by applicants from the counties of Split-Dalmatia and Primorje-Gorski Kotar. The
regional distribution of participating organisations indicates that applicants and beneficiary
organisations from urban areas are highly overrepresented. As presented in Figure 12, in these
counties there is a concentration of urban population, a high number of registered NGOs and
educational institutions (at all levels), and a higher share of the population that has completed higher
education and is at lower risk of poverty. In contrast, counties with a lower share of urban population,
a higher risk of poverty and a higher level of participate less in the programme. The findings of the
RAY Standard Survey (national report) confirm that the YOU is characterized by a prevalence of
participants from urban areas, a low share (10%) of participants from rural areas, and slightly more
representation in Youth Exchange projects. A regression analysis estimating the relationship between
socio-demographic characteristics and the perceived impact of projects shows that the place of
residence (urban/rural) influences the perception of project impact. Participants from larger urban
areas perceive a higher degree of progress in their abilities/skills after programme participation than
do participants coming from less urban areas. In contrast, participants from less urban (i.e., rural)
areas estimate a higher impact of project participation on their future plans and their work in the
Youth field (Gregurović, 2017).
Figure 12: Regional distribution of LLP and E+ beneficiaries and indicators of regional
development
County Poverty rate
(2014)
Share of
population with
completed
higher
education (age
group > 15)
Share of
population
living in urban
area
Share of
population in
rural or other
area
Sucess rate of
contracted
projects in
LLP, YiA
and E+
(2010.-2016.)
%
Bjelovar-Bilogora 20,0 9,3 37,7 62,3 46,81
Brod-Posavina 35,9 9,5 44,3 55,7 42,80
Dubrovnik-Neretva 14,5 18,7 53,2 46,8 43,84
City of Zagreb 9,8 29,0 94,0 6,0 44,10
Istria 11,9 16,6 54,9 45,1 43,75
Karlovac 23,2 12,8 49,6 50,4 42,07
Koprivnica Križevci 20,3 10,7 35,9 64,1 54,13
Krapina-Zagorje 18,8 9,2 16,1 83,9 44,90
Lika-Senj 19,8 10,5 35,3 64,7 42,55
Međimurje 20,8 10,1 23,0 77,0 53,79
Osijek-Baranja 28,0 12,7 49,2 50,8 40,61
Požega-Slavonia 26,5 10,0 44,3 55,7 43,37
Primorje-Gorski Kotar 11,9 20,1 61,8 38,2 38,52
Sisak-Moslavina 23,7 10,5 46,8 53,2 42,75
Split-Dalmatia 19,5 18,0 59,4 40,6 44,64
99
Šibenik-Knin 24,7 13,2 52,5 47,5 44,14
Varaždin 17,1 11,9 34,0 66,0 48,13
Virovitičko-podravska 33,4 8,2 40,7 59,3 51,02
Vukovar-Srijem 31,9 9,5 50,3 49,7 42,03
Zadar 25,2 14,8 57,8 42,2 38,79
Zagreb 16,7 12,4 32,0 68,0 45,70
Total 19,40 16,4 55,3 44,7 35,24
4. Effectiveness24
4.1. Achieving programme objectives
In this section, evidence factors driving or hindering progress towards the achievement of programme
objectives are presented. One of the specific objectives of the Erasmus+ programme, as stated in the
Regulation on establishing the Erasmus+ Programme (Regulation EU No 1288/2013), is “to improve
the level of key competencies and skills, with particular regard to their relevance for the labour market
and their contribution to a cohesive society, in particular through increased opportunities for learning
mobility and through strengthened cooperation between the world of education and training and the
world of work” (European Council/European Commission, 2013).
In the Recommendation on key competencies for lifelong learning, the European Council and the
European Parliament defined key competencies in the following way: “knowledge, skills, and attitudes
that will help learners find personal fulfilment and, later in life, find work and take part in society.
These key competencies include 'traditional' skills such as communication in one's mother tongue,
foreign languages, digital skills, literacy, and basic skills in maths and science, as well as horizontal
skills such as learning to learn, social and civic responsibility, initiative and entrepreneurship, cultural
awareness, and creativity” (European Council/European Parliament, 2006).
In order to draw conclusions about the progress made towards achieving this objective, the
following paragraphs analyse and present the results of the Participant Report (EU Survey), a
survey that was completed by mobile participants (pupils, students and staff) upon returning from
their mobility periods abroad. Findings suggest that end-users’ (i.e., participants’) overall perception
of the influence of mobility on the enhancement of their abilities is positive, although there are
differences across sectoral fields/mobility types.
According to the results of the EU Survey, the overall percentage of Erasmus+ participants declaring
that they have improved their key competencies and/or their skills relevant for employability is high
among both examined groups of VET and HE learners/students. Among KA1 participants (i.e.,
learners) from the VET (n= 1.982), 73,4% and 24,2% recognize a “very positive” and a “positive”
influence, respectively, while among those in the HE (n= 2.223), the corresponding percentages are
50,8% and 44,9%. It should be noted that HE participants from disadvantaged backgrounds view the
24
To what extent have Erasmus+ and its predecessor programmes contributed to the realisation of the Erasmus+ specific
objectives (as listed in point B.2 in annex 3) in your country?
[see below]
100
influence of project participation on skill enhancement more positively (54,3% ) than those who did
not receive top-up grants due to disadvantaged backgrounds (50,8%). These participants view the
overall mobility experience more positively as compared to their colleagues who did not receive
disadvantaged background grants.
A statistical test was applied in order to determine differences between VET and HE participants
(learners). As shown by the The Kruskal-Wallis H test, there is a statistically significant difference
across all of the variables related to key competencies. For all of the examined variables, VET pupils
recognized a more positive influence of their mobility periods than did HE students. VET pupils and
HE students who received disadvantaged background grants had an overall higher degree of general
satisfaction with the mobility experience, and a higher level of agreement with statements regarding
skills enhancement following their mobility periods abroad. It should be noted that VET pupils
embark on mobility in order to undertake vocational training abroad; these learners are hosted either
at a workplace (i.e., in an enterprise or other relevant organisation) or at a VET school (with periods
of work-based learning in an enterprise or other relevant organisation). In contrast, a majority of HE
students spend their mobility periods studying rather than in vocational training. VET learners’
orientation towards training rather than academic studies could influence their perceptions of
improvement with respect to skills deemed relevant for employment, since there is no statistical
difference in perceptions of skill enhancement between HE students, who carried out traineeships
abroad, and VET learners.
In addition to the Participant Reports (EU Survey), the RAY Standard Survey (national report) was
examined in order to assess the perspectives of project leaders and Youth participants. As shown in
Table 13, participants and group leaders alike who took part in Erasmus+ Youth activities expressed a
high degree of agreement with statements about the development of skills/abilities that are relevant
for lifelong learning. The highest degree of agreement was expressed in statements related to cultural
knowledge, communication in foreign languages and cooperation in teams. The enhancement of
teamwork was also highly recognized as an outcome of project participation among project leaders
and project participants. In comparison to other competencies/skills/abilities, group leaders and
participants alike recognize little progress in producing media content (i.e., printed, audiovisual,
electronic).
101
Figure 13: Answers to question „Through my participation in this project I learned better …“
Source: Research-based Analysis and Monitoring of Erasmus+: Youth in Action In Croatia 2015., Gregurović (2017)
However, there are statistically significant differences between participants from different programme
activities in the YOU. Participants in the Y-(TRANS)NAT have the lowest level of agreement with
statements related to communication in foreign languages and understanding people from other
cultures, but they perceive improvement in their ability to discuss political topics seriously.
Participants from the Y-EXCH have lower scores than Y-(TRANS)NAT participants in statements
related to their ability to do something for the local community and those related to logical thinking,
while they have a higher level of agreement with statements related to creative and artistic expression
(Gregurović, 2017). Results indicate that participation in different programme activities can trigger
different interests, depending on the activities that each participant engaged in during the project, and
that participation in projects will not necessarily contribute to the development/enhancement of all key
competencies, but rather will contribute to segmented such development/enhancement.
1%
3%
1%
6%
3%
2%
2%
3%
2%
2%
9%
5%
4%
1%
12%
5%
4%
28%
11%
9%
9%
20%
10%
10%
35%
25%
17%
4%
58%
43%
46%
44%
55%
52%
52%
54%
54%
49%
37%
47%
47%
37%
29%
49%
49%
22%
31%
37%
38%
24%
34%
39%
19%
22%
32%
57%
0% 20% 40% 60% 80% 100%
… to say what I think with conviction in
discussions. (N=1472)
… to communicate with people who speak
another language. (N=1468)
… how to cooperate in a team. (N=1461)
… to produce media content on my own
(printed, audiovisual, electronic). (N=1465)
… to develop a good idea and put it into
practice. (N=1465)
… to negotiate joint solutions when there are
different viewpoints. (N=1468)
… how to achieve something in the interest of
the community or society. (N=1466)
… to think logically and draw conclusions.
(N=1455)
… to identify opportunities for my personal or
professional future. (N=1459)
… how I can learn better or have more fun
when learning. (N=1454)
… to discuss political topics seriously.
(N=1454)
… to plan and carry out my learning
independently. (N=1451)
… to express myself creatively or artistically.
(N=1453)
… to get along with people who have different
cultural background. (N=1458)
Not at all
Not so
much
To some
extent
Definitely
102
4.2. Promoting participation in democratic life in Europe and the labour market
Promoting participation in democratic life in Europe and the labour market is one of the specific
objectives emphasized in the YOU, but also one of the expected outcomes of the KA1 programme
activities that were implemented in E&T. EU Survey results are based on participants’ self-
assessments of the influence of project participation on their interest in social and political life, and
therefore serve as an indication of end-user perspectives about the achievement of progress in
enhancing skills/competencies after participation in projects.
The percentage of Erasmus+ participants claiming to be better prepared for participation in social and
political life after returning from mobility is relatively high, although the results are low compared to
other examined indicators from the EU Survey. Learners were asked if after having taken part in
mobility, they intend to participate more actively in the social and political life of their community. A
majority of HE students (60,3%), VET learners (80,4%) and EVS volunteers (75,5%) reported an
increase in positive attitudes towards social and political participation following mobility abroad. A
Kruskal-Wallis H25
test was conducted in order to determine if there were differences in attitude
towards social participation between HE students (n=3.150), VET learners (n=1.936) and EVS
volunteers (n=278). The distributions of this social participation score were statistically significantly
different26
between groups, H(2) = 234,454, p = .001. Paired comparisons were performed using
Dunn's (1964) procedure, with a Bonferroni correction for multiple comparisons. Statistical
significance was accepted at the p < .001 level. This post hoc analysis revealed statistically significant
differences in scores between the HE students (2903,18) and VET learners (2352,48) (p = .001) as
well as between the HE students and EVS volunteers (2480,28) (p = .001), but not between the EVS
volunteers and VET learners. Overall, HE students express less positive attitudes towards
participation in democratic life than do VET learners and EVS volunteers after returning from
mobility.
Figure 14: Answers to the question „After having taken part in mobility… I intend to
participate more actively in the social and political life of my community
Source: Participant Reports/EU Survey
In the RAY Standard Survey (national report), the impact of Erasmus+ Youth projects on involvement
25 The Kruskal-Wallis H test (sometimes also called the „one-way ANOVA on ranks“) is a rank-based nonparametric test
that can be used to determine if there are statistically significant differences between two or more groups of an independent
variable on a continuous or ordinal dependent variable. 26
Distributions of social participation scores were not similar for all groups, as assessed by visual inspection of a boxplot.
Values are mean ranks unless otherwise stated.
75,54%
60,32%
80,42%
18,71%
27,90%
14,82%
5,76%
11,78%
4,75%
0% 20% 40% 60% 80% 100%
EVS volunteers (YOU)
HE students
VET learners Agree
Neither agree nor
disagree
Disagree
103
of participants in active citizenship and social life has been assessed on a frequency scale with three
response options. Respondents were asked about their behaviour before and after the project. Results
show that a higher share of participants recognized no change in behaviour after the projects in
comparison to behaviour beforehand. There are only two areas – a) appreciation of cultural diversity
(64%) and b) interest in involvement in and development of Youth policy (50%) – in which at least
half of the participants recognized a change after returning from mobility abroad. Further analysis
revealed that there are differences in the self-assessments of participants from different programme
activities. Y-EXCH participants, in comparison with Y-(TRANS)NAT participants, show a higher
degree of appreciation for cultural diversity and environment protection. Conversely, Y-EXCH
participants are less interested in involvement in policy development as compared to Y-TNYW
participants. There is a statistically significant difference between Y-(TRANS)NAT participants and
TCA (Gregurović, 2017).
In addition to students/learners/volunteers, a group comparison of participating staff from the HE,
VET, SCH and YOU was conducted. A Kruskal-Wallis H test was run in order to determine if there
were differences in social and civic competence scores between the four groups. Distributions of
social and civic competence scores were not similar. The distributions of scores were significantly
different between groups, H(3) = 77,854, p = .001. Paired comparisons were performed using Dunn's
(1964) procedure, with a Bonferroni correction for multiple comparisons. Statistical significance was
accepted at the p < .05 level. This post hoc analysis revealed statistically significant differences in
median scores between the HE staff (1), YOU workers (2), VET staff (4) and SCH staff (3), but not
between VET staff and SCH staff. (p = .1). VET staff expressed the highest level of agreement with
having the intention to participate in social and political life after their mobility periods abroad.
4.3. Multilingualism and intercultural dialogue
Among the expected outcomes of mobility are improved language skills and heightened interest in
new languages and cultural diversity. The percentage of Erasmus+ participants in long-term mobility
(Education and Training) or voluntary service (Youth) declaring that they have improved their
language skills is high among Croatian participants across all sectoral fields. Overall, 97,9% of
learners/students/volunteers assess that they have improved their proficiency in the language that they
used abroad, and 78% of participants feel that they have done the same in languages other than the
main language used (most often, this main language is English). However, it should be noted that 14%
of students in the HE declared that they were already fluent when they embarked on mobility. In
terms of language skills, 89,3% (1.715) of VET learners, 80,6% (2.284) of HE students and 78,1%
(89) of young volunteers declared that their language skills did improve after mobility, with
statistically significant differences between groups.27
The proportion of learners from the VET who
stated that they had improved their language skills was significantly higher than that of students from
the HE or volunteers from the YOU, while there was no statistically significant difference between
participants from the HE and the YOU.
However, previous knowledge of a foreign language can have an impact on respondents’ self-
assessments of language skills progress. Participants who had a higher level of language skills prior to
27
A significant difference in proportions was detected by using the Chi-square test, p = .001. Post hoc analysis involved
paired comparisons using the z-test of two proportions, with a Bonferroni correction. The Kolmogorov-Smirnov test of
normality was statistically significant (p = .001) and the subsets of HE, VET and YOU learners/volunteers were not
normally distributed.
104
embarking on mobility perceived less progress than those who had a lower level of knowledge at the
same point in time. OLS assessment28
(i.e., the online language test) results of HE students before
(LA1) and after (LA2) mobility show that there was limited progress with respect to language skills,
since in the first assessment (LA1), 9,6% of students had language knowledge at the B1 level or
lower, and 90,4% had the same at the B2 level or higher, according to the language test. In the second
assessment (LA2), results were almost the same, with just 0,4% improving their level of language
competence. Results show that in the HE, a majority of outbound students already had a high level of
language skills prior to taking part in mobility. However, when a language is spoken in an
international environment or with native speakers, there is also an accumulation of new knowledge
that is not necessarily detected through standardized language testing.
Aside from students/learners/volunteers, participating staff across sectoral fields also estimated their
progress in foreign language communication abilities. Among SCH staff, 61,2% strongly agree that
they have improved their ability to communicate in a foreign language after spending a period of time
abroad. Results vary across sectors, with more than half of VET staff (68,4%) and Youth workers
(59,8%) responding positively, but a relatively small share of HE staff (40%) responding in the same
way (see Figure 15).
Additional evidence with respect to the YOU is available from the RAY Standard Survey (national
report). According to Gregurović (2017), project leaders assessed that they have developed
communication skills, foreign language skills and the ability to cooperate in teams.
Figure 15. Staff: Communication in foreign languages
Source: Participant Reports
A Kruskal-Wallis H test was run in order to determine if there were differences in participants’ self-
assessments of improvement in the area of foreign language communication. The distribution of
scores between SCH staff (n=529), VET staff (n=257) and YOU youth workers (n=1.300) was not
similar. Paired comparisons were performed using Dunn's (1964) procedure, with a Bonferroni
correction for multiple comparisons. Statistical significance was accepted at the p < .05 level. This
post hoc analysis revealed significant differences in median scores between HE staff (1) and SCH
staff (2) (p=.001);29
HE staff and VET staff (2); and HE Staff and YOU youth workers (2) (p=.018).
There is no difference between VET, SCH and YOU workers/staff. Staff from the HE, as compared to
those from other groups, expressed a lower level of agreement with statements about improved
28 All HE students undertake the OLS assessment prior to embarking on mobility. 29 Response scale from -2 = “Strongly Disagree” to 2 = “Strongly Agree.”
3,90%
7,20%
10,50%
19,10%
30,10%
31,60%
27,70%
34,30%
61,20%
68,40%
59,80%
40,00%
0% 20% 40% 60% 80% 100%
SCH Staff
VET Staff
YOU Youth Workers
HE StaffStrongly disagree
Rather disagree
Neither agree nor
disagreeRather agree
Strongly agree
105
foreign language communication skills after participation in mobility.
Further analysis was conducted among staff in order to determine the impact that project participation
had on cultural awareness and cultural expression. A vast majority (i.e., more than two thirds) of
participating staff in all sectors declared that mobility had a positive impact on their cultural
awareness and expression (see Figure 16). A Kruskal-Wallis H test was run in order to determine if
there were differences in perception with respect to cultural awareness and expression scores between
four groups of participants: SCH staff (n=529), VET staff (n=257), YOU youth workers (n=1.300)
and HE staff (n=1.395).30
The distribution of scores was not similar for all groups, as assessed by
visual inspection of a boxplot. Paired comparisons were performed using Dunn's (1964) procedure,
with a Bonferroni correction for multiple comparisons. Statistical significance was accepted at the p <
.05 level. This post hoc analysis revealed significant differences in mean rank between YOU youth
workers (947,84) and SCH staff (1051), as well as between YOU youth workers (947,84) and VET
Staff (1115) (p=0.018). There was no difference between SCH staff and VET staff (p=1). χ2(2) =
26,405, p = .001. YOU workers expressed a lower level of agreement regarding progress with respect
to cultural awareness and cultural expression than did SCH and VET staff.
Figure 16. Cultural awareness and expression
Source: Participant Reports
4.4. Enhancing employment and career opportunities
As regards students, trainees, apprentices, young people and volunteers, mobility activities are meant
to enhance the employability and improve the career prospects of these participants, as well as to
increase their sense of initiative and entrepreneurship. As regards staff, youth workers and
professionals involved in Education and Training as well as Youth, mobility activities are expected to
result in improved competencies that are linked to these participants’ professional profiles (e.g.,
teaching, training, youth work, etc.), as well as to enhance their career opportunities. In Participant
Reports, HE students and VET learners who spent a period of time in traineeships answered a
question about their prospects of finding a better job after spending time abroad. Among VET
learners, 78,4% considered their chances of finding a new or better job very positive, 17,9% declared
their chances positive, 3% declared their chances unchanged, and 0,2% perceived their chances of
finding a better job as negative. Among HE students, there is also a positive perception of future
employment prospects after mobility: 60,6% have very positive expectations and 30,4% have positive
expectations, while 7,4% of students consider their chances unchanged and 0,4% consider their
chances negative. A Mann-Whitney U test was run in order to determine if there were differences in
30
Response scale from -2 = “Strongly Disagree“ to 2 = “Strongly Agree“
3,70%
5,50%
20,40%
19,70%
28,70%
75,00%
80,30%
64,50%
0% 20% 40% 60% 80% 100%
SCH Staff
VET Staff
YOU Youth WorkersStrongly disagree
Rather disagree
Neither agree nor
disagreeRather agree
Strongly agree
106
perception of chances for employment between VET learners and HE students upon returning from
mobility. Distributions of the scores for VET learners and HE students were similar, as assessed by
visual inspection of the histogram. Scores were significantly higher among HE students (mean
rank=2.503) than they were among VET students (1.856), U = 1850.746,00, z = -17,419, p = .001,
using an exact sampling distribution for U. According to the results, HE students have a more positive
perception of their employment opportunities than do VET learners.
In addition to students/learners/volunteers, participating staff also recognize the influence of
programme participation on their career prospects. More than 2/3 of participating staff from all
sectoral fields declared that mobility enhanced their employment and career opportunities (the
distribution of their answers is presented in the table above), while 1/3 considered that it had no
impact or a negative impact. A Kruskal-Wallis H test was run in order to determine if there were
differences in the scores of perceived career opportunities enhancement between SCH staff (n=415),
VET staff (n=258) and YOU youth workers (n=1.300), with levels of perceived enhancement ranging
from -2 (strongly disagree) to 2 (strongly agree). Score distributions were not similar for all groups, as
assessed by visual inspection of a boxplot. There is a statistically significant difference between
groups χ2(2) = 12,070, p = .017. Paired comparisons were performed using Dunn's (1964) procedure,
with a Bonferroni correction for multiple comparisons. Statistical significance was accepted at the p <
.05 level. This post hoc analysis revealed statistically significant differences in mean rank between
SCH staff (1.871) and YOU staff (1.697), (p=.012). SCH staff had a more positive view of career
opportunities enhancement than did YOU staff. There was no statistically significant difference
between other groups.
Figure 17: Participants answers to the question “I have enhanced my employment and
career opportunities”
Source: Participant Report
2,70%
2,70%
2,40%
4,50%
4,80%
20,00%
16,30%
22,70%
17,20%
23,80%
36,10%
47,70%
39,10%
54,50%
34,80%
40,20%
31,40%
34,50%
23,90%
34,20%
0% 20% 40% 60% 80% 100%
SCH Staff
VET Staff
HE Staff
ADU Staff
YOU Staff
Strongly Disagree
Disagree
Neither agree nor
disagree
Agree
Strongly agree
107
4.5. Objectives at the institutional level
Some of the specific programme objectives are tailored to influence educational and youth
organisations, not merely the end-users (i.e., participants) who take part in project activities. One of
the objectives of E&T is to “foster quality improvements, innovation excellence and
internationalisation at the level of Education and Training institutions, in particular through enhanced
transnational cooperation between Education and Training providers and other stakeholders,” while in
the YOU the aim is to “foster quality improvements in youth work, in particular through enhanced
cooperation between youth organisations and other relevant stakeholders”(European
Commission/European Council, 2013).
Even though at the moment there is no evidence as to the impact of Erasmus+ on participating
institutions, there is an in-depth evaluation of the participation of Croatian educational organisations
in LLP projects. According to evidence collected by Ančić and Klasnić, it seems that no matter what
sectoral programme institutions participated in (i.e., LDV, COM or GRU), there is no statistically
significant difference between sectoral programmes when it comes to the use of new pedagogical
methods, recognition of an institution in its local community or the European dimension in education.
Survey respondents recognized progress in all of these dimensions. However, LDV and GRU did
have a stronger impact on the development of project management skills and international
partnerships in educational organisations, as well as on the personal development of end-users (i.e.,
participants). A more detailed review of LLP findings suggests that after programme participation,
there is progress in terms of institutional motivation/readiness to participate in professional
development activities and use new pedagogical methods, as well as in terms of the visibility/status of
educational institutions in their local communities and the European dimension in education. There is
no statistically significant difference between LLP sectoral projects. However, according to the
results, the LDV and GRU sectoral projects had a more significant influence on institutional
capacities for project management, internal organisation and cooperation between staff, and
broadened development of cross-sectoral and international cooperation. One reason why these sectoral
programmes had a stronger influence lies in the content of their activities as well as in the activities
themselves, which implemented a greater number of large scale projects and which were tailored to
influence organisations, not merely individual participants (2016: 152).
The institutional influence when it comes to higher education is more complex, since participation in
the programme demands a certain degree of organisational dedication and a strategic approach that
needs to be elaborated in the Erasmus+ Charter for Higher Education (previously the Erasmus
University Charter). In 2009, when the Erasmus programme was first introduced, Croatian HEIs
started developing procedures, rules, promotional/informational activities directed at students and
teaching staff, courses provided in foreign languages for inbound students, and support mechanisms
for inbound/outbound mobilities. This lead to an increase in the number of mobilities, courses and
bilateral agreements. Preparatory actions also included networking and the transfer of information
between HEIs, as well as the development of problem solving skills among administrative staff
(Baketa, Brajdić Vuković and Klasnić, 2016: 174-178). Therefore, the introduction of the Erasmus
programme inevitably triggered some changes and contributed to the scope and quality of
international cooperation at the institutional level.
There is no evidence as to YiA’s impact at the institutional level, although according to the results of
108
the RAY Standard Survey (national report), in Erasmus+ , YOU project leaders recognized a positive
impact from the projects on an organisational level in terms of cultural diversity acceptance,
international projects and networking at an international level. However, project leaders recognized no
impact on cooperation with the local community, the use of open educational resources or the
frequency of European topic-related work after project participation (Gregurović, 2017).
4.6. Challenges and difficulties in LLP implementation31
Some of the key findings of the LLP Evaluation Study concern institutional obstacles to participation
on the part of adult education providers, primary and secondary schools, and pre-primary institutions
in programme activities. Limited financial resources for project pre-financing (i.e., before the first and
second instalments are paid to the project beneficiary); a high administrative burden from projects that
defers staff from applying for programme funding; the perception that programme participation
demands lots of financial, human and administrative resources that are not available within the
institution; insufficient foreign language skills among staff and pupils; a lack of formal recognition for
project-related teacher training/professional development activities carried out abroad; and an
insufficient level of information about project participation possibilities are institutional obstacles that
have been recognized by the respondents. Further statistical analysis conducted within the LLP
Evaluation confirmed that the more institutional obstacles are recognized within an institution, the
lower the impact of project participation, since the obstacles are negatively correlated with the indexes
of progress on an institutional level. The authors further recognized four intercorrelated groups of
obstacles: a) administrative obstacles; b) a lack of information about the programme; c) a lack of
interest and motivation on the part of staff; and d) a lack of institutional knowledge to successfully
implement projects. (Ančić and Klasnić, 2016).
During the LLP period, HEIs also faced challenges in Erasmus programme implementation.
Examined findings suggest that HEIs differ in the level of human and financial capacities that they
invest in Erasmus programme internationalization and implementation. Some HEIs decide to further
other strategic objectives and tend to move resources from international cooperation to other areas,
resulting in both a lack of human capacities to implement the programme in a quality fashion and
discontinuity in the progress made by previous iterations of management. IRO employees and ECTS
coordinators at HEIs identified an insufficient number of Erasmus scholarships available (i.e., high
demand for limited available funding), insufficient administrative human capacities at HEIs (mostly
the case with major national universities), a low number of courses provided in foreign languages to
foreign students, a lack of formal recognition for teacher training/teaching assignments abroad, and a
lack of support for students of lower socioeconomic status (who are therefore unable to participate in
mobility) as institutional obstacles. Findings also indicate that there is no reciprocity in the number of
outbound versus inbound mobilities between the HEIs that signed bilateral agreements. Some HEIs
signed bilateral agreements that were rarely if ever realized, and only 20% of bilateral agreements
were realized in practice.
A content analysis of HEIs’ strategic documents showed that internationalisation objectives appear to
be unrealistically ambitious, since a majority of HEIs do not dedicate financial resources (apart from
Erasmus funding) to enhancing international mobility at the institutional level. Targets set are often
31 What challenges and difficulties do you encounter while implementing the various actions of Erasmus+? What changes
would need to be introduced in Erasmus+ or its successor programme to remedy these?
109
higher than institutional capacities can handle. Erasmus programme coordination at most Croatian
HEIs is carried out in a centralised fashion by the central Erasmus coordinator (i.e., an IRO
employee). Erasmus coordinators at universities identified a quantity of “invisible work”
underpinning programme implementation, including detailed correspondence with inbound/outbound
staff as well as intra-institutional coordination, which was sufficient to result in a work overload for
Erasmus coordinators. This is often the case at HEIs with larger student populations (Baketa, Brajdić
Vuković and Klasnić, 2016).
Moreover, an insufficient number of courses that are offered in foreign languages can present a
challenge to inbound students in Croatia. Erasmus coordinators/ECTS coordinators emphasize that
teaching staff are often not interested in conducting courses in foreign languages, since there is no
form of valorisation for doing so. Similarly, a lack of valorisation for short-term teaching staff
mobilities (i.e., with a duration of less than 3 months) on an institutional level is recognized as an
obstacle to wider participation of teaching staff at HEIs (Baketa, Brajdić Vuković and Klasnić, 2016:
203).
5. Efficiency32
Efficiency considers the relationship between the resources used by an intervention and the changes
generated by the intervention (which may be positive or negative). Differences in the way an
intervention is approached and conducted can have a significant influence on the effects, making it
interesting to consider whether other choices achieved the same benefits at less cost (or greater
benefits at the same cost).
Among the examined existing evidence, there is insufficient data/information to draw conclusions
about the efficiency of Erasmus+. However, the LLP Evaluation Study did examine HEIs’ financial
and human resources under the predecessor programme, and findings indicate that limited institutional
resources represent one of the main obstacles to growth in the scope and quality of programme
implementation.
According to the findings, HEIs’ human and administrative capacities are insufficient for further
growth of mobilities. IRO staff (especially at larger universities) emphasized the high level of
“invisible work” that is part of implementation, such as detailed information provision to all inbound
and outbound students/staff, and an administrative burden that increases along with the growth in
mobilities. At some HEIs, the direction of human resources away from international cooperation
towards other relevant areas (such as quality or research) results in discontinuity in the
implementation of international cooperation programmes. In the interviews conducted with Erasmus
coordinators, IRO employees emphasized the lack of support from HEIs’ decision makers in regards
to academic mobility and internationalisation in general. Some IRO employees noted that, after the
initial years of programme implementation (in Croatia, implementation started in 2009), the level of
involvement on the part of upper management dropped, while concurrently the number of mobilities
and the requirements of programme implementation grew over time, suggesting the need for a more
strategic approach and stronger programme coordination on the institutional level, and not merely on
32 To what extent is the level of human and financial resources that is available for programme implementation in your
country adequate? What steps did you take to optimise the efficiency of the resources deployed for Erasmus+
implementation in your country? What kind of rationalisation effort did you make in this respect?
110
the IRO office level (Baketa, Brajdić Vuković and Klasnić, 2016).
Among the topics addressed by the LLP Evaluation survey were questions on institutional obstacles
limiting the scope of Programme participation. Project managers from pre-primary, primary and
secondary schools as well as adult education providers perceived a lack of human and financial
capacities33
to implement projects as one of the major obstacles. This is particularly the case among
primary and secondary schools that are financed by regional authorities, which results in different
amounts of funding received by schools in different counties. In addition to such financial constraints,
the administrative burden of LLP projects de-motivated employees from participating in the project.
High employee workloads and insufficient foreign language knowledge also influenced decisions to
take part in the projects.
In open-ended questions posed to participating and non-participating staff within beneficiary
organisations, respondents complained about the inability to find replacement teachers to cover for
them during mobility periods abroad (i.e., staff training or other programme activities), delays in
signing agreements that resulted in delayed payments, and lots of “invisible work” that is often not
valorised or compensated in any way. According to the interviews and focus groups from the LLP
Evaluation Study, the willingness of school principals to motivate and inform their staff about
programme opportunities proved to be crucial for participation: principals who are not interested in
international projects and don’t provide support to employees tend to further deter staff participation
(Ančić and Klasnić, 2016).
6. Relevance34
Relevance looks at the relationship between the needs and problems in society and the objectives of
the intervention. Things change over time: certain objectives may be met or superseded, needs and
problems change, and new needs and problems arise. Interviews with relevant stakeholders and
survey results will provide more insight into the needs of beneficiaries, applicants and relevant
stakeholders, while monitoring data can give insight into target audience participation and the demand
for funding.
In order to answer how successful the programme is in reaching target audiences and groups within
different fields, it is important to understand the scope of participation. The high level of demand for
funding indicates that there is a need for international projects and mobility opportunities in Croatia’s
national context. This is particularly the case among staff from primary, pre-primary and secondary
education institutions in which teaching and non-teaching staff often lack professional development or
lifelong learning opportunities due to the limited financial resources that are available to institutions.
The number of participating primary and secondary education institutions dropped in Erasmus+, due
to a lower success rate and an overall lower number of projects. Around 20% of all educational
institutions (i.e., those in pre-primary, primary, and secondary education) in Croatia took part in the
programme since 2009, leaving lots of space for widened participation. There is also an urban-rural
divide that is recognized in beneficiary distribution. The regional distribution of participating
33 This refers to a lack of financial resources to cover costs prior to the first instalment payment. 34
How successful is the programme in attracting and reaching target audiences and groups within different fields of the
programme's scope? In case some target groups are not sufficiently reached, what factors are limiting their access, and what
actions could be taken to remedy this?
111
organisations and applicants reveals that organisations from more developed regions/counties (i.e.,
those with a higher level of urban population, a higher share of the population that has completed
higher education, and a lower poverty rate) are overrepresented among beneficiaries. This data
indicates that there is still a need for information provision and capacity building in order to widen the
participation of organisations from rural and remote areas, and to achieve more balanced participation
at the national level. Meanwhile, the participation of HEIs in the programme is extremely high, since
46 out of 49 registered HEIs take part in KA1.
Some other groups, such as enterprises as well as local and regional authorities, did not significantly
raise their level of participation in Erasmus+, and even though there was an increase in applications
received from these organisations, they often didn’t qualify for funding. This once again indicated a
need for capacity building and tailor-made information campaigns.
In YiA, a majority of applicant and beneficiary organisations were NGOs, and there was also good
participation on the part of youth organisations. Since 2014, there has been an increase in the number
of NGOs among participating organisations, while some of the traditional LLP beneficiaries, such as
primary schools, have participated less in a new programme. In the YOU, there is still a prevalence of
NGOs, however there is very modest participation on the part of smaller, youth- and volunteer-led
youth organisations.
The LLP and YiA period was characterized by lots of recurring beneficiaries – mainly organisations
with sufficient knowledge of project management to successfully pass the selection process. During
that period, 26% of beneficiaries took part in multiple programme activities or participated in both the
LLP and YiA. There were 15% of the beneficiaries who took part in the same programme activities
more than once. In the last three years, there has been an increase in the number of newcomers, but a
still-significant share (25%) of participating organisations are reoccurring beneficiaries. Erasmus+
offers more opportunities for different organisations to submit application across sectoral fields
(especially in KA2), however limited funding and more demanding projects seem to de-motivate less-
experienced organisations that lack the capacities to apply for or successfully pass the selection
process.
In terms of participation on the part of specific target groups (i.e., participants with fewer
opportunities, special needs or disadvantage backgrounds), there is still a need to widen access by
means of more dynamic, targeted outreach at the institutional and national levels. The introduction of
top-up grants for disadvantaged background students in the HE broadened mobility participation
among students with lower socio-economic backgrounds, and is generally deemed as a positive step
towards greater inclusion and equity. In contrast, the share of participants with fewer opportunities in
the VET and the SCH is relatively low, considering that in the VET there is a high concentration of
students of lower socio-economic status. The introduction of top-up grants, based on national criteria,
could be a useful measure to support the participation of pupils with disadvantaged backgrounds. To
conclude, there is still a need for targeted, tailor-made information provision and capacity building in
order to secure more diverse beneficiaries and applicants.
112
7. European added value and sustainability35
Under this criteria, the definition of an EU-added value is examined, as are changes that are
reasonably attributable to EU intervention, rather than to other factors.
According to the LLP Evaluation Study (Baketa, Brajdić Vuković and Klasnić, 2016), Croatian
universities were already cooperating internationally prior to joining the Erasmus programme in 2009.
Such international cooperation was mostly organized through bilateral agreements, Fulbright
scholarships, Tempus Individual Mobility Grants and the Central European Exchange Programme for
University Studies (CEEPUS). Meanwhile, applied universities and polytechnics had significantly
less internationalisation experience prior to joining Erasmus. However, in all types of HEI research,
participants recognized a widening of partnerships with foreign HEIs, the establishment of procedures
(especially recognition procedures) with those foreign HEIs, an increase in HEI visibility and
attractiveness, and an increase in the introduction of courses conducted in foreign languages since
participation in Erasmus started. Erasmus also triggered an increase in mobility among students as
well as teaching and non-teaching staff. According to the monitoring data analysis, since 2014 there
has been a significant increase in inbound mobilities (the in:out ratio of students being 1:1.24) such
that the number of inbound student and staff mobilities is expected to balance out the number of
outbound mobilities in the coming years. Even though complementary programmes such as Bilateral
scholarships and the CEEPUS continued to coexist with Erasmus, on an annual basis the budget
available through CEEPUS amounts to just 3% of the Erasmus+ funding available for Higher
Education, while the total funding from Bilateral scholarships amounts to 2.85% of the Erasmus+
funding available for Higher Education. Therefore, it is not surprising that Erasmus represented a
significant added value for HEIs, since it is the largest source of funding for international mobilities
on a national level and has significantly increased the scope of participation as compared to the
previous period. Still, with the increase in outbound and inbound mobilites, the institutional obstacles
that could limit further growth are recognized as well. Some of the institutional obstacles recognized
in the LLP Evaluation Study include: a lack of recognition or valorisation of short-term mobilities on
the part of teaching staff, a lack of strategic management of Erasmus funds (there is a discrepancy
between institutional targets and available funding, and some HEIs have no internationalisation
strategy), difficulties in the social and academic integration of inbound students, a lack of dynamic
outreach and information provision towards special groups of students (i.e., those of lower socio-
economic status or with special needs), and a lack of experience/confidence speaking foreign
languages. To conclude, EU funding through Erasmus (2009-2013), and then Erasmus+, has provided
significant added value to the development of international cooperation, although there are
institutional obstacles that limit the further internationalisation of Croatian HEIs.
Unlike the HE, other sectoral fields in education don’t have comparable programmes on a regional or
inter-regional level to support international mobility projects. Even though educational institutions
participate in various EU funded projects that involve international cooperation, information about
cooperation between educational institutions is not available for analysis. Research conducted among
35 a) To what extent Erasmus+ and its predecessor programmes produce impacts that are additional to the impacts that
would have resulted from similar actions initiated only at regional or national levels in your country? b) To what extent
Erasmus+ will be able to absorb in an effective way the sharp increase in the budget that is foreseen in the coming years up
to 2020 in your country. Could the programme use even higher budgets in an effective way? Do you see challenges to
effectively use more money for particular actions or fields of the programme?
113
project participants and non-participants within LLP beneficiary organisations (Ančić and Klasnić,
2016) provides an overview of the added value that LLP projects gave to educational institutions (i.e.,
pre-primary, primary and secondary education institutions). Beneficiary organisation employees
recognize that the implementation of projects did influence educational institutions, with the strongest
impact on the personal development of participants, the introduction of a European dimension in
education and the willingness/motivation of employees to participate in professional development
activities. The weakest impact of projects on the institutional level was recognized on the
development of international and cross-sectoral partnerships, as well as on that of specific skills,
knowledge and language competencies (p. 151). The integrated framework of Erasmus+ did bring
significant changes, but the core values and objectives of the programme remained similar, so the LLP
Evaluation Study, despite its focus on the institutional level, still provides relevant insight into the
programme’s value to educational institutions. As is the case in the HE, in other educational sectors
there are implementation difficulties and challenges of implementation (described on page 31) that
can limit the further development of internationalisation and the scope of impact that the programme
can have on educational institutions.
114
8. Conclusions
In terms of participation on the part of the target groups, the expectations of Erasmus+ when it was
created and launched in 2014 were that the integrated framework would bring not only more
cooperation between sectors, but also the involvement of new types of organisations across sectors,
such as enterprises, more and different social partners, and public bodies. Another difference in
comparison to the predecessor programme was a more institutional approach, which excluded the
possibility for individuals to apply directly for grants without some institutional affiliation. In
Croatia’s national context, the transfer to the new programme resulted in an increased degree of
participation from NGOs and HEIs, and a decrease in participation from pre-primary and primary
schools. The integrated framework resulted in a lower number of contracted projects, while demand
remained at a high level. Consequently, there is a high discrepancy between demand and available
funding. More complex project management followed an increase in the project budget, and
organisations with fewer resources were unable to pass the selection process or were de-motivated
from applying. There is no significant increase in diversity among participating organisations. The
higher degree of NGO participation is mainly a result of the integration of Education and Training
with Youth, while at the same time there is no increase in participation on the part of enterprises,
public bodies, or local and regional authorities, and there is a decline in participation on the part of
pre-primary and primary schools.
In terms of progress made towards achieving the specific and general objectives defined in the
Regulation EU No. 1288/2013, OJ L 347, this report emphasizes the combined findings from the
Participant’s Report (EU Survey), the RAY Standard Survey (national report) and the LLP Evaluation
Study. All of the findings indicate that the predecessor programme (i.e., the LLP) and the current
programme (i.e., Erasmus+) are successful in contributing to quality and a stronger international
dimension within educational institutions (according to project participants and non-participants from
beneficiary organisations), while project participation on the part of individuals contributes to the
development of abilities relevant for lifelong learning (according to end-users). However, some
specific objectives are very broadly defined. For example, the term “key competencies” is a
multidimensional concept that is defined by the European Commission as “the basic set of knowledge,
skills and attitudes which all individuals need for personal fulfilment and development, active
citizenship, social inclusion and employment,” as described in Recommendation 2006/962/EC of the
European Parliament and of the Council.36
A closer group comparison between different types of
participants (i.e., VET learners, HE students, YOU participants) reveals that the greatest recognized
areas of mobility impact are cultural knowledge and communication in a foreign language, while
certain attitudes, such as interest in active citizenship, are less influenced by project participation. A
more detailed review of the findings revealed that, although the results are mainly positive, there are
some differences in impact on end-users across mobility type. Rather than having broad objectives,
the definition of more focused objectives that are tailored to specific target groups and that take into
account differences in age, education level and programme activity content are recommended.
According to the Better Regulation Guidelines, “efficiency considers the relationship between the
resources used by an intervention and the changes generated by the intervention (which may be
positive or negative). Differences in the way an intervention is approached and conducted can have a
36 Recommendation 2006/962/EC of the European Parliament and of the Council of 18 December 2006 on key competencies
for lifelong learning (OJ L 394, 30.12.2006, p. 10).
115
significant influence on the effects, making it interesting to consider whether other choices achieved
the same benefits at less cost (or greater benefits at the same cost).” (European Commission, ch. 6,
2015). More evidence about the benefits of actions will be available in Analytical Report 2 (survey
results) and Analytical Report 3 (interviews). Results presented in this report indicate that the cost of
the programme differs across sectors. As in the predecessor programme, the highest share of the
budget is allocated to the HE, which also has the highest cost (i.e., funding) per participant. In
Erasmus+, the cost per participant grew particularly in the HE due to more than 75% of outbound
mobilities taking place in countries with a high or medium-high cost of living, an increase in monthly
grant rates, and the introduction of top-up funding for students with lower socio-economic
backgrounds. An increase in the cost of participation resulted in slower growth of the number of
outbound students. However, limited growth appears to be adequate in the Croatian context, since the
LLP Evaluation Study identified multiple institutional obstacles within participating HEIs, including
insufficient administrative and human capacities to administer a higher increase in outbound and
inbound mobilities. In other sectoral fields, there is insufficient quality data about the Comenius,
Leonardo da Vinci and Grundtvig sectoral programmes to draw conclusions about the efficiency of
Erasmus+ actions in comparison to those of the previous generation,
The extent to which EU interventions resulted in benefits to the EU that were above and beyond those
that would have resulted from interventions initiated at a regional or national level by both public
authorities and the private sector was also examined in this report. As presented in the country
overview (page 6-7), in the Erasmus+ programme there is a strong demand for funding across all
fields, and especially in the YOU, which is partially due to a lack of similar initiatives in the national
context. One exception is the HE. In addition to participating in Erasmus+, most HEIs participate in
regional and bilateral academic mobility programmes, namely, the Bilateral scholarship program,
which is based on bilateral agreements between Croatia and foreign countries, and the Central
European Exchange Programme for University Studies (CEEPUS). However, on an annual basis, the
budget available through CEEPUS amounts to just 3% of the Erasmus+ funding available for Higher
Education, and total funding from Bilateral scholarships amounts to just 2,85% of Erasmus+ annual
funding available for Higher Education. In other sectoral fields, there are no comparable programmes
on a regional or inter-regional level that provide support for international mobility projects, even
though educational institutions participate in various EU-funded projects that have an international
dimension.
116
9. Literature
Regulation (EU) No 1288/2013 of the European Parliament and of the Council of 11 December 2013
establishing 'Erasmus+': the Union programme for education, training, youth and sport.
European Commission. OJ L 347, 20.12.2013, p. 50–73.
Ančić, B., Klasnić, K., 2016. Evaluation of the Comenius, Grundtvig and Leonardo Da Vinci Sectoral
Programmes. In: Milanović-Litre, I. Puljiz, I., Gašparović, F., ed. Towards
Internationalisation of Education – Participation of the Republic of Croatia in the Lifelong
Learning Programme, Zagreb: Agency of Mobility and EU Programmes, pp. 60-150.
Brajdić Vuković, M., Klasnić, K., Baketa, N., 2016. Evaluation of Erasmus Sectoral Programme. In:
Milanović-Litre, I. Puljiz, I., Gašparović, F., ed. Towards Internationalisation of Education –
Participation of the Republic of Croatia in the Lifelong Learning Programme. Zagreb:
Agency for Mobility and EU Programmes, pp.159-226.
Milanović-Litre, I. Puljiz, I., Gašparović, F., ed. (2016). Towards Internationalisation of Education –
Participation of the Republic of Croatia in the Lifelong Learning Programme, 1st ed. [online]
Zagreb: Agency for Mobility and EU, pp.50-227. Available at:
http://www.mobilnost.hr/cms_files/2016/12/1481199381_k-internacionalizaciji-obrazovanja-
web.pdf [Accessed 23 Feb. 2017].
European Commission, (2015). Erasmus+ Programme Annual Report 2014. [online] European
Commission. Available at
http://ec.europa.eu/dgs/education_culture/repository/education/library/statistics/erasmus-plus-
annual-report_en.pdf [Accessed 23 Feb. 2017].
Ferencz, I., Mitić, M. and Wächter, B. (2016). Decentralized implementation - first experiences.
Brussels: European Parlament.
European Commission (2015). Education and Training Monitor 2015. (2015). 1st ed. Brussels,
Belgium: European Commission.
Gregurović, M. (2017). ERASMUS+: MLADI NA DJELU Rezultati istraživanja RAY Research-based
Analysis and Monitoring of Erasmus+: Youth in Action u Hrvatskoj 2015. Agencija za
mobilnost i programe EU. [unpublished report]
Erasmus+ Programme Guide. (2016). 1st ed. European Commission.
European Commission (2007), Inclusion strategy of the “Youth in Action” programme (2007-2013),
Brussels, Council of the European Union.
Education and Training Monitor 2015. (2017). 1st ed. Brussels, Belgium: European Commission.
European Commission/Directorate General for Education and Culture, (2014). Erasmus+ - Inclusion
and Diversity Strategy in Youth field. [online] Available at: https://www.salto-
youth.net/downloads/4-17-3103/InclusionAndDiversityStrategy.pdf [Accessed 24 Feb. 2017].
Recommendation 2006/962/EC of the European Parliament and of the Council of 18 December 2006
on key competencies for lifelong learning (OJ L 394, 30.12.2006, p. 10).
117
Appendix VI: Questionnaire [in Croatian]
ANKETA O PROVEDBI I UČINKU PROGRAMA ERASMUS+
Poštovani,
U tijeku je provedba anketnog istraživanja o provedbi i učinku programa ERASMUS+ te Vas ovim
putem molimo da sudjelujete u istraživanju. Tijekom listopada 2016. godine je na vašu e-adresu ili e-
adresu organizacije u kojoj radite poslan poziv za sudjelovanje u istraživanju (tekst dopisa dostupan je
ovdje: (http://www.idi.hr/wp-content/uploads/2016/11/Molba_za_suradnju_MZO.pdf).
Upitnikom koji ste upravo primili prikupljamo informacije o Vašim očekivanjima i perspektivi
programa ERASMUS+ Zanima nas Vaše mišljenje o nekim od aspekata programa ERASMUS+ i
programa koji su mu prethodili poput Programa za cjeloživotno učenje, Mladi na djelu, Tempus,
Erasmus Mundus.
Vi osobno ste izabrani da sudjelujete u ovom istraživanju jer imate određena iskustva s
prijavljivanjem i/ili provođenjem projekata u sklopu nekih od navedenih programa bilo kao zakonski
predstavnik/ica organizacije ili kao projektni voditelj/ica (osoba koja je kontaktirala s Agencijom za
mobilnost i programe EU), Stoga nam je važno da nam pomognete u ovom istraživanju ispunjavajući
ovaj anketni upitnik.
ČAK I AKO STE SAMO PRIJAVLJIVALI PROJEKTE ŽELIMO ČUTI VAŠE MIŠLJENJE.
Vaša anonimnost u potpunosti je zajamčena. Prikupljene podatke koristit ćemo isključivo u
istraživačke svrhe. Molimo Vas da na pitanja odgovarate iskreno, jer se jedino tako može osigurati
uspješnost, objektivnost i znanstveni karakter istraživanja.
Ispunjavanje upitnika traje od 5 do 15 minuta.
Ukoliko imate kakvih pitanja slobodno nas kontaktirajte.
U svrhu ispravnog pristupa upitniku molimo Vas da u svom pregledniku omogućite uporabu kolačića
(cookies).
U ime provoditelja istraživanja
Dr.sc. Branko Ančić
--------------------
znanstveni suradnik / research associate
Institut za društvena istraživanja u Zagrebu // Institute for Social Research in Zagreb
Centar za istraživanje društvenih nejednakosti i održivosti (CIDNO) // Centre for Research in Social
Inequalities and Sustainability (CRiSIS)
Amruševa 11/II - 10 000 Zagreb
Hrvatska / Croatia
T/F +385 01 49 222 999
M +385 098 885 024
www.idi.hr
Postoji 35 pitanja u ovom upitniku.
118
1. SUDJELOVANJE U ERASMUS+, PROGRAMU ZA CJELOŽIVOTNO UČENJE I PROGRAMU
MLADI NA DJELU
1.a. Je li organizacija u kojoj radite, u razdoblju od 2009. do 2016., dodijeljena financijska potpora od strane
Agencije za mobilnost i programe EU iz sredstava Programa za cjeloživotno učenje, Mladi na djelu, Tempus,
Erasmus Mundus ili programu Erasmus+?
[Samo jedan odgovor]
a. Da FILTER → idi na PITANJE
1.b.1
b. Ne, podnijeli smo prijavu/e ali sredstva nisu bila dodijeljena FILTER → idi na PITANJE
1.b.2 nakon toga na PITANJE 2.a.
1.b.1. U okviru kojih programa ste Vi osobno i/ili Vaša organizacija prijavljivala i/ili provodila projekte:
[Moguće je odabrati više odgovora.]
a. Program za cjeloživotno učenje (Life Long Learning Programme (LLP) uključuje potprograme
Erasmus, Leonardo da Vinci, Comenius i Grundtvig
b. Mladi na djelu
c. ERASMUS+
d. ostalim programima međunarodne suradnje (TEMPUS, ERASMUS MUNDUS)
e.
1.b.2. U okviru kojih programa ste Vi osobno i/ili Vaša organizacija prijavljivala i/ili provodila projekte:
[Moguće je odabrati više odgovora.]
a. Program za cjeloživotno učenje (Life Long Learning Programme (LLP) uključuje potprograme
Erasmus, Leonardo da Vinci, Comenius i Grundtvig
b. Mladi na djelu
c. ERASMUS+
d. ostalim programima međunarodne suradnje (TEMPUS, ERASMUS MUNDUS)
1.c. Koja je bila Vaša uloga u projektu/ima koje je Vaša organizacija provodila?
[Samo jedan odgovor]
a. Projektni voditelj / osoba koja je kontaktirala s Agencijom za mobilnost i programe EU
(AMPEU)
b. Zakonski predstavnik organizacije
c. Projektni voditelj / osoba koja je kontaktirala s Agencijom za mobilnost i programe EU
(AMPEU) i zakonski predstavnik organizacije
1.d. U okviru kojih programa/sektorskih područja/ključnih aktivnosti je Vaša ustanova koristila financijsku
potporu AMPEU?
[Moguće je odabrati više odgovora.]
[Odgovaraju Zakonski predstavnici organizacije – FILTER: 1.c. samo koji su odgovorili pod b i c]
1. Pogram za cjeloživotno učenje - ERASMUS
2. Pogram za cjeloživotno učenje - COMENIUS
3. Pogram za cjeloživotno učenje - GRUNDTVIG
4. Pogram za cjeloživotno učenje - LEONARDO DA VINCI
5. Pogram za cjeloživotno učenje - TRANSVERZALNE AKTIVNOSTI (studijski posjeti)
6. Mladi na djelu
7. Erasmus+: Comenius - aktivnosti Programa koje su isključivo povezane s područjem školskog
obrazovanja
119
8. Erasmus+: Erasmus za aktivnosti Programa koje su isključivo povezane s područjem visokog
obrazovanja
9. Erasmus+: Erasmus Mundus - združeni studiji
10. Erasmus+: Leonardo da Vinci - za aktivnosti Programa koje su isključivo povezane s
područjem strukovnog obrazovanja i osposobljavanja
11. Erasmus+: Grundtvig - za aktivnosti Programa koje su isključivo povezane s područjem
obrazovanja odraslih
12. Erasmus+: Mladi na djelu - za aktivnosti Programa koje su isključivo povezane s područjem
neformalnog i informalnog učenja mladih
13. Erasmus+: Jean Monnet - za aktivnosti Programa koje su isključivo povezane s područjem
studija o Europskoj uniji
14. Erasmus+: Sport - za aktivnosti Programa koje su isključivo povezane s područjem sporta.
1.e. Molimo Vas da označite u kojim aktivnostima u sklopu Erasmus+ programa je Vaša organizacija
sudjelovala? [Moguće je odabrati više odgovora.]
[Odgovaraju – FILTER: 1.d. samo koji su odgovorili od 7-14]
1. ERASMUS+ - Ključna aktivnost 1
2. ERASMUS+ - Ključna aktivnost 2
3. ERASMUS+ - Ključna aktivnost 3
1.f. U okviru kojih programa/sektorskih područja/ključnih aktivnosti ste vi osobno sudjelovali?
[Moguće je odabrati više odgovora.]
[Odgovaraju Projektni voditelj / osoba koja je kontaktirala s Agencijom za mobilnost i programe EU (AMPEU)
– FILTER: 1.c. samo koji su odgovorili pod a i c]
1. Pogram za cjeloživotno učenje - ERASMUS
2. Pogram za cjeloživotno učenje - COMENIUS
3. Pogram za cjeloživotno učenje - GRUNDTVIG
4. Pogram za cjeloživotno učenje - LEONARDO DA VINCI
5. Pogram za cjeloživotno učenje - TRANSVERZALNE AKTIVNOSTI (studijski posjeti)
6. Mladi na djelu
7. Erasmus+: Comenius - aktivnosti Programa koje su isključivo povezane s područjem školskog
obrazovanja
8. Erasmus+: Erasmus za aktivnosti Programa koje su isključivo povezane s područjem visokog
obrazovanja
9. Erasmus+: Erasmus Mundus - združeni studiji
10. Erasmus+: Leonardo da Vinci - za aktivnosti Programa koje su isključivo povezane s
područjem strukovnog obrazovanja i osposobljavanja
11. Erasmus+: Grundtvig - za aktivnosti Programa koje su isključivo povezane s područjem
obrazovanja odraslih
12. Erasmus+: Mladi na djelu - za aktivnosti Programa koje su isključivo povezane s područjem
neformalnog i informalnog učenja mladih
13. Erasmus+: Jean Monnet - za aktivnosti Programa koje su isključivo povezane s područjem
studija o Europskoj uniji
14. Erasmus+: Sport - za aktivnosti Programa koje su isključivo povezane s područjem sporta.
1.g. Molimo Vas da označite u kojim aktivnostima u sklopu Erasmus+ programa ste Vi osobno sudjelovali?
[Moguće je odabrati više odgovora.]
[Odgovaraju – FILTER: 1.f. samo koji su odgovorili od 7-14]
1. ERASMUS+ - Ključna aktivnost 1
2. ERASMUS+ - Ključna aktivnost 2
120
3. ERASMUS+ - Ključna aktivnost 3
1.h. Koliko ste puta Vi osobno vodili/administrirali projekte za koje ste dobili potporu od AMPEU-a?
[Samo jedan odgovor]
[Odgovaraju Projektni voditelj / osoba koja je kontaktirala s Agencijom za mobilnost i programe EU (AMPEU)
– FILTER: 1.c. samo koji su odgovorili pod a i c]
a. jedanput
b. dvaput
c. triput
d. četiri puta
e. pet i više puta
1.i. Tijekom vođenja/administriranja projekta u sklopu ERASMUS+ programa jeste li sudjelovali u sljedećim
aktivnostima:
[Samo jedan odgovor]
[Odgovaraju Projektni voditelj / osoba koja je kontaktirala s Agencijom za mobilnost i
programe EU (AMPEU) – FILTER: 1.c. samo koji su odgovorili pod a i c]
DA NE
Sudjelovao/la sam u prijavi projekta
(Su)organizirao/la sam sve ili neke projektne aktivnosti
Sudjelovao/la sam u izvještavanju o projektima
1.j. Jeste li vi osobno proveli razdoblje mobilnosti uz potporu dodijeljenu od strane AMPEU-a?
[Definicija mobilnosti: Boravak u drugoj zemlji sudionici Programa radi pohađanja nastave, stjecanja radnog
iskustva, provedbe ostalih aktivnosti vezanih uz učenje, podučavanje ili osposobljavanje te srodne
administrativne aktivnosti. U mobilnost je uključena i virtualna mobilnost, odnosno niz aktivnosti
potpomognutih informacijskom i komunikacijskom tehnologijom, uključujući e-učenje, kojima se ostvaruju ili
olakšavaju međunarodna, suradnička iskustva u kontekstu poučavanja, osposobljavanja ili učenja.]
[Samo jedan odgovor]
[Odgovaraju Projektni voditelj / osoba koja je kontaktirala s Agencijom za mobilnost i programe EU (AMPEU)
– FILTER: 1.c. samo koji su odgovorili pod a i c]
a. Da
b. Ne
1.k. Jeste li vi osobno ostvarili mobilnosti uz potporu dodijeljenu od strane AMPEU-a?
[Samo jedan odgovor]
[Odgovaraju Projektni voditelj / osoba koja je kontaktirala s Agencijom za mobilnost i programe EU (AMPEU)
– FILTER: 1.j. samo koji su odgovorili pod a]
a. Fizička mobilnost
b. Virtualna mobilnost
c. Kombinirana mobilnost
121
2. REALIZACIJA CILJEVA PROGRAMA ERASMUS+
2.a. ERASMUS+ najveći je program Europske unije za obrazovanje, osposobljavanje, mlade i sport te kao takav
sadrži određene ciljeve. U nastavku se nalazi popis ciljeva ERASMUS+ programa.
Na temelju Vaše upoznatosti s ERASMUS+ programom, molimo Vas da procijenite koliko po Vama Erasmus+
omogućuje ostvarenje sljedećih ciljeva:
OBRAZOVANJE I OSPOSOBLJAVANJE
[Samo jedan odgovor]
[Odgovaraju SVI]
1
NE
omogućuje
2
Niti
omogućuje
niti ne
omogućuje
3
Omogućuje
9
Ne znam /
Ne mogu
procijeniti
a. poboljšanje ključnih kompetencija i vještina
općenito („ključne kompetencije“ = osnovni
skup znanja, vještina i stavova koje svi
pojedinci trebaju za osobno ispunjenje i razvoj,
aktivno građanstvo, socijalnu uključenost i
zapošljavanje)
b. poboljšanje ključnih kompetencija i vještina s
obzirom na potrebe tržišta rada
c. doprinos društvenoj koheziji (društvenom
povezivanju)
d. stvaranje prilika za mobilnost u svrhu
formalnog/neformalnog/informalnog
obrazovanja
e. jačanje suradnje između svijeta
obrazovanja/osposobljavanja i svijeta rada
f. na razini ustanova za obrazovanje i
osposobljavanje omogućuje poticanje
poboljšavanja kvalitete, izvrsnosti u
inovacijama i internacionalizacije
g. promicanje nastanka europskog prostora
cjeloživotnog učenja i podizanja svijesti o tom
prostoru
h. unaprijediti međunarodnu dimenziju
obrazovanja i osposobljavanja
i. ostvarivanje suradnje ustanova EU u području
strukovnog obrazovanja i osposobljavanja
j. ostvarivanje suradnje ustanova EU u području
visokog obrazovanja
k. jačanje privlačnosti europskih ustanova
visokog obrazovanja
l. poboljšavanje podučavanja i učenja jezika te
poticanje široke jezične raznolikosti u Uniji i
međukulturalne osviještenosti
MLADI
a. poboljšanje ključnih kompetencija i vještina
mladih („ključne kompetencije“ = osnovni
skup znanja, vještina i stavova koje svi
122
pojedinci trebaju za osobno ispunjenje i razvoj,
aktivno građanstvo, socijalnu uključenost i
zapošljavanje)
b. uključivanje mladih koji imaju manje prilika
odnosno nailaze na neke prepreke koje im
onemogućavaju aktivan pristup obrazovanju,
osposobljavanju i radu.
c. promicanje sudjelovanja u demokratskom
životu u Europi
d. promicanje sudjelovanja na tržištu rada
e. aktivno građanstvo
f. međukulturni dijalog
g. društvenu uključenost
h. ostvarivanje solidarnosti
i. povećavanje prilika za obrazovnu mobilnost
mladih, za osobe koje rade s mladima i/ili
organizacijama mladih i za mlade lidere
j. jačanje veza između područja povezanih s
mladima i tržišta rada
k. suradnju između organizacija u području
povezanih s mladima i/ili drugih
zainteresiranih strana
l. nadopunjavanje reformske politike na lokalnoj,
regionalnoj i nacionalnoj razini
m. razvoj znanja i politika za mlade temeljeno na
dokazima (evidence based)
n. priznavanje neformalnog i informalnog učenja
o. jačanje međunarodne dimenzije aktivnosti
mladih
p. jačanje uloge socio-pedagoških djelatnika i
organizacija kao strukture potpore za mlade
123
3. RELEVANTNOST CILJEVA ERASMUS+ PROGRAMA
3.a. Na temelju Vašeg poznavanja ERASMUS+ programa i ciljeva koje ste upravo mogli pročitati u prethodnim
pitanjima, molimo Vas da procijenite u kojoj mjeri oni odgovaraju potrebama Vaše organizacije i područja u
kojem Vi i Vaša ustanova djelujete:
[Samo jedan odgovor]
[Odgovaraju SVI]
1
Cilj je
relevantan za
moj sektor
2
Cilj nije
relevantan za
moj sektor
3
Ne znam / Ne
mogu
procijeniti
a. poboljšanje ključnih kompetencija i vještina općenito
(„ključne kompetencije“ = osnovni skup znanja,
vještina i stavova koje svi pojedinci trebaju za osobno
ispunjenje i razvoj, aktivno građanstvo, socijalnu
uključenost i zapošljavanje)
b. poboljšanje ključnih kompetencija i vještina s obzirom
na potrebe tržišta rada
c. doprinos društvenoj koheziji (društvenom povezivanju)
d. stvaranje prilika za mobilnost u svrhu
formalnog/neformalnog/informalnog obrazovanja
e. jačanje suradnje između svijeta
obrazovanja/osposobljavanja i svijeta rada
f. na razini ustanova za obrazovanje i osposobljavanje
omogućuje poticanje poboljšavanja kvalitete, izvrsnosti
u inovacijama i internacionalizacije
g. promicanje nastanka europskog prostora cjeloživotnog
učenja i podizanja svijesti o tom prostoru
h. unaprijediti međunarodnu dimenziju obrazovanja i
osposobljavanja
i. ostvarivanje suradnje ustanova EU u području
strukovnog obrazovanja i osposobljavanja
j. ostvarivanje suradnje ustanova EU u području visokog
obrazovanja
k. jačanje privlačnosti europskih ustanova visokog
obrazovanja
l. poboljšavanje podučavanja i učenja jezika te poticanje
široke jezične raznolikosti u Uniji i međukulturalne
osviještenosti
q. poboljšanje ključnih kompetencija i vještina mladih
(„ključne kompetencije“ = osnovni skup znanja,
vještina i stavova koje svi pojedinci trebaju za osobno
ispunjenje i razvoj, aktivno građanstvo, socijalnu
uključenost i zapošljavanje)
r. uključivanje mladih koji imaju manje prilika odnosno
nailaze na neke prepreke koje im onemogućavaju
aktivan pristup obrazovanju, osposobljavanju i radu.
s. promicanje sudjelovanja u demokratskom životu u
Europi
t. promicanje sudjelovanja na tržištu rada
124
u. aktivno građanstvo
v. međukulturni dijalog
w. društvenu uključenost
x. ostvarivanje solidarnosti
y. povećavanje prilika za obrazovnu mobilnost mladih, za
osobe koje rade s mladima i/ili organizacijama mladih i
za mlade lidere
z. jačanje veza između područja povezanih s mladima i
tržišta rada
aa. suradnju između organizacija u području povezanih s
mladima i/ili drugih zainteresiranih strana
bb. nadopunjavanje reformske politike na lokalnoj,
regionalnoj i nacionalnoj razini
cc. razvoj znanja i politika za mlade temeljeno na
dokazima (evidence based)
dd. priznavanje neformalnog i informalnog učenja
ee. jačanje međunarodne dimenzije aktivnosti mladih
ff. jačanje uloge socio-pedagoških djelatnika i
organizacija kao strukture potpore za mlade
3.b. Koje ciljeve bi ERASMUS+ program ili njegov nasljednik trebao imati u narednom razdoblju? [Odgovaraju
SVI]
____________________________________________________________ (OTVORENI ODGOVOR)
125
4. UČINKOVITOST PROGRAMSKIH AKTIVNOSTI U OSTVARENJU CILJEVA
4.a. Ciljevi ERASMUS+ programa trebaju se ostvariti kroz aktivnosti. ERASMUS+ program je strukturiran
prema sljedećim aktivnostima:
a) Ključna aktivnost 1 – Mobilnost u svrhu učenja za pojedince
b) Ključne aktivnosti 2 - Suradnja za inovacije i razmjenu dobre prakse, omogućena je suradnja među
ustanovama kroz aktivnosti Strateških partnerstava
c) Ključne aktivnosti 3 - Podrška reformi politika
Temelje informacija koje imate o prve tri godine prvedbe programa, molimo Vas procijenite pridonose li neke
ključne aktivnosti realizaciji programskih ciljeva u većoj mjeri nego druge. Molimo Vas navedite procjenu za
Ključnu aktivnost 1 (Mobilnost u svrhu učenja za pojedince), Ključnu aktivnost 2 (Suradnja za inovacije i
razmjenu dobre prakse, omogućena je suradnja među ustanovama kroz aktivnosti Strateških partnerstava) i
Ključnu aktivnost 3 (Podrška reformi politika).
Molimo izaberite odgovarajući odgovor za svaku stavku.
[Samo jedan odgovor]
[Odgovaraju – FILTER: 1.a. samo koji su
odgovorili a; 1.b.2. odgovor c]
1
U manjoj
mjeri
2
Osrednje
3
U većoj mjeri
Ne znam / Ne
mogu
procijeniti
a. Ključne aktivnosti 1 - Mobilnost u svrhu
učenja za pojedince
b. Ključne aktivnosti 2 - Suradnja za
inovacije i razmjenu dobre prakse,
omogućena je suradnja među
ustanovama kroz aktivnosti Strateških
partnerstava
c. Ključne aktivnosti 3 - Podrška reformi
politika
126
5. PERCEPCIJA DJELOTVORNOSTI INTEGRIRANOG PROGRAMA ERASMUS+
5.a. Program ERASMUS+ zamijenio je programe kao što su Program za cjeloživotno učenje (s
potprogramima Erasmus, Leonardo da Vinci, Comenius i Grundtvig) i program Mladi na djelu.
Niže navedene tvrdnje odnose se na djelotvornost integriranog programa ERASMUS+. Molimo da na
ljestvici od 1 do 3 označite u kojoj mjeri se slažete sa navedenim tvrdnjama o programu ERASMUS+, pri
čemu 1 – U manjoj mjeri a 3- U većoj mjeri.
[Odgovaraju projektni voditelji Erasmus+ /Programa za cjeloživotno učenje, Mladi na djelu programa.]
[Odgovaraju – FILTER: 1.a. koji su odgovorili a / 1.c. dogovor NE / 1.b.2 odgovor DA]
[Samo jedan odgovor]
U odnosu na prethodnu generaciju programa kao što su
Program za cjeloživotno učenje ili Mladi na djelu
ERASMUS+ je…
1
U manjoj
mjeri
2
Osrednje
3
U većoj
mjeri
Ne znam / Ne
mogu procijeniti
a) Standardizacijom administrativnih pravila i
dokumenata olakšao upravljanje projektom.
b) Pojednostavio financijsko upravljanje
projektnim budžetom.
c) Iznose financijskih potpora u većoj mjeri prilagodio
stvarnim potrebama korisničkih organizacija.
d) ERASMUS+ je poboljšao prilike za međusektorsku
suradnju
5.b. Na temelju Vašeg iskustva s ERASMUS+ programom molimo Vas da procijenite u kojoj mjeri se slažete sa
sljedećim tvrdnjama. Molimo da na ljestvici od 1 do 3 označite u kojoj mjeri se slažete sa navedenim tvrdnjama
o programu ERASMUS+, pri čemu 1 - NE slažem se, a 3- Slažem se.
[Odgovaraju projektni voditelji Erasmus+ /Programa za cjeloživotno učenje, Mladi na djelu programa.]
[Odgovaraju – FILTER: 1.a. koji su odgovorili a / 1.c. dogovor NE / 1.b.2 odgovor DA]
[Samo jedan odgovor]
1
Ne slažem se
2
Niti se
slažem niti
se ne slažem
3
Slažem se
Ne znam / Ne
mogu
procijeniti
a) Procedure prijave su jasne i prilagođene
korisnicima.
b) Kriteriji vrednovanja projekata i njihova
primjena su transparentni.
c) Pravila za upravljanje financijskim
sredstvima projekta (računovodstvena
pravila, opravdanost troškova, uvjeti
isplate) su jasna.
d) Vrijeme za provedbu projekta je u skladu
s realnim mogućnostima korisnika.
e) Vrijeme za izvještavanje o rezultatima
projekta je primjereno.
127
6. DISEMINACIJA PROGRAMSKIH REZULTATA
6.a. Niže navedene tvrdnje odnose se na širenje i korištenje informacija vezanih za program ERASMUS+.
Molimo naznačite kako biste ocijenili sljedeće aspekte vezane uz širenje informacija vezanih za ERASMUS+
program, pri čemu je 1 – loše, a 5 – izvrsno.
[Samo jedan odgovor]
6.b. Molimo Vas da nam napišete svoje ideje oko širenja i korištenja informacija vezanih za ERASMUS+
program:
(OTVORENI ODGOVOR)
6.c. U kojoj se mjeri se susrećete sa sljedećim preprekama za sudjelovanje u projektima programa ERASMUS+?
[Samo jedan odgovor]
[Odgovaraju – FILTER: 1.a. samo koji su odgovorili
od a]
1
loše
2
dovoljno
3
dobro
4
vrlo
dobro
5
izvrsno
Ne mogu
procijeniti
Cjelokupna vidljivost rezultata programa Erasmus+ na
nacionalnoj razini
Promocija primjera dobre prakse od strane Agencije za
mobilnost i programe EU
Regionalna rasprostranjenost informativno-
promotivnih aktivnosti Agencije za mobilnosti i
programe EU
Dostupnost promotivnih materijala sa primjerima
dobre prakse.
Vidljivost rezultata programa u Vašem području rada
1
Uopće
ne
2
Većinom
ne
3
Osrednje
4
Većinom
da
5
Izrazito
da
Ne mogu
procijeniti
Premali interes zaposlenika za sudjelovanje u
projektima
Nedovoljno znanje stranih jezika
Nepriznavanje sudjelovanja na projektima u
svrhu profesionalnog napredovanja
Nemogućnost dobivanja dopuštenja za
odlazak na mobilnost
Nemogućnost pronalaska zamjene
zaposlenicima koji bi htjeli ići na mobilnosti
Nedovoljna informiranost zaposlenika o
mogućnostima odlaska na mobilnost
128
6.d. Postoje li još neke prepreke ili problemi s kojima ste se susretali prilikom prijave ili provedbe projekata u
okviru programa ERASMUS+, a koje nisu navedene u ovom upitniku? Molimo Vas navedite:
(OTVORENI ODGOVOR)
7. ADEKVATNOST IT ALATA
7.a. Niže navedene tvrdnje odnose se na adekvatnost informacijsko komunikacijskih alata (poput Moblity
Tool+, URF, VALOR-Dissemination Platform [diseminacijska platforma], Mrežna jezična potpora [Online
Linguistic Support - OLS]) koji se koriste za administriranje i praćenje rezultata programa ERASMUS+.
Molimo da označite u kojoj mjeri se slažete sa navedenim tvrdnjama za svaki od navedenih informacijsko
komunikacijskih alata, pri čemu 1 - Da, a 2- Ne.
[Odgovaraju – FILTER: 1.a. samo koji su odgovorili od a / 1.b.2. odgovor Da]
[Moguće je odabrati više odgovora.]
Moblity Tool+ Da Ne Ne mogu procijeniti
IT alati je prilagođen informatičkom znanju korisnika.
IT alat je potpuno funkcionalan
Postoje jasne smjernice za korištenje IT alata
URF Da Ne Ne mogu procijeniti
IT alati je prilagođen informatičkom znanju korisnika.
IT alat je potpuno funkcionalan
Postoje jasne smjernice za korištenje IT alata
Preveliko radno opterećenje
Izostanak vrednovanja rada na
međunarodnim projektima unutar moje
institucije
Preslaba podrška partnerskih institucija u
inozemstvu
Nedostatak financijskih sredstava institucije
za pokrivanje vlastitih troškova vezanih za
sudjelovanje u projektima
Nedostatak znanja administrativnog osoblja
za provedbu projekata
Sudjelovanje u projektima zahtijeva previše
administrativnih, ljudskih i financijskih
resursa
Administrativne formalnosti povezane sa
sudjelovanjem u projektima odvraćaju
zaposlenike od uključivanja u mobilnost
Zbog specifičnosti naših programa vrlo nam
je teško pronaći partnerske ustanove sa
sličnim programima
129
VALOR-Dissemination Platform [diseminacijska platforma] Da Ne Ne mogu procijeniti
IT alati je prilagođen informatičkom znanju korisnika.
IT alat je potpuno funkcionalan
Postoje jasne smjernice za korištenje IT alata
Mrežna jezična potpora [Online Linguistic Support - OLS] Da Ne Ne mogu procijeniti
IT alati je prilagođen informatičkom znanju korisnika.
IT alat je potpuno funkcionalan
Postoje jasne smjernice za korištenje IT alata
7.b. Jeste li se susreli sa poteškoćama prilikom korištenja informacijsko komunikacijskih alata programa
Erasmus+ (npr. Mobility Tool+, Valor, URF, Mrežna jezična potpora - OLS)? Molimo
obrazložite._____________________________________(OTVORENO)
8. ADMINISTRATIVNI, FINANCIJSKI I LJUDSKI KAPACITETI ZA IMPLEMENTACIJU
PROGRAMA
8.a. Molimo procijenite administrativne, financijske i ljudske kapacitete u Vašoj organizaciji za provedbu
projekata u okviru Erasmus+ programa.
[Samo jedan odgovor]
[Odgovaraju SVI] Nedovoljni
kapaciteti
Dovoljni
kapaciteti
Ne mogu procijeniti
a. Znanje i vještine zaposlenika vezano uz
administrativno praćenje projekta (priprema dopisa,
projektnog prijedloga i pripadajuće dokumentacije).
b. Praktično iskustvo upravljanja projektima.
c. Znanje stranih jezika u usmenom i pismenom obliku.
d. Upravljačke vještine rukovoditelja
e. Suradnja i komunikacija u organizaciji.
f. Korištenje informacijsko-komunikacijskih alata.
g. Razvijeni partnerski odnosi s ustanovama iz
inozemstva
h. Raspoloživa financijska sredstva za predfinanciranje
provedbe projektnih aktivnosti.
i. Raspoloživo vrijeme djelatnika za sudjelovanje u
projektima.
j. Stručna znanja računovodstvenih djelatnika potrebna
za financijsko praćenje EU projekata.
130
9. ORGANIZACIJSKE I SOCIO DEMOGRAFSKE KARAKTERISTIKE
9.a. Kakvog je tipa organizacija u ime koje ste se prijavljivali za financijsku potporu?
a. tijelo državne uprave
b. javna ustanova
c. nevladina organizacija
d. profitna organizacija
e. neformalna grupa mladih
f. Ostalo
9.b. Vaša dob? Molimo Vas da upišete godinu rođenja_____________
9.c. Kojeg ste spola?
a. Ženski
b. Muški
9.d. Koji je Vaš najviši završeni stupanj obrazovanja?
a. Bez završene osnovne škole
b. Završena osnovna škola
c. Završena trogodišnja strukovna škola (škola za industrijska, obrtnička, zanatska zanimanja,
ŠUP – Škola učenika u privredi)
d. Završena četverogodišnja strukovna škola (tehnička, ekonomska medicinska, umjetnička itd.)
e. Završena gimnazija
f. Završeno petogodišnje strukovno srednjoškolsko obrazovanje, majstorski ispit,
g. Završena viša škola u trajanju od dvije godine, završen dvogodišnji stručni studij (viša škola,
visoka škola, veleučilište) (stručni pristupnik)
h. Završena prva razina visokog obrazovanja (sveučilišni studij, visoka škola, veleučilište)
(prvostupnik)
i. Završena druga razina visokog obrazovanja ili dodiplomski četverogodišnji studij ili
integrirani preddiplomski i diplomski studij (sveučilišni studij, visoka škola, veleučilište)
(magistar struke, stručni specijalist,)
j. Završen poslijediplomski specijalistički studij (sveučilišni specijalist)
k. Završen znanstveni magistarski studij (akademski stupanj magistar znanosti - mr. sc.)
l. Završen poslijediplomski doktorski studij (akademski stupanj doktor znanosti - dr. sc.)
m. Bez odgovora
9.e. U kojoj županiji se nalazi organizacija u kojoj ste zaposleni?
I ZAGREBAČKA
II KRAPINSKO-ZAGORSKA
III SISAČKO-MOSLAVAČKA
IV KARLOVAČKA
V VARAŽDINSKA
VI KOPRIVNIČKO-KRIŽEVAČKA
VII BJELOVARSKO-BILOGORSKA
VIII PRIMORSKO-GORANSKA
IX LIČKO-SENJSKA
X VIROVITIČKO-PODRAVSKA
XI POŽEŠKO-SLAVONSKA
XII BRODSKO-POSAVSKA
XIII ZADARSKA
XIV OSJEČKO-BARANJSKA
131
XV ŠIBENSKO-KNINSKA
XVI VUKOVARSKO-SRIJEMSKA
XVII SPLITSKO-DALMATINSKA
XVIII ISTARSKA
XIX DUBROVAČKO-NERETVANSKA
XX MEĐIMURSKA
XXI GRAD ZAGREB
9.f. U kojem području ste zaposleni:
a. Predškolski odgoj i osnovnoškolsko obrazovanje
b. Srednjoškolsko obrazovanje
c. Visoko obrazovanje
d. Znanost
e. Mladi
f. Ostalo
132
Appendix VII: Interview Protocol [in Croatian]
VODIČ POLUSTRUKTURIRANI INTERVJU: ERASMUS+ EVALUACIJA
Predstavljanje istraživačice. Informirani pristanak.
1. Uvodni dio
Molim Vas da mi se ukratko predstavite, recite koja je Vaša trenutna funkcija u Agenciji?
Na koje ste sve načine trenutno povezani s provođenjem programa Erasmus+? Što je sve u
Vašoj nadležnosti? Jeste li na neki način sudjelovali u provođenju Programa za cjeloživotno
učenje i Mladi na djelu na europskoj i nacionalnoj razini?
Kako ste kroz proteklo razdoblje bili povezani? Što je sve bilo u Vašoj nadležnosti?
Ukratko, ako biste trebali objasniti u par rečenica, u cijeloj hijerarhiji zaduženja povezanih s
programom Erasmus+ u Republici Hrvatskoj, koja je funkcija Agencije, a koja (bila) Vas
osobno?
Imate li Vi osobno dužnosti povezane s nekim drugim programima – projektima EU?
Navedite molim neke, kao primjer.
Dosta ćemo razgovarati o učinkovitosti programa Erasmus + i njegovih ključnih aktivnosti,
bilo bi dobro kada bismo odmah na početku razjasnili što za vas znači da je program
učinkovit?
Pritom pod učinkovitosti smatramo: (engl. effectiveness) ocjenu odnosa između postignutog
rezultata i postavljenog cilja. Programske aktivnosti smatraju se učinkovitim ako njihovi
rezultati pridonose realizaciji postavljenih ciljeva. Pod djelotvornosti (engl. efficiency)
smatramo cijenu postignutog rezultata – koristi se naročito za uspoređivanje dvije ili više
mjera/aktivnosti ili metoda u rješavanju problema.
2. Suradnja i podjela zadataka među tijelima i organizacijama zaduženim za provođenje
programa Erasmus+
Provođenje programa Erasmus + ovisi o sustavu suradnje i podjele zadataka među mnoštvom
tijela i organizacija, Europske komisije, Izvršne agencije, nacionalnih agencija, nacionalnih
tijela (u HR to su MZO i MDOMSP), neovisnih tijela za reviziju i Erasmus+ odbora. Recite
kako su raspodijeljene odgovornosti između dvaju ministarstava i Agencije za mobilnost i
programe EU?
Kada razmišljamo o provođenju programa u Hrvatskoj, čini li vam se da taj sustav dobro
funkcionira? Što bi se u tome sustavu u budućnosti moglo poboljšati? Na koji način? Koje bi
promjene u tom sustavu mogle pojednostavniti ili čak poboljšati implementaciju programa?
3. Doprinos provođenja programa posebnim i osnovnim ciljevima programa Erasmus+ na
razini RH.
133
Pogledajmo prvo zajedno posebne Ciljeve koje je program Erasmus + imao na razini naše
države, a koji se odnose na Obrazovanje i osposobljavanje te Mlade i sport. Recite, prema
vašim saznanjima i cjelokupnom viđenju situacije, u kojoj su mjeri program Erasmus+ i
programi prethodnici bili uspješni u realizaciji posebnih ciljeva programa Erasmus+ ? Možete
li potkrijepiti nekim primjerima? Kako je bilo zamišljeno posebni ciljevi trebali su se
ostvarivati kroz tri razine, individualnu (osobna mobilnost pojedinaca), institucionalnu (razvoj
institucionalnih kapaciteta) i policy razinu (razvoj pravila, procedura i politika), što biste rekli
kroz koju su razinu ciljevi najviše ostvareni i kako?
Recite, smatrate li da postoje razlike po područjima (osnovnoškolsko obrazovanje, strukovno,
visoko, obrazovanje odraslih) u ispunjavanju posebnih ciljeva, ukoliko da, na koji način i što
mislite zašto (što su uzroci) da neka područja imaju više uspjeha od drugih? Da li se to moglo
izbjeći? Na koji način?
Erasmus+ ima i neke zajedničke opće ciljeve, koji se uglavnom odnose na ispunjavanje
ciljeva strategije Europa 2020. u području obrazovanja, ciljeve strateškog okvira za europsku
suradnju u obrazovanju i osposobljavanju („ET 2020.”), ukupne ciljeve obnovljenog okvira za
europsku suradnju u području mladih (2010. - 2018.), cilju razvijanja europske dimenzije u
sportu, posebno sportu na lokalnoj razini, u skladu s programom rada Unije za sport; te
promicanju europskih vrijednosti u skladu s člankom 2. Ugovora o Europskoj uniji. Kakav je
doprinos programa E+ i prethodnika u realizaciji tih strateških (EU ciljeva)? Što mislite u
kojoj mjeri je ispunjavanje posebnih ciljeva o kojima smo maloprije razgovarali doprinio
realizaciji glavnih ciljeva programa Erasmus+? Pokušajte procijeniti u pojedinim domenama
osnovnih ciljeva, realizacija kojih posebnih ciljeva je najviše pomogla.
U kojoj mjeri su ključne aktivnosti (mobilnost pojedinaca (1), Suradnja za inovacije i
razmjenu dobre prakse (2) te Podrška reformi politika (3)) programa Erasmus utjecale na
razvoj politike u domeni obrazovanja i osposobljavanja, mladih i sporta kod nas? Recite
pobliže, koja od aktivnosti vam se čini da je bila najutjecajnija? A koja je imala najmanji
utjecaj? Koje su aktivnosti bile najučinkovitije u tome? Postoje li razlike između područja
(osnovnoškolsko, strukovno, visoko)? Objasnite.
S posebnim i općim ciljevima nastoje se riješiti neke specifične potrebe i probleme, u kojoj
mjeri su to i Hrvatske potrebe i problemi? Jesu li te potrebe i problemi još uvijek relevantni u
Hrvatskoj? Bi li program koji će naslijediti Erasmus + trebalo u ciljevima nekako prilagoditi
da bude primjereniji i hrvatskim potrebama?
Kada razmišljamo o tome kako je u proteklom razdoblju program Erasmus + kako ste
zadovoljni informiranjem i diseminacijom informacija o programu? Smatrate li da je trud u
tom području učinjen od strane 'Eramus tima' doprinio učinkovitosti u postizanju posebnih
ciljeva? Na koji način?
4. Učinkovitost pojedinih aktivnosti programa Erasmus+
Porazgovarajmo kratko u učinkovitosti pojedinih aktivnosti programa. Pod učinkovitosti
smatramo: (engl. effectiveness) ocjenu odnosa između postignutog rezultata i postavljenog
cilja. Programske aktivnosti smatraju se učinkovitim ako njihovi rezultati pridonose
realizaciji postavljenih ciljeva. Kada promatrate skup ključnih aktivnosti, mobilnost,
partnerski projekti, policy orijentirani projekti, centralizirane aktivnosti, što biste rekli koje su
aktivnosti bile učinkovitije od ostalih? Koje najmanje učinkovite? Kako se to razlikuje prema
134
područjima (visoko obrazovanje itd.)? Kako biste to objasnili, razlike među učinkovitosti
aktivnosti kao i razlike u učinkovitosti aktivnosti u različitim područjima?
2014. s prelaskom na Erasmus +, više je programa integrirano u jedan zajednički program,
recite kako je ta integracija utjecala na učinkovitost programa u Hrvatskoj? U kojoj je mjeri
integracija više programa ishodila dobicima ili gubicima u djelotvornosti povezanoj s
implementacijom programa u Hrvatskoj? Ponoviti definiciju djelotvornosti. Smatrate li da bi
nešto drugo moglo program učiniti učinkovitijim? A djelotvornijim? Kako bi izgledale
promjene u strukturi programa koje bi ga mogle učiniti učinkovitijim i djelotvornijim?
Opišite.
Samo AMPEU: a) U kojoj su mjeri pristupi i alati korišteni za diseminaciju i iskorištavanje
rezultata programa Erasmus+ i programa prethodnika u vašoj državi bili učinkoviti? b) Gdje
vidite mogućnosti za poboljšanje?
Samo AMPEU i udruge: a) Smatrate li da je implementacija određenih aktivnosti programa
djelotvornija od drugih? b) Postoje li razlike po područjima? Koje su dobre prakse
djelotvornijih aktivnosti programa koje se mogu prenijeti u druge aktivnosti?
Samo AMPEU: a) U kojoj su mjeri povezane različite aktivnosti objedinjene programom
Erasmus+? b) Možete li identificirati neku postojeću ili moguću sinergiju između aktivnosti
unutar programa Erasmus+? c) Možete li identificirati napetosti, nekonzistentnosti ili
preklapanja između aktivnosti unutar programa Erasmus+?
5. Prilagođenost programa potrebama korisnika
U programu sudjeluju različiti akteri iz različitih sektora, što biste rekli u kojoj mjeri su
njihove potrebe adresirane ciljevima programa Erasmus +? Kako se to razlikuje prema
sektorima i korisnicima, odnosno prema trima ključnim skupinama: odgojno-obrazovnim
ustanovama, visokim učilištima i organizacijama civilnog društva? Jesu li neki korisnici bolje
pokriveni u smislu potreba, od nekih drugih? Opišite.
Nije pitanje samo u adresiranju potreba, već vjerujemo i u prepoznavanju programa od strane
potencijalnih korisnika. Kako ste time zadovoljni? Čini li vam se da je program uspio privući
korisnike kojima je namijenjen? Kako se to razlikuje prema područjima, obrazovanju i
osposobljavanju, mladima i sportu? Opišite. Koji faktori su utjecali na to da neki od korisnika
nisu dovoljno prepoznali program? Ima li tome u budućnosti lijeka?
6. Komplementarnost programa drugim programima
U Hrvatskoj se provode i neki drugi međunarodni i nacionalni programi namijenjeni istim
korisnicima i u svrhu rješavanja dijelova istih potreba i problema. Primjerice CEEPUS i
Bilateralne stipendije koje se provode u području visokog obrazovanja. Recite, smatrate li da
su programi koji se provode komplementarni Erasmus + programu, ili tu vidite nekih
mogućih dodatnih poboljšanja? Koje su komplementarnosti, a koje nekonzistentnosti ili
napetosti prisutne?
Erasmus + adresirao je neke potrebe i ima učinke u sektoru visokog obrazovanja u kojima su i
drugi programi doprinijeli i mogli bi i dalje doprinositi, smatrate li da postoji sinergija u tom
doprinosu ili je taj doprinos nezavisan? Što se tiče drugih sektora, smatrate li da postoji
135
sinergija između velikih projekata koji će se financirati kroz ESF i programske aktivnosti?
Koliko se Erasmus + uklapa u zajedničke napore u ostvarenju ciljeva europskih obrazovnih
politika koje su nadnacionalne i zapravo služe kao „nadopuna“ ili „dodana vrijednost“
nacionalnih obrazovnih politika? Koja su to područja u kojima program može
komplementirati nacionalne napore/politike? Postoje li područja u kojima se razilaze
nacionalne politike i eu politike (nisu komplementarni)?
7. Financiranje programa Erasmus+ i ljudski resursi
Kada govorimo o godišnjoj i višegodišnjoj alokaciji novčanih sredstava Hrvatskoj, jeste li
zadovoljni iznosom koji dobivamo s obzirom na nacionalni doprinos koji HR izdvaja iz
proračuna za program? Smatrate li da je iznos novčanih sredstava alociran za provođenje
programa Erasmus+ prikladan i proporcionalan za ono što se Programom želi postići?
Objasnite. A kada govorimo o različitim područjima i aktivnostima programa, je li
distribucija sredstava napravljena na način da je postignuta zadovoljavajuća razina
učinkovitosti? Bi li drugačija distribucija bila bolja? Na koji način?
Kada razgovaramo o financijskim i ljudskim resursima dostupnim za implementaciju
Erasmus + programa na razini države, radi li se o adekvatnim resursima? Objasnite (na koji
način da, ili ne). Koliko se djelotvorno upravljalo tim resursima? Jeste li zadovoljni time?
Postoje li neke mjere koje biste predložili kako bi se djelotvornost resursa razmještenih za
implementaciju programa u Hrvatskoj povećala?
Kada razmišljamo o tome kako je u proteklom razdoblju program Erasmus + bio promoviran
u Hrvatskoj, recite koje ste posebne aktivnosti poduzeli u tu svrhu, poput sufinanciranja,
promocije itd. Recite mi više o tome. Smatrate li da je trud u tom području učinjen od strane
'Eramus tima' doprinio učinkovitosti u postizanju posebnih ciljeva? Na koji način?
Predviđeno je povećano financiranje programa Erasmus + u sljedećem razdoblju u Hrvatskoj.
S obzirom na dosadašnja iskustva, smatrate li da će Hrvatska s dosadašnjim kapacitetima,
resursima, podjelom poslova i aktivnostima moći učinkovito i djelotvorno upravljati
povećanim resursima? Smatrate li da će visoka učilišta s obzirom na njihove kapacitete moći
upravljati većim iznosima za mobilnost? Koje biste mjere predložili za povećanje
djelotvornosti i učinkovitosti upravljanja ovim resursima?
8. Izazovi i poteškoće implementacije programa Erasmus+
Jeste li se susreli s poteškoćama tijekom praćenja programa i donošenja strateških odluka o
programu na nacionalnoj razini? Opišite. S čime su te teškoće najviše povezane? Što bi
pomoglo u njihovu otklanjanju? Bi li se moglo u program u budućnosti unijeti neke preinake
koje bi pomogle da se poteškoće i izazovi umanje ili premoste u cjelini? Kroz evaluaciju LLP
detektirale određeni izazovi u provedbi i razvijen je set preporuka. Na koji način planiraju
adresirati te nacionalne izazove? Koliko je program Erasmus + doprinio realizaciji Akcijskog
plana za internacionalizaciju obrazovanja 2015.-2016.? Objasnite.
