European guidelines for validating non-formal and informal learning

Draft (print version 30/April/2015) - for external distribution and comments

1

CHAPTER 1. Introduction

The European Guidelines for validating non-formal and informal learning are written for those individuals and institutions that are responsible for the initiation, development, implementation and operation of validation. These stakeholders operate at different levels (at European, national, sectoral and local) and in different contexts (in public, private and voluntary sectors; in education and training and in labour market services). The ambition of the Guidelines is to clarify the conditions for implementing validation, pointing to the critical choices to be made by stakeholders at different stages of the process.

A first set of European Guidelines for validating non-formal and informal learning were published jointly by the European Commission and Cedefop in 2009. Acknowledging the positive reception of these, the 2012 Council Recommendation on validation of non-formal and informal learning invited the European Commission – in consultation with Member States and stakeholders – to regularly review the Guidelines.

The guidelines are structured according to the themes promoted by the 2012 Recommendation on validation of non-formal and informal learning. For each of the main theme introduced by the Recommendation, detailed guidelines have been developed. The purpose is to clarify some of the choices facing stakeholders when implementing validation arrangements and to point to possible steps which can be taken and the implications of these. When writing the Guidelines it is acknowledged that validation arrangements must be fit for purpose and designed according to the particular context it operates in. The Guidelines will thus not promote single ‘correct solutions’ but strive to identify relevant alternatives for action.

The electronic version of these guidelines (to be presented at Cedefop’s webpage) will contain systematic links to the European inventory on validation of non-formal and informal learning (2010 and 2014 versions) and provides detailed examples related to the issues dealt with by the Guidelines. This will allow the Guidelines and the Inventory to act together as an integrated tool directly supporting the work of policy makers and practitioners.

1.1. THE LINK BETWEEN THE 2012 RECOMMENDATION ON VALIDATING NON-FORMAL AND INFORMAL LEARNING AND THE EUROPEAN GUIDELINES

The 2012 Recommendation on validation provides a strong platform for intensified European cooperation in the field of validation of non-formal and informal learning. Member States, education and training institutions, social partners and other relevant stakeholders have been invited to intensify work in this area and, by 2018, put in place appropriate national arrangements allowing individuals to value and make visible the outcomes of learning at work, at home, during leisure time and in voluntary activities.

The Recommendation identifies a limited number of critical issues, of both political and practical character, which have to be addressed for validation to become fully integrated and accepted at

2

national level. In order to add detail and value to the Recommendation, the revised Guidelines closely mirror these themes. The main themes covered by the Recommendation are the following:

The themes identified by the Recommendation should not be read in isolation from each other but should be seen as building blocks which, when combined, can provide the basis for a coherent approach to validation of non-formal and informal learning. When implementing validation. The following key questions, linked to each theme, are of crucial importance:

Key questions regarding the implementation of validation

Have the purpose (2.1.1) of the validation initiative been clarified? How does the validation initiative respond to the interests of the individual citizen (2.1.2)? Have steps been taken to coordinate and target guidance and counselling services (2.1.3)? Are mechanisms for coordination of relevant stakeholders in place, so as to avoid

fragmentation and ensure a coherent approach (2.1.4)? Are validation arrangements linked to national qualifications frameworks (2.1.5) and how

does this impact transparency and access? Are the outcomes of validation referring to the same or equivalent standards as those used

for formal education (2.1.6) and how does this affect its value and currency? Are validation arrangements linked to quality assurance arrangements (2.1.7) and how does

this influence trust and credibility? Which steps have been taken to strengthen the professional competences of validation

practitioners (2.1.8)? What is the role of validation in education and training systems (2.1.9.1); in relation to the

labour market (2.1.9.2); and in the voluntary sector (2.1.9.3)? What is the relationship between validation arrangements areas and sectors (2.2)? Which tools and instruments can be used (and combined) for identification, documentation

and assessment of learning (2.1.10)?

In the following chapters (3-6) these themes will be discussed in more detail, allowing for the identification of issues to be considered by those involved in developing and implementing validation across Europe.

3

CHAPTER 2. THE BASIC FEATURES OF VALIDATION

In essence, validating non-formal and informal learning is about making visible and valuing the outcomes of the diverse and rich learning that takes place outside formal education and training institutions. A key challenge for validation is to be able to capture this diversity and richness in a reliable, valid and credible way – enabling the individual to make full use of the experiences s/he has made in education, work and in society in general. The following chapter explores some of the pre-conditions for making validation a reality.

2.1. The four stages of validation The 2012 Recommendation identifies four main stages that validation arrangements have to consider and allow for:

‘IDENTIFICATION of an individual's learning outcomes acquired through non-formal and informal learning; DOCUMENTATION of an individual's learning outcomes acquired through non-formal and informal learning; ASSESSMENT of an individual's learning outcomes acquired through non-formal and informal learning; CERTIFICATION of the results of the assessment of an individual's learning outcomes acquired through non-formal and informal learning in the form of a qualification, or credits leading to a qualification, or in another form, as appropriate.’. (point 2 a-d)

The validation process can appear a complex one, sometimes confused by the terminology used. An important purpose of the 2012 Recommendation has been to clarify the main functions being part of validation, pointing to the significance of what has been identified as four distinct stages of validation – identification, documentation, assessment and certification. In some cases validation is only about Identification and documentation of learning, for example when a job-seeker needs to make visible his or her previous experiences in a better way. In other cases, for example when somebody seeks a formal qualification, confirming that their learning meets agreed standards, all these four stages will be required.

Case – validation steps at national level In Iceland, when validating non-formal and informal learning of adults, all four stages must be included. More limited approaches, for example on identification and documentation, may be used in other sectors. This is exemplified by the third sector where the focus tends to be on initial guidance, reflection and recognising and identifying skills, and gathering evidence in portfolios.

The emphasis on the four stages makes it clear that validation is not only and not always about acquiring a formal qualification. While formal qualifications will be important in many instances, a limited identification and documentation may positively influence the self-awareness and self-esteem of an individual; directly improving opportunities in the labour market and in further learning. While the two first stages – identification and documentation - will have a formative character, the two final stages – assessment and certification – are summative in character and will

4

require the involvement of external actors (assessors, certification bodies) offering official recognition of the learning outcomes in question.

The four stages of the validation process requires that practical arrangements are structured and presented in a way which clarifies their different purposes and allows individuals to choose the form of validation which is best suited for their particular needs. In the diagram below (figure 1) the validation process is shown, highlighting that a validation process has many possible outcomes. To put it simple, the further to the right in the figure, the more extensive the process from the viewpoint of the candidate and all professionals involved:

Figure 1: The validation process

When validation is part of a formal qualification process the four stages have to be seen in relation to a relevant qualification standard (see 4.2). If the purpose is (for example) skills audit, limited to the identification and documentation of learning experiences, qualification standards are optional but not obligatory.

In summary, an individual interested only in having his or her knowledge and skills made more visible, or better documented, might decide that a self-assessment is sufficient, and not seek official certification. In these cases the individual can, without the intervention of any external actor, decide to present the outcomes of the process as informal proof of achievements. This is what we may term

5

‘representation’ (1)- the process of combining formal and informal proofs in a way which clarifies and promotes the profile of the candidate.

Key questions regarding the four stages of validation

Clarifying the stages of the process makes it possible to distinguish between the different purposes served by validation (formative and summative), and the following questions provide a starting point for this clarification: Have the different stages of the validation process been clearly defined and communicated to the

individual candidates? How is guidance and counselling integrated in the support of a validation process, specially

identification and documentation? Are candidates informed on how to use the outcomes of validation, for further education and

training and/or employment?

2.1.1. Identification and Documentation Validation necessarily starts with the identification of the knowledge, skills and competences acquired. This initial phase can be seen as a formative phase where the individual becomes increasingly aware of the knowledge, skills and competences s/he has acquired through past experiences. The identification stage is of particular importance outside the education and training context (for example employment services, private companies, the voluntary sector) as these are less likely to opt for a formal qualification. While this identification in some countries is supported by the use of standardised ICT tools allowing self-assessment to take place, this stage will normally require the active involvement of advisers and counsellors able to support the individual and direct him/her to the adequate options and tools. At this initial stage the individual must be made aware of the costs and benefits of validation, as compared to (for example) further education and training. If validation is the preferred option, the appropriate form has to be chosen. While ICT based tools may reach more people and can be cheaper to use, they can be less open to the individual character of learning outcomes and fail to identify and value these particulars. Using interviews and dialogue based approaches in this stage can be more costly but potentially of greater value to the candidate (see also chapter 6).

Documentation will normally follow the stage of identification, but not always, since in some countries it might be done throughout the whole process, as an accompanying measure. Documentation consists of the provision of evidence of the learning outcomes acquired. This is normally carried out through the collection of a portfolio that tends to include a CV and a career history of the individual, with documents (in some cases, where this is relevant, also work samples) that attest to their learning achievements. It is important that this evidence provides appropriate insight into the learning outcomes acquired, simply listing job-titles or positions will not be sufficient. Common templates for the presentation of learning experiences, as demonstrated by Europass, will facilitate transfer and promote a common understanding of the outcomes in question. This portability of evidence is of crucial importance; for validation to support progress within education and towards employment, a certain level of national and European harmonization is needed. The

1 For a detailed discussion on ‘representation’, see: Cedefop (2010), Changing qualifications. http://www.cedefop.europa.eu/en/publications-and-resources/publications/3059

6

development of the classification of European Skills/Competences, qualifications and Occupations (ESCO) exemplifies an approach supporting better and more standardised descriptions of knowledge, skills and competences – promoting transferability and portability. In several countries, the identification and documentation stages are grouped together, viewed as one stage of collecting evidence to build the dossier that will be assessed by an external evaluator.

Key questions regarding identification and documentation

For the identification and documentation stages, the following questions needs to be asked: Can the identification and documentation of non-formal and informal learning be carried out as

a separate and independent stage? Which tools support identification and documentation (ICT-based, dialogue based)? Which formats are used for documenting non-formal informal learning (and how transferable are

these)?

2.1.2. Assessment and Certification Assessment is normally referred to as the stage in which an individual’s learning outcomes are compared against specific standards. This might refer to the evaluation of documentary evidence – as discussed above - but might also involve evaluation of other forms of evidence. The assessment stage is of crucial importance to the overall credibility of validation of non-formal and informal learning. These processes, and their outcomes, are sometimes perceived as inferior to the assessments carried out in formal education and training settings. To counter this scepticism, which also reflects the relative novelty of validation of non-formal and informal learning, assessment tools and the way they are applied and combined must be presented to the candidate - and the general public - in a transparent way.

Case – validation steps at national level In Norway, the stages depend on the needs of the individual and the purpose of the application. It will also depend on the sector – e.g. in post-secondary VET and in HE validation is more commonly used to support access, so there is less focus on certification, whereas assessment / certification is the focus of the VET sector, since the Education Act permits individuals to take a final craft or journeyman’s examination based on practical work experience, rather than solely on education and training in school and/or the apprenticeship which would normally be required. (It is not possible to acquire a full Craft or Journeyman’s Certificate through validation. The candidates must still take the final exam.).

Many of these tools used for assessing non-formal and informal learning will be similar to those used in formal education and training. As validation is about capturing individual learning experiences, assessment tools need to capture what is specific to each learner and the relevant learning context. This is different from formal learning where assessment tools frequently are applied across large cohorts of students and where it can be difficult to adapt methods to the needs of sub-groups or

7

individuals. Tools must for example be able to reflect practical skills or theoretical reflections in varying degrees. Like in formal education, the individual specificity of learning outcomes concerned may require more than one tool, for example a combination of written tests and practical challenges. Tools will frequently need to apply practical demonstrations, simulations or gathering of evidence from past practices.

The final stage of validation is the Certification (or Recognition) of the results of the assessment. This can take different forms, but will normally be linked to the award of a formal qualification. In the context of enterprises or economic sectors, certification may also involve the issuance of a license allowing the individual to carry out specific tasks etc. Whatever the case, a validation process reaching the stage of formal certification requires a summative assessment confirming the achievement of learning outcomes against a specified standard. What is crucial is that this process is managed by a credible authority or organisation. The value – or the currency - of a certificate or qualification acquired through validation largely depends on the legitimacy of the awarding body or authority. The use of summative approaches for validating non-formal and informal learning needs to be strongly linked – preferably integrated – into national qualifications systems (see chapter 4.4). Some countries have chosen to issue separate certificates or qualifications for non-formal and informal learning. While this might be appropriate in some settings, the risk is to create ‘A’ and ‘B’ certificates where certification through validation is seen as inferior.

Key questions regarding assessment and certification

For the assessment and certification stages, the following questions needs to be asked: Are the conditions for assessment and certification clearly defined and communicated to the

candidate? Which tools are used? Which standards are used? What is the formal status (and thus currency) of the standard?

2.2. The Centrality of the individual The 2012 Recommendation underlines that the individual must be at the focus of validation arrangements:

(point 1 a-b) ‘The arrangements for the validation of non-formal and informal learning enable individuals to a) have knowledge, skills and competences which have been acquired through non-formal and informal learning validated, including, where applicable, through open educational resources; (b) obtain a full qualification, or, where applicable, part qualification, on the basis of validated non-formal and informal learning experiences..’.

Validation arrangements aim at enabling the individual to acquire a qualification (or part of it) and/or to make visible their knowledge, skills and competences. Validation aims at empowering the individual and can serve as a tool for providing second chance opportunities to disadvantage individuals. Validation is (normally) voluntary and it is up to the individual to take the first step and decide whether s/he wants to enter into a process of identification, documentation assessment and certification of learning. The individual should be able to take control of the validation process and

8

decide at what stage to end the process. However, the individual has to receive adequate provision of information and guidance through the entire process. Communication strategies about the costs and benefits of validation and explanations on how the system works can motivate the individual to proceed.

Case – entitlements at national level In Denmark, the legislation for VET (2003) entitles individuals to assessment in relation the preparation of personal education plans. The personal education plan is based on an assessment of the trainees’ competences and outlines an individual pathway through the VET system. Since 2007, individuals can gain access to short-cycle and medium-cycle Higher Education (Bachelor-level degrees) based on an individual competence assessment. The key legislation on validation of prior learning in Denmark is Act no. 556 of 6 June 2007

A decision to establish validation of non-formal and informal learning as a normal route to qualifications – parallel to the traditional route going via formal education and training – may require that individuals have a legal right to access and make use of validation, in the same way as they have a right to access and use formal education and training. This might take several forms and will necessarily depend on the legal constrains of the country (Figure 2 below illustrates the situation in 2014 as regards the legal basis of validation).

Figure 2: Legal framework for validation?

The validation arrangement has to comply with standard individual data-protection and privacy rights; confidentiality and trustworthiness are closely connected. For this to happen, and to avoid conflicts of interests, a clear distribution of roles and responsibilities has to be ensured. Those who manage the process of validation must not enter into personal deliberations beyond the point which makes the individual vulnerable or uncomfortable.

9

The results of validation will (normally) remain the property of the individual. In cases where validation is carried out by enterprises (e.g. for recruitment or employee appraisal) this might however not be the case. As the main owner and actor of the validation process, individual’s privacy must be assured, protected and respected.

In summary, equal and fair treatment must be guaranteed across the whole process, ensuring transparent procedures, the avoidance of conflicts of interest, and the possibility for appeal. To this end, division of roles and responsibilities in the four stages of validation needs to be clear: The assessment role should normally be separated from the guidance or training role. This may pose a challenge in some cases as the guidance practitioner might act also as the assessor.

Key questions regarding individuals rights and obligations

The individual is at the focus of validation processes and his or her rights and obligations must be treated with care. The following questions provide a starting point: To what extent is validation an individual right?

o If yes, which are the financial and organisational implications? o If no, which are the financial and organisational implications?

Is the privacy and personal integrity of the candidates protected throughout the validation process?

Have explicit procedures been put in place to guarantee confidentiality? Have ethical standards been developed and applied? Are the outcomes of the process the exclusive property of the candidate?

o If no, which are the implications? Which arrangements have been put in place to guarantee fair and equal treatment?

CHAPTER 3. CONDITIONS FOR DEVELOPING AND IMPLEMENTING VALIDATION

Referring to the themes introduced by the 2012 Recommendation, the following sections seeks to clarify main conditions for introducing high quality and trusted validation arrangements.

3.1. Information, Guidance and counselling The 2012 Recommendation pays particular attention to the role of guidance and counselling in taking forward validation:

(points 3 b and 3 e) Member States should provide within validation arrangements (3 b)’… information and guidance on the benefits of, and opportunities for validation, as well as on the relevant procedures, are available to individuals and organizations, and (3 e) the validation of non-formal and informal learning is supported by appropriate guidance and counselling and is readily accessible.. .

National experience indicates that potential candidates and those in the process of receiving validation should have access to impartial, comprehensive and informed advice. As it is not possible to establish one single process for validation that will accommodate the needs of all potential candidates, coordination of information on validation is of key-importance and must be offered

10

close to where people live and work, at the time when they need it, and together with adequate guidance and counselling support.

Even before a decision is made to seek validation, candidates need to know what to expect, what standards have to be met and which evidence is needed. Similarly the individual needs to be informed on the costs and benefits of validation, notably how the outcomes of the process can influence progress in learning and work. Box 1 lists the main areas where information and advice is needed.

Box 1: Information to be provided to individuals Candidates will need information and advice on the following aspects: Existing alternative validation forms available (formative as well as summative) timelines for validation, costs, procedure, forms of evidence of learning outcomes, requirements for evidence, quality and standards, presentation of evidence, assessment and how best to approach the process, support available, appeal procedures

Career guidance is, ideally, a continuous process enabling citizens at any age and at any point in their lives to identify their capacities, competences and interests, to make educational, training and occupational decisions and to manage their individual life paths in learning, work and other settings in which those capacities and competences are learned and/or used. Guidance includes a range of individual and collective activities such as provision of information, counselling, competence assessment (identification), mentoring, and training in career management skills. Since guidance can originate from a range of services which include education, training, employment services and local administration, cross sector coordination becomes fundamental. Guidance, thus, needs to be coordinated and integrated with validation procedures. The linkages between employment services and education & training are particularly important.

Guidance and counselling service for validation shares common aspects with guidance and counselling services for other purposes. Adequate guidance and counselling will necessarily include (one or more of) the stages explained in chapter 3. The phases of identification and documentation are, in fact, integral part of any guidance and counselling process. Countries might opt for different ways of integrating validation and guidance and counselling:

• co-ordinated networks of provision; • one organisation with central responsibility for provision of guidance and counselling; • provision of information in relation to occupational or educational sectors, for example

offered by chambers of commerce and industry;

A main challenge facing networks of information and guidance providers is to ensure quality and consistency of services. This will particularly be the case where networks include organisations from different sectors, for example seeking to link education and training, social services and employment

11

services. Such networks, however, will normally be able to operate on a decentralised level, offering services close to where people live and work. Providing information and guidance on validation through one single organisation makes coordination and the implementation of new measures simpler, but will in most cases lack the capacity to operate close to the end-users. Reliance on one single organisation faces further challenges because, in practice, these are rarely supported by structures that make mutual learning, experimentation and transfer of good practice possible. Such learning is more common in networked models. Information and guidance in the private sector faces particular problems and will in many cases be limited to particular sectors or even single companies. The role of sectoral organisations and chambers can be important, also for linking validation in the private and public sector (See also Cedefop 2013).

Information and guidance can be delivered in different ways. Countries tend to combine Web-based delivery and face-to-face provision. Web-based approaches might also include self-assessment tools enabling individuals to take a first step towards validation. Face-to-face provision might be provided on a one-to-one basis or in groups. The European Inventory provides detailed information on the solutions chosen by countries in this area (Overview).

Case – Guidance and counselling strategies

In Spain, information, advice and guidance on validation of professional competences is provided by the education and employment administrations (at national and Autonomous Community level), local governments, social partners, chambers of commerce, and other authorised institutions. For example the web platform TodoFP porta l- was designed to provide information and advice to individuals. Information is also provided in the website of the National Public Employment Service (SEPE, Ministry of Employment). ACREDITA is the tool that the Ministry of Education offers individuals to see if they can apply for the accreditation of competencies and to explain them the process.

Guidance should be tailored to each individual, departing from an assessment of individual needs. Guidance and counselling is of particular importance for reaching disadvantaged groups and for releasing their inherent potential. Existing guidance methods and tools devised to respond to the identified needs of specific target groups based on age, employment situation, socioeconomic background or migrant status can be utilised in validation initiatives to assist in the definition of the purpose of the validation process.

The findings of an on-line survey for the 2014 Inventory update showed that 64 per cent of respondents have used more than one method of guidance (with an average of 2.8 methods per respondent). Figure 1.1 below illustrates the methods of guidance that projects and/or practices employ. With just under half of respondents (48 per cent), written information is the most commonly used method to guide validation participants. The next most cited guidance method was electronic tools (36 per cent), closely followed by group support from a trainer / teacher (34 per cent).

12

Figure 3: Modes of delivering guidelines

Source: 2014 Inventory Update: Online Survey of Projects and Practices, number of respondents, 42

A coordinated approach between validation and guidance is important to assure rationalization of resources; especially because funding depends very much on the level of complexity at which the guidance is provided. A high level of coordination between initial guidance and validation can reduce procedural costs and add value to service provision.

Key questions regarding information, counselling and guidance

To analyse the situations as regards guidance and counselling, the following questions provide a starting point: Are already existing career guidance and counselling services, for example in education and

training, labour market and social services, providing information and advice on validation? Do they provide information on the benefits of validation? Are there particular target groups

that should be prioritise? How can the role of existing services be strengthened to take into account validation? How can existing career guidance and counselling service network to improve the services

offered to candidates? Is there a need for one coordinating institution to make sure that candidates are served

where they live, study and work? How can public and private stakeholders cooperate to offer better information and advice on

validation? Is there a need for specialised guidance and counselling for validation? Are tools supporting (in particular) identification and documentation of non-formal and

informal learning being shared between different guidance and counselling services?

3.2. Coordination of stakeholders The 2012 Recommendation emphasizes the importance of coordination of and appropriate information on validation:

13

(Points 4 a-b and 5) The Member States are invited to ‘…promote the involvement in the development and implementation of the elements and principles […] of all relevant stakeholders, such as employers, trade unions, chambers of industry, commerce and skilled crafts, national entities involved in the process of recognition of professional qualifications, employment services, youth organizations, youth workers, education and training providers, and civil society organizations. The Member States are furthermore (point 5) called to promote coordination on validation arrangements between stakeholders in the education, training, employment and youth sectors, as well as between those in other relevant policy area.’

Validation is a complex process that requires the involvement of many different actors, with different responsibilities and functions. As indicated above, coordination is necessary to permit well-functioning information and guidance services. The 2014 inventory demonstrates significant differences in the use of validation in education and training sector compared to the private and the voluntary sectors. The inventory also shows that there are significant differences within the education and training sector as regards the implementation and use of validation. These differences make coordination difficult. In addition, while a number of countries are introducing new legislation and new governance arrangements to support validation, other countries lack a national strategy or a legal framework, making it difficult to identify and mobilise stakeholders willing to drive validation policy at national level.

Figure 4: Does the country have a national (regional where appropriate) institution in charge of overseeing validation practices?

The above obstacles are important and need to be addressed. While the situation will vary between countries, table 2 lists the main stakeholders involved at different levels and their respective roles. Putting in place validation arrangements meeting the needs of the (diverse) end-users requires that existing stakeholders are able to communicate and coordinate their activities. Table 1 illustrates this complexity.

14

Table 1: Validation and Stakeholders

This complexity is reflected in the organisation of national validation arrangements. The 2014 inventory shows that few countries have a single organisation in place at the national level in charge of overseeing validation. In most countries, the responsibility for validation is attributed to several national organisations (including several ministries). Countries thus need to reflect on their own institutional framework and the overall division of roles and tasks to permit coordination. Working towards a better coordination of validation arrangements requires require a focus on a number of system-characteristics.

First, the legal framework matters. The existence of different and to some extent competing legal systems may in some cases hinder developments. The introduction of centralized solutions has,

Who is involved? Example of main functions

European Level

• EU Commission and Council • EU Agencies, Cedefop and ETF • European programmes • European Social Funds • European social partners organisations • European employers • European projects

• Provide common guidelines • Provide common EU tools for validation • Support transparency and portability • Facilitate policy learning and best practices transfer

National public level

• Ministries (of Education, Labour etc.) • Education, training and qualification

Authorities • National projects • Public employment services • Social services

• Provide adequate legislation • Establish procedures • Determine role and responsibilities • Coordinate institutional actors • Provide national guidelines • Establish QA mechanisms

Regional and local

public level • Regional public authorities in

education, labour and social services etc. • Local public authorities in education,

employment and social services • Regional and local projects

• Provide support to institutions • Establish procedures • Adapt guidelines to local environment • Coordinate among regional actors • Provide information and support • Carry out identification, documentation, assessment

and certification

Education and training

institutions

• Vocational Schools • Universities • Private education institutions • Assessment centres and specialist

recognition centres • Projects

• Carry out assessment and certification • Provide support for identification and documentation,

including information and guidance • Support individuals

Business sector • Enterprises • Sector or Branch associations • Trade Unions • Employer organisations • Chambers of commerce and Industry • Projects

• Provide information • Carry out assessment and certification • Carry out identification and documentation of

competences

Voluntary sector • Non-governmental organisations • Projects

• Provide information and guidance • Carry out identification and documentation of

competences

15

however, to be balanced with the need to develop targeted arrangements for specific areas and stakeholders.

Case – national coordination The situation in the Netherlands highlights a combined, multi-level approach, whereby information and guidance concerning validation is made available at a number of different levels. Aside from training providers being obliged to offer information and advice on why, how and when to embark on the validation process other channels of support are present at a:

■ Macro-level: through the Ministry of Education and the social partners;

■ Meso-level: by the sector partners (employers and trade unions);

■ Micro level: within organisations by HR departments and internal ‘ambassadors’, also with any registered provider.

As noted above, training providers offer guidance and counselling, and providers are the main source of information for people interested in validation of prior learning. Private training providers form a key part of the information and guidance sector, and include former Government-funded advisory training centres or AOBs, many of which were privatised after 2000, as well as independent organisations and larger consultancies6. In addition, a regional structure has been set up with 35 Leer Werk Loketten (Counter for services around the issue of 'working and learning') across the country.

Second, the scale and size of arrangements must be taken into account. In large systems, for example, the certification function is usually strictly separated from the Identification, documentation and assessment stages. This separation of stages and functions may be more difficult to achieve in smaller systems. Third, the position of dominant stakeholder can be decisive. The 2014 inventory has shown that in general terms, education and training bodies have a dominant position in the validation process. This is helpful from the point of view of comparability of standards between the informal and formal systems. However it is possible that the dominance of the formal education and training system could inhibit the development of assessment practices respecting the particularities of non-formal and informal learning. It is likely that the responsiveness of the validation process to the needs of candidates is dependent on the different types of institution offering validation of the knowledge, skills and competences to individuals. Fourth, attention should be paid to coordinating organisations. Most countries have no single coordinating organisation that takes lead responsibility for validation; instead the responsibility is shared across a number of different ministries, or other national authorities, making validation a transversal issue. Countries need to consider whether such a coordinating organisation should be identified and appointed.

Case – national coordination

The Agency for Lifelong Learning (Vox) maintains an overview at national level of the Norwegian approach to validation of non-formal and informal learning for adults, while the Directorate for Education and Training supervises validation in lower and upper secondary education. The Norwegian Agency for Quality Assurance in Education (NOKUT) supervises education above upper secondary level

16

Fifth, the needs of particular groups have to be addressed. The majority of countries seem to tailor validation systems to the needs of particular groups; notably unemployed or low qualified. While there may be good reason for choosing this approach it also runs the risk of placing validation outside mainstream qualifications policies – running the risk of creating ‘A class ’ and ‘B class’ certificates depending on the route to certification.

Key questions regarding coordination of stakeholders

Coordination of validation must primarily take place at national level, addressing the division of roles and responsibilities between public as well as private stakeholders. To analyse the situation, the following questions can be asked: Which validation arrangements exist and what is their legal and political basis? Has a single legal framework been put in place Which stakeholders are involved? How are these frameworks and stakeholders related to each other and who is responsible for

coordination at different levels? Which information services and resources have been put in place; are these related and

coordinated; does information reach individuals where they live, work and study? Which administrative processes are in place (contact procedures, applications, recording of

results, appeals); are these related and/or coordinated? How are services related and coordinated; how can they reach individuals where they live, work

and study? Are roles and responsibilities (information, guidance, assessment, certification and appeals)

divided in a transparent and credible manner, how can this be improved?

3.3. National qualifications frameworks The 2012 Recommendation stresses the importance of linking validation arrangements to national qualifications frameworks:

Member states should enable individuals to (1b)‘[…] obtain a full qualification or, where applicable, part qualification on the basis of validated non-formal and informal learning experiences’. They should ensure that (3 a) ‘[…] validation arrangements are linked to national qualifications frameworks and are in line with the European Qualifications Framework’, that (3j)‘[…] synergies exist between validation arrangements and credit systems applicable in the formal education and training system such as ECTS and ECVET’ as well as ensure that (4 b) ‘[…] education and training providers facilitate access to formal education and training on the basis of learning outcomes acquired in non-formal and informal settings and, if appropriate and possible, award exemptions and/or credits for relevant learning outcomes acquired in such settings.’

The development of validation of non-formal and informal learning and National qualification frameworks (NQFs) share a common objective: enabling individuals to make progress in their learning careers on the basis of their learning outcomes and competences; not on the basis of the duration and location of a particular learning programme. The concept of a national qualifications system is now widely understood as all aspects of a country’s activity that result in the recognition of learning. These systems include the means of developing and operationalising national or regional policy on qualifications, institutional arrangements, quality assurance processes, assessment and awarding processes, skills recognition and other mechanisms that link education and training to the

17

labour market and civil society. Arrangements for validation forms an important and integrated part of these qualifications systems. By integrating, politically and legally, the validation of non-formal or informal learning with the national qualifications system, the validation of non-formal and informal learning becomes more transparent through a clearer legal status, governance and financing. Most importantly, validation supports – in a practical and concrete way- progression between different levels and types of education and training

Case – validation and NQFs

The methodology for recognition of prior learning is based on the 12-level Scottish Credit and Qualifications Framework (SCQF). This Framework aims to be all-inclusive and to take account of all types of learning, including formal, non-formal and informal learning across all sectors (public, private and the voluntary sector). However, the framework is not intended to be used solely as a way of categorising and understanding qualifications or to formally credit prior learning. It is also intended to be used to support learners to identify their level of competences and skills against the framework and plan their learning and career development accordingly. The framework is therefore intended to support formative recognition, as well as summative recognition.

An integration of validation into the national qualification system requires that qualifications are opened up to a broader set of learning pathways and that validation arrangements are established as an accepted and normal route to a certificate or qualification. In practical terms this requires that a shift to learning outcomes is carried out, reducing the emphasis on particular learning forms and approaches. All EU Member States are now in the process of developing and implementing learning outcomes based national qualifications frameworks (NQFs) and the majority of these aim for comprehensive frameworks covering all levels and types of qualifications. The design of national qualifications frameworks and learning outcomes based qualifications provides an opportunity to mainstream validation and make it become an accepted route to qualifications (Overview).

An objective shared by most NQFs is to better relate different qualifications to one another and thereby reduce barriers between education and training sectors and promote access, transfer and accumulation of learning outcomes. Methods and systems for validation of non-formal and informal learning contribute directly to this objective. If introduced on a systematic basis, as a part of the overall qualifications system, validation of non-formal and informal learning will not only open up qualifications to a broader set of learning experiences but also make it easier for individuals to progress across institutional, sectoral and national borders.

Case – validation and NQFs

Validation leading to of a partial or full qualification (VAE) is possible for all qualifications covered in the French NQF.

Attributing an explicit role to validation in the context of NQFs may promote overall flexibility of education and training systems. This is particularly the case if validation can support not only formal access to courses and programs (as is the case in several countries), but also make it possible for candidates to be exempted from parts of a programme so as to avoid repeating parts already

18

known. This will however require cooperation across education and training sectors to ensure that the principles applied are shared.

In the development of NQFs it will be important to consider how a more systematic integration of validation can be combined with credit transfer arrangements. We observe, for example in relation to the ongoing development of the European credit system for VET (ECVET), that similar functions (identification, documentation, assessment and certification) are being pursued by validation and credit arrangements respectively. Given that validation arrangements involve putting in place extensive practical arrangements, a separate development of similar arrangements for credit purposes could prove negative.

The emerging national qualifications framework can be used to influence the way standards are formulated and used. Until recently, the description of national qualifications levels have been implicit and based on duration and location of programmes of education and training. The rapid development and implementation of NQFs can be used to actively promote the development of explicit, coherent, learning outcomes based standards for qualifications that could accommodate outcomes of learning in non-formal and informal settings.

Key questions regarding national qualifications frameworks

National qualifications frameworks are now being implemented across Europe. These frameworks may facilitate the introduction of validation and the following questions point to some key issues to be addressed: Are validation arrangements (all, only some?) seen as an integrated part of the national

qualifications system and as a normal route to qualifications? What is the relationship between validation and the national qualifications framework (NQF)? Can validation be used to support progression between all types and levels of qualifications in the

NQF? How does validation relate to qualifications not included into the NQFs? Is there a link established between validation and (possible) credit transfer and accumulation

arrangements?

3.4. Standards and learning outcomes The 2012 Recommendation asks Member States to assure that:

‘[…]qualifications or, where applicable, parts of qualifications obtained by means of the validation of non- formal and informal learning experiences comply with agreed standards that are either the same as, or equivalent to, the standards for qualifications obtained through formal education programs’. (point 3 h)

Awarding a certificate on the basis of non-formal and informal learning requires an agreed reference point - either in the form of an official qualifications standard, an occupational standard or in the form of an approved education programme or curricula. While it is possible to envisage the identification and documentation stages of validation - for example in a skills audits – to be carried out without a formalised standard, assessment and certification aiming for a qualification cannot be carried out without reference to formal reference point and standard.

19

It is crucial whether a standard is written in learning outcomes or not. A standard referring to the specific inputs of a particular learning context can make it difficult to value learning having taken place in a different context and according to a different logic of learning. A learning outcomes based standard - expressing what a candidate is expected to know, and be able to do - will in most cases provide a better reference point for validation - implicitly acknowledging that the same outcome can be reached in a number of different ways. The use of learning outcomes, however, provides no guarantee of success: If written too narrowly there is a risk that important facets of the individual learning experience may be lost; if written in too general a way assessment may lose orientation and result in lack of consistency and reliability. The writing of learning outcomes based standards furthermore requires attention to be paid to the balance between job/task-specific and transversal skills and competences.

Case – learning outcomes based qualifications levels

In Norway, the learning outcomes approach is now applied in all sectors and at all levels of education and training. This is exemplified by the VET sector where each qualification is defined at national through a distinction between specific and basic skills and competences. The first category refers to the occupational specific requirements; the second to the basic skills in communication, in numeracy as well as digitally. These requirements can be used both for traditional courses and for the assessment of Real competences (validation).

An argument against awarding qualifications on the basis of non-formal and informal learning is that the resulting qualifications are inferior in quality to those delivered by the formal education and training system. This scepticism is partly inked to standards and the way they are developed and applied:

• Lack of visible standards - it is not clear to the users which standards are applied. • Too weak standards - main stakeholders have not been involved in the definition of

standards. • Outdated standards – standards are not seen as relevant. • Different standards are used for formal education and training and for validation – implicitly

signalling differences in value and status of the resulting certificate or qualification.

The standard is thus of key importance to the validation process and the resulting outcomes. Validation referring to visible standards defined and supported by the main, relevant stakeholders will greatly increase the acceptance among users – individuals as well as employers – of these practices. In general, qualifications – and validation of non-formal and informal learning – relate to two main categories of standards (2) - occupational and education-training standards. These two

2 In addition to occupational and educational standards which may be considered to be primarily concerned with the content (or knowledge, skills and competence) of learning, some countries (for example the UK) operate with standards that apply specifically to the process of assessment, validation and certification of learning.

20

categories operate according to different logics, reflecting different sets of priorities, motivations and purposes:

• Occupational standards: Following the logic of employment, these standards will focus on what people need to do, how they will do it, and how well they do it in an occupational context. They exist in all European countries but each nation has its own style of derivation and presentation of the standards. Occupational standards form a bridge between the labour market and education because educational standards (syllabuses and pedagogies) can be developed from them.

• Education-training standards: following the logic of education and training, these standards will focus on what people need to learn, how they will learn it, and how the quality and content of learning will be assessed. Traditionally these standards have been formulated in terms of input (subject, syllabus, teaching methods, process and assessment) but the ongoing shift to learning outcomes in most European countries means that educational standards are increasingly applying principles typically found in occupational standards.

Figure 5: How is validation linked to formal qualifications? (Number of countries,

2014) (Link to inventory)

Many approaches to validation of non-formal and informal learning tend to relate to the second category of standards, those designed specifically for the education and training system. As mentioned already, the critical question is whether these standards are defined through the specification of teaching input or learning outcomes? The experiences from validation of non-formal and informal learning may be seen as providing important feed-back to standards used in the qualifications system, in particular if these experiences can influence the dialogue between stakeholders involved in defining and reviewing standards. The initial writing of standards, as well as their continuous review and renewal, can draw on the experiences gained from validation. The complexities of non-formal and informal learning make it possible to critically judge whether standards support overall validity and reliability of assessments.

Key questions related to standards and learning outcomes

21

Are qualifications awarded on the basis of non-formal and informal learning referring to the same or equivalent standards as those used for formal education and training?

If no, which other standards are used and how do they relate to formal standards? Are standards used for validation written in learning outcomes?

If no, which are the implications for validation? Who developed the standards and in reference to which sources (education or occupation)? Are there mechanisms in place to provide feed-back on the relevance of existing standards to

validation?

3.5. Quality assurance The 2012 Recommendation asks Member States to assure that:

(Point 3f) ‘[…] transparent quality assurance measures in line with existing quality assurance frameworks are in place that support reliable, valid and credible assessment methodologies and tools’.

Quality assurance in validation must be systematic, take place on a continuous basis and be an integrated part of the process;

• Systematic quality improvement requires an explicit and agreed quality strategy; • A system for feed-back from users/customers should always be considered; • A quality plan/strategy must be known to the public, including candidates; • A quality/plan strategy can prepare the ground for external quality assessment and review.

The overall quality of validation depends on a range of factors reflecting the character and complexity of the validation process. Ensuring and improving the quality of validation is thus complex but need to be applied on a systematic and continuous basis (following the principles of the Quality circle – PLAN, DO, CHECK and CHANGE).

Case – quality assurance

In Wales, quality standards have been developed for the validation pillar of the Welsh qualification framewo9rk (CQFW) which ensure that the quality assurance processes are comparable with the other learning pillars. In general, validation (or Recognition of prior learning RPL) process should be subject to the same quality criteria as other assessment methods.

While the concrete form of the quality process will vary between countries and contexts, the following issues will have to be considered when developing a quality strategy for validation: Fitness for purpose is of critical importance for ensuring quality of validation. There are many methods for judging evidence of learning. Not only should any one chosen method be suited to the form of the learning but methods in combination should create a sensitive and trustworthy toolbox of methods of assessing learning. The safety, security and confidentiality of the process must be ensured and continuously improved. The initial and ongoing engagement with the validation process

22

from identification to certification must not be compromised by lack of trust and consequential deterioration in motivation to proceed. The process of validation must be reliable, and lead to trusted results. If the settings for learning and validation vary greatly then the process of validation must allow for these differences but should the process be repeated then the outcome must be the same. The evidence documented for an individual must be valid and be directly related to the standards being used for validation. The candidate must feel confident that the interpretation of evidence and standards is thorough and not based on arbitrary judgements. Standards are the basis of measuring learning outcomes – and thus validation. They must exist in a clear and unambiguous form that has the confidence of the key stakeholders. Systematic quality assurance of standards, and how these are defined and reviewed, is of critical importance to validation and the way these generate reliability, validity and trust. Costs need to be measured in relation to the expected returns to the stakeholders concerned. It is generally the case that validation processes for non-formal informal learning do not have the benefit of large scale application (large cohorts of learning being assessed in similar ways). When measuring and improving the cost efficiency of validation the economies of scale (at least until now) have been limited. Quality assurance arrangements should support the long term implementation of validation arrangements. Sustainability is a must for validation processes to be trusted. Going through a validation process is often expensive for individuals and it is therefore important to put in place permanent arrangements which are known to and valued in society at large and/or in the particular sector. Transparency and visibility of the validation process and what this entails is a fundamental feature for supporting trust to validation. The transparency of the process of using established standards is particularly important and should be supported by quality assurance arrangements.

The above factors need to be taken into account when putting in place validation, be this at national, regional, sectoral or local level.

Key questions regarding quality assurance of validation

Have explicit and integrated quality assurance measures been put in place for validation? o If existing, are these measures reflecting an explicit and agreed quality strategy?

How are quality assurance arrangements divided between internal and external assurance and control?

Has a system for feed-back from users/customers been put in place or considered? Is there a monitoring system for validation that permits to see the users, beneficiaries and

outcomes of the validation process? Is there a systematic evaluation of results and costs?

3.6. The professional competences of validation practitioners The 2012 Recommendation maintains that

Point (3g) ‘[…] provision is made for the development of the professional competences of staff involved in the validation process across all relevant sectors’. The overall trust towards validation will to a large extent depend on the work carried out by the ‘front-line’ practitioners and professionals directly involved with validation candidates at different stages of the process. These validation practitioners cover all aspects of validation and include those that offer information, advice and guidance (orientation), those that carry out assessment, the

23

external observers of the process, the managers of assessment centres/procedures and a range of other stakeholders that have an important but less direct role in the validation process. These groups of practitioner are evident in validation systems across countries. It is not always possible to say that each of these five groups are distinct from one another, in practice one person or one kind of body can have roles that cross this classification.

Existing national data shows that the guidance and counselling function is crucial to the success of validation processes (see also 4.1). The work of a counsellor starts with the process of reaching out to engage potential candidates for validation, then preparing the candidate for assessment, the role continues by guiding the candidate after the assessment decision. An important part of the role of the counsellor is often to work with the candidate to appraise the breadth and depth of evidence of learning (helping to develop self-awareness). Some would refer to this as competence mapping (see chapter 5.2), pointing to the critical role of counsellors in skills audits and enterprise internal competence mapping.

To fulfil his/her role, the counsellor has to have a clear understanding of the validation context. If the candidate aims for a formal qualification, it is essential that the counsellor is aware of the relevant standards and is able to advice on whether existing evidence is sufficient or not. The counsellor has also to prepare the candidate for the assessment process, informing them of procedures, how to present evidence of learning, respond to questions, expectations in terms of behaviour, possible outcomes and so on. This also requires the counsellor to have a thorough knowledge of the assessment process. A distinctive part of the Counsellors role is their independence from the actual assessment process for an individual and their ability to offer impartial but useful advice.

Box 2 Key knowledge and skills of counsellors A thorough knowledge of the education system; a thorough knowledge of the validation process; an understanding of the labour market (expected standards and post assessment advice); a list of contacts (experts) to answer specific technical questions (social partners and other

sector experts).

The job of an assessor is to seek and review evidence of an individual’s learning and judge what meets specific standards. They must be familiar with the standards and the potentially useful assessment methods that might be used to reference evidence against standards. Assessors should be acknowledged as professionals in their sector as this leads to trust and credibility in the assessment process itself. The authenticity of the assessment situation is likely to be improved when sectoral experts can direct the use of an assessment instrument or judges the outcomes of its use. Assessors should not be linked to the candidate or their work or social life in any way. The credibility of the validation process very much depends on the credibility – and neutrality – of the assessor.

Evidence from the Inventory shows the qualifications and experience of assessors as a key element in the quality assurance of validation projects. Assessors are generally required to have more than five years of experience in the respective field. Experience in the specific field of work is seen as essential and assessors could be recently retired professionals, senior managers in the specific field of work, expert representatives of the Social Partners or teachers in the specific technical field, with

24

direct experience in the sector. Training in the validation procedure is also required. Networks of assessors should be set up - where and when possible - to assure professional development and coherent practices.

Box 3 Key knowledge and skills of assessors To be familiar with the validation process (validity and reliability); have no personal interest in the validation outcome (to guarantee impartiality & avoid

conflicts of interest); be familiar with different assessment methodologies; able to inspire trust and to create a proper psychological setting for the candidates; committed to provide feedback on the match between learning outcomes and validation

standards/references (via support systems); be trained in assessment and validation processes and be knowledgeable about quality

assurance mechanisms; be operating according to a code of conduct

The third key group of practitioners are the managers of the validation process. The function of these people is to manage the process, the people and possibly a physical or virtual centre where candidates, counsellors and assessors come together. Process managers can have responsibilities for the public profile of the validation centre, for ensuring equality of access to validation, managing an appeals process and ensuring external review. One key role is the financial management of the validation process. Whether privately or publicly funded, the task of minimising costs and creating a sustainable operation is challenging.

External observers provide a quality check on validation procedures, training of practitioners and outcomes for candidates. The counsellors and the assessors have distinct roles when engaged with the candidate and the external observer reflects on the maintenance of separation of these roles. In some settings the external observer is an advisor to counsellors and assessors and helps them to learn from their experience and that of others. The external observer may have a role in reviewing the efficiency of the process and checking that resource use is optimised. S/he might not necessarily be expert in the given profession/activity, but needs to be trained in quality assurance procedures. The observer can be considered a source for advise and operate as an external auditor, who does not have regular presence in the process.

It is not possible to focus on the practitioners involved in validation processes without referring to a group of supporting stakeholders who counsel, assess or manage centres. These stakeholders have an interest in the successful operation of validation and they include:

• responsible people in public bodies that fund the process; • responsible people in public bodies that have agreed a policy for validation; • managers of human resources for private companies; • community leaders that seek engagement of groups of individuals in learning and working; • education services in the formal sector.

Key questions regarding validation practitioners

What are the formal requirements for: o Counsellors and guidance personnel

25

o Assessors o Other practitioners involved with validation

Is there a strategy in place for the professional development of these practitioners? Is the professional development of validation professionals coordinated between different

sectors and arrangements? Can a community of practise for validation professionals be developed, supporting

networking and professional developments?

26

CHAPTER 4. THE CONTEXTS OF VALIDATION

In line with the messages of the 2012 Recommendation the following sections address the different validation settings/contexts and the challenges involved in each of these.

4.1. Validation in education and training The 2012 recommendation recognizes the key-role played by education and training institutions in taking forward validation:

(point 4b)‘[…] education and training providers should facilitate access to formal education and training on the basis of learning outcomes acquired in non-formal and informal settings and, if appropriate and possible, award exemptions and/or credits for relevant learning outcomes acquired in such settings;

The main conditions for integrating validation in education and training – as a normal route to qualifications – have been discussed in chapter 4.3 and 4.4 when discussion national qualification frameworks and standards.

The 2014 inventory shows that education and training sector is still the key promoter of validation of non-formal and informal learning in Europe and that in most cases, Ministry of education and educational institutions play a crucial role for validation. However, at the same time, education and training sector might constitute an important obstacle to the wider implementation of validation arrangements. This somewhat contradictory role is linked to the different characteristics of the sub-sectors and their attitudes towards validation.

Overall, validation is of crucial importance to adult education and training and to facilitate lifelong learning. The age group 25-45 is in most countries an important the main user of validation, indicating that these arrangements play an important role in facilitating transitions from employment to education and back again. However, validation of non-formal learning might come at a cost to adult education providers as their financing might be linked to to enrolment. In addition, in many countries adult education provision is not very regulated which makes the assurance of the validity of the validation process more challenging.

In several countries, adult education is linked to the vocational education and training (VET) sector. The VET sector has been a main proponent of validation of non-formal and informal learning in Europe. Its close relationship to the labour market and strong traditions as regards work based learning has facilitated the openness to validation. In addition, the standards are normally aligned with occupational standards that are easier to relate to previous work experience. It is reasonable to expect that the VET sector also in the future will play an important role in validation.

Validation in relation to higher education qualifications is far less common than in VET. Higher education institutions are normally more autonomous and determine the scope and possibilities for validation on their own. The Bologna process has put some pressure on creating methods and possibilities for validation of non-formal and informal learning. The 2014 inventory has shown that

27

there is certain increase in later years, especially concerning access to university on the basis of validation. In general, higher education institutions made limited use of validation for awarding exemptions to parts of a learning programme and only in very few cases can we observe that full qualifications has been awarded in this way. However, many of these processes are embedded into recognition of prior formal education and are ill prepared for learning acquired outside formal institutions.

The use of validation in the context of initial, general education is normally limited. One difficulty is that in many instances, general educational programmes are not described in terms of learning outcomes. Portugal represents an exception; the large scale ‘New opportunities programme’ was directed towards adults lacking formal qualifications at this initial level. The ‘take up’ of validation in general education will in addition, and importantly, be linked to the lack of information, legal rights and appropriate provisions and services.

Validation must be developed according the needs of the individual learners, not only according to the needs and interests of the institutions and systems. There will be a need to further reflect on these attitudes and consider whether the effectively prevent learners from progressing in an efficient way.

Key questions regarding validation and education and training

The following questions are important when addressing validation in the context of education and training: Can validation be offered in all parts of the education and training system? Are validation arrangements in the different parts of the education and training system building

on similar or differing principles? Can the validation arrangements in the different parts of education and training ‘work together’

and facilitate progress across types and levels of education? Is there a link between validation and credit transfer arrangements?

4.1.1. Validation and Open Educational Resources The 2012 Recommendation states that the knowledge skills and competences acquired through open educational resources should be addressed by validation arrangements: (point 1a) ‘The arrangements for the validation of non-formal and informal learning enable individuals to have knowledge, skills and competences which have been acquired through non-formal and informal learning validated, including, where applicable, through open educational resources.’

The reference to Open Educational Resources in the 2012 Recommendation on validation reflects the rapid expansion of on-line learning opportunities in recent years, in particular promoted by higher education institutions. OERs, in particular exemplified by the so-called MOOCs (massive open, on-line courses), are seen as important supplements to traditional education and training programmes by reducing overall cost and increasing accessibility, and by allowing individuals to follow their own pace and organise learning according to other obligations like work and family. A further argument for MOOCs is that they give access to high quality (‘world-class’) teaching to a

28

broad group of learners and thus contributes to an overall strengthening of the quality of education and training.

As OERs become more common and are offered by a broader range of institutions, the differences between high and low quality offerings increase. It is therefore important to consider how the outcomes of this learning can be appropriately documented and assessed and how current practises on validation can take them into account.

Crucially it should be underlined that the outcomes of OERs have to be treated with the same care and degree of scrutiny as any other learning outcomes. Given the inevitable variation of quality of OER provisions along with the varying success of learners to adapt to on-line learning, more attention has to be given – at national and in particular at European and international level - to the process of documentation, assessment and certification of outcomes from OERs. For OERs to be taken into account in validation requires first and foremost transparency. As long as the results of OERs are described through learning outcomes and the status of standards and testing arrangements is clarified, no particular treatments of these courses in validation arrangements seem to be warranted.

Box 4 Requirements to Documentation and assessment The outcomes of OER (and MOOCS) must be described in the form of learning outcomes, In cases where the OER brings with it some form of ‘ internal’ credit’ , for example ‘badges’,

these must explained and documented in a transparent way which encourages trust; The standards and/or reference points underpinning credits or badges must be clearly

explained The arrangements for quality assurance underpinning OEDs must be transparently

presented The methods for assessment/testing must be transparently explained.

The role of OERs in the overall qualification system may require particular solutions though. A key question is whether OERs can link to existing credit systems, notably the ECTS. If there’s a positive answer to this, the requirements to transparency, learning outcomes and clarity as regards standards and tests listed above becomes even more important.

In addition, on-line systems for assessing and recognition of learning are becoming more and more popular and used. Web-based platforms that allow for recognition and assessment of specific skills require careful consideration vis a vis the existing systems of validation as to allow adequate quality assurance and rationalization of efforts.

Key questions regarding open educational resources

The following questions are important when addressing open educational resources: Are the methods for validating outcomes of MOOC (for example) the same as for other

learning outcomes? How are internal credits (‘for example ‘Badges’) taken into account by validation?

4.2. Validation in enterprises The 2012 Recommendation underlines the importance of pursuing validation at the workplace and recommends to

29

(point 4) ‘[…] promote the involvement in the development and implementation of the elements and principles referred to in points 1-4 of all relevant stakeholders, such as employers, trade unions, chambers of industry and commerce and skilled crafts’ . Furthermore (4a), ‘[…] to foster participation in this process, employers should promote and facilitate the identification and documentation of learning outcomes acquired at work’.

The recent publication by Cedefop (2014a) on validation in enterprises shows that this is a field of increasing importance. A key question when taking forward validation arrangements in Europe is whether the outcomes of company-internal arrangements can be presented in a format which allows them to be used outside enterprises, for example when somebody wants/needs to shift job or seeks further education and training. Finding a solution to this could potentially be of big importance for the overall development of validation and competence assessment in Europe. Currently, competence assessments in enterprises normally address the following functions:

Recruitment constitutes one of the most important and traditional areas of competence assessment. Selection of inappropriate candidates can be extremely costly for companies. The need for assessment arises because formal qualifications do not match the required skills sets. Personnel & competence development is another important part of developing competitive capacity. In general terms, this refers to up-grading and adapting the competences of employees. It is also necessary to define for each employee the skills gap and the required training or learning measures.

Basically, the purpose of the above is to measure an individual’s competences against the requirements of a higher job position and to establish her/his potential as well as possible personnel development measures. It also enables the individual to understand his/her skills gaps and how the skills and competences are and could be used and improved. Career planning systems require a certain hierarchy in the company. Therefore, validation in the context of career planning will exist mainly in larger companies. Knowledge management systems have become more widespread in recent years, supported by the introduction of ICT tools. Although enterprise competence maps are rather about assessing the competences of the company as a whole, and to identify respective gaps, it is certainly ultimately based on measuring the competences of individual employees. Employees leaving a company may wish to have their competences formally validated and certified. This is frequently done by way of testimonials. Employees may even have a legal right for such certification. Here transferability (outside the company) is crucial for the individual and has an impact on the functioning of the labour market in general.

Box 5 Benefits of competence assessments in companies Increase motivation and interest in workplace practice on the part of the employee/learner; Reduce the amount of time needed to complete a qualification and therefore require less

time away from the workplace; Generate new ideas and developments in the workplace as a result of process of reflection

on practice by employee/learner; Improve employee retention and reduce recruitment and training costs.

In some cases the company needs to demonstrate that its workforce is highly qualified – for example when competing for contracts or seeking insurance for safe working on an assignment. When this is the case the normal duration of professional training programmes can make such qualifications very

30

expensive. Validation can lead to exemptions for some learning and thus reduce direct training costs and indirect opportunity costs for losing people from the workplace during training.

It is possible that the drive for visibility and upgrading of an individual’s competences will not be seen as entirely beneficial from the individual’s perspective. For example the employee may see themselves as not in control of the validation of their own competences. They may also feel unable to challenge validation decisions in the way they might do if validation is the responsibility of an independent jury (as often in the formal education system). These two cases illustrate conflicts of interest between company and employee that can challenge the integrity of a company scheme. The 2004 European Principles for the validation of informal and non-formal learning (see also chapter 3.1) make clear the importance of avoiding conflicts of interest such as these by putting emphasis is on transparency of the validation process and the close involvement of a range of stakeholders.

Notwithstanding a number of challenges, it remains the case that the need for employer involvement in national validation processes is crucial. Without employer involvement capitalizing on the workplace as a learning arena is reduced and the implementation of large scale validation of learning in a population is restricted. Sustainability can also be supported by more systematic intervention at sector or branch level, for example by the introduction of supporting competence frameworks and standards, providing a reference point and facilitating transfer. The involvement of small and medium-sized companies (SMEs) in the validation process present special challenges because resources and capacity for developments are limited. However the SME is an organisation that typically experiences a continuous need to develop skills in its workers. Research shows that SMEs are positive about the process of validation but that the intervention by external agencies in this field needs special consideration.

Box 6 Cooperation on competence assessment in companies

Cooperation need to be industry- or sector focussed in order to ensure a sufficient homogeneity of tasks and competence requirements;

The development of common competence standards (based on job requirements), including precise and unambiguous descriptors will be important;

Building a pool of qualified assessors in firms through common trainings and instructions; or independent external assessors if appropriate;

Standardised and informative documentation of assessment results, made available also to employees;

Taking into account the various quality factors identified in this study, including employee participation and involvement.

For progress to be made in this area (see Box 6 above) increased cooperation between enterprises is necessary. Today only a very small percentage of companies (Cedefop, op.cit.) are involved in some form of collaborative initiative. However, evidence indicates that there is significant interest among companies to engage in collaborative activities on competence assessment, in particular with a view to assessments related to management positions as well as in certain sectors. Apart from increasing the transferability of assessment results, cooperation can also make more formalised and sophisticated appraisal practices affordable especially for small businesses. The formation of inter-firm initiatives could be promoted by relevant public and semi-public institutions at national and

31

European levels (governments, social partners, PES, associations in the HR area etc.) through awareness raising, provision of advice, guidance and training, or also financially.

Key questions regarding validation in enterprises

The following questions are important when addressing validation in enterprises: Can competence assessment carried out in enterprises be used outside the enterprise in

question? To what extent can increased networking support further development of methods and

standard for competence assessment? How can methods for competence assessment be made better accessible for SMEs? How can competence assessment in enterprises be made available for a broader scope of

employees? Can there be a stronger link between validation in the public sector and competence

assessment in enterprises?

4.3. Skills audit and the labour market The 2012 Recommendation furthermore states (3 c) that

‘[…]disadvantaged groups, including individuals who are unemployed and those at risk of unemployment, are particularly likely to benefit from the validation arrangements, since validation can increase their participation in lifelong learning and their access to the labor market’. It further states that (3 d) ‘[…] individuals who are unemployed or at risk of unemployment have the opportunity, in accordance with national legislation and specificities, to undergo a ‘skills audit’ aimed at identifying their knowledge, skills and competences within a reasonable period of time, ideally within six months of an identified need’. (point 3c)

According to the 2014 update of the European Inventory, the definition and interpretation of ‘skills audit’ varies among European countries. However, the reasons for developing and implementing skills audits seem to be shared:

• To help individuals to reflect on their own career background (competences, learning), their motivations and to plan a career path or prepare for the validation of non-formal and informal learning;

• To help the unemployed increase their employability prospects; • To help disadvantaged/vulnerable groups (migrants, women) to increase their employability

and develop their career; • To help employees to plan / move forward in their career or to change their career path; • To help the low-skilled / low-qualified to identify and understand their skills and

competences, and where they can go next with these; • To provide users with a means of marketing their skills and competences (e.g. to employers); • To map needs for further training, taking into account the importance of matching the

individuals’ competences with labour market needs.

Some countries have for many years operated systems of ‘Bilan de competence’ which addresses the objectives listed above. While the evidence produced through the ‘Bilan de competence’

32

sometimes can be used towards a formal qualification, this is normally not the main purpose in any of the countries listed. While the Bilan de competence approach is the best known, also other countries have developed skills audit arrangements of different types. The diversity of approaches currently in existence in Europe points to some critical issues which have to be addressed in the coming period:

Skills audits in Poland In Poland, skills audits are carried out under the provisions of regulations on the Education system and on the Promotion of Employment and Institutions of the Labour Market. These fall under two different systems of career counselling:

One system is organised within the formal education system, where the tasks associated with skills audits are generally carried out by the school professional counsellors and the psychological and pedagogical guidance services. It should be noted that the school system is aimed primarily at formal education, analysing the achievement of learning outcomes set by the curriculum and special educational needs. The services of school counsellors are open only for students of public and non-public schools.

The second guidance system is operated by the district labour offices, where the tasks associated with skills audits are carried out by job counsellors. In this case, the services provided by counsellors are available for unemployed people and jobseekers who are registered with the employment office. This system should be considered more universal than the school system and targeted primarily on the analysis of competences and social skills in demand from employers. As a result of the skills audit, an unemployed person or job seeker receives an action plan including, inter alia, the scope of professional qualifications, skills and competencies that must be completed or acquired in order to find employment. These skills and competencies can be acquired both in the formal, informal and non-formal context. Skills audits can be also carried by the non-public employment agencies on the same basis as in the case of labour offices.

Skills audits can be implemented at different levels and with different coverage. One possibility will to develop a single tool at national level to be used by all unemployed or at risk of unemployment. Possibly being run by Employment services, this could build on and scale up existing labour market counselling, for example by increasing the overall capacity for assessment. It is also possible to envisage a more decentralised approach where the development and implementation of these instruments is delegated to regional and sectoral stakeholders, as demonstrated by Sweden. A third option is to build on existing methods applied within sectors and enterprises.

The choice of methodology for the audit is important and a question is how to balance and combine ICT-based self-assessment with face-to-face, dialogue based approaches. The use of (on-line) self-assessment tools is wide-spread (and increasing) in Europe. The added value of these tools compared to (more costly) dialogue and counselling oriented approaches must be considered. Given that skills audit, as reported by some countries, have an important motivational effect, the way self-assessment and one-to-one assessment are combined will influence the impact of these activities on the employability of individuals.

The orientation of the skills/competence audits needs to be considered. The purpose of an audit is to make visible the achievements of the individual in question, pre-set surveys can overlook and distort important individual achievements. How will, for example, the focus on job-specific skills and competences be balanced with a focus on digital, linguistic as well as transversal skills and

33

competences? While the use of dialogue-based approaches in a face-to-face situation makes it possible to cover all these aspects, the design of self-assessment tools must consider this aspect seriously. It is important to avoid unintended bias created by the tool itself.

A key question is related to the transferability of the outcomes of the skills audit. Will the outcomes be documented in a format which is known to and recognised by employers? Will the outcomes of the audit be accompanied by a training or development plan, as is the case in some European countries?

Taking forward skills audit requires increased cooperation within and between countries. Within countries we see a tendency for skills audits to be carried out in isolation from other forms of validation (normally linked to qualifications and education and training systems). Given the emphasis of the Recommendation on the four stages of validation (Identification, documentation, assessment and certification) it will be important to consider the relationship between the (mainly) employment led audits and the (mainly) education and training led validation arrangements. In some countries we furthermore observe competing arrangements having been introduced by different stakeholders at different levels and contexts. While decentralised approaches may be cost-efficient, lack of clarity regarding the status and quality of outcomes may reduce value to individuals seeking employment. The 2012 recommendation provides an opportunity to increase cooperation between countries, within Europe, in particular allowing for exchange of audit tools and the strengths and weaknesses experienced when using these. European cooperation on a common format for documenting results of audits could also be considered, for example in the context of Europass.

Key questions regarding skills audit

The following questions are important when addressing skills audit: Which existing identification and documentation arrangements exist for people seeking

unemployment or risking to lose their job? Can existing approaches be better coordinated? Can there be cooperation between public and private sector stakeholders? Which are the appropriate methods to be used; how to balance the need for dialogue and

standardised testing? How can the outcomes of skills audit be made visible and credible to employers and others

who receive them?

4.4. Validation in the voluntary sector The 2012 recommendation (4 a) points to the importance of actively involving the voluntary sector in the implementation of validation

‘[…] youth organisations and civil society organisations should promote and facilitate the identification and documentation of learning outcomes acquired at work or in voluntary activities, using relevant Union transparency tools such as those developed under the Europass framework and Youthpass’. (point 4a)

The voluntary (or ‘third’) sector plays an important role in promoting validation of non-formal and informal learning. Non-governmental organisations involved with adult and lifelong learning is a

34

good example of this, as are the organisations working in the youth sectors. All these organisations see non-formal and informal learning as important outcomes of their activities that need to be made more visible.

The distinction between identification and documentation on the one hand and assessment and certification on the other hand is important in this particular context. It is commonly asserted that the learning experiences from voluntary work should be valued in their own right and not assessed according to the standards developed for the formal education and training system. A number of concrete approaches have been made within this sector aiming in particular at identification and documentation of learning (for example the ‘Youthpass’).

Others argue that the experiences from the voluntary sector are highly relevant for progressing in formal education and training as well as in employment and that the experiences from the voluntary sector must be taken into account also for summative assessment and certification. The differentiation between the stages of validation and a clarification of the ultimate purpose of the validation process can in principle accommodate these different possibilities.

Key questions regarding validation in the voluntary sector

The following questions are important when addressing validation in the context of the voluntary sector? In which cases should validation be limited to identification and documentation; in which

cases should validation apply assessment and certification in a summative approach? How can the validation initiatives in the voluntary sector interact with and strengthen

arrangements in the public sector, In particular in education and training?

35

CHAPTER 5. VALIDATION TOOLS

The 2012 Recommendation emphasizes the need for appropriate tools and instruments allowing for validation of non-formal and informal learning, also drawing attention to the relevance of common European tools for transparency and recognition and their possible support to the process:

‘[…] the use of Union transparency tools, such as the Europass framework and Youthpass, is promoted in order to facilitate the documentation of learning outcome’; and asks Member States to assure that (3 j) ‘[…] synergies exist between validation arrangements and credit systems applicable in the formal education and training system, such as ECTS and ECVET’; as well as asking Member States to foster participation of stakeholders that (4 a): ’[…] should promote and facilitate the identification and documentation of learning outcomes acquired at work or in voluntary activities, using relevant Union transparency tools such as those developed under the Europass framework and Youthpass’. (point 3i)

The tools used for validation are important as they will influence the overall quality - the validity and reliability - of the validation process and its outcomes. The validation tools very much influence the way individual learners experience the validation process and determines whether their experiences are captured or not. The following chapter discusses in some detail the

5.1. Selecting tools fit for purpose The 2012 recommendation draws attention to the need to develop and share appropriate tools for validation. Making progress in validation requires that the distinction between formative and summative assessment is clarified. Formative approaches to assessment aim to provide feedback to the learning process or learning career, indicating strengths and weaknesses and providing a basis for personal or organizational improvement. Formative assessment fulfils a very important role in numerous settings ranging from guidance and counselling to human resource management in enterprises. Summative approaches to assessment and validation aim explicitly at the formalization and certification of learning outcomes and are linked to, and integrated into, institutions and bodies authorized to award qualifications. The formative approach to assessment is, thus, important as it draws attention to the identification of knowledge, skills and wider competences, a crucial part of lifelong learning. Summative approaches to assessment need to have a clearly defined and unambiguous link to the standards used in the national qualifications system (or framework). The boundaries between formative and summative assessment, however, are not always clear-cut, in some cases tools van be used for both purposes

Following an initial determination of the learning in question – for example its breadth, depth, currency, complexity and authenticity - it is possible to examine the fitness for purpose of different assessment tools. The following criteria need to be considered for each potentially useful assessment tool:

• validity – the tool must measure what it is intended to measure; • reliability – the extent to which identical results would be achieved every time a candidate is

assessed under the same conditions;

36

• fairness – the extent to which an assessment decision is free from bias (context dependency, culture and assessor bias);

• cognitive range – does the tool enable assessors to judge the breadth and depth of candidate’s learning;

• fitness for purpose of the assessment – ensuring the purpose of the assessment tool matches the use for which they are intended.

It is helpful to distinguish the methods that aim to extract evidence (tests and examinations, conversational methods, declarative methods, observations, simulations, evidence extracted from work) and the methods for documenting and presenting evidence (such as ‘live evidence’, CVs, third party declarations and portfolios). Although this differentiation is not clear-cut (the production of a portfolio may be considered proof of certain competences in itself) it captures the difference in nature between methods that primarily aim to make visible individual competences and those methods that present the collected evidence –for instance, portfolios can be integrally composed of evidence collected through observations and test results or records of participation in learning activities.

Key questions regarding validation tools

5.2. Tools for extracting evidence

5.2.1. Tests and examinations Tests and examinations have the advantage of being a familiar method, which is socially recognized as valid and reliable. Tests are also relatively cheap and quick to administer, when compared to some of the other methods. They have a relatively high degree of ‘authenticity’ of the evidence of the candidates’ own learning outcomes, unlike other methods, such as the declarative method. Tests and examinations can be linked to educational standards more straightforwardly than some other methods.

THE COMBINATION OF TOOLS AND INSTRUMENTS

A combination of methods is frequent in the VAE system in France, where the portfolio method is often combined with interviews and debates. Other methods can be used as part of the assessment/certification phase (related to the type of learning outcomes to be assessed) such as observation of real or simulated working activities. Tests and examinations, on the other hand, are not common practice as part of the VAE, emphasising its focus on alternative assessment.

in Switzerland, the main method used to identify, document and assess learning outcomes acquired through non-formal and informal learning is the validation dossier. The evaluation dossier is a collection of descriptions and documents of what the candidate has learned in

Before the assessment tool can be selected it is important to look at the learning to be assessed. It is generally accepted that the following criteria need to be considered: breadth of knowledge, skills and competences to be assessed; depth of learning required; how current or recent knowledge, skills and competence are; sufficiency of information for an assessor to make a judgement; Authenticity of the evidence being the candidate’s own learning outcomes.

37

non-formal and informal contexts. Often in addition to this, an interview with the assessor is carried out. In some cases, other forms of assessment are used. Interviews are also used by counsellors to help candidates identify implicit skills acquired in non-formal and informal situations, of which candidates are often unaware.

It is because mainstream procedures for validation often aim to result in the award of formal qualifications, that tests and examinations are often used. Employing these methods enhances stakeholder trust and parity of esteem between the qualifications obtained through validation and those obtained through regular formal learning pathways. These methods, correspondingly, tend to be more important in those countries where state recognized qualifications are most important in the labour market. Tests can however be intimidating for those individuals who have had negative experiences in the formal education sector or more generally have poor verbal/ writing skills. Some country reports in the 2014 inventory have argued that this method measures relatively superficial knowledge and learning, and that some competences acquired through non-formal and informal learning may not be picked up and validated through this method. For some trades, for instance journalism or law, however, tests and examinations based on case studies should be able to provide an assessment of the competences of individuals. In other trades, where practical skills and competences are essential, the potential of tests to assess competences is more limited. This, in turn would have implications for the degree of validity of tests and examinations: if knowledge is what they are intended to measure validity will be higher than if they are intended to measure skills and competences or ‘learning’ more wholly.

5.2.2. Dialogue or conversational methods Conversational methods of assessment are divided into two main types: interviews and debates (or ‘discussions’). A presentation (normally focused on declarations or on evidence collected from work) followed by an interview/ debate is also relatively common. Interviews can be used to extract further information documented through other means and probe documented knowledge, skills and competences. It has been argued that in this respect interviews could be considered to have a “supporting function”, which allows for further exploration, instead of being a primary means to elict non-formal and informal learning.

However, interviews can have an important role in themselves at various stages of validation, and they can indeed be very useful tools at the time of identification of acquired competences. When they take place early on in the assessment process they can be used as a screening tool, to check whether further mechanisms to extract evidence should be applied – this can be an important aid to the effectiveness of the system. Such early interviews can have some summative elements, but they can also concentrate on clarifying options, standards and other key system aspects to the applicant. In that respect their formative character can also be pronounced, and lead these interviews to be close to ‘structured guidance sessions’.

Interviews can have a higher degree of validity than tests and examinations as they enable dialogue that can avoid misunderstandings in the formulation of questions and also probing. On the other hand they can be less reliable than tests and examinations unless appropriate protocols are implemented as different interviewers (given their experience, personal characteristics, interviewing style, etc.) may affect the interview outcome. They can also be less fair than exams (in particular

38

when the assessor does not have any previous information about those who are being examined, which can be the case in validation initiatives), as assessors can be influenced by the personal characteristics of those being interviewed. In order to achieve high levels of fairness it is important that a comfortable environment is created for those being examined, in order to decrease stress, and to focus on open-ended questions. Assessors’ experience, communication and facilitation skills as well as thorough knowledge of the assessed learning outcomes (so that relevant and appropriate information can be extracted) are also vital.

5.2.3. Declarative methods Declarative methods are based on the individuals’ independent identification and recording of competences (sometimes against given criteria and sometimes not). However the declaration is normally signed (verified) by a third party. The results of the assessment process are then recorded, for instance in an individual book of competences, but this could also be in a CV or a similar document. Declarative methods are often well suited for formative assessments or as preparation for the identification of competences before summative assessment take place.

Declarative methods are versatile and low-cost. Strengths of declarative methods are their low costs as well as their flexibility, enabling individuals to reflect on their knowledge, skills and competences at their own pace. However, sometimes these processes may suffer from a lack of validity and reliability, due to the absence of external objective assessment. In practice, the validity and reliability of these methods depends on the existence of clear guidelines or standards for the individual to use, on the provision of support or 'mentoring' during the preparation phase, and on the individual’s ability to provide a realistic assessment of his/ her own competences. Help from counsellors can increase the fairness of this method, in particular as individuals from different backgrounds may have different ways in presenting their skills and competences. Counsellors can help to moderate the importance of such biases in the reporting of individuals´ own learning.

Thus, one of the main weaknesses of the declarative method on its own is that it rarely can lead to a clear mapping to existing qualifications or standard frameworks, in particular in the absence of guidance, and rarely lead to the award of qualifications. Third sector organisations in particular do not always have the resources to provide such guidance.

5.2.4. Observations Observations consist of the extraction of evidence of competences from individuals while they are performing everyday tasks at work, which are judged by a third party. The method has relatively greater usage in the private sector (where people have developed real skills in the workplace), but is spreading to other areas as well.

Potentially, the validity of observations is high and it can give access to competences that can be difficult to capture through other means. Observations, as simulations, have the advantage that sets of skills can be assessed simultaneously, and the measurement can be highly valid. They are also fair as individuals are not detached from their usual work environment and placed under additional stress before the assessment. Nevertheless, assessors´ bias may exist as personal characteristics of individuals and their workplace are revealed during the assessment process which may influence the assessment outcome.

39

However, observations are not always possible due to characteristics, safety, time constraints and other factors. Observations can be time consuming, in particular if there is more than one assessor. Moreover, because observations are grounded in an everyday practice, the information obtained through them for the assessment of an individual may be context specific rather than subject to generalization.

5.2.5. Simulations In simulation methodologies, individuals are placed in a situation that fulfils all the criteria of the real-life scenario in order to have their competencies assessed. In some cases they are used when observations are not possible. Their use, however, is constrained by several aspects, in particular the costs. Some situations cannot be observed in real life, for security or other reasons (e.g. reaction of aircraft or bus pilots to extreme weather conditions or a motor/ engine failure).

In some countries, either observations or simulations are incorporated into most validation methods because of their high level of assurance regarding the competence of an individual, as opposed, for instance, to the presentation of evidence produced during work or in other practices. Simulations can, however, be more complex to organise and more expensive than other validation methods. They require a large amount of studies and job analysis to be prepared properly and often involve judgment by a third party

In general, the higher the level of ‘realism’ of the simulation, the more effective the assessment will be. In this respect, it could be argued that simulations should only be used when observations are not possible or have strong potential drawbacks, as already highlighted. Simulations, on the other hand, can solve part of the problems of observations undertaken at work as they can place individuals in a variety of different contexts, to increase the validity of the assessment. The reliability and fairness of this method are often considered high.

5.2.6. Evidence extracted from work or other practice Here a candidate collects physical or intellectual evidence of learning outcomes from work situations, voluntary activities, family or other settings. This evidence then forms the basis of the validation of competences by a third party. Evidences from work can also include written work, such as essays or transcript reviews. Evidence extracted for work is thus different from observations in that the candidate selects the evidence to be assessed, and how that evidence has been produced is not necessarily observed by a third party –in this sense the validity of the method may be lower than the validity of observations, unless it is complemented by other checks that the evidence presented is indeed the product of the work of the candidate. Evidence extracted from work can be usefully accompanied by a declaration (see section on declarative methods above) explaining the knowledge, skills and competences demonstrated in the evidence to facilitate the assessment and make it more valid and reliable.

Because of the way in which evidence is selected (by the learner), assessors need to be aware that in this way they are likely to be judging the best of the work of the candidate, rather than his/her average performance. The fairness of this method is generally deemed to be similar to that of observations. Evidence extracted from work is most often used in the validation of professional competences.

40

5.3. Tools for presenting evidence Evidence extracted through the methods outlined in the previous section needs to be documented during the validation process. Documentation is not only enables assessment but can be seen as an independent outcome of validation, for example supporting job-seekers. A key challenge is to develop documentation methods that have credibility and legitimacy across different sectors and institutions, for example both in the workplace and in the education system. The different learning arenas have different criteria, sometimes making it difficult to establish ‘equivalent competences’ across sectors.

5.3.1. Live evidence The most basic form of presentation of evidence is the use of samples of work or other forms of what is called ‘live evidence’, where individuals and assessors can see or experience the knowledge, skills or competence in question. It should be noted that a live method to present evidence can coincide in time and space with a method to extract evidence. For instance, a presentation can be a method to extract evidence (check a set of competences) and a platform to present live evidence for observation. The mapping of the actual delivery to the set of competences to validate can occur after a period of analysis and reflection that follows the observation of the actual performance. These are, therefore, conceptually two different aspects.

5.3.2. CVs and individual statement of competences CVs are probably the most common way to document individual knowledge, skills and competences. These are often used in job as well as educational applications. CVs can be mere declarations or statements of individuals’ qualifications as well as experience or also identify the competences derived from these –as for instance some sections of the Europass CV do. Individual statements of competences can be aided by competence checklists, or in CV formats that include structured competence sections – cf. Europass CV. It is important to mention that some CV formats, such as the Europass CV can actually be based on portfolios.

5.3.3. Third party reports Third party reports for the validation of non-formal and informal learning can adopt a variety of forms. These can include, for instance, a reference letter, audio declaration or video from supervisors, employers and colleagues, salary level, performance appraisals or certificates (e.g. of attendance to a non-formal training course). Examples include the use of employer statements for labour market applications or as part of the validation processes for admission into HE, or database records kept by companies on the competences of their employees. It is clear that employees sometimes have difficulties in proving their real level of work experience because very often the employment title that they hold does not correspond to the responsibilities and tasks they actually perform -employees tend to perform tasks above their job title level. Employer reports can help to document the actual tasks performed.

5.3.4. Portfolios

Portfolios are one of the most complex and frequently used methods to document evidence for validation purposes. Portfolios aim to overcome the risk of subjectivity by introducing a mix of instruments to extract evidence on the individuals’ competences and can incorporate assessments by third parties. Moreover, they provide the audience with comprehensive insights into the achievements and successes of the learner and cognitive depth. There is evidence of a recent increase in the importance of portfolios. The portfolio method tends to be process-orientated. There

41

is much evidence that the selection process included in portfolio building promotes self-assessment and focuses students' attention to quality criteria. Some countries that provide national guidelines for validation, rather than prescribe validation methods, recommend a stage in the validation process which involves some form of assessment of the content of the portfolio by a third party (e.g. the juries) in order to ensure greater validity. Still, the introduction of third party assessment does not solve all problems. It is important that quality assurance processes are in place to ensure the consistency and transparency of the third party assessment and equality and fairness in the validation process for all candidates.

VALIDATION METHODS IN ACOMPETENCE BASED QUALIFICATION SYSTEM

In the case of Competence based qualifications (CBQ) in Finland, the applicant is interviewed in the initial phase (identification) and the qualification options or possibly qualification requirements are discussed depending on the needs of the candidate. Self-assessments are commonly used. An e-tool (www.osaan.fi) (FI), has been developed which comprises of all CBQs and the assessment criteria. The candidate may make a self-assessment to verify if s/he has the required competences to take the competence tests. The candidate is also asked to provide any evidence of his/her competences related to the learning outcomes of the qualification in question.

In cases where the candidate has reliable documented evidence of competences the documentation is delivered to the Qualification Committee for assessment. The Committee may recognize a qualification unit or units to the entire qualification on the basis of the evidence. In cases where there is no documented evidence or one cannot verify competences on the basis of the documentation, competence tests will take place.

The competence tests are in most cases carried out in an authentic work environment. The candidate has drafted an individual plan on how to make his/her competence test and how it complies with the qualification criteria. The test situation is monitored and assessed by an employer representative, an employee representative and a representative from education (tripartite assessment). The assessors record the results on an assessment sheet, in which the learning outcomes and assessment criteria are described. After the competence test the assessors and the candidate review the test situation and give feedback to the candidate. All assessors must reach a consensus of the assessment results. The documented assessment results are delivered to the Qualification Committee, which will award the qualification and the certificate.

As highlighted, portfolios can include evidence extracted through a combination of methods. It is argued that the kind of reflection and investigation associated with portfolio methods empowers people undergoing validation, which aids them to later obtain jobs or choose appropriate further education. Portfolios can be developed to help disadvantaged people out of social exclusion or into employment by taking into account the specific characteristics of these groups. Building a portfolio is a time-consuming exercise from the point of view of the applicant although it is nevertheless a popular method with applicants, as they have the possibility to show their competences in different ways. It is also time-consuming from the point of view of assessment, as portfolios are often judged by panels and contain different sources of evidence. It is also often the case that assessors need to identify the most relevant evidences in portfolios when these are not well organized. Portfolio assessment is often dependent on good written documentation of the individual’s skills and undocumented or tacit knowledge is harder to identify through this method. The portfolio method also is considered difficult, in particular for some target groups such as immigrants and people with learning difficulties. Some forms of digital portfolios can be helpful to these populations, for whom it

42

is not straightforward to use methods that rely heavily on written documentation, as use can be made of video recording, CD players and related elements.

The most serious risk in the preparation of portfolios is the lack of focus that can occur when applicants prepare them alone or with little mediation from one tutor. Tutor aid and sufficient time for self-reflection are, therefore, crucial in relation to this method’s effectiveness and fairness. In the process of self-assessment against the curriculum standards, guidance should be at hand to explain the theoretical concepts and help the transfer from theory to practice.

The competence assessments carried out by private enterprises are based on some of the same tools as listed above. Some differences can be spotted though, and main tools are presented in box 7 below.

COMPETENCE ASSESSMENT IN ENTERPRISES - INSTRUMENTS Cedefop (3) identifies the following tools as most commonly used by enterprises for competence assessment: Screening curriculum vitae: The act of screening a CV may generally be divided into

three steps, the first pass or scanning for keywords, the second pass which includes reading the CV to evaluate the candidate against the job requirements and the final pass, a full review including a subjective qualitative review of the candidate’s job history.

Interview concentrating on the candidate’s job history, personality, work style and other factors relevant to the job. In panel interviews the candidate is interviewed by a group representing various stakeholders. Panel interviews are becoming more popular in corporate sector. These interviews can include presentations to the panel or debates with experts. This type of interviews offer the opportunity to assess the depth of adequate knowledge of a subject, communication and social skills as well as analytical skills and ability to structure complex information clearly.

Personal review: annual meetings between managers and employees talking about agreements on objectives, assessment of achievement, further education, career development, personal feedback etc.

Observation: A third party assesses the (potential) employee’s behaviour in a particular setting: there is an opportunity to observe real practice. Assessment criteria are set in advance.

Simulation: individuals are placed in a situation that fulfils all the criteria of the real-life scenario to have their competences assessed. The method allows for testing complex interacting skills sets.

(Psychometric) Tests: written, oral or other tests as a means of determining the suitability or desirability of a job applicant. Tests refer to the measurement of knowledge, abilities, attitudes and personality traits and provide direct assessment of specific know-ledge and skills.

Employee surveys: quantitative surveys among their employees. Such surveys especially focus on the satisfaction of the employees with the management, the further education, the flow of information and communication in the enterprise, the

3 Cedefop (2014). The use of validation by enterprises: http://www.cedefop.europa.eu/en/publications-and-resources/publications/3065

43

co-operation in teams, the working conditions etc. Portfolios are also used by enterprises and follows the same principle as identified in

chapter 6.3.4 above

6.5 General considerations regarding tools The quality of any method depends on those implementing it. The level of qualifications/ experience of assessors is one key element in the quality assurance of validation methods. The wide range of available tools can be considered as positive but will also require that counsellors and assessors reflect on what is appropriate and fit for purpose application. It is important to keep in mind that the tools referred to above capture different aspects of the learning experiences in question, for example being able to reflect practical skills or theoretical reflections in varying degrees. Like in formal education, the individual specificity of learning outcomes concerned may require more than one tool, for example a combination of written tests and practical challenges.

44

CHAPTER 6. Concluding remarks

These guidelines have made it clear that the development and implementation of validation arrangements rely on a number of interconnected steps to be taken. Figure 6 illustrates, in a simplified way, how these steps are connected and how they -to a large extent depends on each other.

Figure 6 Developing and implementing validation - interrelationships

While all these steps will have to be taken into account when aiming for national validation arrangements, intiatives linked to particular sector or user-groups will be able to concentrate on a more limited selection of issues and steps. Combined with the questions/check-lists developed for each of the above steps it is our hope that the European Guidelines will prove useful for policy makers and practitioners alike.

45

ANNEX – key questions relevant to the development and implmenetation of validation of non-formal and informal learning

Key questions regarding the four stages of validation

Clarifying the stages of the process makes it possible to distinguish between the different purposes served by validation (formative and summative), and the following questions provide a starting point for this clarification: Have the different stages of the validation process been clearly defined and communicated to the

individual candidates? How is guidance and counselling integrated in the support of a validation process, specially

identification and documentation? Are candidates informed on how to use the outcomes of validation, for further education and

training and/or employment?

Key questions regarding individuals rights and obligations

The individual is at the focus of validation processes and his or her rights and obligations must be treated with care. The following questions provide a starting point: To what extent is validation an individual right?

o If yes, which are the financial and organisational implications? o If no, which are the financial and organisational implications?

Is the privacy and personal integrity of the candidates protected throughout the validation process?

Have explicit procedures been put in place to guarantee confidentiality? Have ethical standards been developed and applied? Are the outcomes of the process the exclusive property of the candidate?

o If no, which are the implications? Which arrangements have been put in place to guarantee fair and equal treatment?

Key questions regarding information, counselling and guidance

To analyse the situations as regards guidance and counselling, the following questions provide a starting point: Are already existing career guidance and counselling services, for example in education and

training, labour market and social services, providing information and advice on validation? Do they provide information on the benefits of validation? Are there particular target groups

that should be prioritise? How can the role of existing services be strengthened to take into account validation? How can existing career guidance and counselling service network to improve the services

offered to candidates? Is there a need for one coordinating institution to make sure that candidates are served

where they live, study and work? How can public and private stakeholders cooperate to offer better information and advice on

validation? Is there a need for specialised guidance and counselling for validation? Are tools supporting (in particular) identification and documentation of non-formal and

informal learning being shared between different guidance and counselling services?

46

Key questions regarding coordination of stakeholders

Coordination of validation must primarily take place at national level, addressing the division of roles and responsibilities between public as well as private stakeholders. To analyse the situation, the following questions can be asked: Which validation arrangements exist and what is their legal and political basis? Has a single legal framework been put in place Which stakeholders are involved? How are these frameworks and stakeholders related to each other and who is responsible for

coordination at different levels? Which information services and resources have been put in place; are these related and

coordinated; does information reach individuals where they live, work and study? Which administrative processes are in place (contact procedures, applications, recording of

results, appeals); are these related and/or coordinated? How are services related and coordinated; how can they reach individuals where they live, work

and study? Are roles and responsibilities (information, guidance, assessment, certification and appeals)

divided in a transparent and credible manner, how can this be improved?

Key questions regarding national qualifications frameworks

National qualifications frameworks are now being implemented across Europe. These frameworks may facilitate the introduction of validation and the following questions point to some key issues to be addressed: Are validation arrangements (all, only some?) seen as an integrated part of the national

qualifications system and as a normal route to qualifications? What is the relationship between validation and the national qualifications framework (NQF)? Can validation be used to support progression between all types and levels of qualifications in the

NQF? How does validation relate to qualifications not included into the NQFs? Is there a link established between validation and (possible) credit transfer and accumulation

arrangements?

Key questions related to standards and learning outcomes

Are qualifications awarded on the basis of non-formal and informal learning referring to the same or equivalent standards as those used for formal education and training?

If no, which other standards are used and how do they relate to formal standards? Are standards used for validation written in learning outcomes?

If no, which are the implications for validation? Who developed the standards and in reference to which sources (education or occupation)? Are there mechanisms in place to provide feed-back on the relevance of existing standards to

validation?

47

Key questions regarding quality assurance of validation

Have explicit and integrated quality assurance measures been put in place for validation? o If existing, are these measures reflecting an explicit and agreed quality strategy?

How are quality assurance arrangements divided between internal and external assurance and control?

Has a system for feed-back from users/customers been put in place or considered? Is there a monitoring system for validation that permits to see the users, beneficiaries and

outcomes of the validation process? Is there a systematic evaluation of results and costs?

Key questions regarding validation practitioners

What are the formal requirements for: o Counsellors and guidance personnel o Assessors o Other practitioners involved with validation

Is there a strategy in place for the professional development of these practitioners? Is the professional development of validation professionals coordinated between different

sectors and arrangements? Can a community of practise for validation professionals be developed, supporting

networking and professional developments?

Key questions regarding validation and education and training

The following questions are important when addressing validation in the context of education and training: Can validation be offered in all parts of the education and training system? Are validation arrangements in the different parts of the education and training system building

on similar or differing principles? Can the validation arrangements in the different parts of education and training ‘work together’

and facilitate progress across types and levels of education? Is there a link between validation and credit transfer arrangements?

Key questions regarding open educational resources

The following questions are important when addressing open educational resources: Are the methods for validating outcomes of MOOC (for example) the same as for other

learning outcomes? How are internal credits (‘for example ‘Badges’) taken into account by validation?

Key questions regarding validation in enterprises

The following questions are important when addressing validation in enterprises: Can competence assessment carried out in enterprises be used outside the enterprise in

question? To what extent can increased networking support further development of methods and

48

standard for competence assessment? How can methods for competence assessment be made better accessible for SMEs? How can competence assessment in enterprises be made available for a broader scope of

employees? Can there be a stronger link between validation in the public sector and competence

assessment in enterprises?

Key questions regarding skills audit

The following questions are important when addressing skills audit: Which existing identification and documentation arrangements exist for people seeking

unemployment or risking to lose their job? Can existing approaches be better coordinated? Can there be cooperation between public and private sector stakeholders? Which are the appropriate methods to be used; how to balance the need for dialogue and

standardised testing? How can the outcomes of skills audit be made visible and credible to employers and others

who receive them?

Key questions regarding validation in the voluntary sector

The following questions are important when addressing validation in the context of the voluntary sector? In which cases should validation be limited to identification and documentation; in which

cases should validation apply assessment and certification in a summative approach? How can the validation initiatives in the voluntary sector interact with and strengthen

arrangements in the public sector, In particular in education and training?

Key questions regarding validation tools

Before the assessment tool can be selected it is important to look at the learning to be assessed. It is generally accepted that the following criteria need to be considered: breadth of knowledge, skills and competences to be assessed; depth of learning required; how current or recent knowledge, skills and competence are; sufficiency of information for an assessor to make a judgement; Authenticity of the evidence being the candidate’s own learning outcomes.

49