Methodological Guidelines for Assessment of Profiles by Vocation October 2006

PHARE 2003-BG 2003/2004 937 05.03

Magda Kirsch, Yves Beernaert & Luk Indesteege

1

Methodological Guidelines for Assessment of Profiles by Vocation

October 2006

PHARE 2003-BG 2003/2004 937 05.03


2

Objectives

• Methodological assistance

• For assessement of workers in the workplace

• Based on ‘learning outcoms’ from the national Bulgarian SER’s

• Linked to European VET policy

PHARE 2003-BG 2003/2004 937 05.03


3

Definition of assessment

Assessment means :

the process of collecting evidence and making judgements on whether competency has been achieved to confirm that an individual can perform to the standard expected in the workplace as expressed in the relevant endorsed industry or enterprise competency standards or the learning outcomes of an accredited course.

PHARE 2003-BG 2003/2004 937 05.03


4

Competences

A competence is an integrated set of knowledge, skills and attitudes that can be employed efficiently by the candidate

• Knowledge Knowing facts,understand

• Skills Being able, apply

• Attitudes Wanting, behave

PHARE 2003-BG 2003/2004 937 05.03


5

Process of Assessment

When discussing the process of assessment of vocational competences we want to focus on – Why?– What? – How?– Who? – When?

PHARE 2003-BG 2003/2004 937 05.03


6

Why assessment ?

• To check whether the workers meet the professional standards

• To certify workers achievements in their profession

• To diagnose the learning needs• To have feedback for future instruction• To select for the next stage in work• To encourage workers• To evaluate AP(E)L

PHARE 2003-BG 2003/2004 937 05.03


7

What to Assess?

Classroom based assessment changes:

• Learning outcomes• Workplace based• Independent of

learning pathway

PHARE 2003-BG 2003/2004 937 05.03


8

WHAT do we assess?

We assess the core competences described in the SER.A competence is an integrated set of knowledge, skills and attitudes that can be employed efficiently by the candidate

• Knowledge Knowing facts,understand• Skills Being able, apply• Attitudes Wanting, behave

PHARE 2003-BG 2003/2004 937 05.03


9

WHAT do we assess?

• We assess the learning outcomes described in the SER.

• Learning outcomes are the basis for an assessment program that focuses on what workers have to do to be successful in their job.

• Assessment identifies what action the worker should be able to perform. It also denotes the conditions under which the workers should demonstrate mastery.

PHARE 2003-BG 2003/2004 937 05.03


10

Example: Refrigeration technician in food industry Unit 1: Electrical maintenance

Core competences

Learning outcomes: To achieve this unit the learner should be able to:

1.1. Read electrical diagrams

To indicate the parts of electrical circuits on a diagram and in reality

To achieve electrical measurements based on diagrams

1.2. Operate switch-boards

To distinguish between main circuits and steering circuits To measure if a circuit is working correctly

1.3 Operate electrical drives

To connect a 3-phase AC motor correctly To explain the thermal and magnetic safety circuits on a drive

1.4 Work safely at electrical installations

To apply the safety signals correctly To switch in a safe way according the procedures To explain the different ground connection systems

PHARE 2003-BG 2003/2004 937 05.03


11

WHAT do we assess?We can also assess the level of masteryLevel 1 - vocations involving routine activities performed

under routine conditions

Level 2 - a range of complex activities performed in changing conditions

Level 3 - a range of complex activities performed in changing conditions, as well as responsibilities for the performance of others

Level 4 - a wide range of complex activities performed in changing conditions, as well as managerial responsibilities for the performance of others and for allocation of resources.

PHARE 2003-BG 2003/2004 937 05.03


12

HOW do we assess?

Principles of assessment

– based on understanding of workers’ learning + workplace– accommodate for individual differences (flexible)– clearly explained to workers (e.g. criteria)– valid and have a reliable and fair process – allow for timely feedback– allow learners and trainers to reflect on their learning

PHARE 2003-BG 2003/2004 937 05.03


13

HOW do we assess?Key elements of assessment:• Assess in a congruent way; how the competence is implemented in the

professional situation: Plan, Do, Check, Act ;• Assess authenticity of products developed, of assignments or activities

carried out in real professional situations;• Assess on the basis of competence evidence demonstrated by the

worker and which can be compared with the SER;• Assess in an integrated way: theory and practice have to be assessed

together.• Criterion-related assessment: assessment is based on criteria taken

from the SER;• Assessment independent from the learning pathway: based solely on

the demonstration of the competences by the worker • Assessing in a specific professional context by a professional assessor

PHARE 2003-BG 2003/2004 937 05.03


14

HOW do we assess?Assessment tools should be:

– Valid, really linked to the prfile

– Reliable, repeatable

– Practical, usable

PHARE 2003-BG 2003/2004 937 05.03


15

Valid assessment?To be valid, assessment tools should:

– Be appropriate to the SER– match the type of skills listed in the LO– assess a broad range of the LO– have a weighting that reflects the relative

importance of the range of skills in the specific LO. – have a good balance between theory and practice

PHARE 2003-BG 2003/2004 937 05.03


16

Reliable and practical assessment?

Reliable Consistant results with different candidates Consistant results with different assessors

PracticalTakes account of available resources and

equipment Uses an acceptable amount of time

PHARE 2003-BG 2003/2004 937 05.03


17

Assessment tools 1. Observation in real or live work

situation

2. Simulation3. A brief case (scenario) focusing on

action and activity in a context 4. An extended case or scenario 5. Assignments for presentations 6. 360° feedback 7. Portfolio

PHARE 2003-BG 2003/2004 937 05.03


18

360° feedback

Assessee

CustomerLevel+1

Peer

level-1suplier.Level-1

Peer

Level+1

PHARE 2003-BG 2003/2004 937 05.03


19

Assessment tools Assessment tools :

– Checklists– Questions and Answers– Multiple choice questions– True/false questions– Observation in real or live work situation – Simulations– Cases– Scenarios– ……

PHARE 2003-BG 2003/2004 937 05.03


20

Important remarks on HOW to assess

• Assessment tools or techniques should be linked to the learning outcomes

• Attitudes and personal competences are difficult to assess because of subjectivity Therefore use a measuring scale

• Assess those personal competences that are relevant

PHARE 2003-BG 2003/2004 937 05.03


21

Linking LO to assessment criteria

Learning outcomes

Methods, tools

Features Issues Criteria

Practical skills• making and producing

things• manipula-ting materials

and objects

-Observation

-Micro tasks (smaller components assessed separately and built up over time)

- Checklists

- Practical tests

- Progress sheets

- Scenarios

Direct and indirect evidence

- Assessment increasingly involves workplace assessors and joint -assessment decisions.

- Criteria need to be discussed among assessors.

-Range and scope of practical skills are specified

- Simplicity or complexity of materials

- Working to time scales and deadlines

PHARE 2003-BG 2003/2004 937 05.03


22

SAM: Scale for Attitude Measurement example 1.1 initiative

Insufficient Low Good Very good

Notices no work and

takes no initiative.

Notices only work

when the manager is

around and takes

initiative in that case.

Notices werk, and

takes initiative only

after some trigger.

Notices work and takes

initiative

spontaneously.

Has only attention for

assignments that she/ he

likes.

Executes sometimes

assignments that

he/ she doesn’t like.

Executes all

assignments with

quality and devotion

Executes all

assignments

spontaneously and with

devotion, even those

he/ she doesn’t like.

Waits for others to take

action and does nothing

him/ herself.

Takes only action

when told to or when

the manager is around

Takes action

spontaneously after

assignment of tasks

that are part of the

job description.

Takes action

spontaneously after

assignment of tasks,

even if they are no part

of the job description.

Notices no problems,

mentions no problems.

Notices problems, but

don’t react.

Notices problems and

mentions them.

Notices problems and

looks spontaneously for

solutions if possible.

PHARE 2003-BG 2003/2004 937 05.03


23

How to assess with scenarios

1. Group the competences into families of situations with professional significance

2. Select a family of situations which can be integrated into a scenario and its components

3. Choose an integrative situation in relation with the selected family of situations

4. Decide on the criteria and the indicators5. Decide on the weighing to be used6. Make references to all the competences involved

PHARE 2003-BG 2003/2004 937 05.03


24

WHEN do we assess?

Assessing is a continuous process

For strategic reasons we distinguish:– Diagnostic assessment: before entry of procedure– Recognition : AP(E)L– Formative assessment; continuous, more

process than event;– Summative assessment; at the end of period of

assessment

PHARE 2003-BG 2003/2004 937 05.03


25

Accreditation of PRIOR …..

• Accreditation of prior learning (APL)• Accreditation of prior certificated

learning (APCL)• Accreditation of prior experiential

learning (APEL)• Accreditation of prior certificated and/or

experiential learning (AP E/C L)• Accreditation of prior learning and

achievement (APL&A)

PHARE 2003-BG 2003/2004 937 05.03


26

Importance of AP(E)L in assessment

• Recognition of experience

• Allowing people without formal qualifications to acquire qualifications

• Enhancing the chances of low-skilled or workers

• Allowing career changes

Respect European principles for validation of non-formal and informal learning

PHARE 2003-BG 2003/2004 937 05.03


27

Common European Principlesfor the identification and validation

of non-formal and informal learning

Elements of the identification and validation of non-formal and informal learning

• Individual entitlements

• Obligations of stakeholders

• Confidence and trust

• Credibility and legitimacy

PHARE 2003-BG 2003/2004 937 05.03


28

Who is involved in assessment

of competences in the workplace?

• the worker himself (self-assessment)• his peers (peer-assessment) • mentors in the company• accredited assessors : experienced practitioners

who are present during practical tests or to assess portfolios of workers

The assessment procedure is controlled by a verifier from the ministry of labour, the ministry of education or the leading industrial body

PHARE 2003-BG 2003/2004 937 05.03


29

Example of an assessment gridAssessed competence : Name of the candidate :

Performance criteria

Method of assessment

Details of evidence

Successful or not

Assessors Date

1.1

1.2

1.3

1.4

PHARE 2003-BG 2003/2004 937 05.03


30

Documents

Methodological Guidelines for Assessment of Profiles by Vocation October 2006