How to measure ‘soft’ outcomes to demonstrate ‘hard’ impact

Gail Dervish, WCVA Economic Inactivity Initiatives Manager/ Morgan Armstrong WCVA ILM Development & Support Manager

Definitions & Delivery

Soft OutcomesSoft IndicatorsDistance Travelled

Based on ESF funded WCVA ILM Scheme

Soft Outcomes

Outcomes from training, support or guidance interventions, which cannot be measured directly or tangibly

They may include achievements relating to:• Interpersonal skills such as social skills and coping with

authority• Organisational skills such as personal organisation and the

ability to order and prioritise• Analytical skills such as the ability to exercise judgement,

managing time or problem solving• Personal skills such as insight, motivation, confidence,

reliability and health awareness

Soft Indicators

There is interplay between indicators and outcomes – • Indicators are the means by which we can measure whether

the outcomes have been achieved • The term soft indicators can be used when referring to the

achievements that indicate acquisition or progress towards an outcome

A project, for example, will need to explore whether an individual’s motivation has increased over the length of the scheme –

• This is a subjective judgement but indicators such as improved levels of attendance, improved time keeping and improved communication skills, can suggest strongly that motivation has increased

Distance Travelled

• The progress that a participant makes towards employability or harder outcomes as a result of the scheme intervention

• The acquisition of certain soft outcomes may seem insignificant, but for certain individuals the leap forward in achieving these outcomes is immense

• A consideration of distance travelled is very important in contextualising participants’ achievements

• Indicators (or measurements) of soft outcomes can be used as tools for measuring distance travelled towards labour market participation

Importance of Measuring Soft Skills & Distance Travelled

• Should be an intrinsic element of any scheme evaluation & be considered good practice

• Accrues benefits to the scheme and participant• Improves the process of working with participants

& service delivery• Provides an important context for client needs &

successes

Importance of Measuring Soft Skills & Distance Travelled (General)

• Hard outcomes (jobs, qualifications, etc.) do not show the success of the scheme as a whole

• They are incomplete as an indicator of a participant’s increased employability

• Participants facing multiple barriers may be a long way from acquiring a job or qualification

• Soft outcomes for such participants is a critical indicator of success

Importance of Measuring Soft Skills & Distance Travelled (Participant Level)

• Employers are interested in soft skills, valuing personal attributes & attitudes

• Participants will be at an advantage if they are able to evidence such skills & attributes gained during the scheme

• Staff will be able to demonstrate to participants that they have pre-existing skills & attributes that they did not know existed

Importance of Measuring Soft Skills & Distance Travelled (Participant Level)

• Working with participants to record & monitor distance travelled can give them a sense of ownership over their own personal development

• If participants are made aware of the distance they have travelled, this boosts confidence & motivation

• Participants will understand that the recognition & development of soft skills is part of their integration into the labour market

Types of Soft Skills & Examples of Indicators – Key Work Skills

• Improved team working, problem solving, numeracy, ICT

• Increased language & communication• Completion of work placements• Lower rates of sickness related absence

Types of Soft Skills & Examples of Indicators – Attitudinal Skills

• Increased levels of motivation• Increased levels of confidence• Recognition of prior skills• Increased feelings of responsibility• Increased levels of self-esteem• Higher personal & career aspirations

Types of Soft Skills & Examples of Indicators – Personal Skills

• Improved personal appearance & presentation

• Improved levels of attendance• Improved timekeeping• Improved personal hygiene• Better health & fitness• Greater levels of concentration &

engagement

Types of Soft Skills & Examples of Indicators – Practical Skills

• Ability to complete forms• Ability to write a CV• Ability to understand & follow instructions• Improved ability to manage money• Improved awareness of rights &

responsibilities

How do we do it?

There are numerous & varied methods of measuring soft outcomes.

On the ILM scheme one tool we use is a soft skills questionnaire of up to 77 questions

Have a go!

Baseline Assessment

Baseline assessment is crucial in order to establish a starting point from which a participant may start to demonstrate progress in soft skill outcomes. This can normally be done during the initial assessment phase when client needs are established, barriers to employability identified & targets set

It is important to ensure that methods used for collecting & recording information on soft outcomes & distance travelled are rigorous & targeted to the client group. Selecting & implementing a range of methods to collect data is more likely to capture a full picture of individual progress

Collection & Recording Methods

Action-planning:

This is carried out at initial assessment & periodically reviewed to gauge whether objectives have been met. The plan can include personal goals, priorities & reflections on progress

Reviews:

Improvements over time can be noted on a regular formal & informal basis. This system is reliant on sound judgement from the client & staff member. It will not provide an absolute or formal measure on distance travelled

Collection & Recording Methods

Daily diary or personal journal –

Clients can be encouraged to write about progress towards soft outcomes. Issues of confidentiality should be considered

In-depth reflection –

This could include weekly, work placement & end of course evaluations. Questionnaires are also an important tool for this purpose. This would allow the participant to consider & review their progress. It could provide valuable portfolio evidence & is easily compared against baseline data

Collection & Recording Methods

Recorded observations - It is important to have comprehensive documentation systems

that will allow for the recording of anecdotal evidence of progress & outcomes. This method requires a good level of observer skill, lack of observer bias & an approach that does not influence participant behaviour during the process

Portfolio compilation –

This would include evidence of tasks completed successfully or progress towards them. An evidence based portfolio would be a concrete output that could be presented to an employer

Case Study: ILM Scheme

External Evaluation Report:

From: Wavehill Second Annual Report:

External Evaluation Report:

External Evaluation Report:

External Evaluation Report:

Soft Outcomes: Further details Outcomes Star. Developed by Triangle Consulting for use by homelessness

organisations. Take a look at http://www.outcomesstar.org.uk/

WEFO’s own: A practical guide to measuring soft outcomes and distance travelled

The Institute for Employment Studies have produced ‘A guide to measuring soft outcomes and distance travelled’ Http://www.esf.gov.uk/_docs/distance1.pdf

Many other systems out there: Catching Confidence tool, SOUL Record, Spirit Level etc.

Want to share thoughts? ilm@wcva.org.uk