Embed Size (px)
Citation preview
Improving functional skills qualifications
Bryan HorneFriday 13th February 2015
Improving functional skills qualifications
1. Functional skills – background
2. Why we did our work
3. What we did
4. Improvements being made
5. What next
BACKGROUNDThe market, assessment and use
Functional skills - history
• Functional skills qualifications are applied qualifications in English, mathematics and Information and Communications Technology (ICT).
• Available from entry level through to level 2 in England.
• Allow students to demonstrate they have practical skills in literacy, numeracy and IT that help them to live and work confidently, effectively and independently.
• Developed and piloted from 2007-2010 by awarding organisations and the Qualifications and Curriculum Authority.
• The content and the regulatory criteria were published in 2009 for qualifications that were available for first teaching in 2010.
Functional skills – criteria
• The ‘skills standards’ are set out in the regulatory criteria - based on the Adult Literacy and Numeracy Curriculum.
• The Criteria were reissued in 2011 and are available on Ofqual’s website: https://www.gov.uk/government/collections/functional-skills-qualifications-requirements
• Made up of components: ICT and mathematics have a single component and English three components: reading; writing; and speaking, listening and communication.
• ‘Size’ of qualification - 45 guided learning hours.
• Assessment is on a ‘pass-fail’ basis – there are no grades. The criteria set out the skills that must be demonstrated to pass each subject.
• Assessment methodologies used by different awarding organisations vary.
Functional skills - market
• Used for apprenticeships, in adult education, and as a free-standing qualifications for 14-19 year olds.
• Some awarding organisations also cater for their use at Key Stage 3 (11-14 year olds) as a ‘stepping stone’ to a GCSE, although the qualifications were not designed for this purpose.
• Represent ~7% of all regulated qualifications taken in England; more functional skills qualifications are awarded than any other type of qualification apart from GCSEs.
• Number of qualifications achieved has increased from just under 300,000 certifications per year in 2010/11 to just over a million in 2013/14.
Functional skills - market
Functional skills certifications 2009-14 at entry level, level 1 and level 2
0
50,000
100,000
150,000
200,000
250,000
300,000
350,000
400,000
450,000
2009-10 2010-11 2011-12 2012-13 2013-14
Nu
mb
er o
f ce
rtif
icat
ion
s
Academic year (Oct-Sept)
Functional Skills market (all subjects): 2009-14
Entry
1
2
Source: Ofqual certification data 2009-2014
Functional skills - market
Functional skills qualifications: market share 2013-14
41%
38%
8%
6%
7%
Functional Skills total market share by awarding organisation: 2013-14
Pearson Education Ltd
City and Guilds of LondonInstitute
OCR
AQA Education
Others
Source: Ofqual certification data 2009-2014
WHY WE DID OUR WORKOur rationale
Functional skills – high profile
Functional skills – rationale for our work
• Validity of qualifications is at the heart of the way we regulate.
• A valid qualification effectively and reliably measures what it is designed to measure. With a clear purpose established in consultation with end-users – employers, or higher or further education providers.
• Purpose drives content and the assessment methodology used
• Content, assessment and performance standards must be consistent, so that the level of skill needed to gain a particular grade or pass is comparable over time and between awarding organisations.
• Standards can come under pressure when qualifications are put to a use for which they are not designed.
• Principles as relevant to functional skills as to any other qualifications – a key driver for our work.
WHAT WE DIDHow we approached our work
Functional skills – our approach
• From May to August 2014 we worked with each of the 13 awarding organisations offering all 28 level 2 English and mathematics functional skills qualifications.
• We looked at assessment materials from each of the qualifications.
• We met each of the awarding organisations during and following our review to discuss their systems, resources and processes.
• Three of the 16 awarding organisations offering functional skills do not offer English or mathematics qualifications at level 2.
Functional skills – our approach
* In 2011 EDI merged with Pearson. Both awarding organisations offered functional skills qualifications. Pearson EDIs functional skills qualifications will be discontinued by mid-2015.
Awarding organisation
City & Guilds of London Institute
Pearson Education Ltd (including Pearson EDI)*
AQA Education
Ascentis
EAL (Excellence, Achievement and Learning Limited)
IQ
NCFE
NOCN (National Open College Network)
OCR
SQA (Scottish Qualifications Authority)
Skillsfirst Awards Ltd
VTCT
WJEC
Functional skills – our approach
0
10,000
20,000
30,000
40,000
50,000
60,000
Nu
mb
er
of
ce
rtif
ica
tio
ns
Awarding organisation
Level 2 Functional Skills market share by awarding organisation: 2013-14
English
ICT
Mathematics
Source: Ofqual certification data 2009-2014
Functional skills – our approach
• A four-week call for evidence on functional skills qualifications during June 2014.
• A short survey was publicly available on our website and advertised to a range of stakeholders.
• Questions were designed to gather stakeholder views about how far the qualifications met their intended purpose of improving skills - for life and learning, and also how far they met employers' needs.
IMPROVEMENTS BEING MADE
Functional skills – improvements
Key areas where improvements are being made
Examples of how awarding organisations are doing this
Changes students, teachers and others might see
1. Improving the quality of assessment materials, including:
More higher level / stretch questions
Better coverage of the range of the skills as specified in the regulatory criteria
Better aligned question papers and mark schemes
Reviewing and strengthening test development procedures
Development of improved test specifications
Strengthened quality assurance processes for assessment materials
Increased pre-testing of questions
Questions covering a wider range of areas and in some areas providing additional challenge
More realistic context to questions
More open questions that allow students to demonstrate problem-solving skills
Fewer question paper errors
Functional skills – improvements
Key areas where improvements are being made
Examples of how awarding organisations are doing this
Changes students, teachers and others might see
2. Reducing the risks of malpractice and maladministration
Increased controls over internal (school and college-based) assessment of speaking, listening and communication (SLC)
Increased security of assessment materials
Review and update of security policies and procedures
Risk-based sampling of live SLC assessment and risk-based monitoring visits
Increased monitoring of SLC results, particularly in comparison to reading and writing elements, to enable identification and investigation of test centres with unexpected results
Improved standardisation procedures between different centres
New training and procedures for moderators
More regular refreshes of test content, for example through the introduction of a paper / item bank
Reviewing whether the approach to test windows might put the security of test materials at unacceptable risk
Updated guidance to schools and colleges on security of assessments
Updated guidance on internal assessment of SLC
More observation of SLC assessments
More centre visits by awarding organisations
More regular refreshes of tasks and question papers
Possible reduction in the length of test windows
Functional skills – improvements
Key areas where improvements are being made
Examples of how awarding organisations are doing this
Changes students, teachers and others might see
3. Improving standard setting procedures
Greater scrutiny at awarding to make sure skills standards are effectively met; for example, more full script inspections and borderline checks at panels
Revised exemplar materials for centres
Additional examiner training
Appointment of separate principal examiners for each subject
Updated guidance to centres
Functional skills – improvements
Key areas where improvements are being made
Examples of how awarding organisations are doing this
Changes students, teachers and others might see
4. Improving evaluation of whether qualifications are meeting user needs
Commissioning user research
Development and implementation of a stakeholder engagement plan
Analysis of destination data and tracking of candidate progression
Introduction of subject reviews
End-users have greater opportunity to feedback views to awarding organisations
Functional skills – call for evidence
• A significant level of confidence in the qualifications, and that they met employers’ needs.
• Support was often due to the value attributed to the real-life focus of assessments.
• The four features of functional skills qualifications most often mentioned as ‘very important’ by respondents were: • consistency of marking (91%), • prompt delivery of results (90%), • accessible tests (88%), and • frequent opportunities to take the test (84%).
• Analysis of responses identified themes for change or improvement that could be made in several areas.
Given the numbers that responded and the self-selecting nature of the sample, we cannot assume that responses are representative.
WHAT NEXTNext steps and future plans
Next steps - 1
• Awarding organisations are working to improve each of their qualifications – based on feedback from our work.
• We will take additional steps to help secure greater consistency of standards between different awarding organisations. We will:– set out guidance on good and poor practice in functional skills
question design, both for on-line and paper-based approaches, and
– work with awarding organisations to secure their commitment to a consistent approach to setting and maintaining standards between different awarding organisations.
• We will also consider the impact of flexible and on-demand assessment on standards and comparability.
Next steps - 2
• In the autumn we will carry out a further review of functional skills qualifications and we will report our findings.
• We will follow up on the changes made by awarding organisations and check whether they are fully meeting our regulatory requirements.
• We will not limit our audit to level 2 English and mathematics but will look across subjects and levels.
• Where there is evidence that our requirements are not being met, we will consider formal regulatory action.
Next steps -3
• Should we find that the provision of additional informal guidance is not sufficient to achieve greater validity and consistency of standards, we will consider whether we need to take further action.
• We will review the evidence from our audit, including input from stakeholders, and the findings from the ETF review, to determine how we can best secure the quality and validity of functional skills qualifications in the longer term.
Questions
Bryan Horne Associate Director Standards for Vocational Qualifications and Apprenticeships, Ofqual T: 02476 716729 M: 0791 2775036
