Value Added and Distance Travelled and the Learner Achievement Tracker

Yvonne FullwoodNew Measures of SuccessLSC, National Office

Context of the New Measures

• Fair, valid and ‘fit for purpose’ measures

• Four partners: LSC, DfES, ALI, OFSTED

• ‘Basket’ of measures to contextualise performance

• Application across the sector

• No additional bureaucratic burden

New Measures of Success

1. Qualification success rate (QSR)

2. Value Added and Distance Travelled measures (VA/DT)

3. Recognising and recording progress and achievement (RARPA)

4. Learner satisfaction

• Learner destinations

• Provider responsiveness to employers

• Value for money (VFM)

• Staff measures

Purpose of VA and DT for 16-19 learners

Improvement and inspection:• LSC: Value Added and Distance Travelled data for 16-19 learners• Learner Achievement Tracker (LAT)• Provider Gateway

DfES accountability:

• VA for 16 – 19 Learners included in the School and College Achievement Tables pilot

Overview (1)

• Measures the progression in qualifications of 16-19 learners based on their prior attainment

ie, how well do learners do post-16 based on their achievement at Key Stage 4?

• Cover the majority of larger, approved qualifications taken by 16-19 learners, including vocational and ungraded qualifications

Overview (2)

• VA for 16-19 learners will measure achievement in Level 3 graded qualifications, for example:

•GCE A and AS level•AVCE•BTEC National Diploma

• DT for 16-19 learners will include other approved Qualifications at levels 1,2 and 3 for example:

•NVQ •Foundation and Intermediate GNVQ•GCSE•BTEC First

Learner AchievementTracker (LAT)

• Produces reports for use by teachers, managers, inspectors and LSC

• Accessible via the Provider Gateway

• Uses most robust statistical methods available and includes full national sample

• Builds on proprietary systems currently available

• Software and data free of charge to all LSC-funded providers and LAs

VA and DT data

• Data used to calculate the VA and DT for 16 – 19 learners is gathered directly from the the awarding bodies

• Additional data from the ILR is required to calculate DT for 16 – 19 learners

• Providers will not be involved in collecting data for the measure

• Providers will be involved in checking the accuracy of the VA data

Issues for further exploration

• Learners with missing or no prior attainment

• Context variables

• Customising the LAT

• Treatment of AS levels in the VA measure

• Treatment of fails in the VA measure

• Use of success rate for DT

LAT piloting arrangements

January 2006 Updated Quality Improvement Packs availableJanuary 2006 Pilot LAT will be available via the Provider

Gateway. This will contain Work-Based Learning Distance Travelled (DT) data and un-amended Value Added (VA) data for schools and colleges

March 2006 DT data for colleges and amended VA data for schools and colleges.

January 2006

to April 2006

Short term evaluation.

November 2006 The LAT software released for reflection and quality improvement purposes. Ofsted continues to pilot the LAT.

More information

• http://www.lsc.gov.uk/nms• Measuring Success Newsletter• The Quality Improvement Pack• Implementing New Measures of Success• Local LSC

The Learner Achievement Tracker (LAT) Reports

LAT Output reports

Type of LAT report What the report shows

Summary Charts Overview of performance across different

qualifications and across different subjects

within a given qualification type compared to

the national average

National comparison

graph

Performance in a given qualification/subject

compared to the national average across

the full range of prior attainment

National chances charts

Chances of a learner with a given prior

attainment achieving a particular outcome

Summary Chart for all Qualifications

A Level Summary Chart

National Comparison Graph: A Level D&T Product Design

National Comparison Graph: A Level D&T Product Design

A Level Mathematics

National Comparison Graph: A Level Maths

A Level Psychology

National Comparison Graph: A Level Psychology

A Level D&T Product Design Chances Chart (40-46)

34%

A Level D&T Product Design Chances Chart (40-46)

A Level Mathematics Chances Chart (40-46)

A Level Psychology Chances Chart (40- 46)

Output type: Distance Travelled national comparisonchart (fails included)Organisation: fictional organisationQualification type: NVQ level 2Subject/ Subject group: Engineering, Technology andManufacturingProvider sample size: 40National sample size: 4,267Data source: 16-19 learners completing in 1900/1901

Prior attainment of learners, banded(Average QCA points at GCSE/ GCSE equivalent (40=C))

22-28 28-34 34-40 40-46 46-52 52-58

DT

sc

ore

(%

)

25% 23% 23% 19% 20% 22%

Overall DT score across allprior attainment: 22%

We are 95% confident thatthis score lies betweenand

6%36%

DT score = the percentage differencein the chances of success in thisorganisation, compared to anationally average organisation,taking prior attainment into account.

0%= national average

0%

-100%

100%

50%

-50%

25%

75%

-25%

-75%

Key

Performance is significantlyabove or below the nationalaverage, taking priorattainment into account

Performance is similar to thenational average, takingprior attainment intoaccount.

95% confidence intervals

Number inbold

DT score for band of priorattainment

The number of learners isfewer than five, so nojudgement can be made.

*

Example of a Distance Travelled National Comparison Graph

Further Information

• The Quality Improvement Pack contains detailed information about the interpretation and application of each type of LAT reports

• www.lsc.gov.uk/nms