Mike TreadawayDirector of ResearchFischer Family Trust

Using FFT Live

Secondary Schools

Contents

Data Landscape2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012

Value-Added

Contextual Value-Added

3 Year Summaries

Estimates

FFT

DfE

Whether or not to Contextualise?Type of Analysis Advantages Disadvantages

Raw Outcomes e.g. %L4+ Provides a view as to whether pupils are attaining the

knowledge and skills needed for their next stage of

education?

When used to compare schools, depends strongly

upon attainment of the school’s intake.

Value-Added Enables a focus upon progress irrespective of context e.g.

How well are our FSM pupils doing compared to our non-

FSM pupils?

Comparing value-added outcomes for schools with

different contexts and make-up of intakes can be

misleading.

Contextual Value-Added Aims to take account of factors which are outside the school’s control and which,

from analysing national data, can be shown to influence

progress.

If used to look ahead (estimates) then danger of ‘self-fulfilling prophecy’ for

pupils in groups who currently under-perform nationally.

Differences – VA and CVA• Compare PA (Value-added) and SX (Contextual Value-Added)

outcomes for KS2->4 in 2009/10.– Use KS4 Capped Points Score as outcome measure.– Look at how many change significant state.– Analyses by grouping schools in different socio-economic contexts.

School VA compared to CVA

Context Better Worse

Q1 30% 2%

Q2 9% 11%

Q3 7% 17%

Q4 14% 13%

School Change in VA Score

Context Avg Min Max

Q1 +7 -19 +25

Q2 -1 -53 +38

Q3 -3 -62 +32

Q4 0 -54 +34

To get some idea of the likely change for your school, compare PA and SX value-added outcomes on FFT Live or in FFT Self-Evaluation reports.

To get some idea of the likely change for your school, compare PA and SX value-added outcomes on FFT Live or in FFT Self-Evaluation reports.

In Lambeth, most schools have VA better than CVA – this is mainly related to the progress of ethnic

minority pupils.

In Lambeth, most schools have VA better than CVA – this is mainly related to the progress of ethnic

minority pupils.

Recommendations

Expected Progress• DfE Approach

– Whole Levels, Subject Specific Input– So, for example, 68% L4 in Mathematics at KS2 attained grade C or

higher at KS4 in 2011

• FFT Approach– Differentiate within levels (using Fine Grades)– Take into account prior attainment in English and Mathematics

• DfE Approach– Whole Levels, Subject Specific Input– So, for example, 68% L4 in Mathematics at KS2 attained grade C or

higher at KS4 in 2011

• FFT Approach– Differentiate within levels (using Fine Grades)– Take into account prior attainment in English and Mathematics

KS2 Maths KS4 C+ Maths

4C 47%

4B 69%

4A 86%

KS2 English C+ KS4 Maths

3 50%

4C 61%

4B 69%

4A 75%

5 80%

FFT Live

Includes:• Latest

updates• Future

releases• Help Files

Includes:• Latest

updates• Future

releases• Help Files

SETS

• Key Changes for 2011/12– Default changed to PA (Prior Attainment)– SX (Contextual VA) continues to be available– Options to SORT and FILTER (Pupil Groups) available on range of reports– Default layout for KS4 and KS5 Subject Summary VA Simplified

• Key Changes for 2011/12– Default changed to PA (Prior Attainment)– SX (Contextual VA) continues to be available– Options to SORT and FILTER (Pupil Groups) available on range of reports– Default layout for KS4 and KS5 Subject Summary VA Simplified

Self-EvaluationSignificant Areas Grid• Range of pupil groups and indicators• KS2->3, KS2->4, KS3->4• 3 Year summary, Overall + Trends

Significant Areas Grid• Range of pupil groups and indicators• KS2->3, KS2->4, KS3->4• 3 Year summary, Overall + Trends

Significant Areas Detail• Shows actual attainment and VA/CVA scores for

each year• Can be accessed by clicking on Significant Areas grid

Significant Areas Detail• Shows actual attainment and VA/CVA scores for

each year• Can be accessed by clicking on Significant Areas grid

Pupil Value-Added• Overall Indicators KS2->3, KS2->4, KS3->4• Individual Subjects KS2->4, KS3->4, KS4->5 • VA or CVA (KS3 and KS4 outcomes)

Pupil Value-Added• Overall Indicators KS2->3, KS2->4, KS3->4• Individual Subjects KS2->4, KS3->4, KS4->5 • VA or CVA (KS3 and KS4 outcomes)

Subject Summary Reports• KS4 and KS5 Individual Subjects• Summary Report (most users find this is sufficient)• Detail Report

Subject Summary Reports• KS4 and KS5 Individual Subjects• Summary Report (most users find this is sufficient)• Detail Report

EstimatesSchool Summary Estimates• Range of Indicators and Pupil Groups• A – PA 50th Percentile; B – SE 50th Percentile,• D – SE 25th Percentile; S – School Previous

School Summary Estimates• Range of Indicators and Pupil Groups• A – PA 50th Percentile; B – SE 50th Percentile,• D – SE 25th Percentile; S – School Previous

School Subject Estimates• Core Subjects including Expected Progress for English and

Mathematics• Outcomes for all grades (A* to G)• Some pupil groupings available

School Subject Estimates• Core Subjects including Expected Progress for English and

Mathematics• Outcomes for all grades (A* to G)• Some pupil groupings available

Pupil Summary Estimates• Types A, B, D• Range of indicators (e.g. 5A*C, 5A*CEM, Points)

Pupil Summary Estimates• Types A, B, D• Range of indicators (e.g. 5A*C, 5A*CEM, Points)

Pupil Subject Estimates• Show likelihood of attaining all grades (A* to G or A* to E)• Also provides option to show ‘most likely grade’ (GM)

Pupil Subject Estimates• Show likelihood of attaining all grades (A* to G or A* to E)• Also provides option to show ‘most likely grade’ (GM)

Using Estimates

• Estimate -> Prediction -> Target• Consider option of range - realistic and ambitious• Don’t add up ‘estimated levels / grades’

• Estimate -> Prediction -> Target• Consider option of range - realistic and ambitious• Don’t add up ‘estimated levels / grades’

Two basic approachesTwo basic approaches

• Start with pupil data• Take into account other

factors• Add up and compare with

overall estimates• Revise as necessary to

arrive at overall targets

• Start with pupil data• Take into account other

factors• Add up and compare with

overall estimates• Revise as necessary to

arrive at overall targets

• Use overall estimates and evaluation of past performance to draft overall targets

• Use ‘RANK’ option in FFT School Subject Estimates to set overall figure at your overall targets

• Click on link to pupil estimates

• Use overall estimates and evaluation of past performance to draft overall targets

• Use ‘RANK’ option in FFT School Subject Estimates to set overall figure at your overall targets

• Click on link to pupil estimates

Adding up estimated levels – why not?• Consider a group of 50 ‘identical’ pupils• They all have the same estimates:

• Consider a group of 50 ‘identical’ pupils• They all have the same estimates:

G F E D C B A A* GM HGM GA

0% 0% 10% 20% 40% 20% 10% 0% C 30% B

If we add up the ‘GM’ gradesIf we add up the ‘GM’ gradesE D C B A

0 0 50 0 0

If we average the ‘chances’ for each level and multiply by

number of pupils

If we average the ‘chances’ for each level and multiply by

number of pupils

E D C B A

10% 20% 40% 20% 10%

5 10 20 10 5

Note : Adding up predicted or target grades is fine – assuming that they are not simply just using

unmoderated estimates!

Note : Adding up predicted or target grades is fine – assuming that they are not simply just using

unmoderated estimates!

KS4 Estimates

• GM – Grade likely to be achieved by 50% of more ‘similar’ students nationally

• HGM – likelihood of attaining above GM• GA – one grade above GM• EP – Expected Progress from KS2->KS4 (English and Mathematics only)

• GM – Grade likely to be achieved by 50% of more ‘similar’ students nationally

• HGM – likelihood of attaining above GM• GA – one grade above GM• EP – Expected Progress from KS2->KS4 (English and Mathematics only)

DefaultDefault

ExtendedExtended

Other Reports• Export Section• Provides additional data (e.g. UPN,

FFT_ID) which is not available on SETS reports

• Removes formatting and colours, so easier to import into other systems

• Export Section• Provides additional data (e.g. UPN,

FFT_ID) which is not available on SETS reports

• Removes formatting and colours, so easier to import into other systems

• Student Explorer• Select students by range of characteristics• Look at individual pupil history• Developments:

• FSM as pre-set group (can currently select using drop-down menus)

• Pilot estimate report which takes into account factors such as EAL and when arrived in UK

• Student Explorer• Select students by range of characteristics• Look at individual pupil history• Developments:

• FSM as pre-set group (can currently select using drop-down menus)

• Pilot estimate report which takes into account factors such as EAL and when arrived in UK

Questions