Evidence-based policy:Hopes, Truths, and Lessons

19 November 2008

Lee Elliot MajorThe Sutton Trust

EBP: the ideal?

• Objective, agenda free academic pursues research• Academic works through policy implications and

main messages in detail• Publishes peer reviewed research

• Policy makers take on research amongst other work and consider policy change

• Pilot policy reform is evaluated, and then expanded or changed, or halted

The reality: social mobility claims

Social mobility "on the rise”- Reuters News UK, UK - 3 Nov 2008

‘We have consulted widely…to ensure this paper recognises wide range of perspectives in this area’

- Liam Byrne, Minister of the Cabinet Office

Social mobility claims

• Politicians only cite research that backs their position, but claim EBP

• Doesn’t matter if research isn’t published or peer reviewed

• Press like simple stories; nuances, caveats are always lost

• Never underestimate the power or speed of politics

Selective research

• Rutter research and Tory education policies in 1980s

• PISA results and Labour

• Feinstein -- early years inequalities

• Heckman and Obama

What in common?

• Specialist schools

• Academies

• Diplomas

• Choice and competition

• League tables

Successes (NPD-PLASC)

• HE participation driven by early attainment (NCEE)

• Competition does not benefit standards in the current system

• Highly segregated schools

• Impact of the EMA

Media flaws

• Counter-factuals

• Complexity• Statistics

• Regressions

• Controversy

• Fleeting interest

Incessant reformSecretary of State for Education and ScienceKenneth Baker (1988)John MacGregor Kenneth Clarke Secretary of State for EducationJohn Patten Gillian Shepherd Secretary of State for Education and EmploymentGillian Shepherd David Blunkett Secretary of State for Education and SkillsEstelle Morris Charles Clarke Ruth Kelly Alan Johnson Secretary of State for Children, Schools and FamiliesEd Balls (2008)Secretary of State for Innovation, Universities and SkillsJohn Denham (2008)

Academic imperfections

• Policy afterthought

• Inexpert views

• Fixed views and agendas

• Selective interpretation

• Media afterthought

Lessons

• Everyone has prior value views – so an uphill task for evidence to overcome these

• Academic and policy worlds are polar extremes

• Interpretation/dissemination is key

• Know the way it works

What to do?

• Policy analysis as part of academic papers/joint conferences

• Recognition for policy impact• Accessible research summaries• Involve policy expertise/discussion earlier• Be simple, timely and tactical• Be consistent

Questions

• Should academics delve in policy development?

• Should policy reform be a key rationale for publicly funded research?

• Should non-peer reviewed research influence policy?

• Policy peer review?• Should all research be made available?