Embed Size (px)
DESCRIPTION
Presentation by Paulo Santiago at the 10th annual meeting of the Senior Budget Officials Performance and Results Network held on 24-25 November 2014. Find more information at http://www.oecd.org/gov/budgeting
Citation preview
Performance Information in the Education Sector
Session 4a – Performance Informed Budgeting in Practice using a Sectoral Perspective 10th Annual Meeting of the OECD Senior Budget Officials Performance and Results Network Paris, 24-25 November, 2014
From Education System Evaluation to Funding in Education
Paulo Santiago, Senior Analyst Directorate for Education and Skills
Outline of Presentation
1. Expenditure in School Education
2. Education System Evaluation: Generating Performance Information
3. The OECD School Resources Review
4. Funding of School Education
1. Expenditure in School Education
In 2011, OECD countries spent an average of 3.9% of their GDP on
primary, secondary and post-secondary non-tertiary education Expenditure on educational institutions as a percentage of GDP (2011). From public and
private sources, by level of education and source of funds
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
5.5
Ne
w Z
ea
land
Arg
en
tin
a
Ice
lan
d
Un
ite
d K
ing
do
m
Ire
lan
d
De
nm
ark
Be
lgiu
m
Isra
el
Co
lom
bia
Ko
rea
Au
str
alia
Fin
lan
d
Sw
itze
rla
nd
Ne
the
rla
nd
s
Ca
na
da
Mexic
o
Fra
nce
Sw
ed
en
Slo
ve
nia
Un
ite
d S
tate
s
Ch
ile
Po
rtu
ga
l
Au
str
ia
EU
21
ave
rag
e
Pola
nd
Esto
nia
Lu
xe
mb
ou
rg
Spain
Ge
rma
ny
Ita
ly
La
tvia
Ja
pa
n
Czech
Re
pu
blic
Slo
va
k R
ep
ub
lic
Ru
ssia
n…
No
rwa
y
Bra
zil
% of GDP
Primary, secondary and post-secondary non-tertiary education
Public expenditure on education institutions Private expenditure on education institutions
OECD average (total
expenditure)
In 2011, 13% of total public spending was devoted to education
Total public expenditure on education as a percentage of total public expenditure (1995, 2005,
2011)
0
5
10
15
20
25
Ne
w Z
ea
land
Me
xic
o
Bra
zil
Ko
rea
Sw
itzerland
Ice
lan
d
De
nm
ark
No
rwa
y
Au
str
alia
Isra
el
Esto
nia
Un
ite
d S
tate
s
Ca
na
da
Sw
ed
en
Ire
lan
d
OE
CD
ave
rag
e
Be
lgiu
m
Fin
lan
d
EU
21
ave
rag
e
United K
ingdom
Ne
the
rla
nd
s
Au
str
ia
Po
land
Slo
ve
nia
Ge
rma
ny
Ru
ssia
n F
ed
era
tio
n
Po
rtu
ga
l
Slo
va
k R
ep
ub
lic
Spain
Czech
Re
pu
blic
Fra
nce
Hu
ng
ary
Ja
pa
n
Ita
ly
% of total public expenditure 2011 2005 1995
Source: PISA 2012 Results: What makes schools successful? Resources, policies and practices, Volume IV, Figure IV.1.8.
Slovak Republic
Czech Republic Estonia
Israel
Poland
Korea
Portugal
New Zealand
Canada Germany
Spain
France
Italy
Singapore
Finland
Japan
Slovenia Ireland
Iceland
Netherlands
Sweden
Belgium
United Kingdom
Australia Denmark
United States
Austria
Norway
Switzerland
Luxembourg
Viet Nam
Jordan
Peru
Thailand
Malaysia
Uruguay
Turkey
Colombia
Tunisia
Mexico Montenegro
Brazil
Bulgaria
Chile
Croatia
Lithuania Latvia
Hungary
Shanghai-China
R² = 0.01
R² = 0.37
300
350
400
450
500
550
600
650
0 20 000 40 000 60 000 80 000 100 000 120 000 140 000 160 000 180 000 200 000
Math
em
ati
cs p
erf
orm
an
ce (
sco
re p
oin
ts)
Average cumulative spending per student from the age of 6 to 15 (USD, PPPs)
Above a certain level of investment, the key factor is how
to spend available funding most effectively Student performance and average spending per student
2. Education System Evaluation: Generating Performance
Information
OECD educational policy reviews
OECD Reviews of Evaluation and Assessment in Education (Schools)
Highlights the importance of context. Different systems are at different
stages in developing an evaluation and assessment framework.
Expansion of evaluation in
school systems
Greater reliance on
educational standards
Increased importance of
measurement and
indicators
Accountability purpose is
gaining importance
Building capacity for
Evaluation and Assessment
(E&A)
Aligning goals of education
system with E&A
Going beyond measurement
Effectively conceiving
accountability
Main trends Policy priorities
Education System Evaluation: Generating Performance Information
• Drivers:
– The rising importance of education in a global world – The growing imperative of an efficient use of public resources – Greater decentralisation and school autonomy – Greater accountability in the public sector – including in budgetary procedures – The growing importance of evidence-based policy
• Purposes: – To monitor:
• Student outcomes at a given point in time (differences among different regions within the education system and given student groups) and changes in student outcomes over time
• Broader outcomes of education systems (e.g. labour market outcomes; social outcomes)
• The impact of given policy initiatives or educational programmes
• Demographic, administrative and contextual data which are useful to explain the outcomes of the education system
– To generate and feedback information for different agents in the education system;
– To use the generated information for analysis, development and implementation of policies.
Education System Evaluation:
Reference standards
– National education goals and objectives, e.g. • Provide high-quality education to students; promote national values
and civic responsibilities; develop skills in the economy.
– Specific priorities: e.g. improve equity; goals for specific groups
– References used in national assessments • National curriculum goals; National learning progressions; National
standards; National curriculum goals and standards
– Specific targets set to be achieved over a certain timeline • Also supra-national, e.g. EU benchmarks
Education System Evaluation:
Specific targets
Examples of targets: – Mexico (2007-2012): education system evaluation framed by Education Sector
Programme with 6 policy objectives (e.g. promotion of ICT in education) and 41 indicators (each with a target and the respective measure)
– Northern Ireland: Programme for Government 2011-15 includes high-level targets for the performance of the education system by 2015 (e.g. 66% of young people achieve at least 5 General Certificates in Secondary Education with a mark of A to C in mathematics, English and three other subjects)
– Supra-national, e.g. EU benchmarks to be achieved by 2020 • At least 95% of children between the age of 4 and the age for starting primary
education should participate in early childhood education
• The share of 15-year-olds with insufficient abilities in reading, mathematics and science should be less than 15% (measure by PISA)
• The share of early leavers from education and training should be less than 10%
• The share of 30-34 year-olds with tertiary educational attainment should be at least 40%
Education System Evaluation:
Procedures
• Instruments: – Indicator frameworks;
– Tools to monitor student outcomes (national assessments; longitudinal research and surveys; international assessments);
– Qualitative reviews of particular aspects of the school system (ad hoc reviews; evaluative information generated via external school system reviews);
– Stakeholder surveys; and
– The evaluation of specific programmes and policies
Education System Evaluation: Prominence of international student assessments
• The profile of the results from international student assessments has been significantly raised in national policy discussions
– Perceived as indicators of future economic competitiveness
– Highlighted the importance of monitoring student outcomes
– Aspirational targets for performance are established (e.g. in PISA)
• A number of countries has established PISA targets
– Denmark (2010): Danish students to be in the top five countries as judged in international assessments.
– Australia (National Plan for School Improvement, 2012): to be among the top five school systems in the world by 2025 in mathematics, science and reading achievement.
Student performance and equity
Peru
Chile
Bulgaria
Hungary
Slovak Republic Portugal
Luxembourg
France
Uruguay
New Zealand
Chinese Taipei
Belgium
Costa Rica
Romania Israel
Germany
Indonesia Colombia
Tunisia
Argentina Brazil
Malaysia Turkey
Greece Lithuania
Latvia Russian Fed. Spain
UK
Czech Republic
Denmark
Slovenia
Ireland
Austria
Viet Nam
Switzerland
Singapore
Shanghai-China
Poland
United States
Croatia
Netherlands
Montenegro
Serbia
Hong Kong-China
Estonia Finland
Thailand
Japan
Sweden
Australia Canada
Jordan
Macao-China
U.A.E. Kazakhstan
Iceland
Qatar
Norway
Mexico
Liechtenstein
Korea
Italy
300
350
400
450
500
550
600
650
051015202530
OE
CD
ave
rag
e
OECD average
Percentage of variance in performance explained by ESCS (r-squared x 100)
Me
an
ma
them
ati
cs
pe
rfo
rma
nc
e
Below average mathematics performance
Below average impact of socio-economic
background
Above average mathematics performance
Below average impact of socio-economic
background
Below average mathematics performance
Above average impact of socio-economic
background
Above average mathematics performance
Above average impact of socio-economic
background
Relationship between mathematics performance and variation in
performance explained by students’ socio-economic status
Education System Evaluation - Example
Australia
References and standards (National level) • Council of Australian Governments (COAG) National Productivity Agenda
– Includes a set of aspirations, outcomes, progress measures, and policy directions for education
• Melbourne Declaration on Educational Goals for Young Australians – Articulates future directions and aspirations for Australian schooling
• Australian Curriculum • Priority areas / Education targets [e.g. Increased proportion of young Australians attaining
secondary education; halve the gap for indigenous students within a decade]
Methods and instruments (National level)
• National Assessment Program – NAPLAN – Literacy and Numeracy: full cohort tests in reading, writing, spelling, grammar and
punctuation, and numeracy at Year levels 3, 5, 7 and 9. – Sample assessments: Cyclical sample surveys in science, ICT, civics and citizenship in Years 6
and 10. – International assessments: PISA, TIMSS, PIRLS
• Measurement Framework for National Key Performance Measures • Independent reviews
OECD Indicators of Education Systems
(INES) – Education at a Glance
The INES organising framework includes three major policy perspectives:
• quality of educational outcomes and educational provision;
• equality of educational outcomes and equity in educational
opportunities; and
• adequacy, effectiveness and efficiency of resource management.
Comparative
information
for different levels of
education on:
Student enrolment,
entrance and
graduation rates
Educational
personnel
Educational
finance
Comparative
information:
Teaching and
learning
environments
(TALIS 2008;
PISA surveys)
Student outcomes at age
15
Reading, Mathematics and
Science
(PISA 2000, 2003, 2006,
2009, 2012….)
Survey of Adult Skills
(16-65)
(Literacy, Problem-Solving,
Numeracy, Skills-Use)
(PIAAC)
Education System Evaluation:
Policy Options
Governance: Being systematic and strategic for better informed policy making
• Ensure a broad concept of education system evaluation within the E&A framework
• Ensure policy making is informed by high-quality measures, but not driven by their availability
• Situate education system evaluation in the broader context of public sector performance requirements
Procedures: Developing an approach to learn from a broad evidence base • Develop a national education indicator framework
• Design a national strategy to monitor student learning standards
• Ensure the collection of: qualitative information; and contextual information to monitor equity
• Assure the monitoring of changes over time and progress of particular student cohorts
3. OECD School Resources Review
OECD School Resources Review:
Objectives Overarching policy question:
“What policies best ensure that school resources are effectively used to improve student outcomes?”
• Key issues for analysis
o Funding of school education [including budgeting in education] Level of resources; sources of revenue; education budgeting procedures; forecasting resource needs; resource strategy; distribution of funding across administrative levels, education levels, sectors and individual schools; monitoring of resource use; capacity for resource management; transparency and reporting.
o Management of human resources (e.g. teachers, school directors)
o Organisation of school network (location, size of schools)
The Review will:
• Synthesise research-based evidence on effective resource use in the school sector and disseminate this knowledge among countries
• Identify innovative and successful policy initiatives and practices
• Identify policy options for policy makers to consider.
Context
Economic
Demographic
Political
Cultural
Governance
OECD School Resources Review:
Analytical framework and Key Issues
Optimal outputs
Access, participation, completion
Learning outcomes
Labour market & social outcomes
Resource Distribution
Resource Utilisation
Resource Management
Education System Goals
OECD School Resources Review:
Participation
• Austria
• Belgium (Flemish Community)
• Chile
• Czech Republic
• Denmark
• Estonia
• Kazakhstan
• Lithuania
• Slovak Republic
• Uruguay
Country Reviews
• Belgium (French Community)
• Iceland
• Luxembourg
• Spain
• Sweden
Country Background
Reports
4. Funding of School Education
Funding of School Education
A diversity of
channels for
funding distribution:
Central
government to line
ministries
From Ministry of
Education to local
authorities
To schools
through funding
formulas
A range of factors used in Funding Formulae, including:
• Number of students, class size, characteristics of student population,
qualifications of teachers
• Type of educational offer (e.g. VET vs General programmes), school
location, size of school
Performance measures typically not used
Requires high-quality and reliable data at the school level
• Comparative indicators of school performance can miss important aspects of school quality – Standardised performance measures
• Often limited to discrete areas of student learning objectives • Often cross-sectional and do not allow monitoring of school progress • Often do not identify impact of the school on student outcomes (value-added
measures are needed)
• Use of performance measures for accountability: • Not strongly linked to financial rewards/sanctions • Very different policies on how these feed into decisions on possible school
closure • Closer supervision of schools with quality concerns • Reward of more autonomy to schools with good evaluation results
• Care with accountability uses of school performance measures • Over emphasis on what is assessed in performance measures; narrowing of
the curriculum; teaching to the test; hinder innovation. • Stigmatisation of particular schools and unintended impacts on parental
choice of schools. Complacency of schools performing well on such measures.
School Evaluation: Use of performance
measures
Thank you for your attention!
www.oecd.org/edu/evaluationpolicy
www.oecd.org/edu/school/schoolresourcesreview.htm
