View
39
Download
0
Category
Preview:
DESCRIPTION
10 things policymakers should know about learning goals and assessment. Andreas Schleicher St. Petersburg, May 2014. Benchmark globally. 2. Lesson 1 - PowerPoint PPT Presentation
Citation preview
OECD EMPLOYER BRANDPlaybook
1
10 things policymakers should know about learning goals and assessment
Andreas SchleicherSt. Petersburg, May 2014
22 Benchmark globally
Lesson 1In a global economy, the benchmark for educational success is no longer solely
improvement by national standards, but the best performing education systems internationally
A world of change – higher education
0 10 20 30 40 50 60 700
5,000
10,000
15,000
20,000
25,000
30,000
1995
Expe
ndit
ure
per
stud
ent
at t
erti
ary
leve
l (U
SD)
Tertiary-type A graduation rate (%)
Cost
per
stud
ent
Graduate supply
A world of change – higher education
0 10 20 30 40 50 60 700
5,000
10,000
15,000
20,000
25,000
30,000
1995
Expe
ndit
ure
per
stud
ent
at t
erti
ary
leve
l (U
SD)
Tertiary-type A graduation rate (%)
Cost
per
stud
ent
Graduate supply
United States
A world of change – higher education
0 10 20 30 40 50 60 700
5,000
10,000
15,000
20,000
25,000
30,000
2000
Expe
ndit
ure
per
stud
ent
at t
erti
ary
leve
l (U
SD)
Tertiary-type A graduation rate
United Kingdom
A world of change – higher education
0 10 20 30 40 50 60 700
5,000
10,000
15,000
20,000
25,000
30,000
2001
Expe
ndit
ure
per
stud
ent
at t
erti
ary
leve
l (U
SD)
Tertiary-type A graduation rate
A world of change – higher education
0 10 20 30 40 50 60 700
5,000
10,000
15,000
20,000
25,000
30,000
2002
Expe
ndit
ure
per
stud
ent
at t
erti
ary
leve
l (U
SD)
Tertiary-type A graduation rate
A world of change – higher education
0 10 20 30 40 50 60 700
5,000
10,000
15,000
20,000
25,000
30,000
2003
Expe
ndit
ure
per
stud
ent
at t
erti
ary
leve
l (U
SD)
Tertiary-type A graduation rate
A world of change – higher education
0 10 20 30 40 50 60 700
5,000
10,000
15,000
20,000
25,000
30,000
2004
Expe
ndit
ure
per
stud
ent
at t
erti
ary
leve
l (U
SD)
Tertiary-type A graduation rate
A world of change – higher education
0 10 20 30 40 50 60 700
5,000
10,000
15,000
20,000
25,000
30,000
2005
Expe
ndit
ure
per
stud
ent
at t
erti
ary
leve
l (U
SD)
Tertiary-type A graduation rate
A world of change – higher education
0 10 20 30 40 50 60 700
5,000
10,000
15,000
20,000
25,000
30,000
2006
Expe
ndit
ure
per
stud
ent
at t
erti
ary
leve
l (U
SD)
Tertiary-type A graduation rate
A world of change – higher education
0 10 20 30 40 50 60 700
5,000
10,000
15,000
20,000
25,000
30,000
2007 Ex
pend
itur
e pe
r st
uden
t at
ter
tiar
y le
vel
(USD
)
Tertiary-type A graduation rate
A world of change – higher education
0 10 20 30 40 50 60 700
5,000
10,000
15,000
20,000
25,000
30,000
2008
Expe
ndit
ure
per
stud
ent
at t
erti
ary
leve
l (U
SD)
Tertiary-type A graduation rate
A world of change – higher education
0 10 20 30 40 50 60 700
5,000
10,000
15,000
20,000
25,000
30,000
2009 Ex
pend
itur
e pe
r st
uden
t at
ter
tiar
y le
vel
(USD
)
Tertiary-type A graduation rate
A world of change – higher education
0 10 20 30 40 50 60 700
5,000
10,000
15,000
20,000
25,000
30,000
2010
Expe
ndit
ure
per
stud
ent
at t
erti
ary
leve
l (U
SD)
Tertiary-type A graduation rate
IcelandPoland
UK
A world of change – higher education
0 10 20 30 40 50 60 700
5,000
10,000
15,000
20,000
25,000
30,000
2010
Expe
ndit
ure
per
stud
ent
at t
erti
ary
leve
l (U
SD)
Tertiary-type A graduation rate
US
1717 Don’t sacrifice validity gains for efficiency gains
Lesson 2The kinds of things that are easy to
teach and test are also easy to digitise, automate and outsource
18
1960 1970 1980 1990 2000 2006 200935
40
45
50
55
60
65
70
Routine manualNonroutine manualRoutine cognitiveNonroutine analyticNonroutine interpersonal
Mean task input in percentiles of 1960 task distribution
Changes in the demand for skillsTrends in different tasks in occupations (United States)
Source: Autor, David H. and Brendan M. Price. 2013. "The Changing Task Composition of the US Labor Market: An Update of Autor, Levy, and Murnane (2003)." MIT Mimeograph, June.
1919 Monitor excellence and equity jointly
Lesson 3
The false choice betweenequity and excellence
410
420
430
440
450
460
470
480
490
500
510
520
530
540
550
560
570
580Mean score
High mathematics performance
Low mathematics performance
… Shanghai-China performs above this line (613)
… 12 countries perform below this line
Average performanceof 15-year-olds in
MathematicsFig I.2.13
Socially equitable distribution of learning
opportunities
High mathematics performance
Low mathematics performance
Average performanceof 15-year-olds in
mathematics
Strong socio-economic impact on student
performance
02468101214161820222426
AustraliaAustria
Belgium Canada
Chile
Czech Rep.Denmark
Estonia
Finland
France
Germany
Greece
Hungary
IcelandIreland
Israel
Italy
Japan
Korea
Luxembourg
Mexico
Netherlands
New Zealand
Norway
Poland
Portugal
Slovak Rep.
Slovenia
Spain Sweden
Switzerland
Turkey
UK
US
Singapore
Hong Kong-ChinaChinese Taipei
Macao-ChinaLiechtenstein
Viet Nam
Latvia
Russian Fed.Lithuania
Croatia
SerbiaRomania
Bulgaria United Arab Emirates
KazakhstanThailand
Malaysia
2012Shanghai-China
Socially equitable distribution of learning
opportunities
Strong socio-economic impact on student
performance
AustraliaAustria
Belgium Canada
Chile
Czech Rep.Denmark
Estonia
Finland
France
Germany
Greece
Hungary
IcelandIreland
Israel
Italy
Japan
Korea
Luxembourg
Mexico
Netherlands
New Zealand
Norway
Poland
Portugal
Slovak Rep.
Slovenia
Spain Sweden
Switzerland
Turkey
UK
US
AustraliaAustriaBelgiumCanadaChileCzech Rep.DenmarkEstoniaFinlandFranceGermanyGreeceHungaryIcelandIrelandIsraelItalyJapanKoreaLuxembourgMexicoNetherlandsNew ZealandNorwayPolandPortugalSlovak Rep.SloveniaSpainSwedenSwitzerlandTurkeyUKUS
2012
Socially equitable distribution of learning
opportunities
Strong socio-economic impact on student
performance
AustraliaAustria
Belgium Canada
Chile
Czech Rep.Denmark
Estonia
Finland
France
Germany
Greece
Hungary
IcelandIreland
Israel
Italy
Japan
Korea
Luxembourg
Mexico
Netherlands
New Zealand
Norway
Poland
Portugal
Slovak Rep.
Slovenia
Spain Sweden
Switzerland
Turkey
UK
US
AustraliaAustriaBelgiumCanadaChileCzech Rep.DenmarkEstoniaFinlandFranceGermanyGreeceHungaryIcelandIrelandIsraelItalyJapanKoreaLuxembourgMexicoNetherlandsNew ZealandNorwayPolandPortugalSlovak Rep.SloveniaSpainSwedenSwitzerlandTurkeyUKUS
2525 Spend wisely to make a difference
Lesson 4
Not more money but better spending choices make a difference
Portu
gal
Spain
Switz
erlan
d
Belg
ium
Kore
a
Luxe
mbo
urg
Germ
any
Gree
ce
Japa
n
Aust
ralia
Unite
d Ki
ngdo
m
New
Zeala
nd
Fran
ce
Neth
erlan
ds
Denm
ark
Italy
Aust
ria
Czec
h Re
publ
ic
Hung
ary
Norw
ay
Icela
nd
Irelan
d
Mexic
o
Finlan
d
Swed
en
Unite
d St
ates
Polan
d
Slov
ak R
epub
lic
-10
-5
0
5
10
15
Salary as % of GDP/capita Instruction time 1/teaching time 1/class size
Contribution of various factors to upper secondary teacher compensation costs, per student as a percentage of GDP per capita (2004)
Percentage points
Difference with OECD average
AustraliaAustria
Belgium Canada
Chile
Czech Rep.Denmark
Estonia
Finland
France
Germany
Greece
Hungary
IcelandIreland
Israel
Italy
Japan
Korea
Luxembourg
Mexico
Netherlands
New Zealand
Norway
Poland
Portugal
Slovak Rep.
Slovenia
Spain Sweden
Switzerland
Turkey
UK
US
AustraliaAustriaBelgiumCanadaChileCzech Rep.DenmarkEstoniaFinlandFranceGermanyGreeceHungaryIcelandIrelandIsraelItalyJapanKoreaLuxembourgMexicoNetherlandsNew ZealandNorwayPolandPortugalSlovak Rep.SloveniaSpainSwedenSwitzerlandTurkeyUKUS
Singapore
Shanghai
Singapore
2003 - 2012 Germany, Turkey and Mexico improved both their mathematics performance and equity levels
2828 Keep track of ‘added value’
Lesson 5The country where students go to class matters more than what social class students come from
29School performance and socio-economic background: Brazil
AdvantagePISA Index of socio-economic backgroundDisadvantage
Stu
dent
per
form
ance
Brazil
30
Vietnam
Desempeño del colegio y nivel socioeconómico
VentajaÍndice de PISA de nivel socioeconómicoDesventaja
Des
empe
ño d
e lo
s es
tudi
ante
s
31School performance and socio-economic background: Brazil
AdvantagePISA Index of socio-economic backgroundDisadvantage
Stu
dent
per
form
ance
Brazil
3232Resilience in educationPISA performance by decile of social background
Mex
ico
Gre
ece
Swed
en
Isra
el
Unite
d St
ates
Denm
ark
Aust
ralia
Unite
d Ki
ngdo
m
Cana
da
Aust
ria
Liec
hten
stei
n
Esto
nia
Slov
enia
New
Zea
land
Net
herl
ands
Switz
erla
nd
Belg
ium
Mac
ao-C
hina
Kore
a
Chin
ese
Taip
ei300
325
350
375
400
425
450
475
500
525
550
575
600
625
650
675
Source: PISA 2012
33
33Provide a data-rich school environmentto combat inequities
National and school data
School-based strategies for
analysis
Data for sup-porting learning
3434Le
sson
s fro
m h
igh
perfo
rmer
s
Catching up with the top-performers
Low impact on outcomes
High impact on outcomes
Low feasibility High feasibility
Money pits
Must haves
Low hanging fruits
Quick wins
3535Le
sson
s fro
m h
igh
perfo
rmer
s
Low impact on outcomes
High impact on outcomes
Low feasibility High feasibility
Money pits
Must haves
Low hanging fruits
Quick wins
Commitment to universal achievement
Gateways, instructional systems
Capacity at point of delivery
Incentive structures and accountability
Resources where they yield most
A learning systemCoherence
3636Le
sson
s fro
m h
igh
perfo
rmer
s
Low impact on outcomes
High impact on outcomes
Low feasibility High feasibility
Money pits
Must haves
Low hanging fruits
Quick wins
Commitment to universal achievement
Gateways, instructional systems
Capacity at point of delivery
Incentive structures and accountability
Resources where they yield most
A learning systemCoherence
Lesson 6: Believe that all children can achieve Universal educational standards and personalization as
the approach to heterogeneity in the student body…… as opposed to a belief that students have different
destinations to be met with different expectations, and selection/stratification as the approach to heterogeneity
Clear articulation who is responsible for ensuring student success and to whom
3737Le
sson
s fro
m h
igh
perfo
rmer
s
37
37 High expectations for all students
-0.60 -0.40 -0.20 0.00 0.20 0.40 0.60 0.80 1.00 1.20300
350
400
450
500
550
600
650
481.366786279212
517.501096817955
561.241096454551
391.459888954175
499.749902827587
452.973426858907
409.291567937716
493.934230896316
520.545521676786518.750335282979
394.329333356314
471.131460759248
490.571021411359
481.644744006327489.845098037208
513.525055819928
478.823277433358
505.540743249801
498.95788231768
559.824796201498
494.98467432064426.737491293011
536.406918234208
447.984414978954 478.260635903011
477.044455015488504.150766311124
466.48143014931
518.078519433354
501.497460196644438.738259877415
385.595556395556
422.632355405519
538.134494733918
U.A.E.
514.745238582901522.971758192682
484.319297801971
388.431709907139
375.114451681749
500.026756625414
431.798408505078
368.102547127357
406.999866988793
530.931003950397
409.626613284347
387.824629620249
492.795697239492
501.127422390953
376.4483986347
573.468314296641
487.063181343903
489.373070348755
376.488601072821
420.512967619054
413.281466667708
534.96508297892
553.766659143613
448.859130247604
Russian Fed.
444.554242787643
511.338207501182
485.321181012553
612.675536305453
f(x) = 138.160916953927 x + 477.587612682211R² = 0.368631715648504
PISA mean index of mathematics self-efficacy
Mea
n m
athe
mat
ics
perfo
rman
ce
OEC
D a
vera
ge
Countries where students have stronger be-liefs
in their abilities perform better in mathe-matics
Source: PISA 2012
38 Perceived self-responsibility for failure in mathematics
Percentage of students who reported "agree" or "strongly agree" with the following statements:
I’m not very good at solving mathematics problems
My teacher did not explain the concepts well this week
This week I made bad guesses on the quiz
Sometimes the course material is too hard
The teacher did not get students interested in the material
Sometimes I am just unlucky
0 10 20 30 40 50 60 70 80 90
Russian Federation United States OECD average
%
Fig III.3.6
39
39
39 A continuum of supportBe demanding for every student
Ensure that students feel valued and included and learning is collaborative
4040Le
sson
s fro
m h
igh
perfo
rmer
s
Low impact on outcomes
High impact on outcomes
Low feasibility High feasibility
Money pits
Must haves
Low hanging fruits
Quick wins
Commitment to universal achievement
Gateways, instructional systems
Capacity at point of delivery
Incentive structures and accountability
Resources where they yield most
A learning systemCoherence
Lesson 7: Have clear ambitious goals that are shared across the system and aligned with high stakes gateways and instructional systems
Well established delivery chain through which curricular goals translate into instructional systems, instructional practices and student learning (intended, implemented and achieved)
High level of metacognitive content of instruction …
4141Le
sson
s fro
m h
igh
perfo
rmer
s
Low impact on outcomes
High impact on outcomes
Low feasibility High feasibility
Money pits
Must haves
Low hanging fruits
Quick wins
Commitment to universal achievement
Gateways, instructional systems
Capacity at point of delivery
Incentive structures and accountability
Resources where they yield most
A learning systemCoherence
Lesson 8: Build capacity at the point of delivery Attracting, developing and retaining high quality
teachers and school leaders and a work organisation in which they can use their potential
Instructional leadership and human resource management in schools
Keeping teaching an attractive profession System-wide career development …
4242Le
sson
s fro
m h
igh
perfo
rmer
sLu
xem
bour
g
Thai
land
Sha
ngha
i-Chi
na
Col
ombi
a
Chi
le
Mex
ico
Vie
t Nam
Uru
guay
Kaz
akhs
tan
Bel
gium
Mal
aysi
a
Bra
zil
U.A
.E.
New
Zea
land
Sw
itzer
land
Mac
ao-C
hina
OE
CD
ave
rage
Arg
entin
a
Aus
tria
Irela
nd
Fran
ce
Uni
ted
Kin
gdom
Alb
ania
Can
ada
Latv
ia
Uni
ted
Sta
tes
Cro
atia
Mon
tene
gro
Hun
gary
Slo
veni
a
Ser
bia
Bul
garia
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
3
Bottom quarter of this index
Mea
n in
dex
Teacher shortage Fig IV.3.5
43
43
43 Prepare for work in disadvantaged schools
Preparation
• Reinforce initial teacher training including curriculum content for disadvantage
• Strengthening diagnostic capacity• Include practical field experience
• Both new and experienced teachers benefit
• Pedagogical and relational strategies
4444Le
sson
s fro
m h
igh
perfo
rmer
s
Low impact on outcomes
High impact on outcomes
Low feasibility High feasibility
Money pits
Must haves
Low hanging fruits
Quick wins
Commitment to universal achievement
Gateways, instructional systems
Capacity at point of delivery
Incentive structures and accountability
Resources where they yield most
A learning systemCoherence
Lesson 9: Align autonomy with accountability Aligned incentive structures
For students How gateways affect the strength, direction, clarity and nature of the incentives
operating on students at each stage of their education Degree to which students have incentives to take tough courses and study hard Opportunity costs for staying in school and performing well
For teachers Make innovations in pedagogy and/or organisation Improve their own performance
and the performance of their colleagues Pursue professional development opportunities
that lead to stronger pedagogical practices A balance between vertical and lateral accountability Effective instruments to manage and share knowledge and spread
innovation – communication within the system and with stakeholders around it
A capable centre with authority and legitimacy to act
4545Le
sson
s fro
m h
igh
perfo
rmer
s
45
45 Align autonomy with accountability
The question is not how many charter schools you have but how you enable every teacher to assume charter-like autonomy
4646Le
sson
s fro
m h
igh
perfo
rmer
s
46
46
-1.5 -1 -0.5 0 0.5 1 1.5300
350
400
450
500
550
600
650
531.551979302783
414.947431329217
430.53288984921
423.795593172672
484.685067484024
507.375949559565
493.913526079401
557.719613495498
454.493852942216459.674291542381
419.468595641077
488.357558008343
404.86657067849406.81928697245
410.692469685374
455.967032005237
396.468122669645
431.953772561969416.098738598916
300.849653448456
527.668467891543
404.539944308878
440.111661967012
474.054187560775
464.989161819408
547.743708881437
626.566663790363
452.789179885987
529.511834268283
497.071637137884
453.49524309675
482.577394045123
532.465311188924
506.274697797594
488.818411796174
402.907104971934
498.55233132561486.358212456265
502.809277446549
485.011835724539
525.143096315803
466.514022482625
460.853234111852
488.150072840935484.3703865799
468.514073102546
499.317279833724
438.810335285436
499.440165643771501.844010272146
478.664970193416480.554307802789
498.658254792673
481.116171960251
503.011259906496490.67709912419
463.432481043829
552.313972933536
478.845972683071R² = 0.133981453407518
Index of school responsibility for curriculum and assessment (index points)
Mat
hem
atic
s pe
rform
ance
(sco
re p
oint
s)Countries that grant schools autonomy over cur-
ricula and assessments tend to perform better in mathe-
matics
Source: PISA 2012
Schools with more autonomy perform better than schools with less autonomy in systems with more collaboration
Less school autonomy
More school autonomy
455
460
465
470
475
480
485
Teachers don't participate in management
Teachers participate in management
Score points
School autonomy for resource allocation x System's level of teachers participating in school managementAcross all participating countries and economies
Fig IV.1.17
Schools with more autonomy perform better than schools with less autonomy in systems with more accountability arrangements
Less school autonomy
More school autonomy
464
466
468
470
472
474
476
478
School data not public
School data public
Score points
School autonomy for curriculum and assessment x system's level of posting achievement data publicly
Fig IV.1.16
Less school autonomy
More school autonomy
455
460
465
470
475
480
485
No standardised math policy
Standardised math policy
Schools with more autonomy perform better than schools with less autonomy in systems with standardised math policies
Score points
School autonomy for curriculum and assessment x system's extent of implementing a standardised math policy (e.g. curriculum and instructional materials)
Fig IV.1.16
50
Written specification of the school's curriculum and educational goals
Written specification of student-performance standards
Systematic recording of data, including teacher and student attendance and graduation rates, test results and professional development of teachers
Internal evaluation/self-evaluation
External evaluation
Written feedback from students (e.g. regarding lessons, teachers or resources)
Teacher mentoring
Regular consultation with one or more experts over a period of at least six months with the aim of improving the school
Implementation of a standardised policy for mathematics
0 10 20 30 40 50 60 70 80 90 100
Percentage of students in schools whose principal reported that their schools have the following for quality assurance and improvement:
Singapore OECD average
%
Quality assurance and school improvement Fig IV.4.14
51
51
51 Square school choice with equity
Financial incentives
for schools
Assistance for disadvantaged
parents
Controlled choice
Financial incentives
Inform par-ents
Foster col-laboration
among teachers and
schools
Use student and school
assessments
5252Le
sson
s fro
m h
igh
perfo
rmer
s
Low impact on outcomes
High impact on outcomes
Low feasibility High feasibility
Money pits
Must haves
Low hanging fruits
Quick wins
Commitment to universal achievement
Gateways, instructional systems
Capacity at point of delivery
Incentive structures and accountability
Resources where they yield most
A learning systemCoherence
Lesson 10: Invest resources where they can make most of a difference
Alignment of resources with key challenges (e.g. attracting the most talented teachers to the most challenging classrooms)
Effective spending choices that prioritise high quality teachers over smaller classes
5353 Align the resources with the challenges
-0.500.511.5300
350
400
450
500
550
600
650
700R² = 0
Equity in resource allocation (index points)
Mat
hem
atic
s pe
rform
ance
(sco
re p
oint
s)
Greater equityLess equity
Adjusted by per capita GDP
Countries with better performance in mathematics tend to allocate educational resources more equitably
Source: PISA 2012
5454 Adequate resources to address disadvantage
Disadvantaged schools reported more teacher shortage
Advantaged schools reported more teacher shortage
Kor
eaEs
toni
aIs
rael
Latv
iaSl
oven
iaIta
lyPo
land
Sing
apor
eA
rgen
tina
Net
herla
nds
Portu
gal
Col
ombi
aFr
ance
Finl
and
Tuni
sia
Mac
ao-C
hina
Spai
nG
reec
eSw
itzer
land
Nor
way
Rus
sian
Fed
.Ja
pan
Aus
tria
Mon
tene
gro
Cro
atia
Can
ada
OEC
D a
vera
geG
erm
any
Den
mar
kH
unga
ryU
nite
d K
ingd
omLu
xem
bour
gH
ong
Kon
g-C
hina
Bel
gium
Icel
and
Viet
Nam
Irela
ndU
nite
d St
ates
Chi
leC
zech
Rep
ublic
Serb
iaTu
rkey
Mex
ico
Indo
nesi
aU
rugu
aySh
angh
ai-C
hina
Slov
ak R
epub
licSw
eden
Bra
zil
New
Zea
land
Aus
tralia
Chi
nese
Tai
pei-0.5
-0.3
-0.1
0.1
0.3
0.5
0.7
0.9
1.1
1.3
1.5Difference between socio-economically disadvantaged and socio-economically advantaged schools
Mea
n in
dex
diffe
renc
e
A shortage of qualified teachers is more of concern in disadvantaged schools
55
55
55 Reduce tracking and grade repetition
Both vertical and horizontal stratification hurt equity
5656Le
sson
s fro
m h
igh
perfo
rmer
s
Low impact on outcomes
High impact on outcomes
Low feasibility High feasibility
Money pits
Must haves
Low hanging fruits
Quick wins
Commitment to universal achievement
Gateways, instructional systems
Capacity at point of delivery
Incentive structures and accountability
Resources where they yield most
A learning systemCoherence
A final thought Alignment of policies
across all aspects of the system Coherence of policies
over sustained periods of time Consistency of implementation Fidelity of implementation
(without excessive control)
CAN
5757Le
sson
s fro
m h
igh
perfo
rmer
s
57
57 Thank you
Find out more about our work at www.oecd.org– All publications– The complete micro-level database
Email: Andreas.Schleicher@OECD.orgTwitter: SchleicherEDU
and remember:Without data, you are just another person with an opinion
Recommended