Upload
tristian-lovern
View
215
Download
0
Tags:
Embed Size (px)
Citation preview
1
Research and DevelopmentFrom CEM
CEM conference: Improving Pupil Assessment
London 7th June 2011
2
CEM’s Achievements
The largest educational research unit in a UK university (70 staff)
1.1 million assessments are taken each year
More than 50% of UK secondary schools use one or more CEM system
CEM systems used in over 40 countries Largest provider of computerised adaptive
tests outside US
3
Changes in proportion gaining five A*-Cs at GCSE122
.6
22.9
23.5
23.7
23.7
24.0
25.0
26.1
26.2
26.7
26.9
26.7
26.4 29
.9 32.8 34
.5 36.8 38
.3 41.2 43
.3
43.5
44.5
45.1
46.3 47
.9 49.2
50.0 51
.6 52.9
53.7 56
.3 58.5 60
.9
64.8
0
10
20
30
40
50
60
70
197
519
76
197
719
78
197
919
80
198
119
82
198
319
84
198
519
86
198
719
88
198
919
90
199
119
92
199
319
94
199
519
96
199
719
98
199
920
00
200
120
02
200
320
04
200
520
06
200
720
08
Pe
rcen
tag
e o
f '1
5 ye
ar o
lds'
2
Rising standards
4
Grade slippage at A level
Average grade achieved by students with the same ability (ITDA=50)
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
A le
vel g
rade
BiologyEnglish (Lit)FrenchGeographyHistoryMathsWeighted avg of 40 subjs
U
E
D
C
B
-----
------
Cu
rricu
lum
200
0 an
d ne
w TD
A -
------
------
------
---
5
Performance of England in international surveys
Maths (age 10, TIMSS)
Maths (age 14, TIMSS)
Reading (age 11, PIRLS)
Science (age 10, TIMSS)
Science (age 14, TIMSS)
Reading literacy (age 15, PISA)
Mathematical literacy (age 15, PISA)
Scientific literacy (age 15, PISA)
480
490
500
510
520
530
540
550
560
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
Year
Sta
nd
ard
ise
d t
es
t s
co
re
6
Maths performance from 1976-2008
Hodgen et al, 2009
7
Can existing research help you to improve your school?
8
School Effectiveness Research (= Lists of characteristics of ‘effective’ schools)o Can we really identify effective schools?o Can we change these characteristics?o Will this lead to real improvement?
School Improvement Research (= Descriptions of change efforts)o Did important outcomes really improve?o Do we know why?o Are the changes (and any real effects) replicable?
Existing research
9
Set high expectations Are good at planning Employ a variety of teaching strategies Have a clear strategy for pupil
management Manage time and resources wisely Employ a range of assessment methods Set appropriate homework Keep pupils on task
Effective Teachers (according to Hay McBer, 2000)
(For which the DfEE paid £3m)
10
The secret of success(according to J. Paul Getty)
Rise early Work late Strike oil
(This advice was given for free)
11
Wait for a bad year and/or choose a bad school to start with. Things can only get better.
Take on any initiative, and ask everyone who put effort into it whether they feel it worked. No-one wants to feel foolish.
Define ‘improvement’ in terms of perceptions and ratings of teachers. DO NOT conduct any proper assessments – they may disappoint.
Only study schools that recognise a problem and are prepared to take on an initiative. They’ll improve whatever you do.
How to produce ‘school improvement’ (1)
12
Conduct some kind of evaluation, but don’t let the design be too good – poor quality evaluations are much more likely to show positive results
If any improvement occurs in any aspect of performance, focus attention on that rather than on any areas (or schools) that have not improved or got worse
Put some effort into marketing and presentation of the school. Once you start to recruit better students, things will improve.
How to produce ‘school improvement’ (2)
13
Use high-quality assessment and monitoring systems to track a range of valued outcomes
Take account of the best available research evidence about the effectiveness of different approaches
Experiment and adapt to local needs, contexts and capacities, with rigorous evaluation.
What would an evidence-driven school do?
14
15
‘Best buy’ strategies
Cost per pupil
Effe
ct S
ize
(m
on
ths
ga
in)
£00
10
£1000
Feedback
Meta-cognitive
Peer tutoring Pre-school
1-1 tutoringHomework
ICT
AfLParental
involvementSports
Summer schools
After school
Individualised learning
Learning styles
ArtsPerformance
payTeaching assistants
Smaller classes
Ability grouping
16
Probably worth investing ino Feedbacko Meta-cognitive & self-regulation strategieso Peer tutoring
Less effective (or not good value)o More teachers / TAs (smaller classes)o 1-1 support (ECaR, ECC)o Setting / streamingo PRP
If our aim is to promote learning
17
CEM’s Aim
To help educators improve educational outcomes, througho Assessments that support learningo Monitoring and feedback systems for
self-evaluationo Rigorous evaluation of the impact of
different approacheso Promotion of evidence-based practices
and policies
18
Use high-quality assessment and monitoring systems to track a range of valued outcomes
Take account of the best available research evidence about the effectiveness of different approaches
Experiment and adapt to local needs, contexts and capacities, with rigorous evaluation.
What would an evidence-driven school do?