View
52
Download
5
Category
Tags:
Preview:
DESCRIPTION
Introduction to CEM and the Baseline Tests. Robert Clark Neil Defty Nicola Forster. Who / What is CEM ?. Centre for Evaluation & Monitoring Not for Profit Organisation Part of Durham University Close links to School of Education Established 1985 Nursery / Reception Post-16 - PowerPoint PPT Presentation
Citation preview
Introduction to CEM and the Baseline Tests
Robert ClarkNeil Defty
Nicola Forster
Who / What is CEM ?
Centre for Evaluation & Monitoring• Not for Profit Organisation• Part of Durham University• Close links to School of Education• Established 1985• Nursery / Reception Post-16• Monitoring Systems, Research and Evaluation Projects• UK & International• Over 1/3 Secondary & Post-16 Establishments• Informed by Evidence from Research
The Systems
The Systems
Year 7Year 8
Year 9MidYIS Computer Adaptive Baseline Test
Year 10Year 11 Yellis
Computer Adaptive Baseline Test
Year 12Year 13 Alis
GCSE Computer Adaptive Baseline Test
GCSE
A / AS / Btec / IB etc
INSIGHT Combines curriculum tests with developed ability
KS3
Typical TimelineMeasure BaselineSTART:
Autumn Term
Feedback
Reports
Collect and Send Results
Value-Added
Feedback
August
END:
September
The baseline score is measured as early as possible and the information sent to CEM
The institution then receives feedback from CEM: baseline test results and predictions
The public examination results are sent to CEM when received by the institution
The value added data is available from September via the Institution’s website at CEM
Typical Timeline
Early Testing & Predictions
Testing is available from June at end of previous academic year
e.g. Y6 / Y9 / Y11
Baseline Assessments of Developed Ability
Baseline Assessment
• Year 7 (8)• Year 9• Year 10 (11)• Year 12 (13+)
Computer Adaptive Baseline Test
Vocab Maths Non Verbal
What is an Adaptive test ?
• All questions allocated to groups of different difficulty• All students in a year group get common difficulty starting question• If answer correct, next question harder• If answer wrong, next question easier• Test looks at all answers in a section and homes in on questions of a
suitable difficulty for student
Try it yourself at www.intuproject.org/demos
X
Standardisation
• Test scores are standardised; Mean = 100, SD = 15
Standardised Score
National Percentage
Comment
>130 Top 2.5% Traditional classification of ‘mentally gifted’
>120 Top 10%
>100 Top 50%
<80 Bottom 10%
<70 Bottom 2.5% Potential special educational needs ??
The Computer Adaptive Baseline Test:
• Adaptive components common to MidYIS, Yellis, and Alis are:– Vocabulary– Maths
• Non- adaptive sections:– Non-verbal (MidYIS, Yellis, Alis)– Skills (MidYIS only)
• Assessments are ‘curriculum free’: i.e. students are not taught prior to the test
• Measure of acquired ability• There is no preparation for the test
Difficult vocabulary
Difficult maths
Non-Verbal: Cross-sections
Non-Verbal: Blocks
PicturesNon-Verbal: Pictures
Skills: Proof Reading
Skills: Perceptual Speed and Accuracy
Additionally for INSIGHT:Developed Ability
• Vocabulary
• Non verbal, Skills
• Attitudinal measures
Curriculum-based assessments covering:
Maths - Number & Algebra- Handling Data- Space, Shape & Measurement
Science - Biology, Chemistry, Physics, - Attitudes to Science
Reading - Speed Reading- Text Comprehension- Passage Comprehension
INSIGHT
INSIGHT - Maths
INSIGHT - Maths
Plus Attitude to Science: Perception of Science, Relevance to Self, Environmental Actions, Enjoyment of Science
INSIGHT - Science
INSIGHT – Reading(Text Comprehension)
INSIGHT – Reading(Passage Comprehension)
INSIGHT – Reading(Speed Reading)
INSIGHT – Developed Ability
Measuring AbilityKey Stage Data
or Baseline Test ?
Key Stage Baselines – KS2; GCSEStrengths WeaknessesRelated to curriculum Dependent on teaching effectiveness (prior Value-
Added) as well as student ability
Automatically available Open to manipulation
In Depth Not all students have KS data
Linked to student learning experience
Reliability of KS measure
Consistency of KS score (i.e. GCSE & Btec First etc)
Independent Baseline Test – CEM Adaptive Test
The Effect of Prior Value Added
Beyond Expectation+ve Value-Added
In line with Expectation0 Value-Added
Below Expectation-ve Value-Added
Average GCSE = 6 Average GCSE = 6 Average GCSE = 6
Do these 3 students all have the same ability ?
Baseline Feedback
Feedback from Baseline Test
• Intake Profiles• Individual Pupil Records (IPRs)• Predictions & Chances Graphs
Individual Pupil Record Sheets(IPRs)
Look for sections that are inconsistent
Available in all systems pre & post 16
Predictions
.
Subject X
020406080
100120140
4 5 6 7 8
Baseline
Out
com
e
0 2 4 6 8
Subject X
How CEM ‘Predictions’ are made…
A* / A
C
Regression Line (…Trend Line, Line of Best Fit)
Outcome = gradient x baseline + intercept
Correlation Coefficient (~ 0.7)
40
60
80
100
120
140
5 6 7 8
Average GCSE
Gra
de
PhotographySociologyEnglish LitPsychologyMathsPhysicsLatin
Some Subjects are More Equal than Others….
>1 grade
A*ABC
A
A*
B
C
D
E
F
E
D
C
B
A
A*
Test Score
GC
SE G
rade
s
Art & DesignBiologyChemistryEconomicsEnglishFrenchGeographyGermanHistoryIctMathematicsMedia StudiesMusicPhysical EducationPhysicsReligious StudiesScience (Double)Spanish
Some Subjects are More Equal than Others….GCSE (MidYIS or Yellis)
1 grade
Predictions – MidYIS example
Similar spreadsheets available from Yellis & INSIGHT
2 3
10
23
33
23
5
1 00
5
10
15
20
25
30
35
40
U G F E D C B A A*
Perc
ent
Grade
Individual Chances Graph for Student no.5 - GCSE EnglishMidYIS Score 82 MidYIS Band D
Most likely grade
Prediction/expected grade: 3.8 grade D
Predictions - Alis example
Why KS Baselines do not always tell the whole story
Student 1
Student 2
Student 2 - IPR
Student 3
Student 3 - IPR
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 330
5
10
15
20
25
30
Vocab Item Responses
Question
Resp
onse
Tim
e (s
)
Correct
Wrong
Thank YouRobert Clark – robert.clark@cem.dur.ac.uk
Neil Defty – neil.defty@cem.dur.ac.ukNicola Forster – nicola.forster@cem.dur.ac.uk
Recommended