Great teachers change lives – but can we prove it?

Pearson Morning for English Teachers of Cambridge Exams

Great teachers change lives – but can we prove it?

Ian Wood

It’s all about outcomesWhere every journey starts

12

What do we value?

From the values exercise, Ian identified his core values as follows: •Making a meaningful difference

What difference do we make?

What difference does Pearson make?

“By 2018, we will report publicly on our impact on learner outcomes, in a rigorous and externally audited way, across Pearson’s whole business. We will be setting ourselves challenging targets and holding ourselves publicly to account for delivering on them.”

John Fallon, Pearson CEO

ef fi ca cy‧ ‧ ‧ [uncountable + of] formal the ability of something to produce the right result [= effectiveness]

https://neo.pearson.com/videos/22533

How good is my English?And other awkward questions

26

How good is my English?

What do I need to learn?

Am I making progress?

How much longer?How many more $$

$s?

You’re intermediate.An intermediate book.Of course you’re learning!!One of our courses/books…

Qual

ity

Quantity

Two basic dimensionsHow many

different tasks?

How effective?How efficient?

Language development

Combining quantity and quality

A slippery slope

C2

C1

B2

B1

A2

A1

Below A1

The Council of Europe scale

Prof

icie

nt

Use

r C2

Can understand with ease virtually everything heard or read. Can express him/herself spontaneously, very fluently and precisely.

C1 Can use language flexibly and effectively for social, academic and professional purposes.

Inde

pend

ent

Use

r

B2 Can understand the main ideas of complex text on both concrete and abstract topics, including technical discussions in his/her field of specialisation.

B1 Can understand the main points of clear standard input on familiar matters regularly encountered in work, school, leisure, etc.

Bas

ic

Use

r A2 Can understand sentences and frequently used expressions related to areas of most immediate relevance.

A1 Can interact in a simple way provided the other person talks slowly and clearly and is prepared to help.

Asp

irin

g

Use

r

<A1 Can use and understand some isolated words. Communication is hardly possible.

Basic Characteristics of the CEF ‘Levels’

Need for formal instruction extremely rare

Learner becomes autonomous

Most learners require instruction and guidance

Continuous relevant input required to even start learning

CEF

C2

C1

B2

B1

A2

A1

• Disagreement on what it means to be “at a level”

• Levels are too wide to be of practical use

• All learners start below A1

• Limited set of descriptors

Disadvantages of the CEF

Start learning “Perfect”

Learner Descriptor

<A11

32

CEF boundaries

A1A2

B1

B2

C1C2

Back to the slippery slope

Increasing language proficiency and CEF descriptors

Measuring progressThe CEF – the original data

DIFFICULTY ORIGINAL CFR LABEL CEFR

3.90 - Mastery C2

2,80 3.90 Operational efficiency C1

1.74 2.80 Vantage Plus B2+

0.72 1.74 Vantage B2

-0.26 0.72 Threshold Plus B1+

-1.23 -0.26 Threshold B1

-2.21 -1.23 Waystage Plus A2+

-3.23 -2.21 Waystage A2

-4.29 -3.23 Breakthrough A1

-5.39 -4.29 Below A1 <A1

…ut you’ll get there later!

CEFR CEFR LABEL CEFR

C2 MasteryProficient User

C1 Operational efficiency

B2 VantageIndependent User

B1 Threshold

A2 WaystageBasic User

A1 Breakthrough

Common European Framework of Reference for Languages: Learning, Teaching and Assessment

, Cambridge University Press, 2001

Measuring progressThe CEF – the original data


3.90 - Mastery C2







-3.23 -2.21 Waystage A2


-5.39 -4.29 Below A1 <A1





B2 VantageIndependent User

B1 Threshold


A1 Breakthrough

So, how many levels?

Measuring progressThe CEFR – the original data


3.90 - Mastery C2







-3.23 -2.21 Waystage A2


-5.39 -4.29 Below A1 <A1



3.90 - Mastery C2







-3.23 -2.21 Waystage A2


-5.39 -4.29 Below A1 <A1




B2 Vantage

Independent User

B1 Threshold


A1 Breakthrough


Ten even, or six uneven ones – your choice – but either way




Ten even, or six uneven ones – your choice – but either way

the ‘intermediate plateau’ = 4 whole levels of difficulty.




B2 Vantage

Independent User

B1 Threshold


A1 Breakthrough



3.90 - Mastery C2







-3.23 -2.21 Waystage A2


-5.39 -4.29 Below A1 <A1

Glo

bal

Sca

le o

f Englis

h

Measuring progressThe Global Scale of English

30

43

59

76

85

90

22

13

10

C2

C1

B2+

B2

B1+

B1

A2+

A2

A1

T

Glo

bal

Sca

le o

f Englis

h


3.90 - Mastery C2







-3.23 -2.21 Waystage A2


-5.39 -4.29 Below A1 <A1

90

10

● Finer grained scale from 10 to 90● A psychometric definition of what

is means to be “at a level”● Starts below A1

CEF

C2

C1

B2

B1

A2

A1

A granular Global Scale of English

0%10%20%30%40%50%60%70%80%90%

100%

10 20 30 40 50 60 70 80 90

Likelihood Correct Performance

GSE Task Difficulty

A learner at 25 on GSE

0%10%20%30%40%50%60%70%80%90%

100%

10 20 30 40 50 60 70 80 90


GSE Task Difficulty


0%10%20%30%40%50%60%70%80%90%100%

10 20 30 40 50 60 70 80 90


GSE Task Difficulty


GSE90

10

B1 MATURA

GSE 59

PEARSON Courses

C2

C1

B2

B1

A2

A1

90

10

90

10

Progress

B2

43

3013 to go

GSE 43B1

PTEAcademic

59= 87 TOEFL= 6.5 IELTS

CEF

C2

C1

B2

B1

A2

A1

CEF

C2

C1

B2

B1

A2

A1

Map the inputsWhat do I need to learn?

327

‘The Sea’Proficient User

C1-C2

‘The Delta’Independent User

B1-B2+

‘The River’Basic User

A1-A2+

‘The Stream’‘Tourist’Below A1

How realistic are we really about the ‘width’ of the CEF levels and how many hours it takes to progress from one to the next?

Measuring outcomes with the CEF

Matching the old world and the new…


20th century levels

Beginner

Elementary

Pre-intermediate

Intermediate

Upper-intermediate

Advanced

21st century levels

A1

A2

B1

B2

C1

C2

Publishers traditionally used six level names and the CEFR reduced its nine equal levels into six – so a simple match, right?

…or the 6 into 5 problem…


20th century levels

Beginner

Elementary

Pre-intermediate

Intermediate

Upper-intermediate

Advanced

21st century levels

A1

A2

B1

B2

C1

Everyone agrees A1 = Beginner, A2 = elementary & C1 = Advanced, but…

three ‘intermediate’ levels don’t go into two CEFR levels.

…or the 6 into 5 problem…


In reality, very few courses cater for genuine beginners or genuine C1-C2 level learners. Consider:

Look at the following CEFR can-do statement. Which level is it?

Can use the language fluently, accurately and effectively on a wide range of general, academic, vocational or leisure topics, marking clearly the relationships between ideas. Can communicate spontaneously with good grammatical control without much sign of having to restrict what he/she wants to say, adopting a level of formality appropriate to the circumstances.

● Original CEF descriptors – with values

● 200+ new descriptors – rated by teachers

● Starts below A1

● Triple the descriptors

Available now at:

english.com/gse

Global Scale of English: Learning Objectives

GSE Learning Objectives: 43-50 Spoken Interaction


● General Adult (Available now)

● Academic (Late 2015)

● Professional (Late 2015)

● Young Learners (Late 2015)

Available at:

english.com/gse


More accurate book mapping

More accurate book mapping

Measure the outputsWhat have I learnt?

439

40

The measuring toolProgress test wish list

What would we want from a progress test?

• Accuracy

• Reliability

• Mapped to CEFR

• All four skills

• Grammar and vocabulary

• Quick results

• Convenience

• Affordability

Progress

Access / ExperienceThe measuring tool

Introducing Progress

42

online & adaptive

integrated skills & course agnostic: to measure real language ability rather than course coverage

ready-made & standardised: for time-saving and consistent testing

robustly linked to the Global Scale of English and the CEFR: to accurately measure small amounts of progress within a level

3 tests taken at start, middle and end of a course

personal score report: highlights areas for focus to inform teaching & learning

automatically and objectively scored: delivers rapid results

institutionally administered: can be conveniently taken at school or at home

The measuring toolIntroducing Progress

43

The measuring toolProgress item types

44

45

46

47

Gradebook /

GSE score

Individual

Skills

Intervention

The measuring toolFeedback

The measuring toolFeedback

Can generally identify the topic of discussion around them when conducted slowly and clearly

• Outcomes need to be clear, understood and relevant to the learner

• Outcomes are more likely to be improved if they can be articulated,

made meaningful and measured

• Expectations of progress need to be realistically achievable in the

time available

• Learners and teachers need regular feedback on progress to allow for

timely intervention

• Inputs and outputs need to be calibrated to the same scale

In summaryHow NOT to set ourselves up to fail

The Global scale of English

Thank you

[email protected]

Education

Great teachers change lives – but can we prove it?