. . . . . . . . . . . . . . . . . . .

Manitoba Reading Scores on National and International Assessments

Adolescent Literacy Summit

April 14-15, 2011

Purpose

• Manitoba’s scores and rankings in reading on the most recent published results from the Pan-Canadian Assessment Program (PCAP) and from the Programme for International Student Assessment (PISA) will be presented, followed by an opportunity for discussion regarding what these results mean for Manitoba.

The Assessments

2000 (Reading), 2003, 2006, 2009 (Reading); 15-year-olds; OECD plus ‘partner countries’; each province

2007 (Reading; 13-year-olds), 2010 (Math; Grade 8); follow-up to SAIP (13- and 16-year olds; since mid-90s)

Nature of the assessments

PISA• 390 minutes of testing

material organized into different combinations of test booklets

• 210 minutes in length

120 min: reading

90 min: mathematics

90 min: science• Student and administration

questionnaires

PCAP• Similar to PISA• Student, teacher and

administration questionnaires

Defining ‘Reading’

PISA• Accessing and retrieving:

locate, retrieve distinct pieces of information.

• Integrating and interpreting: make internal sense of a text.

• Reflecting and evaluating: draw upon knowledge, ideas, attitudes beyond the text to relate it to one’s own conceptual/experiential reference frames.

PCAP• Comprehension: explicit and

implicit info; vocabulary, parts, elements, events

• Interpretation: analyze, synthesize to develop broader persp., ident. theme/thesis

• Response to text (includes response and reflection): personal connections, emotional response, evaluative stance

Scales and Levels

PISAMean (OECD): 500

Standard Deviation: 100

Levels

Level 1 (a, b) to Level 6

• Level 2 (407 to 479): ‘baseline level of proficiency, at which students begin to demonstrate the reading literacy competencies that will enable them to participate effectively and productively in life’

PCAPMean (pan-Can.): 500

Standard Deviation: 100

Levels

Below Level 1 to Level 3/4

• Level 2 (380 to 575): ‘acceptable level of performance for 13-year-olds’

PCAP 2007 Reading Results

The first, most evident and ‘impactful’ result.

20 points ~ 3 percentage points on a provincial test.

Manitoba – Canada difference: 28 points

PCAP 2007 Reading Results

Less ‘impactful’, but illustrative:

Proportion of students at Level 2 or above:MB – 84%Can – 88%

Previous Manitoba ranksPCAP/SAIP (13-year-olds)

Reading (SAIP)- 1998: 10th

Writing (SAIP)- 1998: 10th

- 2002: 3rd (82.7% at Level 2+; 83.5% for Canada)

PCAP – Reading- 2007: 6th

- 2010: ? (spring 2011 report)

PISA 2009 Reading Results (15-year-olds)

20 points ~ 3 percentage points on a provincial test.

Manitoba: 495 (9th)Canada: 524OECD: 496

Manitoba – Canada difference: 29 points

PISA 2009 Reading Results

Proportion of students at Level 2 or above:MB – 82.4%Can – 89.7%OECD - ~82%

Below Level 2

Level 5+

PISA 2009 – Contextual AnalysisImmigrant Status (preliminary)

Province

Percentage of students by immigrant status PISA 2009

2006 2009

2006 to 2009 Change – 1st generation

Combined reading score

Non 2nd gen.

1st gen. Non 2nd

gen.1st

gen. Non 2nd gen 1st gen Diff: 1st

- Non

Abs RelProv X 0% 0% 530 502 478 -52Prov Y -1% -7% 536 524 529 -7

Manitoba 88 7 5 80 9 11 6% 120% 502 488 475 -27

Prov Z 4% 67% 532 551 534 2Prov XX 2% 14% 527 526 525 -2Canada 79 11 10 76 14 11 1% 10% 527 522 520 -7

Other provinces have proportions of immigrant students of 6% or less in 2009.

PISA 2009 – Contextual AnalysisSocio-economic Status (preliminary)

0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65460

470

480

490

500

510

520

530

540

Reading Score vs SES - PISA 2009 (15 year-olds)

Socio-economic status (OECD = 0.0)

Rea

din

g S

core

(O

EC

D =

496

)

NF NB MB PE QC NS SK ON BC AB

PISA 2009 – Contextual AnalysisSocio-economic Status (preliminary)

-1 -0.5 0 0.5 1 1.5 2450

470

490

510

530

550

570

590

PISA 2009 Reading Score vs Student SES

Canada

Newfoundland and Labrador

Prince Edward Island

Nova Scotia

New Brunswick

Quebec

Ontario

Manitoba

Saskatchewan

Alberta

British Columbia

SES (OECD average = 0.0)

PIS

A 2

009

Rea

din

g S

core

Previous Manitoba ranksPISA (15-year-olds)

Reading

2000: 5th

2003: 6th

2006: 5th

2009: 9th

QuestionsWhat do these results mean for Manitoba?