©2016 Northwest Evaluation Association. PISA-based … · ©2016 Northwest Evaluation Association. ... Czech Republic France United Kingdom Iceland ... •North Star Academy, Newark

©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®


PISA-based OECD Test for Schools: A Blueprint

for School Improvement

Presented by

Dr. Terri L. Breeden, Shaker Heights City SD

and

Rob Johnson, NWEA Regional Manager



Thoughtful Questions to Consider about our Schools

OECD: Organisation for Economic CO-operation and Development

PISA


Know how students feel about their own learning, their confidence for learning and how their belief systems have a

direct correlation to their performance?


Know where your school fits nationally and internationally

regarding students’ processing, analyzing and thinking?


PISA in brief5

• Over half a million students…

– representing 28 million 15-year-olds in 72 countries/economies

… took an internationally agreed upon 2-hour test…

– Goes beyond testing whether students can reproduce what they were taught…

– Mathematics, reading, science, problem-solving, financial literacy

… and responded to a 30 minute questionnaire on…

– their personal background, their schools and their engagement with learning and school

• Parents, principals and system leaders provided data on…

– school policies, practices, resources and institutional factors that help

explain performance differences .


The structure of the PISA assessment

6

2000 2003 2006 2009 2012 2015

Reading Reading Reading Reading Reading Reading

Mathematics Mathematics Mathematics Mathematics Mathematics Mathematics

Science Science Science Science Science Science

Problem Solving Digital ReadingProblemSolving

Collaborative Problem Solving

PISA helps Countries Learn

The tests are designed to assess to what extent students at the end of compulsory education, can apply their knowledge to real-life situations and be equipped for full participation in society.


PISA-based OECD Test for Schools

• Requested by schools

• Spearheaded by America Achieves

• Developed with grant funds

• 2012 Paper/Pencil pilot started

• Taken by over 400 US schools


Includes International PISA PISA-Based OECD Test for Schools

Utilizes the common PISA scale

Administration Country level; online via thumb drive School-level &Online

Sample size at each school 35 students No more than 85 students

Test intervals Every three years On-demand

Includes student survey questions on

engagement, school culture and

climate

Country level comparable data to

“Participating PISA countries

comparison data”

School level reporting None 159-Page reports for schools


5048

53 > 20 schools

10-19 schools

1-9 schools

31

34

8

1

2

12

21

1

16

5

10 206

6

3

48

14

40

5

10

12

5

15

11

1

2

2

OECD Test for Schools (based on PISA)Participation Through 2015 In The United States

What are schools learning from the OECD Test for Schools?


LEADING

COMPANY

Each School Receives Actionable Information

• 159 page report for each school

• Detailed analyses of how your school(s) compare on

multiple measures- math, reading, science

• Not just scores, but context including climate and

culture


Overview of Results Report

Student Engagement and the Learning Environment at Your School

Disciplinary climate

Teacher/student relations

Students’ reading habits

Instrumental motivationSelf-efficacy

Your School Compared With Similar Schools in Your Country

Your School’s Results in an International Context

14


Your school

US average

Below International Baseline

Medium-High Performers

Top Performers Internationally


Are my students critical thinkers? Are they able to conceptualize, generalize, utilize and link information sources and flexibly translate between them?

OECD VS Consortia Proficiency Levels

Comparison.xlsx

OECD VS Consortia Proficiency Levels Comparison.xlsx


OECD Sample Test Item



Sample Question: Science

Reverse Osmosis

One way of producing drinking water from seawater is by reverse osmosis. Reverse osmosis is a type of filtration. Seawater is pushed through a semi-permeable membrane. Pressure is applied to the seawater. Semi-permeable means salt is trapped on one side of the membrane, but water can pass through. The trapped salts form a ‘seawater concentrate’ on one side of the membrane.

QuestionWhere do the trapped salts go?

A. Ocean

B. Storage tank

C. Delivery pipeline

D.Water distribution system

See more sample questions at: www.oecd.org/pisa/pisa-basedtestforschools.

https://www.oecd.org/pisa/test/


Why do schools care?Evolution of employment in occupational groups defined by PIAAC problem-solving skills

-20

-15

-10

-5

0

5

10

15

20

25

%

Employment of workers with advanced

problem-solving skills

Employment of workers with poor problem-solving skills

Employment of workers with medium-low problem-solving skills (PIAAC)

Source:PIAAC 2011




How We Compare to our Peers Nationally




Exploring Reader ProfilesHow is your reading Instruction addressing the deep and wide?


What types of readers are my

students?


Comparison of

Reader Profiles








What Students Say…

The test was simple, and

the navigating through

them was easy. The test

also had some interesting

articles and facts I

enjoyed reading

It challenged my mind to solve real-world problems that I felt like were relevant.

Some questions allowed for a broad typed out open-ended answer. This allowed me to better portray my thoughts through each answer.


How can we learn from other schools?

• Access to the Global Learning Network

• Facilitated by America Achieves

• Participate in Virtual, Regional and National Convenings

• Engage in peer-to-peer learning

©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®globallearningnetwork.americaachieves.org

GOALS FOR THE CONVENING OF

WORLD-LEADING SCHOOLS

Participants identify concrete

actions that will further this

work in their schools and

districts

Create a sense of community among high-

achieving and committed

individuals to establish a

peer network

Improve participants’

understandingof their OECD

Test for Schools results

Build understanding

of how this work fits in with other efforts to improve student

outcomes


Let’s hear from a school

https://www.youtube.com/watch?v=Cd1TIJRPGyM&ebc=ANyPxKpw5fp92TqSujBQ8bCUieCfAfUzf-CPmAJ_hRunwGpgAaH4i7V3xvRAaQRaJS2cejiMCovoLY2jLwjGZCn_IJjiDlTWwA


Singapore

Hong Kong-ChinaChinese Taipei

Korea

Macao-China

Japan Liechtenstein

Switzerland

Netherlands

Estonia Finland

CanadaPoland

Belgium

Germany Viet Nam

AustriaAustralia

IrelandSlovenia

DenmarkNew Zealand

Czech RepublicFrance

United Kingdom

IcelandLatvia

Luxembourg Norway

Portugal ItalySpain

Russian Fed.Slovak Republic United StatesLithuania

SwedenHungary

Croatia

Israel

Greece

Serbia Turkey

Romania

Bulgaria

U.A.E.

Kazakhstan

Thailand

ChileMalaysia

Mexico

410

420

430

440

450

460

470

480

490

500

510

520

530

540

550

560

570

580

Mean score

High mathematics performance

Low mathematics performance

… Shanghai-China performs above this line (613)

… 12 countries perform below this line

Average performance

of 15-year-olds in

MathematicsFig I.2.13

US

Massachusetts

Connecticut

Florida

26% of American 15-year-olds do not reach PISA

Level 2 (OECD average 23%, Shanghai 4%, Japan 11%, Canada 14%, Some estimate long-term

economic cost to be US$72 trillion )


• North Star Academy, Newark NJ

• Math & Science below average

• Student interest in math & science below average

• Revamped curriculum

• One year later: >50% of seniors planned to major in STEM

Are my students engaged and motivated?

http://hepg.org/hel-home/issues/30_3/helarticle/quietly,-a-new-test-gains-

advocates

http://hepg.org/hel-home/issues/30_3/helarticle/quietly,-a-new-test-gains-advocates


What Students Say…

The test was simple, and

the navigating through

them was easy. The test

also had some interesting

articles and facts I

enjoyed reading

It challenged my mind to solve real-world problems that I felt like were relevant.

Some questions allowed for a broad typed out open-ended answer. This allowed me to better portray my thoughts through each answer.


Thank YouQuestions?

Documents

©2016 Northwest Evaluation Association. PISA-based … · ©2016 Northwest Evaluation Association. ... Czech Republic France United Kingdom Iceland ... •North Star Academy, Newark