The Ohio State University The Ohio State University College of Education College of Education

School of Educational Policy and Leadership School of Educational Policy and Leadership

Policy Implications of Policy Implications of PISA, TIMSS, PISA, TIMSS,

 & World-Class University  & World-Class University Rankings Rankings 

Chuing Prudence Chou (Chuing Prudence Chou ( 周祝瑛周祝瑛 ))Professor, Cheng-chi University, TaiwanProfessor, Cheng-chi University, Taiwan

Email: Email: iaezcpc@nccu.edu.twiaezcpc@nccu.edu.tw May 17th, 2007May 17th, 2007

YOU ARE GOING TO SHARE ABOUT…YOU ARE GOING TO SHARE ABOUT…

What are the Policy Implications of What are the Policy Implications of PISA, TIMSS and the World-class PISA, TIMSS and the World-class University Rankings?University Rankings?

What lessons can we learn from What lessons can we learn from these global rankings from a these global rankings from a comparative perspective?comparative perspective?

ARE YOU IN OR NOT?ARE YOU IN OR NOT?

PISA (OECD)PISA (OECD) TIMSS (IEA) TIMSS (IEA) World-class University Rankings World-class University Rankings

IS THIS EVERYBODY’S BUSINESS?IS THIS EVERYBODY’S BUSINESS? The Finnish education phenomenonThe Finnish education phenomenon The German PISA shockThe German PISA shock Mexican’s indifferent attitudeMexican’s indifferent attitude (For PISA and TIMSS)(For PISA and TIMSS)

Brain Korea 21 (BK21)Brain Korea 21 (BK21) Japanese public university incorporationJapanese public university incorporation China’s 211 and 985 Project China’s 211 and 985 Project Taiwanese world-class university funding Taiwanese world-class university funding

program, etcprogram, etc (For World-class University Rankings)(For World-class University Rankings)

WHAT ARE THESE BENCHMARKS?WHAT ARE THESE BENCHMARKS?

PISA (Program for International PISA (Program for International Student Assessment) (OECD), first in Student Assessment) (OECD), first in 2000 and then held every 3 years. 2000 and then held every 3 years.

Three domains: Three domains:

reading literacy, reading literacy,

mathematical literacy, &mathematical literacy, &

scientific literacy.scientific literacy.

TIMSSTIMSS((Trends in International Mathematics and Science StudyTrends in International Mathematics and Science Study))

IEAIEA Focus on international mathematics Focus on international mathematics

and science achievement assessment. and science achievement assessment. First data collected in 1995, and 1999, First data collected in 1995, and 1999,

2003 and 2007. The United States is 2003 and 2007. The United States is collecting TIMSS data now.collecting TIMSS data now.

PISAPISA TIMSSTIMSS Sponsor Sponsor OECD OECD IEA IEA

FramewoFramework rk

Expert-driven Expert-driven Mathematics Mathematics

curriculum-curriculum-drivendriven

Top-down Top-down approach approach

Bottom-up Bottom-up

approachapproach

Real-world Real-world

application application

Classroom Classroom math math

knowledgeknowledge

Problem-Problem-solving solving

Curriculum contCurriculum contentent

TargetTarget groupgroup

An age-baseAn age-based d

samplesample

A grade-basA grade-based ed

samplesample 15-year- old15-year- old

s s Grade 3 & 4, Grade 3 & 4, 7 & 7 &

8 8 + high + high

school school seniorsseniors

WHAT ARE THE TEST OUTCOMES?WHAT ARE THE TEST OUTCOMES?

There is a greater difference between There is a greater difference between Asian countries/regions and other Asian countries/regions and other countries in TIMSS rather than in PISA. countries in TIMSS rather than in PISA.

Most English-speaking countries are Most English-speaking countries are ahead of Eastern European counterparts ahead of Eastern European counterparts in. PISA, but fall behind in TIMSS.in. PISA, but fall behind in TIMSS.

WHY SUCH DIFFERENCES EXIT? WHY SUCH DIFFERENCES EXIT?

TIMSS items tend to be more TIMSS items tend to be more curriculum-based (more universal), curriculum-based (more universal),

PISA items are more application-PISA items are more application-based (more real-life experience and based (more real-life experience and cultural-bound)cultural-bound)

TIMSS TIMSS (1997)(1997)(13-year-old's average score, Third International Math and Science Study)(13-year-old's average score, Third International Math and Science Study)

Countries:Countries:(sample)(sample)

GlobalGlobalrankrank

MathMath ScienceScience

ScoreScore RankRank ScoreScore RankRank

SingaporeSingapore 11 643643 11 607607 11

JapanJapan 22 605605 33 571571 33

South KoreaSouth Korea 33 607607 22 565565 44

Czech RepublicCzech Republic 44 564564 66 574 574 22

EnglandEngland 1818 506506 2525 552552 1010

ThailandThailand 2020 522522 2020 525525 2121

GermanyGermany 2222 509509 2323 531531 1919

FranceFrance 2323 538538 1313 498498 2828

United StatesUnited States 2424 500500 2828 534534 1717Source: 1997 TIMSS, in The Economist, March 29th 1997.Source: 1997 TIMSS, in The Economist, March 29th 1997.

TIMSS TIMSS (2003)(2003)(13-year-old's average score, Third International Math and Science Study)(13-year-old's average score, Third International Math and Science Study)

Countries:Countries:(sample)(sample)

GlobalGlobalrankrank

MathMath ScienceScience

ScoreScore RankRank ScoreScore RankRank

SingaporeSingapore 11 605605 11 578578 11

TaiwanTaiwan 22 585585 44 571571 22

South KoreaSouth Korea 33 589589 22 558558 33

Hong KongHong Kong 44 586586 33 556556 44

JapanJapan 55 570570 55 552552 55

NetherlandsNetherlands 77 536536 77 536536 99

EnglandEngland 1010 498498 1818 544544 77

United StatesUnited States 1212 504504 1515 527527 1111

MalaysiaMalaysia 1818 508508 1010 510510 2121

ItalyItaly 2323 484484 2222 491491 2222

Sources:Sources:TIMSSTIMSS Math 2003 Math 2003 and TIMSS Science 2003 and TIMSS Science 2003

Trends in Trends in PISA PISA

math achievementmath achievement

2000 2000 2003 2003

JapanJapan 557557 534534

KoreaKorea 547547 542542

NetherlandNetherland 537537 538538

FinlandFinland 536536 544544

AustraliaAustralia 533533 524524

CanadaCanada 533533 532532

SwitzerlandSwitzerland 529529 527527

UKUK 529529

BelgiumBelgium 520520 529529

FranceFrance 517517

OECDOECD 500500

GermanyGermany 490 490 503503

RussiaRussia 478478

New Zealand New Zealand USUS

493493

523523

There were 38 participating countries in There were 38 participating countries in 2003 PISA2003 PISA

US rankedUS ranked：： 7th in reading literacy 7th in reading literacy (15th in 2000 out of 32 countries)(15th in 2000 out of 32 countries) 24th in math literacy24th in math literacy (19th in 2000)(19th in 2000) 9th in science literacy 9th in science literacy (14th in 2000)(14th in 2000) 20th in problem solving 20th in problem solving

Trends in Trends in PISA PISA

science achievementscience achievement

2000 2000 2003 2003

JapanJapan 548548 550550

FinlandFinland 548548 538538

KoreaKorea 538538 552552

AustraliaAustralia 525525 528528

NetherlandNetherland 524524 528528

New New ZealandZealand

521521

CanadaCanada 519519 529529

SwitzerlandSwitzerland 513513

FranceFrance 511511 BelgiumBelgium 509509

GermanyGermany 502502

IrelandIreland 513513

UKUK 532532

SwissSwiss 512512

Austria Austria USUS

499499

519519

Trends in Trends in PISA PISA

reading literacy reading literacy achievement 2003achievement 2003

2003 2003

FinlandFinland 543543

KoreaKorea 534534

CanadaCanada 528528

AustraliaAustralia 525525

New ZealandNew Zealand 522522

IrelandIreland 515515

SwissSwiss 514514

NetherlandNetherland 513513

BelgiumBelgium 507507

NorwayNorway 500500

SwitzerlandSwitzerland 499 499

JapanJapan 498498

PolandPoland 497497

FranceFrance 496496

There were 38 participating countries in There were 38 participating countries in 2003 PISA2003 PISA

US rankedUS ranked：： 7th in reading literacy 7th in reading literacy (15th in 2000 out of 32 countries)(15th in 2000 out of 32 countries) 24th in math literacy24th in math literacy (19th in 2000)(19th in 2000) 9th in science literacy 9th in science literacy (14th in 2000)(14th in 2000) 20th in problem solving 20th in problem solving

As the provision of higher educational As the provision of higher educational 　　 opportunities becomes increasingly opportunities becomes increasingly 　　 international, institutional comparison international, institutional comparison 　　 becomes more in demand. The two most becomes more in demand. The two most 　　 frequently cited rankingsfrequently cited rankings：：

　　 1. Shanghai Jiaotong University, 1. Shanghai Jiaotong University, Academic Ranking of World UniversitiesAcademic Ranking of World Universities　　 2. Times Higher Education Supplement, Londo2. Times Higher Education Supplement, Londo

n, n, Times Higher University World RankingsTimes Higher University World Rankings

WHO NEEDS WORLD-CLASS WHO NEEDS WORLD-CLASS UNIVERSITIES? UNIVERSITIES?

FROM WHOES PERSPECTIVES?FROM WHOES PERSPECTIVES?

Becoming international, becoming Becoming international, becoming

competition-drivencompetition-driven More pressure from domestic and More pressure from domestic and

abroadabroad Who can afford joining the "World-Who can afford joining the "World-

Class Club"?Class Club"? Who benefits from these global, world-Who benefits from these global, world-

class competition? class competition?

MYSTERIES UNRESOLVED?MYSTERIES UNRESOLVED?

What does that mean by the "World Class What does that mean by the "World Class University"?University"?

How many world class universities do we How many world class universities do we need? need?

Does every country need one?Does every country need one? How could universities be evaluated and How could universities be evaluated and

compared in an international scope? compared in an international scope? What measures should be taken in order What measures should be taken in order

to establish world class universities? to establish world class universities? Who share the cost and who benefit?Who share the cost and who benefit?

World’s top 15 universities as an average of World’s top 15 universities as an average of THES & SJTU 2005 rankingsTHES & SJTU 2005 rankings

RanRankk

Av. Av. RankRank

InstitutionInstitution CountryCountry Times Times ShanghaShanghaii

11 11 Harvard  Harvard  U.S.U.S. 11 11

22 2.52.5 CambridgeCambridge U.K.U.K. 33 22

33 3.53.5 Massachusetts Inst. of Massachusetts Inst. of TechnologyTechnology

U.S.U.S. 22 55

44 44 StanfordStanford U.S.U.S. 55 33

55 55 University of California, University of California, Berkeley Berkeley

U.S.U.S. 66 44

66 77 California Inst. of TechnologyCalifornia Inst. of Technology U.S.U.S. 88 66

66 77 OxfordOxford U.K.U.K. 44 1010

88 8.58.5 PrincetonPrinceton U.S.U.S. 99 88

99 99 YaleYale U.S.U.S. 77 1111

1010 1313 CornellCornell U.S.U.S. 1414 1212

1111 13.513.5 ChicagoChicago U.S.U.S. 1818 99

1111 13.513.5 ColumbiaColumbia U.S.U.S. 2020 77

1313 17.517.5 University of California, S.F.University of California, S.F. U.S.U.S. 1717 1818

1414 1818 TokyoTokyo Japan Japan 1616 2020

1414 1818 Imperial College LondonImperial College London U.K.U.K. 1313 2323Sources: Shanghai Jiaotong Academic Ranking of World Universities – 2005; Times Higher Education Supplement World University Rankings – 2005.

Sources: Shanghai Jiaotong Academic Ranking of World Universities – 2005; Times Higher Education Supplement World University Rankings – 2005.

RanRankk

Av. Av. RanRan

kk

InstitutionInstitution CountryCountry TiTimemes s

ShShananghghaiai

16 21.5 Duke U.S. 11 32

17 23 Johns Hopkins U.S. 27 19

18 23.5 University of Pennsylvania U.S. 32 15

19 24 Swiss Federal Institute of Technology, Zurich Switzerland 21 27

20 25.5 University of California, L.A. U.S. 37 14

WHAT CAUSES THE GLOBAL RANKINGS WHAT CAUSES THE GLOBAL RANKINGS AND WHAT THE EFFECTS ARE?AND WHAT THE EFFECTS ARE?

The "Impact of Globalization " The "Impact of Globalization " Borrowing and lending reform strategiesBorrowing and lending reform strategies In comparative education, there is a In comparative education, there is a

great need to examine how local great need to examine how local reactions are to these external global reactions are to these external global forces .forces .

In policy studies, a new paradigm for In policy studies, a new paradigm for educational policy-making and school educational policy-making and school reform is expected to be under way. reform is expected to be under way.

What can be learned and imported What can be learned and imported from elsewhere? (borrowing) from elsewhere? (borrowing)

What can be taught and exported What can be taught and exported elsewhere? (lending), elsewhere? (lending),

The phenomena of cross-national policy The phenomena of cross-national policy attraction and educational borrowingattraction and educational borrowing

Global and international convergence Global and international convergence in education at the expense local needs in education at the expense local needs and cultural diversityand cultural diversity

Three types of National Policy ResponsesThree types of National Policy Responses

According to Steiner-Khamsi (2007), According to Steiner-Khamsi (2007), three types of policy reactions after three types of policy reactions after international comparisons: international comparisons:

1.1. Scandalization (to blame one's own Scandalization (to blame one's own weak educational system), weak educational system), 2. Glorification (to praise one's good 2. Glorification (to praise one's good educational system),educational system),3. Indifference to ranking results. 3. Indifference to ranking results.

Typology of Political Reactions to International Typology of Political Reactions to International Comparative StudiesComparative Studies

TIMSSTIMSS PISAPISA World-Class World-Class University University RankingsRankings

ScandalizatioScandalizationn

USAUSA Germany, Germany,

IsraelIsrael

Korea, Korea, Japan, Japan,

TaiwanTaiwan

GlorificationGlorification Japan, Japan, FinlandFinland

UK, UK, FinlandFinland

USA,UKUSA,UK

IndifferenceIndifference MexicoMexico MexicoMexico

IS IT A FAIR GMAE FOR IS IT A FAIR GMAE FOR EVERYBODY TO PLAY?EVERYBODY TO PLAY?

What makes countries to react so differeWhat makes countries to react so differently?ntly?

Has scandalization led to increased policHas scandalization led to increased policy import from other educational systems?y import from other educational systems?

Has glorification led to increased policy eHas glorification led to increased policy export to other educational systems?xport to other educational systems?

Why some remain indifferent to these gloWhy some remain indifferent to these global comparisons?bal comparisons?

DO POLICY MAKERS BENEFIT FROM THESE DO POLICY MAKERS BENEFIT FROM THESE GLOBAL RANKINGS?GLOBAL RANKINGS?

1. More evidence-based research in public policy studies.1. More evidence-based research in public policy studies.

2. More Outcome-based and standard-based criteria 2. More Outcome-based and standard-based criteria in monitoring school reforms in monitoring school reforms 3. A "scientific rationality" in education to appeal to 3. A "scientific rationality" in education to appeal to the tax payers the tax payers 4. A legitimate “self-referential system “ acting as policy 4. A legitimate “self-referential system “ acting as policy

borrowing ( a lesson learned from elsewhere) and a borrowing ( a lesson learned from elsewhere) and a

form form of externalization. of externalization.

WHY GERMAN KIDS FELL BEHIND?WHY GERMAN KIDS FELL BEHIND?

In the PISA study released in 2001, In the PISA study released in 2001, German students scored below the German students scored below the OECD-average with regard to reading OECD-average with regard to reading literacy. literacy.

The release of the PISA study led to a The release of the PISA study led to a major uproar in the media, and calls major uproar in the media, and calls for policy change and school for policy change and school transformation. transformation.

The Finnish Education PhenomenonThe Finnish Education Phenomenon German PISA ShockGerman PISA Shock Brain Korea 21 (BK21) Brain Korea 21 (BK21) Japanese university incorporation Japanese university incorporation China’s 211 and 985 ProjectChina’s 211 and 985 Project Taiwanese fifty-trillion dollars Taiwanese fifty-trillion dollars

within five year’s world-class within five year’s world-class university funding program, etc.university funding program, etc.

India’s brain drain of the university India’s brain drain of the university facultyfaculty

DARE TO SAY NO?!DARE TO SAY NO?!

Problems with global rankings Problems with global rankings : : 1. Credibility and reliability issues of PISA 1. Credibility and reliability issues of PISA & TIMSS& TIMSS 2. Danger of the international convergence 2. Danger of the international convergence of educational institutions and policies of educational institutions and policies 3. Near-sighted government reform policies 3. Near-sighted government reform policies

&& projects aiming at policy borrowing projects aiming at policy borrowing across national boundaries.across national boundaries.

4.4. The assumption of “standardized and The assumption of “standardized and

universal qualifications” within universal qualifications” within universities universities

5.5. Diagnosed rather than prescribed Diagnosed rather than prescribed

formula approach, not aiming for formula approach, not aiming for solving local educational problems. solving local educational problems.

6. Indifference to the ranking impact on 6. Indifference to the ranking impact on students, employers, teachers, and students, employers, teachers, and school officials.school officials.

7. Criticism about these global rankings as 7. Criticism about these global rankings as the “fast capitalism” which benefits the “fast capitalism” which benefits ranking benchmarks ranking benchmarks

8. The advantage of English-speaking 8. The advantage of English-speaking countries in the global university rankings countries in the global university rankings

9. Only the richest and the best survive in the 9. Only the richest and the best survive in the world-class university rankings world-class university rankings

10. The rich get richer, and the poor get 10. The rich get richer, and the poor get poorer.poorer.

11. Increasing polarization of institutional 11. Increasing polarization of institutional resource generation, public funding, and resource generation, public funding, and educational quality nation-wide and world-educational quality nation-wide and world-wide. wide.

12. The cumulative advantage vs. equal 12. The cumulative advantage vs. equal

opportunities for allopportunities for all

13. Most university ranking criteria 13. Most university ranking criteria focus on focus on

academic research productivity academic research productivity rather rather

than human aspect in cultivation than human aspect in cultivation and and

collegiality.collegiality.

14. Higher education shifted from public 14. Higher education shifted from public good good

to private commodity.to private commodity.

15. Many courses and degree programs are 15. Many courses and degree programs are

packaged and marketed just like packaged and marketed just like consumer goods, and students are seen consumer goods, and students are seen as customers with a world of choice in as customers with a world of choice in front of them. front of them.

16. “Exporting” university services to less 16. “Exporting” university services to less developed countries.developed countries.

THE RANKING GAME IS NOT OVER YET, THE RANKING GAME IS NOT OVER YET, BUT THE LECTURE IS GETTING...BUT THE LECTURE IS GETTING...

International rankings and International rankings and comparisons will continue in spite of comparisons will continue in spite of criticism and protest. criticism and protest.

Support from many governments, Support from many governments, mass media, educational institutes mass media, educational institutes and the general public. and the general public.

Benchmarks create huge benefits.Benchmarks create huge benefits.

QUESTIONS REMAIN…QUESTIONS REMAIN…

Why the US has not so satisfactory PISA and TIMSS Why the US has not so satisfactory PISA and TIMSS results, but has established so many world-class results, but has established so many world-class universities in the countries? universities in the countries?

Does this global ranking competition bring real Does this global ranking competition bring real change or improvement to education at home?change or improvement to education at home?

What is the politics of comparison in global ranking?What is the politics of comparison in global ranking?

Does Taiwan need to be part of the Does Taiwan need to be part of the game? Why?game? Why?

THANK YOU ( THANK YOU ( 謝謝）謝謝） && GOD BLESS YOU!!GOD BLESS YOU!!

Comments are welcome!Comments are welcome! Eamil: iaezcpc@nccu.edu.twEamil: iaezcpc@nccu.edu.tw