Paper 4: Ranking Universities (Ince)

Martin Ince Communications LtdMartin Ince Communications Ltd

International Workshop on International Workshop on Performance ManagementPerformance Management

Kent Business SchoolKent Business SchoolPresentation by Martin Ince Presentation by Martin Ince

July 1, 2010July 1, 2010


Ranking the world’s Ranking the world’s universities; why, how, and universities; why, how, and

what happens when you do?what happens when you do?

Presentation by Martin IncePresentation by Martin InceJournalist and media adviserJournalist and media adviserFounder of the THE/QS World Founder of the THE/QS World

University RankingsUniversity RankingsChair of the Advisory Board of the QS Chair of the Advisory Board of the QS

Rankings Rankings


Why rank universities?Why rank universities?

Rapid growth of higher educationRapid growth of higher education Growing expectation of opennessGrowing expectation of openness New students needing knowledgeNew students needing knowledge More need for international More need for international

knowledgeknowledge Ranking things and peopleRanking things and people


Anything can be ranked..Anything can be ranked..

In the UK: schools, local government, In the UK: schools, local government, libraries, hospitals, police forces….libraries, hospitals, police forces….

Kent Business School has the world’s Kent Business School has the world’s ninth-ranked lecturer in information ninth-ranked lecturer in information systemssystems


Fun or…Fun or…

Some of these rankings are for funSome of these rankings are for fun

- rich lists- rich lists

- now dozens of types- now dozens of types

Some are seriousSome are serious

- Over 1000 “failing” institutions - Over 1000 “failing” institutions


National university rankingNational university ranking

US News: 18 main columns of dataUS News: 18 main columns of data Likewise Times GUG, Macleans et alLikewise Times GUG, Macleans et al Rich data in national systemsRich data in national systems

- Library costs- Library costs- Alumni giving- Alumni giving- Completion- Completion

- Course cost- Course cost - Economic impact- Economic impact - Admissions standards- Admissions standards


National dataNational data

AvailableAvailable ConsistentConsistent Has quality controlHas quality control Has an agreed meaningHas an agreed meaning Tells a definite storyTells a definite story

-- eg admission standards -- eg admission standards

dropout ratesdropout rates


International university rankingInternational university ranking

Far less dataFar less data Scheme has to grow out of what we Scheme has to grow out of what we

want to measurewant to measure Who is it for?Who is it for? Main functions of a universityMain functions of a university Less data on specific subjectsLess data on specific subjects


Globalisation of universitiesGlobalisation of universities

What is it?What is it? Is it important?Is it important? How do you measure it?How do you measure it? What happens when you try?What happens when you try?


What is it?What is it?

Universities are the original global Universities are the original global enterprisesenterprises

3 million overseas students (2006)3 million overseas students (2006) Falling home student numbers in Falling home student numbers in

many developed world nationsmany developed world nations Awareness of students’ need to be Awareness of students’ need to be

more globalmore global CollaborationCollaboration


The five flows: Michael PorterThe five flows: Michael Porter

PeoplePeople MoneyMoney Ideas Ideas

ServicesServices GoodsGoods


War for talentWar for talent

Work of Phil Brown et alWork of Phil Brown et al Unskilled jobs mobile for decadesUnskilled jobs mobile for decades IT makes middle class jobs mobileIT makes middle class jobs mobile Microsoft in ChinaMicrosoft in China Design, research, financeDesign, research, finance Pasadenanow.comPasadenanow.com Universities and students know thisUniversities and students know this


Economic impactEconomic impact

Research and innovationResearch and innovation Not just Silicon ValleyNot just Silicon Valley

• BostonBoston• Cambridge UKCambridge UK• FinlandFinland• MunichMunich• BangaloreBangalore


Economic impactEconomic impact

New emphasesNew emphases• Climate change: mitigation and adaptationClimate change: mitigation and adaptation• Population ageing, obesity etc Population ageing, obesity etc • RoboticsRobotics• NanotechnologyNanotechnology• Web 2.0, 3.0…Web 2.0, 3.0…• BiotechnologyBiotechnology• Where will this happen?Where will this happen?

University only one of many factorsUniversity only one of many factors


This is only part of the story..This is only part of the story..

Manufacturing about 13 per cent of Manufacturing about 13 per cent of UK economyUK economy

Growth in:Growth in:• HealthcareHealthcare• LeisureLeisure• FinanceFinance• EducationEducation


Soft powerSoft power

Governments appreciate thisGovernments appreciate this British Council et alBritish Council et al Cheaper and more reliable than warCheaper and more reliable than war China 282 Confucius Institutes, China 282 Confucius Institutes, 1000 by 20201000 by 2020 Universities major poles of soft powerUniversities major poles of soft power


Other pressuresOther pressures

BolognaBologna GATSGATS Asian growthAsian growth BTABTA LambertLambert


QS: measuring top universitiesQS: measuring top universities

Started in 2004Started in 2004 No agreed way to do thisNo agreed way to do this We decided on a novel approachWe decided on a novel approach QS Quacquarelli Symonds and THESQS Quacquarelli Symonds and THES Key issue: data consistency and data Key issue: data consistency and data

relevancerelevance


Instead…Instead…

Look for globally agreed measures Look for globally agreed measures of:of:

Academic qualityAcademic quality

EmployabilityEmployability

Research Research

TeachingTeaching

World reachWorld reach


Universities mustUniversities must

Teach in at least two of the five Teach in at least two of the five areasareas

Teach undergraduatesTeach undergraduates Omit Rockefeller, UCSFOmit Rockefeller, UCSF Include LSEInclude LSE Meant to cover large general Meant to cover large general

institutionsinstitutions


Academics know about universitiesAcademics know about universities

So we ask themSo we ask them 9,386 this time9,386 this time What do you know about?What do you know about? Who is good at it?Who is good at it? Nearly 200,000 data pointsNearly 200,000 data points 40 per cent of possible score40 per cent of possible score


Who are they?Who are they?

Academic Peer Review Responses by J ob Classification

Professor / Associate Professor

Administrator / Functional Manager Senior Lecturer

Lecturer

OtherTeaching AssistantResearch Assistant

Librarian / Library Assistant

Admissions OfficerResearch Specialist

Head of Department

Senior Administrator

Vice-President / Deputy Vice-Chancellor

President / Vice-Chancellor


Balance…Balance…

Subject areasSubject areas

Social sciences Social sciences

Arts and humanities Arts and humanities

Science Science

Technology Technology

Biomedicine Biomedicine LocationsLocations


Likewise employersLikewise employers

Where do you like to recruit?Where do you like to recruit? Major multinational employersMajor multinational employers 3,281 in 20083,281 in 2008 This means another 65,000 data This means another 65,000 data

pointspoints 10 per cent for this. So academic and 10 per cent for this. So academic and

employer opinion make up half your employer opinion make up half your possible scorepossible score


ResearchResearch

Citations/ 5 years from ScopusCitations/ 5 years from Scopus Per person not per paperPer person not per paper Well-known biases in subject, Well-known biases in subject,

language etclanguage etc 20 per cent20 per cent


TeachingTeaching

Staff/student ratioStaff/student ratio Looks simple but..Looks simple but..

Also 20 per centAlso 20 per cent


Global reachGlobal reach

International staffInternational staff International studentsInternational students 5 per cent each5 per cent each Netherlands problemNetherlands problem

Are these people serious about being Are these people serious about being international?international?Is this a place people want to be?Is this a place people want to be?


Ranking Criteria & Ranking Criteria & WeightsWeights

2828

Recruiter Review – Recruiter Review – 10%10%

International Faculty – International Faculty – 5%5%International Students – International Students – 5%5%

Student Faculty Ratio – Student Faculty Ratio – 20%20%

PeerPeerRevieRevie

ww

CitationCitationss

per per FacultyFaculty

RecruitRecruiterer

ReviewReview

Int’l Int’l FacultyFaculty

Int’l Int’l StudentsStudents

StudenStudentt

FacultyFacultyRatioRatio

Citations per Faculty– Citations per Faculty– 20% 20% Peer Review – 40%Peer Review – 40%


Rankings....Rankings....

Are a composite indicator with many Are a composite indicator with many measuresmeasures

Provide a range of ways to excelProvide a range of ways to excel They are not a league tableThey are not a league table


Unlike a league table…Unlike a league table…

Many ways to be successMany ways to be success Good to be in it at allGood to be in it at all Rules far more complexRules far more complex More variety: many university More variety: many university

missionsmissions


QS Classifications make QS Classifications make missions clearermissions clearer

SizeSize Research volumeResearch volume Coverage:Coverage:

CompleteComplete

Complete but no medical schoolComplete but no medical school

SpecialistSpecialist

See at www.topuniversities.comSee at www.topuniversities.com


32323232


What it showsWhat it shows

Harvard is pretty goodHarvard is pretty good US and UK 83/200US and UK 83/200 But 31 nations have a university hereBut 31 nations have a university here Mexico, India, MalaysiaMexico, India, Malaysia


Faculty level analysisFaculty level analysis

The five areasThe five areas• Arts and humanitiesArts and humanities• Social SciencesSocial Sciences• ScienceScience• TechnologyTechnology• BiomedicineBiomedicine


Methodology..Methodology..

Peer reviewPeer review Citations per paperCitations per paper Only top 100Only top 100 Not combinedNot combined

• Shows big country effectShows big country effect


QS and THEQS and THE

Contract ended between these twoContract ended between these two QS has many international media QS has many international media

partnerspartners US News and World ReportUS News and World Report Nouvelle ObservateurNouvelle Observateur Chosun IboChosun Ibo

Publish 200/500Publish 200/500


Times HigherTimes Higher

Work with Thomson ReutersWork with Thomson Reuters Stay with peer reviewStay with peer review And bibliometrics And bibliometrics Teaching as well as researchTeaching as well as research Not employersNot employers


The competitionThe competition

Shanghai Jiao Tong, ARWUShanghai Jiao Tong, ARWU• Nobel Prizes – tiny universe of 802 Nobel Prizes – tiny universe of 802

people worth 40 per cent people worth 40 per cent • Papers in Science and NaturePapers in Science and Nature• But 146/200But 146/200• Cf Ton van RaanCf Ton van Raan


HEEACTHEEACT

PublicationsPublications CitationCitation High citationHigh citation

But still heavy science orientationBut still heavy science orientation


More competitors…More competitors…

Ecole des MinesEcole des Mines5 of top 10 French5 of top 10 French

CSICCSIC ScimagoScimago EU EU OECD (teaching)OECD (teaching)

Measuring something…Measuring something…Media or other producers?Media or other producers?


4141

Indicators & Weights for Indicators & Weights for Established Aggregate Established Aggregate

RankingsRankingsQ

SQ

S

HE

EA

CT

HE

EA

CT

Shanghai

Shanghai

Webom

etr

ics

Webom

etr

ics


First impactFirst impact

International scholarly industryInternational scholarly industry IREGIREG Berlin PrinciplesBerlin Principles Leiden, Tsinghua et alLeiden, Tsinghua et al Hope they’ll say thank you Hope they’ll say thank you


More importantly..More importantly..

Big media interestBig media interest 24 articles in the Mexican press 200424 articles in the Mexican press 2004 Especially from AsiaEspecially from Asia Also continental Europe and UKAlso continental Europe and UK Far less from USFar less from US


Response is…Response is…

Reject the whole idea (German Reject the whole idea (German rectors)rectors)

Complain about their positionComplain about their position Ask how to do betterAsk how to do better


So here is the answer:So here is the answer:

Be visible around the world as an Be visible around the world as an excellent institutionexcellent institution

Produce graduates people want to Produce graduates people want to hirehire

Produce papers people want to citeProduce papers people want to cite Have enough staff to teach your Have enough staff to teach your

studentsstudents Be attractive to overseas staff and Be attractive to overseas staff and

studentsstudents


But the real surprise..But the real surprise..

THE (previously THES) since 1971THE (previously THES) since 1971 Always written about universities and Always written about universities and

HE around the worldHE around the world Nobody ever noticedNobody ever noticed


Because of the Rankings:Because of the Rankings:

Heisenberg in actionHeisenberg in action

THE turned into something that THE turned into something that anyone running a university anyone running a university anywhere in the world has to know anywhere in the world has to know about about


First group: politiciansFirst group: politicians

Do UK universities need more Do UK universities need more money?money?

Or are they fine as they are?Or are they fine as they are?

US/UK comparisonUS/UK comparison


GermanyGermany

Rectors like things as they areRectors like things as they are Government doesn’t agreeGovernment doesn’t agree Only one winnerOnly one winner Excellence initiativeExcellence initiative €€1.9 billion1.9 billion in first tranche, $2.8 bnin first tranche, $2.8 bn Nine institutions + clusters and Nine institutions + clusters and

research centresresearch centres CHECHE


KoreaKorea

Peak of world interestPeak of world interest Presidential candidatesPresidential candidates About eight institutions want to be in About eight institutions want to be in

top 100top 100 Can’t all succeedCan’t all succeed General interest in other people’s General interest in other people’s

views of their societiesviews of their societies Need THE/QS degreeNeed THE/QS degree


Likewise…Likewise…

● ● BruneiBrunei

Study abroad (THE/QS)Study abroad (THE/QS)

● ● NetherlandsNetherlands

Entry visa (THE or Shanghai 150)Entry visa (THE or Shanghai 150)

● ● Middle EastMiddle East


Pushing systems ..Pushing systems ..

TowardsTowards

ConcentrationConcentration

New reward mechanismsNew reward mechanisms

InternationalisationInternationalisation

Absorbing research institutionsAbsorbing research institutions

But still need a wide range of But still need a wide range of

provision for a mass educated societyprovision for a mass educated society


Is it compulsory?Is it compulsory?

No it isn’tNo it isn’t 4,000 universities cannot all be 4,000 universities cannot all be

HarvardHarvard Costs time and moneyCosts time and money Perfectly valid decision not to get Perfectly valid decision not to get

involvedinvolved Risks: eg Chinese provisionRisks: eg Chinese provision


Rival and complementary routesRival and complementary routes Civic engagementCivic engagement

Newcastle in UKNewcastle in UK EmployabilityEmployability

LJMULJMU TeachingTeaching

WilliamsWilliamsFor ProfitFor Profit

eg UEM and other Laureate eg UEM and other Laureate universitiesuniversities

ResearchResearchNon-universities, PerimeterNon-universities, Perimeter


University missionsUniversity missions

Hierarchies exist because of market Hierarchies exist because of market forcesforces

Systems without competition don’t Systems without competition don’t need themneed them

Students, research money etcStudents, research money etc This is true of higher education and This is true of higher education and

other aspects of societyother aspects of society East Asia cf continental EuropeEast Asia cf continental Europe


Possible future issues for RankingPossible future issues for Ranking

Distance/presence learningDistance/presence learning Dispersed universities eg KentDispersed universities eg Kent Engaged on local, regional or world Engaged on local, regional or world

scalescale Producing new professionals?Producing new professionals? Or new knowledge?Or new knowledge? Profit-making or not?Profit-making or not?


In soft power world…In soft power world…

Media studiesMedia studies Links to developing worldLinks to developing world SportSport Teaching – the original university Teaching – the original university

subjectsubject


Performance ManagementPerformance Management

Ups and downsUps and downs Is this university in the company Is this university in the company

you’d expect?you’d expect? Eg US state universitiesEg US state universities National-level representationNational-level representation Can we go beyond this?Can we go beyond this?


• Can be applied to any institutionCan be applied to any institution

• Teaching or business focused institutions can Teaching or business focused institutions can perform wellperform well

• Performance measured against preset thresholdsPerformance measured against preset thresholds

● ● Recognise contextual differences such as Recognise contextual differences such as languagelanguage

• Recognise strengths of specialist institutionsRecognise strengths of specialist institutions• Ratings can measure specific university missionsRatings can measure specific university missions

QS Star RatingsQS Star Ratings

5959


QS Stars rating systemQS Stars rating system

6060


6161

System StrengthSystem Strength

RankRank CountryCountry SystemSystem AccessAccess FlagshipFlagship EconomicEconomic OverallOverall

11 United StatesUnited States 100100 100100 100100 100100 10010022 United KingdomUnited Kingdom 9999 9696 100100 9898 989833 GermanyGermany 9595 9696 9494 9292 959544 AustraliaAustralia 9393 9898 9898 8787 949455 CanadaCanada 9191 9696 9898 8383 929266 JapanJapan 9292 7979 9898 8989 909077 FranceFrance 8787 7979 9797 8383 878788 NetherlandsNetherlands 9191 8383 9595 7575 868699 Korea, SouthKorea, South 7676 7373 9595 7474 8080

1010 SwitzerlandSwitzerland 8383 7171 9898 6262 79791111 SwedenSweden 7575 8383 9393 6262 78781212 BelgiumBelgium 7474 7777 9393 5858 76761313 New ZealandNew Zealand 6666 8282 9494 6060 75751414 ItalyItaly 7070 9494 7676 5959 75751515 ChinaChina 7676 2525 9595 9898 74741616 Hong KongHong Kong 7878 6262 9898 5151 72721717 IrelandIreland 6767 7979 9696 4747 72721818 IndiaIndia 6767 3939 7878 100100 71711919 SpainSpain 6262 8181 7777 4949 67672020 FinlandFinland 5959 7979 8787 4242 6767


The futureThe future

Likely to get betterLikely to get better More subject dataMore subject data More purposes: companies as well as More purposes: companies as well as

governments, students, university governments, students, university managersmanagers

University rankings and reputation University rankings and reputation practice: [email protected]: [email protected]


Really the last slideReally the last slide

Thank you to..Thank you to..

Nunzio Quacquarelli, Ben Sowter and Nunzio Quacquarelli, Ben Sowter and colleagues at QScolleagues at QS

Staff of Scopus Staff of Scopus You for this invitation and for your You for this invitation and for your

attentionattention