The weight of research internationalization indicators

Ülle MustEstonian Research

Councilulle.must@etag.ee

SCIENTOMETRICS Status and Prospects for developmentMoscow, 10-12 October 2013

Visualizing Friendships

By Paul Butler, 13 December 2010http://www.facebook.com/notes/facebook-engineering/visualizing-friendships/469716398919

Changing publication and collaboration patternPublished documents 1996-2012 (in thousands)

Research collaboration: % of documents with more than one country

19961997

19981999

20002001

20022003

20042005

20062007

20082009

20102011

20120

100

200

300

400

500

600

700

800

900

Western Europe Eastern EuropeNorthern America Asiatic Region

19961997

19981999

20002001

20022003

20042005

20062007

20082009

20102011

20120

5

10

15

20

25

30

35

40

Western Europe Eastern EuropeNorthern America Asiatic Region

SCImago. (2007). SJR — SCImago Journal & Country Rank.Retrieved October 09, 2013, from http://www.scimagojr.com

King, C. Multiauthor Papers: Onward and Upward. Retrieved July 1, 2013 from: http://sciencewatch.com/articles/multiauthor-papers-onward-and-upward

Data and methods

Peer Review survey– General information (field, country of affiliation, citizenship, gender, the profile

as researcher);– Experience in the Peer Review process (as applicant or reviewer);– S&T indicators;– Criticism of Peer Review;– The future of the peer review system.

• Respondents belong to 79 countries by nationality; their places of affiliation were in 66 countries. Among respondents, 1337 were men and 652 were women, 125 did not specify their gender. By academic position, postdoctoral research fellows (643) and full professors (422) constituted the largest share of respondents, followed by lecturers/assistant professors (325), associated professors (323). The largest numbers of respondents belong to natural sciences (1096).

S&T indicators (Un-ty Based Res, 2010)

5.1.1. Input indicatorsExternal fundingRecruitment of PhD students 5.1.2. Process indicators Seminar and conference activityInvited keynotesInternational visiting research appointments 5.1.3. Structure indicators Number of PhD studentsResearch collaborations and partnership

Reputation and esteem (position as journal editors, membership of editorial boards and scientific committees and membership in learned academies)

5.1.4. Output indicators PublicationsNon-bibliographical outputs

Number of PhD graduates and completion rates for graduates

5.1.5. Effect indicators CitationsH index

Number of prestigious national and international awards and prizes

External fundingEmployability of PhD graduates (in private sector) 5.1.6. Social indicators Relevance to citizens' concernsRelevance to global societal challengesUsefulness to policy decision makersRelevance to science communication initiativesContributing to science education

Patent developmentPublic outreach (broader dissemination, transfer and exchange of research results)

The rating of the S&T indicators by country groups

External funding

Invited keynotes

International visiting research appointments

Research collaborations and partnership

Number of prestigious national and international awards and prizes

Relevance to global societal challenges

3.3

3.5

3.28

3.75

3.64

3.18

3.49

3.38

3.39

3.85

3.45

3.21

3.46

3.2

3.08

3.8

2.64

3.8

3.67

3.46

3.62

4.24

3.82

3.65

MS-OLD MS-NEW LAT/CARIB EECA

The rating of the S&T indicators by fields

Relevance to global societal challenges

International visiting research appointments

External funding

Invited keynotes

Number of prestigious national and international awards and prizes

Research collaborations and partnership

3.7

3.4

3.7

3.7

3.7

4.1

2.9

3.6

3.2

3.3

3.4

3.7

3.6

3.2

3.6

3.4

3.4

3.8

3.6

3.3

3.4

3.3

3.6

3.7

3.3

3.4

3.5

3.6

3.7

4.0

3.0

3.2

3.5

3.5

3.7

3.7

Agricultural sciences Humanities Medical Sciences Social Sciences Engineering and Technology Natural Sciences

The rating of the S&T indicators by career

Relevance to global societal challenges

International visiting research appointments

External funding

Invited keynotes

Research collaborations and partnership

Number of prestigious national and international awards and prizes

4.4

3.2

3.1

3.5

3.9

3.1

3.3

3.4

3.7

3.5

3.9

3.7

3.3

3.2

3.5

3.4

3.7

3.6

3.0

3.3

3.4

3.5

3.6

3.6

2.8

3.1

3.2

3.4

3.5

3.6

Student Postdoctoral research fellow Lecturer / Assistant professor Associate professor / Reader / Senior lecturer Full professor

The rating of the S&T indicators by gender

Relevance to global societal challenges

International visiting research appointments

External funding

Invited keynotes

Number of prestigious national and international awards and prizes

Research collaborations and partnership

3.1

3.3

3.4

3.5

3.6

3.7

3.2

3.3

3.5

3.5

3.6

3.8

Male Female

Availability of data• Validity and reliability– Fit to goal– Stability and controllability

• Economic efficiency– Availability and the cost of processing the raw data

• The use quality– The uniqueness and clarity of interpretation – The ability to influence the behavior and process

Indicators validity by criteriaIndicators/Criteria Validity and

reliabilityEconomic efficiency

Exploitation quality

Research collaborations and partnership + + +

Number of prestigious national and international awards and prizes

+/- - -

Relevance to global societal challenges +/- - -

Invited keynotes +/- - -International visiting research appointments + + +

External Funding + + +

Conclusions• The choice of indicators depend on the desired goal;• The range of indicators is broad, and the choice of

individual indicators should meet certain criteria;• Various indicators should be weighted differently in

different fields, at different career stages;• Indicators have to be fit for purpose, appropriate ‐ ‐

and verifiable.