October 24, 2005ISPM academic staff meeting1 Research Assessment Exercise Medical Faculty, University of Bern Matthias Egger

October 24, 2005 ISPM academic staff meeting 1

Research Assessment Exercise

Medical Faculty, University of Bern

Matthias Egger


Menu

• Research Assessment Exercises (RAE)

• Important bibliometric indices– The Impact Factor (IF)– The Maximum Rank (MR)

• Methodology of Bern RAE

• ISPM publications 2002-2004

• Results of 2003 RAE

• Conclusions


Rationale for RAE

• Improve quality of research

• Foster accountability and transparency

• Promote strategic thinking and focus

• Guide resource allocation– UK and other countries– Basel– Bern?


Bibliometric indices


Journal impact factor

• The number of citations in one year (for example 2002) to items published in a specific journal in the two preceding years (2000 and 2001) divided by the total number of citable articles (original articles and reviews, but not other items) published in this two-year period.

• Journal specific measure


Journal citation reports (JCR)

• There are two editions– Science JCR– Social Science JCR

• Journals are grouped in subject categories– Typical impact factors vary between categories

• The same journal may appear in more than one subject category

• Updated every year


Subject categories: examples

• Social Science– PUBLIC, ENVIRONMENTAL & OCCUPATIONAL

HEALTH

– HEALTH POLICY & SERVICES

– SOCIAL SCIENCES, BIOMEDICAL

– PSYCHOLOGY, APPLIED

– PSYCHOLOGY, SOCIAL

– SOCIOLOGY

– WOMEN'S STUDIES


Maximum Rank (MR)• Measure of the standing of a journal within a

given field (subject category)

• Controls for differences in impact factors across subject categories

• Defined as: IF-Rank of Journal -1Rank in Subject Category = 1 - ------------------------------ Number of Journals

MR = Maximum of all ranks, if the journal is listed in several subject cateogories


Top Journals (MR=1)

• New England Journal of Medicine (IF=38.57)

• Value in Health (IF=3.66)

• Womens Health Issues (IF=1.352)


MR: Examples

• British Medical Journal

• Social Science & Medicine

• Statistics in Medicine

• J Adolescent Health

• Sex Transm Inf

• http://wok.consortium.ch/


Social Science & MedicineIF=2.088 (2004)

• PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (Social Science Edition)5/60 MR=0.933

• SOCIAL SCIENCES, BIOMEDICAL2/26 MR=0.961


Statistics in MedicineIF=1.389 (2004)

• PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH46/93 MR=0.516

• MEDICAL INFORMATICS6/18 MR=0.720

• STATISTICS & PROBABILITY13/77 MR=0.844


J Adolescent HealthIF=1.571 (2004)

• PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH(Science Edition)46/93 MR=0.580

• PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH(Social Science Edition)12/60 MR=0.817


Sex Transm InfIF=1.181 (2002)IF=2.204 (2004)

• 2002DERMATOLOGY & VENEREAL DISEASES7/40 MR=0.850

• 2004INFECTIOUS DISEASES19/41 MR=0.561


RAE 2003: Methods


Bern RAE 2003

• Separate evaluation of two groups– Clinics– Institutes (n=21)

• Three ranks based on– Number of all original articles / Stellenpunkt– Average MR of in-house original articles– Competitive grant income spent / Stellenpunkt

• Global ranking based on sum of three ranks


Definitions

• Original articles– Pubmed-indexed original articles– www.pubmed.org

• Original articles in-house – first and/or last author from ISPM

• Original articles collaborations

• Other publications

http://www.pubmed.org/


• Institut für Infektionskrankheiten • Institut für Pathologie • Institut für Klinische Pharmakologie • Institut für Rechtsmedizin • Institut für Sozial- und Präventivmedizin • Fakultäre Instanz für Hausarztmedizin • Universitätsspital-Bibliothek• Institut für Medizinische Lehre • Kollegiale Instanz für Komplementärmedizin• Departement Klinische Forschung • Institut für Anatomie • Institut für Physiologie• Institut für Biochemie und Molekularbiologie • Theodor-Kocher-Institut • Institut für Medizingeschichte • Institut für Pathophysiologie • Institut für Pharmakologie • Institut für chirurgische Technologien und Biomechanik• Institut für Evaluative Forschung in orthopädischer Chirurgie• Institut für Dental- und Skelettbiologie • Zahnmedizinische Kliniken

21 I

nsti

tute

s


ISPM publications 2002-2004

• 188 publications listed on website

• 100 original articles indexed in Pubmed

• 92 articles with impact factor

• 57 in-house articles, 52 with impact factor

• 56 journals

• 28 subject categories


Sex Transm Inf 7 Lancet 6Soz Praeventivmed 6BMJ 5Int J Epidemiol 5Clin Oral Implants Res 3Eur J Public Health 3Eur Respir J 3Int J STD & AIDS 3JAMA 3Swiss Med Wkly 3AIDS 2Eur J Clin Nutr 2Eur Spine J 2Gesundheitswesen 2J Clin Epidemiol 2J Infect Dis 2Rheumatology 2

Stat Med 2AIDS Res Human Retrovir 1Acta Paediatr 1Age Ageinig 1Aids Care 1Alcohol 1Am J Epidemiol 1Am J Public Health 1Am J Respir Crit Care Med 1Anesthesiology 1Arch Int Med 1BMC Med Res Methodol 1Clin Chim Acta 1Clin Sci 1Clin Ther 1 Contr Clin Trials 1 Diagnostica 1 Epidemiology 1

Jour

nals


Eur J Cancer 1 Eur J Epidemiol 1 Familial Cancer 1 HIV Med 1 Health Psychol 1 Health Technol Assess 1 Heart 1 Hepatology 1 Human and Ecol Risk Assess 1 Int J Hyg Environ Health 1 Int J Techno Assess Health Care 1 J Clin Endocrin Metabol 1 J Epidemiol Community Health 1 J Occup Environm Med 1 J Public Health 1 P Natl Acad Sci USA 1 Pest Manag Sci 1 Resp Res 1 Scan J Work Environ Health 1 Schweiz Monatsschr Zahnmed 1

Jour

nals


In-house publications in journals indexed in Social Science Edition

• Eur J Public Health (AGF 2, EpStat 1)

• Gesundheitswesen (Epstat 1)


PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH 25 MEDICINE, GENERAL & INTERNAL 18INFECTIOUS DISEASES 10RESPIRATORY SYSTEM 4DENTISTRY, ORAL SURGERY & MEDICINE 3DERMATOLOGY & VENEREAL DISEASES 3NUTRITION & DIETETICS 2ORTHOPEDICS 2PHARMACOLOGY & PHARMACY 2PSYCHOLOGY, CLINICAL 2RHEUMATOLOGY 2STATISTICS & PROBABILITY 2VIROLOGY 2AGRONOMY 1ANESTHESIOLOGY 1CARDIAC & CARDIOVASCULAR SYSTEMS 1CRITICAL CARE MEDICINE 1ENDOCRINOLOGY & METABOLISM 1

Sub

ject

cat

egor

ies


ENVIRONMENTAL SCIENCES 1GASTROENTEROLOGY & HEPATOLOGY 1GERIATRICS & GERONTOLOGY 1HEALTH CARE SCIENCES & SERVICES 1MEDICAL LABORATORY TECHNOLOGY 1MEDICINE, RESEARCH & EXPERIMENTAL 1MULTIDISCIPLINARY SCIENCES 1ONCOLOGY 1PEDIATRICS 1PSYCHOLOGY, CLINICAL 1TOXICOLOGY 1

Sub

ject

cat

egor

ies


010

2030

Fre

quen

cy

0 5 10 15 20 25impactfactor


05

1015

Fre

quen

cy

.2 .4 .6 .8 1maxrank


2002 2003 2004 2002-04

AGF 4 8 5 17

EpStat 21 23 39 83

ISPM 25 31 44 100

Number of original articles


2002 2003 2004 2002-04

AGF 0.147(n=1)

0.392(n=5)

0.647(n=1)

0.393(n=7)

EpStat 0.842(n=11)

0.681(n=13)

0.705(n=21)

0.731(n=45)

ISPM 0.784(n=12)

0.600(n=18)

0.702(n=22)

0.695(n=52)

Average MR in-house original articles


RAE 2003: Results


RAE 2003 ISPM All 21 units

Staff points (Stellenpunkte, SP)

1694 1722

No of original publications

Number / SP

31

0.0183

19.7

0.0114

MR of inhouse publications 0.684 0.819

Competitive grant income / year / SP 762 1366

4

14

3


Core funding of ISPM

• 1694 staff points, corresponding to ~2.1 Mio SFr/year

• 125,000 SFr / year for running costs

• Rent of offices, maintenance


Current distribution of staff points

By administrative unit• Director’s office 250 (15%)• Central services 271 (16%)• AGF 559 (33%)• EpStat 614 (36%)

By research unit• AGF 559 (41%)• Epstat+ME 806 (59%)


Conclusions• We are doing very well in terms of the number

of original articles

• We are doing less well in terms of the Maximum Rank of our in-house publications

• Our output is very heterogeneous– Lack of focus?– Reflection of consultancies with University

Hospital and others?


Conclusions• EpStat contributes substantially more RAE-

relevant output than AGF• There are very few in-house publications in

journals indexed in Social Science Edition• There are many journals that rank highly in the

Social Science Edition and that may be of interest to us

• We need to make sure that the faculty appropriately considers Social Science journals


High ranking journals indexed in Social Science Edition

Addiction (1.0) Soc Health Illness (0.922)Aids Care (0.846) Social Sci Med (0.961)Am J Health Prom (0.900) Sociol Health Ill (0.922)Archives Sex Behav (0.892) Women Health Issues (1.0)Demography (0.941)Health Psychology (0.928)Health Services Research (0.842)J Adolescent Health (0.817)J Health Soc Behav (0.782)J Soc Policy (0.928)Medical Care (0.921)Perspect Sex Repro Health (1.0)Psychosom Med (0.930)


Conclusions• We should generally try to publish our work in

journals that rank high in relevant subject categories (MR>0.7)

• Everyone at ISPM should know these journals for her/his field

• We may want to exclude original publications with very low MRs from the list we submit to the faculty

• We should use our core funding (staff points) wisely, taking into account the RAE (every staff point counts)

Documents

October 24, 2005ISPM academic staff meeting1 Research Assessment Exercise Medical Faculty, University of Bern Matthias Egger