Tanya Wil l iamson, Assistant Librarian

BANG THOSE ROBOTS’ HEADS TOGETHER!

AN INTRODUCTION TO CITATION ANALYSIS

Image credit: Io robot by i k o: https://flic.kr/p/eC4kKX

The aim originally was to reveal important discoveries and allow researchers to navigate the ‘citation network’ to uncover relevant literature

WHAT IS CITATION ANALYSIS FOR?

‘‘Do results of research really become valueless by remaining buried in the

literature?’ If there is anything but a YES answer to this question then perhaps we

should burn our libraries and the literature in them.’

EUGENE GARFIELD

CHEMICAL AND ENGINEERING NEWS 30(50) , 5232 (1952)  HTTP: / /

GARFIELD.L IBRARY.UPENN.EDU/PAPERS/CHEMICALANDENGINEERINGNEWS30%2850%29Y1952.HTML

Since 1975 citation data is being used to: rank journal titles to understand how science works

And more recently measure impact and influence of a work evaluate an author or institution’s influence as a filter (most worrying!)

WHAT IS CITATION ANALYSIS FOR?

Citations are used to calculate ‘Research Influence’ which equals 30% of the overall score in the World University Rankings.

EVALUATE AN INSTITUTION’S INFLUENCE

EVALUATE AN INSTITUTION’S INFLUENCE

Some Panels use citation data and ‘…consider the number of times an output has been cited as additional information about the academic significance of submitted outputs.’

REF Panel criteria and working methods (2012)

First, stop and think! Why do you want to do this? Are citations the best measure? What other information are you going to use?

Are you interested in citations in relation to: A journal title? A particular work e.g. a journal article? An author, or group of authors? A subject category? An institution?

HOW DO I ANALYSE CITATIONS?

Covers approx. 12,000 selected, peer reviewed journal titles. Mostly English language

Citation data from 1945Source data for THE World University Rankings *changing to

Scopus

Source data for the annual Journal Citation Report , and the Journal Impact Factor

Useful for :Tracking citations over timeFollowing citations forward and backward in the

literatureCitation reports for any set of resultse.g. author, topic, article

WEB OF SCIENCE

Reports produced annually based on the data from Web of Science

Subject Categories to enable comparison within a discipline

Many metrics, including the Journal Impact Factor, Eigenfactor

Useful for:Ranking journal titles in subject

categoriesComparing journals within subject

categories

JOURNAL CITATION REPORTS

Covers approx. 22,000 titlesCitation data from 1996Source data for the REFMany metrics including SJR, SNIP, IPP

Useful for:Analysing an author’s citationsVisual citation reports for any set of results e.g. topic,

author, articleDiscovering trends

SCOPUS

Name ambiguity – getting a comprehensive list is not easy (but getting easier)

No one metric can capture all citations or ‘impacts’Early career researchers will be at a disadvantageCitations ≠ endorsements of quality!Each resource will give you diff erent results

WHAT’S WRONG WITH CITATION ANALYSIS?

Source of data Citations

Web of Science 2137

Scopus 2418

Google Scholar 3072

Publisher’s website

1538

Martinez, F. D., Wright, A. L., Taussig, L. et al. (1995). Asthma and wheezing in the first six years of life. New England Journal of Medicine, 332(3), 133-138.

‘no set of numbers is likely to be able to capture the nuanced judgments that the REF process currently provides’

Metrics cannot replace peer review in the next REF, 8 t h July 2015, HEFCE

‘Metrics have proliferated: usually well intentioned, not always well informed, often il l applied. We risk damaging the system with the very tools designed to improve it, as evaluation is increasingly implemented by organizations without knowledge of, or advice on, good practice and interpretation.’

Bibliometrics: The Leiden Manifesto for research metrics, 22 n d Apri l 2015, Hicks et al.

WHAT’S WRONG WITH CITATION ANALYSIS?

“…both admin i s t ra to rs and the management d i sc ip l i ne w i l l be we l l se rved by eff or t s to eva lua te each a r t i c le on i t s own mer i t s ra ther than abd ica te

th i s respons ib i l i t y by us ing j ourna l rank ing as a p roxy fo r qua l i t y .

S i n g h , H a d d a d & C h o w ( 2 0 0 7 : 3 1 9 )

Publish or PerishGoogle ScholarAltmetricsResearch Evaluation tools such as SciVal or InCites

OF FURTHER INTEREST

Harzing, A. W. (2010). The publish or perish book . Melbourne: Tarma Software Research. http://www.harzing.com/popbook/toc.htm

Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). The Leiden Manifesto for research metrics. Nature, 520, 429-431. http://www.nature.com/news/bibliometrics-the-leiden-manifesto-for-research-metrics-1.17351

Singh, G., Haddad, K. M., & Chow, C. W. (2007). Are articles in “top” management journals necessarily of higher quality? Journal of Management Inquiry, 16(4): 319331.

REFERENCES