Can we use
altmetrics at the institutional level?
A case study analysing the coverage
by research areas of four Spanish universities
Can we use altmetrics at the institutional level? A case study analysing the
coverage by research areas of four Spanish
universities
Daniel Torres-Salinas*, Nicolás Robinson-García** and Evaristo Jiménez-Contreras***
•
* Univ. Navarra and Univ. Granada, EC3metrics & Medialab UGR - @torressalinas ** INGENIO (CSIC-UPV), Universitat Politècnica de València - @nrobinsongarcia
*** Universidad de Granada, Grupo EC3 and EC3metrics
To be published in: Proceedings of the 21st International Conference on Science and Technology Indicators València (Spain) | September 14-16, 2016
Full proceeding paper avaible at arxiv: https://arxiv.org/abs/1606.00232
Can we use altmetrics at institucional level?
Introduction
Can we use altmetrics at institucional level?
Introduction: altmetric studies
Social media based indicators or altmetrics have been under scrutiny for the last seven years.
1.- The relation between citations and altmetric indicators
2.- Coverage and diversity of social media sources
3.- Their meaning as impact indicators
Three lines of work can be observed
Can we use altmetrics at institucional level?
Introduction: a gap in altmetric studies
1. Most studies are focus on the understanding of the nature and relation of altmetric indicators with citation data
No study has been found profiling larger units of research such as institutions, especially universities
2.- Most papers are analysing altmetric indicators at paper (paper level metrics) or author level (author level metrics)
.
Can we use altmetrics at institucional level?
Introduction: information sources
On the other hand, many data providers offering social media metrics have been developed
Plum analytics
Impact Story
Altmetric.com
Can we use altmetrics at institucional level?
Introduction: altmetric.com
Altmetric.com has positioned itself as the most used source for altmetric studies
1.- Variety of social media platforms with regard to mentions, readership, etc. of scientific papers
2.- Offers an aggregated indicator or ‘altmetric’ score which is a weighted sum of values based of a given article in different social media
3.- Maintaining the ‘history’ of each altmetric indicator to an article
Advantages of altmetric.com
Objetive 1: Analyse the coverage of Altmetric.com at the institutional and field level and verify its validity as evaluation tool
Objetive 2: Analyse differences when obtaining subject bibliometric profiles from Altmetric.co and explore the possibility to make benchmarking studies at university level
Can we use altmetrics at institucional level?
Objetives: altmetrics and universities
Our objectives are:
Can we use altmetrics at institucional level?
Material & methods
Can we use altmetrics at institucional level?
Material & methods: 4 universities
y Size Staff
Type and main field
Foundation Year
Region & City
University of Granada
2399 Multidisciplinary 1531 Granada
University Pompeu Fabra
288 Specialized
Medicine/Biology 1990
Barcelona
Polytechnic University Valencia
1847 Specialized Engineering
1971 Valencia
University Carlos III
539 Specialized
Social Sciences 1989 Madrid
1.- We will analyze 4 different Spanish universities 2.- Present different sizes and research publication specialization 3.- Different Foundation year and located in different regions
Selection of spanish universities
Can we use altmetrics at institucional level?
Material & methods: Web of Science
1.- Citable papers for the 2014 year indexed in the Web of Science for the 4 universities were retrieved (5922 records)
2.- Records from the Web of Science provide among other information the DOI number of each publication
3.- The DOI number is very important, as it will allow us to query the Altmetric.com API for each document
4.- A total of 93% of the total share from our data set included DOI
Search for Web of Science papers and check DOI coverage
5.- Publications have been assigned to research areas
Can we use altmetrics at institucional level?
Material & methods: Web of Science
1.- We retrieved altmetric data available for our set of papers with DOI.
2.- Then, the share of records with a score assigned by Altmetric.com has been calculated
3.- Altmetric.com covered 2149 records from the Web of Science, representing 36% of our data set
Search for altmetric.com score using DOI
Can we use altmetrics at institutional level?
Results
Can we use altmetrics at institucional level?
Results: altmetric.com coverage
Documents retrieved 4 universities
5922 93% with DOI
5922 web of Science Documents
NOT Indexed in altmetric.com 3773 - 74%
Indexed in altmetric.com 2149 -36%
Indexed = at least with a value of 1 in altmetric score
Can we use altmetrics at institucional level?
Results: altmetric.com coverage
NOT Indexed in altmetric.com 3773 - 74%
There are significance difference in the % of Web of Science documents indexed in altmetric.com
Can we use altmetrics at institucional level?
Results: altmetric.com coverage Altmetric.com score are very concentrate just in
Twitter and Mendeley Readers
Can we use altmetrics at institucional level?
Results: altmetric.com coverage We can compare the “social media” patron for the four universities
Can we use altmetrics at institucional level?
Results: altmetric.com coverage
Important differences by university
Can we use altmetrics at institucional level?
Results: Research field profile
Gra
nad
a P
om
pe
u F
abra
P. V
ale
nci
a C
arlo
s II
I We have obtained the coverage profile (Web of science papers) of altmetric.com per research fields: • Significant differences in
research field coverage (ej: Pompeu Fabra Vs Carlos III)
• Best coverage in all fields in universities with S&T profile
(ej: P. Valencia & Pompeu Fabra) • Dramatic coverage of
Humanities (ej: Granada) and quite well in Social Science
Can we use altmetrics at institucional level?
Results: Research field profile
Tota
l alt
me
tric
.co
m S
core
Nr of Web of Science Papers indexed in Altmetric.com
Map of Web of Science Categories
We can benchmark universities and compare altmetric performance at category level
Multidisciplinary Sciences Pompeu Fabra
Biochemistry Pompeu Fabra Biochemistry
Granada
Multidisciplinary Sciences Granada
Can we use altmetrics at institucional level?
Results: Research field profile – top10
Multidisciplinary Science (Science, Nature, Plos one…) have higher coverage and concentration of altmetric.com score. In the case of Pompeu Fabra 60% of altmetric.com score are related to papers published in multidisciplinary journals.
We can see that social sciences and humanites are not well represented in top categories
Can we use altmetrics at institucional level?
Results: Research field profile – top10
Top papers according Almetric.com Scores and journals where have been published
We can observed the “Nature” and
“Science” effect that determine altmetrics in these case studies
Final results remarks
Can we use altmetrics at institucional level?
Concluding remarks
Can we use altmetrics at institutional level?
Concluding remarks
Can we use altmetrics at institutional level?
YES √ We can make a local bibliometric database combining bibliometric data
from Web of Science and altmetric data from altmetric.com
√ We can calculate altmetric indicators at university level and offer an
altmetric research field profile in different levels (fields and categories)
√ We can compare and make universities benchmarking studies
Can we use altmetrics at institutional level?
Concluding remarks
We observed a low coverage of altmetric indicators with only 36% of all documents have an ‘altmetric’ score
To take into account: Low coverage of web of science papers:
The DOI´s problem: One of the main factors to such low coverage could be the use of DOI in altmetric.com
not all papers indexed in the Web of Science include DOI not all people that mention a paper in social media use the DOI
So using DOI we have a loss of information
Can we use altmetrics at institutional level?
Concluding remarks To take into account:
Multidisciplinary Journals concentrate altmetrics
The Nature and Science effect
We observed that “multidisciplinary sciences” accumulate a big piece of the total altmetric score at university level. So universities publishing in nature, science, pnas, etc.… also have high levels of mentions in social media.
Can we use altmetrics at institutional level?
Concluding remarks To take into account:
Social science & humanities: low coverage and impact
The national factor and language bias. It is necessary to verify if there is a national or linguistic factor influencing the coverage of altmetric indicators in the areas of Socials Sciences and Humanities & Arts.
Another explanation could be that these areas are represented in social media by small communities, limiting their capacity to
disseminate their research papers in social media.
Can we use
altmetrics at the institutional level?
Full proceeding paper avaible at arxiv:
https://arxiv.org/abs/1606.00232