Defining new metrics for library success

  • View

  • Download

Embed Size (px)



Text of Defining new metrics for library success

  • 1. Mindy Thuna Science Liaison Pam King Scholarly Communication Liaison
  • 2. This work is licensed under: You are free to: Share copy and redistribute the material in any medium or format Adapt remix, transform, and build upon the material Under the following terms: Attribution You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. NonCommercial You may not use the material for commercial purposes. ShareAlike If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.
  • 3. Altmetrics in Higher Ed Settings metrics in context - What our researchers are saying
  • 4. Agenda Impact Metrics Altmetrics What our researchers have to say Bringing it back to libraries
  • 6. Impact Metrics are calculations/algorithms that quantify the impact of research or scholarly activity.
  • 7. Research impact defined Impact is usually demonstrated by pointing to a record of the active consultation, consideration, citation, discussion, referencing or use of a piece of research.
  • 8. Basic Premise Number of citations Impact Reach and significance
  • 9. Three main providers Scopus Web of Science Google Scholar
  • 10. Three Levels Journal Level Article Level Author Level
  • 11. Journal Level Metrics Web of Science Impact Factor (2 & 5 year) Cited half- life Eigenfactor Score Article Influence Score Scopus SCImago Journal Rank (SJR) Source Normalized Impact per Paper (SNIP) Google Scholar h5-index h5-median
  • 12. Journal Impact Factor Average number of times articles published in a two year (or five year) period have been cited A = total number of times ALL articles published in 2 (or 5) year period were cited in WofS indexed journals during the next year B = total number of "citable items" (usually articles, reviews, proceedings or notes; not editorials and letters-to-the-editor) published in 2 (or 5) year period Impact factor = A/B
  • 13. Journal Impact Factor Debate Vary across disciplines e.g., variants in time-to-publication Review journals have higher impact fewer articles per journal, cited more Calculations easily skewed heavy editorial/letter/opinion content high/low percentage of review articles
  • 14. Declared Dissent 10,419 Impact factors declared unfit for duty Stephen Curry, LSE Blog Do not resuscitate: the journal impact factor declared dead Brendan Crabb, The conversation Just say no to impact factors Ismael Rafols and James Wilsdon, the guardian DORA: San Francisco Declaration on Research Assessment (Dec. 2012)
  • 15. Author Level Metrics Web of Science h-index Scopus h-index Google Scholar h-index i10-index
  • 16. h index is NOT perfect Paper 1 Paper 2 Paper 3 Paper 4 Paper 5 Author A 3 cites 6 cites 100 cites 4 cites 1 cite Author B 3 cites 6 cites 100 cites Author C 400 cites 150 cites 3 cites 6 cites h index = 3
  • 17. Article Level Metrics Citation Counts - how often a particular journal article is being cited by other articles
  • 19. Web metrics Webometrics - link analysis, web log file analysis, 1990s Bibliometric use downloads, citations Google web page ranking Web 2.0 ->scientometrics 2.0 (Priem & Hemminger 2010)
  • 20. Open Access Producing a highly complex information environment; touches all disciplines Many formats, e.g. websites, nanopubs, blog posts, code, images versions, e.g. preprints, postprints, same article different places, version of record
  • 21. Identification Systems Authors International Standard Name Identifier (ISNI) ORCID (subset of ISNI) Names Project (JISC) Author Claim, RePEc Author Service arXiv Author ID ResearchID (Thomson Reuters) Scopus Author ID (Elsevier) Gravatars (Auttomatic) Objects DOIs, PMIDs URLs, URIs
  • 22. Altmetrics are calculated from newer data sources, e.g. Wikipedia, Mendeley, Twitter, Facebook, Weibo, blogs report the impact of a wider range of research outputs, e.g. presentation slides, data sets, articles, code
  • 23. Altmetrics aggregators/tools/ services PLoS One Article Level Metrics (ALM) ImpactStory Altmetric PlumX and others
  • 24. PLoS Article Level Metrics (ALMs)
  • 25. ImpactStory
  • 26. 30
  • 27. Altmetric (
  • 28. PlumX
  • 29. The Debate Article-Level Metrics: An Ill-Conceived and Meretricious Idea vs Broaden your horizons: impact doesnt need to be all about academic citations (Jeffrey Beall vs Euan Adie, Aug. 2013)
  • 30. Moving forward to move out of its current pilot and proof-of-concept phase Sloan foundation grant for NISO initiative - creation of community- based standards; definitions, calculations. Data classifications and data sharing practices
  • 32. Our Ongoing Study Research Impact Metrics: Use and Understanding by Post- Tenure Faculty
  • 33. What do you think of when you hear the term impact metrics? I don't think that impact factors at the journal level necessarily captures the importance of the paper... I jokingly say that out of all the things I have published, the paper that I expect will have the longest life is the one that's got the lowest impact factor. Biology
  • 34. What do you think of when you hear the term impact metrics? Nothing I dont understand what your talking about. No clue. I understand impact I understand those words separately. But not together. Mathematical and Computational Sciences Means nothing to me zero. Political Science
  • 35. What do you think of when you hear the term impact metrics? I have a picture of that book that you find at the Robarts library on the 4th floor that is a collection of binders in colourful colours and when you open it, it looks like the phone book with all the list of all the articles and researchers so that is my idea of impact metrics. Language Studies
  • 36. Have you considered/do you use any alternative metrics? No. No trust in them. Mathematical and Computational Sciences
  • 37. I give a lot of relevance to those metrics and I will tell you whyThere was an event in the year 1900. It was some popular festivity in England and they were running a contest. There was a cow or a bull and people had to guess it's weight and the person who gets closes to the weight wins the cow, or something like that. And someone kept a record of the answers and it is very interesting. It was very wide-ranging so let's say the weight of the cow was 500 kg. So the range was from 300-1000. The most accurate answer was the aver