Introduction to Altmetrics for Medical and Special Librarians

Linda M. Galloway, MLISLibrarian for Biology, Chemistry and ForensicsBibliographer for the Sciences & Technology

Syracuse University Library, Syracuse, NY

NN/LM MAR Boost Box SeriesMarch 2014

About Me…• Librarian/subject specialist for Biology,

Chemistry & Forensic Science• Evaluate and select content , help people

access the content they need, teach, create web content, various librarian duties

• B.S. Chemistry, MSLIS from Syracuse University

• Bibliographer for Sciences & Technology • Email: galloway@syr.edu

What are Altmetrics??“the study of scholarly impact measures based on activity in online tools and environments”

(Priem, Groth, and Taraborelli 2012

citable and accessible products not limited to publications, data sets, software, patents, and copyrights (“Grant Proposal

Guide, Chapter II” 2013)

Traditional Scholarly Metrics

Quantifying Scholarly Output

via Citation Metrics

Number of PublicationsCitations to Publications

Relative influence of Publications

Traditional ToolsArticle Level Metrics

• Citations to an individual article– Web of Science– Scopus– Google Scholar

• h-index– measures both the productivity and impact of the

published work– Number of an author’s papers that have been cited at

least h times by other publications

Traditional Tools Journal Level Metrics

• Impact Factor – Journal Citation Reports– Avg. time articles from a journal (past 2 yrs.) are

cited in past year.– Web of Science indexed journals & data

• SCImago Journal & Country Rank– Based on Scopus Data, 1996-– Uses GooglePage Rank algorithim– Citable increments include past 3 years– Open Access

Note: there are other indices and measures available within these resources.

What data does a typical researcher/faculty member need?

# Citable products# Citations to those productsh-IndexOther measures of success and influence

Documents: 30Citations: 253H-index: 92009-2014, 3/5/2014

Scholarly Metrics as a proxy for Scholarly Influence…

Documents: 72Citations: 445H-index: 11Since 2009, 3/5/2014

Scholarly Metrics as a proxy for Scholarly Influence…

Limitations to Traditional Metrics

• Take a long time to accumulate• Often behind pay walls• Measure influence narrowly• Don’t capture a publication’s impact or

influence in emerging forms of scholarly communication

• Variability depending on database used to calculate metrics

Altmetrics

Measure diverse impacts from articles, datasets, blog posts, slide shows, etc.

Beyond citation counts!Readership

ViewsSaves

DownloadsScholarly (or popular) Buzz

What can be measured?

“Evidence of Use” – http://impactstory.org

• # of Tweets • # of “Saves” in online reference managers• Scholarly (and popular) blog interest and

activity• Activity in social networking platforms, tools• And…

Meaningful Interactions

CiteULikeDeliciousF1000GitHubMendeleySlideShareTwitter Zotero

What is tracked??

DiscussionsSavesCitationsRecommendationsDownloadsCopies

Altmetrics measures diverse impacts from articles, datasets, blog posts, slide shows, etc.

Altmetric Toolstrack readership & influence

Academia.edu is a platform for academics to share research papers and interests.

CiteULike permits users to store, organize and share scholarly papers

F1000 is a subscription-based recommendation service for curated articles in biology and medicine.

Social networking tools

Altmetric Toolstrack readership & influence

Google Scholar Citations is a service that allows authors to track their publications and influence using Google Scholar metrics. Mendeley is a free reference manager and social network that was recently acquired by Elsevier. Mendeley is described as “one of the world’s largest crowd-sourced research catalogs” Zotero is a robust and growing citation management and sharing resource. Collaborators can share libraries of references, etc.

Make Sense of the Diversity of Research Outputs

Use an aggregator!

Harvest dataAutomatic updates

Showcase scholarly influence

Tools to gather data

Commercial• Altmetric.com –owned by Macmillan Publishers

(also owns the Nature Publishing Group). “Provides article level metrics for researchers and publishers”

• Plum Analytics – startup co-founded by former Summon developers; recently acquired by EBSCO. Collects article-level data for use by different constituencies to compare individuals, departments, universities.

Tools to gather data

Commercial• Mendeley.com – Reference manager, .pdf organizer &

social networking tool for researchers/authors. Collects & displays altmetrics. Recently purchased by Elsevier.

“Mendeley Institutional Edition (MIE) is an analytics tool built on top of Mendeley that helps librarians, research directors and other admins to understand the research activity and scholarship output of their community and to facilitate collaboration within it (Mendeley.com).”

Tools to gather data

Non-profit

• ImpactStory – designed for the individual researcher, tools to visualize impact of research products. Helps “researchers to tell data-driven stories about their impacts” (ImpactStory, 2014).

Images: blog.impactstory.org, chemconnector.com

ImpactStory

Altmetric.com report linked from database citation

PlumXTM - Library Journal’s Most Ambitious Database of 2013

Engaging your users…

 Engaging Constituents

• Don’t assume anyone knows anything about altmetrics

• New (tenure track) scholars & clinicians• Explain limitations of both traditional citation

metrics & altmetrics• Demonstrate the power of a Google Scholar

Profile, institutional profile, and an ImpactStory Profile

Scholars’ Engagement with Social Media

• Important to maintain and manage an online presence

• Outreach to the public – broader impacts criteria – required by some funding agencies

• Mentions in social media seem to lead to enhanced use of publications

• Dizzying array of social media tools

Strategy for Scientific Social Networking

Goals

1. Choose one or two primary platforms– Institutional platform & Google Scholar

2. Accurate attribution of research products– ORCID and other identifiers

3. Keep profile(s) up to date4. Regularly monitor scientific social networks

The “best” social network depends on the discipline and individual preferences.

Platforms:To showcase and highlight research products

Institutional Profiling Service Google Scholar

Valid data = Valid metrics

• Accurate attribution is essential!• Scholarly authors are assigned Scopus Author

Identifiers, Web of Science Researcher ID’s, etc.

• Scholars can claim and make public their Google Scholar profile

• Scholars can (and should) register for a unique ORCID number – can use this identifier when publishing

ORCID

Open Researcher Identifier

Free service that assigns a unique number to each author and links other identification

schemes.

Encourage researchers to use consistent naming conventions and register for an ORCID ID!

Problem: author disambiguation

John F. Dannenhoffer IIISyracuse University

Joan V. DannenhofferSyracuse University

John F. Dannenhoffer IVPhD Candidate, University of Michigan

Joanne V. Dannenhoffer M.D. May 2013

Joanne M. DannenhofferCentral Michigan University

(spouses) (siblings)

(siblings)

Databases see all of these people as:

J DannenhofferJV DannenhofferJF DannenhofferJM Dannenhoffer

Keep profiles up-to-date

• Great deal of unpublished work can be harvested and promoted

• Immediate data can be provided• Recognize that open profiles are the first

impression of both you and your research

What’s new in Altmetrics?

“Altmetrics … give early estimates of the impact of publications or [to] give estimates of non-traditional types of

impact “ (Sud, 2014).

Published literature increasing

Query: altmetrics or citation metricsQuery: altmetrics OR “citation metrics” – in Title-Abstract-Keyword

Vendors capturing & displaying data

New literature

Altmetrics: A 21st-century solution to determining research quality

• Basic overview of altmetrics & citation metrics• Author provides types of altmetrics and their

correlation to citation counts (via published articles)

• An article’s DOI can be used on ImpactStory to discover article’s metrics.

Konkiel, S. (2013). Altmetrics: A 21st-century solution to determining research quality. Online Searcher, 37(4), 10-15.

Evaluating altmetrics

• Altmetrics are important b/c they give early estimates of an article’s impact.

• Citations from the social web may indicate value oriented more towards applications than pure scientific utility.

• Describes statistical methods to evaluate relationship between altmetrics & established tools.

Sud, P., & Thelwall, M. (2014). Evaluating altmetrics. Scientometrics, 98(2), 1131-1143.

Coverage and adoption of altmetrics sources in the bibliometric community

• Reviews current altmetrics literature• Examined used & coverage of social media in

bibliometricians. (quantitatively analyze academic literature).

• Makes the argument that “total readership” is important b/c it reflects pure (non-publishing) uses of publications – docs applied to daily work, support teaching, societal effects

Haustein, Stefanie, Isabella Peters, Judit Bar-Ilan, Jason Priem, Hadas Shema, and Jens Terliesner. "Coverage and adoption of altmetrics sources in the bibliometric community." arXiv preprint arXiv:1304.7300 (2013).

Do altmetrics correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective

• An extensive article!• Large study of 718,315 publications from Web

of Science with altmetric indicators provided by Altmetric.com (excluded Mendeley)

• Used Pearson’s correlation analysis to find connection between altmetrics & bibliometrics

Costas, R., Zahedi, Z., & Wouters, P. (2014). Do altmetrics correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. arXiv preprint arXiv:1401.4321.

Do altmetrics correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective

• Altmetric counts are low (15-24%)& not very frequent in scientific pubs, although presence is increasing

• Social sciences, humanities, and medical & life sciences had highest presence of altmetrics

• Found positive weak correlation between altmetrics & citations – reflecting that altmetrics do not capture the same concepts of impact

• Altmetrics are valued as a complementary tool of citation analysis

Altmetrics & ResearchersMetrics and their relationship to social media:

• Add value to traditionally published content– Crowdsourced peer review– Expose questions and comments– Enhance worth

• Increase readership• Appear to follow the pattern of traditional

metrics

Thank you!!Linda GallowayContributors: Janet PeaseAnne Rauh

Syracuse University Library

ReferencesAdie, Euan, and William Roe. 2013. “Altmetric: Enriching Scholarly Content with Article-level Discussion and Metrics.” Learned Publishing 26: 11–17. doi:10.1087/20130103.

Arslan, E., Akyokus, S., & Ganiz, M. C. (2013). An application of community discovery in academical social networks. In 2013 IEEE International Symposium on Innovations in Intelligent Systems and Applications (INISTA) (pp. 1–5). doi:10.1109/INISTA.2013.6577650

Bik, Holly M., and Miriam C. Goldstein. 2013. “An Introduction to Social Media for Scientists.” PLoS Biol 11: e1001535. doi:10.1371/journal.pbio.1001535. Bushman, Mike, and Andrea Michalek. 2013. “Are Alternative Metrics Still Alternative?” ASIS&T Bulletin (May). http://www.asis.org/Bulletin/Apr-13/AprMay13_Buschman_Michalek.pdf. Cameron, Brian D. 2005. “Trends in the Usage of ISI Bibliometric Data: Uses, Abuses, and Implications.” Portal: Libraries and the Academy 5 (1): 105–125. doi:10.1353/pla.2005.0003. CiteULike. 2013. “Frequently Asked Questions.” Accessed April 29. http://www.citeulike.org/faq/faq.adp.

Costas, R., Zahedi, Z., & Wouters, P. (2014). Do altmetrics correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. arXiv:1401.4321 [cs]. Retrieved from http://arxiv.org/abs/1401.4321 Eysenbach, G. 2011. “Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact.” Journal of Medical Internet Research 13: e123. Faculty of 1000. 2013. “About.” Accessed April 29. http://f1000.com/. Fenner, M. (2013). What Can Article-Level Metrics Do for You? PLoS Biol, 11(10), e1001687. doi:10.1371/journal.pbio.1001687 Gonzalez-Pereira, Borja, Vicente Guerrero-Bote, and Felix Moya-Anegon. 2009. “The SJR Indicator: A New Indicator of Journals’ Scientific Prestige.” arXiv:0912.4141. http://arxiv.org/abs/0912.4141.

Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., & Terliesner, J. (2013). Coverage and adoption of altmetrics sources in the bibliometric community. Scientometrics, 1–19. doi:10.1007/s11192-013-1221-3

Haustein, S., Peters, I., Sugimoto, C. R., Thelwall, M., & Larivière, V. (2013). Tweeting biomedicine: An analysis of tweets and citations in the biomedical literature. Journal of the Association for Information Science and Technology, doi:10.1002/asi.23101

Hirsch, J. E. 2005. “An Index to Quantify an Individual’s Scientific Research Output.” Proceedings of the National Academy of Sciences of the United States of America 102: 16569–16572. doi:10.1073/pnas.0507655102.

References Jacso, Peter. 2006. “Deflated, Inflated and Phantom Citation Counts.” Online Information Review 30: 297–309. doi:http://dx.doi.org.libezproxy2.syr.edu/10.1108/14684520610675816. Kaur, J., Radicchi, F., & Menczer, F. (2013). Universality of scholarly impact metrics. Journal of Informetrics, 7(4), 924–932. doi:10.1016/j.joi.2013.09.002

Konkiel, S. (2013). Altmetrics: A 21st Century Solution to Determining Research Quality. Retrieved from https://scholarworks.iu.edu/dspace/handle/2022/17147

Mendeley Ltd. 2012. “Mendeley.” http://www.mendeley.com/. ORCID Inc. 2012. “ORCID.” http://about.orcid.org/. Piwowar, Heather. 2013. “Altmetrics: Value All Research Products.” Nature 493: 159–159. doi:10.1038/493159a.

PLOS Biology: What Can Article-Level Metrics Do for You? (2013). Retrieved February 10, 2014, from http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001687

Priem, Jason. 2013. “Scholarship: Beyond the Paper.” Nature 495: 437–440. doi:10.1038/495437a. Priem, Jason, and Heather A. Piwowar. 2013. “ImpactStory: Tell the Full Story of Your Research Impact.” Accessed April 9. http://www.impactstory.org/. Priem, Jason, Dario Taraborelli, Paul Groth, and Neylon, Cameron. 2010. “Altmetrics: a Manifesto – Altmetrics.org.” Altmetrics: a Manifesto. 26. http://altmetrics.org/manifesto/. SCImago. 2007. “SJR - Scimago Journal & Country Rank.” http://www.scimagojr.com/. Shuai, Xin, Alberto Pepe, and Johan Bollen. 2012. “How the Scientific Community Reacts to Newly Submitted Preprints: Article Downloads, Twitter Mentions, and Citations.” arXiv:1202.2461. http://arxiv.org/abs/1202.2461.

Sud, P., & Thelwall, M. (2014). Evaluating altmetrics. Scientometrics, 98(2), 1131–1143. doi:10.1007/s11192-013-1117-2 Thomson Reuters. 2012. “Journal Citation Reports Help.” http://admin-apps.webofknowledge.com.libezproxy2.syr.edu/JCR/help/h_toc.htm.