24
Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein , Timothy D. Bowman, Kim Holmberg, Vincent Larivière, Isabella Peters, Cassidy R. Sugimoto, & Mike Thelwall

Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Embed Size (px)

Citation preview

Page 1: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Empirical analyses of scientific papers and researchers on Twitter: Results of two studies

Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent Larivière, Isabella Peters, Cassidy R. Sugimoto, & Mike Thelwall

Page 2: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Background• when Garfield created SCI, sociologists of science

analyzed meaning of publications and citations (Merton, Zuckerman, Cole & Cole, etc.)

• sociological research• What is it to publish a paper?• What are the reasons to cite?

• empirical bibliometric research• disciplinary differences in publication

and citation behavior• delay and obsolescence patterns

Page 3: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Background• empirical studies helped sociologists to understand

structure and norms of science

• for bibliometricians, studies provided a theoretical framework and legitimation to use citation analysis in research evaluation

• knowledge about disciplinary differences and obsolescence patterns helped to normalize statistics and create more appropriate indicators

Page 4: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Background• recently social-media metrics have become

important in the scholarly world

• suggestions to complement (or even replace) citation analysis by so-called ”altmetrics“• broader audience (not just citing authors)• more timely

• however, similar to bibliometrics in the 1960s, little is known about the actual meaning of various social-media counts

Page 5: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Research questions• What is the relationship between social-media and

citation counts?

• How do various social-media metrics differ?

• Why are papers tweeted, bookmarked, liked…?

• Who tweets (bookmarks, likes…) scientific papers?

• How do these aspects differ across scientific disciplines?

Two case studies on Twitter• large-scale analysis of tweets of biomedical papers

• in-depth analysis of astrophysicists on Twitter

Page 6: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Aim of the study• large-scale analysis of tweets of biomedical papers

• Twitter coverage• Twitter citation rates (tweets per paper)• correlation with citations

• discovering differences between:• documents• journals• disciplines & specialties

providing empirical framework to understand the extent to which biomedical journal articles are tweeted

Study I: Tweeting biomedicine

Haustein, S., Peters, I., Sugimoto, C.R., Thelwall, M., & Larivière, V. (in press). Tweeting biomedicine: an analysis of tweets and citations in the biomedical literature. Journal of the American Society for Information Science and Technology, http://arxiv.org/abs/1308.1838.

Page 7: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Data sets & methods• 1.4 million PubMed papers covered by WoS

• publication years: 2010-2012• document types: articles & reviews• matching of WoS and PubMed

• tweet counts collected by Altmetric.com• collection based on PMID, DOI, URL• matching WoS via PMID

• journal-based matching of NSF classification

• tweets per article, Twitter coverage and correlationwith citations for:• journals• NSF disciplines and specialties

Study I: Tweeting biomedicine

Page 8: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Data sets & methods: framework

Study I: Tweeting biomedicine

Page 9: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Data sets & methods: correlations

Study I: Tweeting biomedicine

PY=2010 PY=2011 PY=2012

Page 10: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Results: documents

Study I: Tweeting biomedicine

Publication year

Twitter coverage

Papers (T≥1)

Spearman's ρ Mean Median Maximum

T2010

2.4% 13,763 .104** 2.1 1 237C2010 18.3 7 3,922

T2011

10.9% 63,801 .183** 2.8 1 963C2011 5.7 2 2,300

T2012

20.4% 57,365 .110** 2.3 1 477C2012 1.3 0 234

T2010-2012

9.4% 134,929 .114** 2.5 1 963C2010-2012 5.1 1 3,922

• Twitter coverage is quite low but increasing

• correlation between tweets and citations is very low

Page 11: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Results: documents

Study I: Tweeting biomedicine

Article Journal C T

Hess et al. (2011). Gain of chromosome band 7q11 in papillary thyroid carcinomas of young patients is associated with exposure to low-dose irradiation PNAS 9 963

Yasunari et al. (2011). Cesium-137 deposition and contamination of Japanese soils due to the Fukushima nuclear accident PNAS 30 639

Sparrow et al. (2011). Google Effects on Memory: Cognitive Consequences of Having Information at Our Fingertips Science 11 558

Onuma et al. (2011). Rebirth of a Dead Belousov–Zhabotinsky Oscillator Journal of Physical Chemistry A -- 549

Silverberg (2012). Whey protein precipitating moderate to severe acne flares in 5 teenaged athletes Cutis -- 477

Wen et al. (2011). Minimum amount of physical activity for reduced mortality and extended life expectancy: a prospective cohort study Lancet 51 419

Kramer (2011). Penile Fracture Seems More Likely During Sex Under Stressful Situations Journal of Sexual Medicine -- 392

Newman & Feldman (2011). Copyright and Open Access at the Bedside New England Journal of Medicine 3 332

Reaves et al. (2012). Absence of Detectable Arsenate in DNA from Arsenate-Grown GFAJ-1 Cells Science 5 323

Bravo et al. (2011). Ingestion of Lactobacillus strain regulates emotional behavior and central GABA receptor expression in a mouse via the vagus nerve PNAS 31 297

Top 10 tweeted documents: catastrophe & topical / web & social media / curious story scientific discovery / health implication / scholarly community

Page 12: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Results: journals• 97.7% of 3,812

journals at least tweeted once

• two-thirds of journals have coverage below 20% and Twitter citation rate < 2.0

• high Twitter citation rates often caused by few papers

• high coverage and Twitter citation rates for general journals

Study I: Tweeting biomedicine

Page 13: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Results: disciplines

Study I: Tweeting biomedicine

Page 14: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Results: specialties

Study I: Tweeting biomedicine

• specialties differ in terms of coverage, Twitter citation rate and correlations with citations

• 47 of 61 specialties show low positive, 3 negative and 13 no correlation

bubb

le s

ize

= T

witt

er c

itatio

n ra

te

Page 15: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Aim of the study• in-depth analysis of astrophysicists on Twitter

• number of tweets, followers, retweets• characteristics of tweets: RTs, @messages,

#hashtags, URLs

• comparison with scientific output• publications• citations

• comparison of tweet and publication content provide evidence in how far astrophysicists on Twitter

use Twitter for scholarly communiation

Study II: Astrophysicists on Twitter

Haustein, S., Bowman, T.D., Holmberg, K., Larivière, V., & Peters, I., (submitted). Astrophysicists on Twitter: An in-depth analysis of tweeting and scientific publication behavior. Aslib Proceedings.

Page 16: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Data sets & methods• 37 astrophysicists on Twitter identified by

Holmberg & Thelwall (2013)

• web searches to identify person behind account

• publications in WoS journals• publication years: 2008-2012• author disambiguation

• Twitter account information

• 68,232 of 289,368 tweets downloaded and analyzed:• number of RTs per tweet• % of tweets that are RTs• % of tweets containing #hashtags, @usernames, URLs

Study II: Astrophysicists on Twitter

Holmberg, K., & Thelwall, M. (2013). Disciplinary differences in Twitter scholarly communication. In: Proceedings of ISSI 2013 – 14th International Conference of the International Society for Scientometrics and Informetrics, Vienna, Austria (Vol. 1, pp. 567-582).

Page 17: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Data sets & methods• grouping astrophysicists according to tweeting and

publication behavior

• analyzing differences of tweeting characteristics between user groups

Study II: Astrophysicists on Twitter

Selected astrophysicists(N=37)

tweet rarely(0.0-0.1 tweets per day)

tweet occasionally(0.1-0.9)

tweet regularly(1.2-2.9)

tweet frequently(3.7-58.2)

total(publishing activity)

do not publish(0 publications 2008-2012) -- -- 1 5 6

publish occasionally(1-9) 4 3 4 2 13

publish regularly(14-37) -- 5 5 3 13

publish frequently(46-112) 1 3 1 -- 5

total (tweeting activity) 5 11 11 10 37

Page 18: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Data sets & methods• comparison of tweet and publication content

• extraction of noun phrases from tweets and abstracts• limited to 18 most frequently publishing astrophysicists

to ensure certain number of abstracts• analyzing overlap of character strings• calculating similarity with cosine per person and overall

Study II: Astrophysicists on Twitter

Selected astrophysicists(N=37)

tweet rarely(0.0-0.1 tweets per day)

tweet occasionally(0.1-0.9)

tweet regularly(1.2-2.9)

tweet frequently(3.7-58.2)

total(publishing activity)

publish regularly(14-37) -- 5 5 3 13

publish frequently(46-112) 1 3 1 -- 5

total (tweeting activity) 1 8 6 3 18

Page 19: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Results: correlations• comparison of Twitter and publication activity and impact

• publications and tweets per day: ρ=−0.339*• citation rate and tweets per day: ρ=−0.457**• citation rate and RT rate: ρ=0.077

Study II: Astrophysicists on Twitter

Page 20: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Results: characteristics

Study II: Astrophysicists on Twitter

Mean share of tweets containing at least one user name orURL per person per group

Page 21: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Results: content similarity

Study II: Astrophysicists on Twitter

• overall similarity between abstracts and tweets is low• cosine=0.081• 4.1% of 50,854 tweet NPs in abstracts• 16.0% of 12,970 abstract NPs in tweets

• Twitter coverage among most frequent abstract terms is high, although this differs between users• 97,1% of 104 most frequent noun phrases on Twitter

Page 22: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Conclusions• Twitter coverage of biomedical papers is low but increasing

• number of tweets per paper varies between journals, disciplines, specialties and from year to year tweet counts need to be normalized accordingly

• correlations between tweet and citation counts are low (biomedical papers) or even moderately negative (astrophysicists) tweets cannot replace citations as measures of

scientific impact challenge is to differentiate between high tweet counts

because of value (to scientists and/or the general public)and curiosity

Page 23: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Outlook• user surveys and qualitative research to investigate who is

using scholarly content on social media and why

• empirical large-scale studies on other metrics

Page 24: Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent

Haustein, S., Peters, I., Sugimoto, C.R., Thelwall, M., & Larivière, V. ( in press). Tweeting biomedicine: an analysis of tweets and citations in the biomedical literature. Journal of the American Society for Information Science and Technology.Haustein, S., Bowman, T.D., Holmberg, K., Larivière, V., & Peters, I., (submitted). Astrophysicists on Twitter: An in-depth analysis of tweeting and scientific publication behavior. Aslib Proceedings.

Stefanie Haustein

Thank you for your attention!

[email protected]@stefhaustein