Upload
jonah-bonnell
View
215
Download
0
Tags:
Embed Size (px)
Citation preview
Empirical analyses of scientific papers and researchers on Twitter: Results of two studies
Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent Larivière, Isabella Peters, Cassidy R. Sugimoto, & Mike Thelwall
Background• when Garfield created SCI, sociologists of science
analyzed meaning of publications and citations (Merton, Zuckerman, Cole & Cole, etc.)
• sociological research• What is it to publish a paper?• What are the reasons to cite?
• empirical bibliometric research• disciplinary differences in publication
and citation behavior• delay and obsolescence patterns
Background• empirical studies helped sociologists to understand
structure and norms of science
• for bibliometricians, studies provided a theoretical framework and legitimation to use citation analysis in research evaluation
• knowledge about disciplinary differences and obsolescence patterns helped to normalize statistics and create more appropriate indicators
Background• recently social-media metrics have become
important in the scholarly world
• suggestions to complement (or even replace) citation analysis by so-called ”altmetrics“• broader audience (not just citing authors)• more timely
• however, similar to bibliometrics in the 1960s, little is known about the actual meaning of various social-media counts
Research questions• What is the relationship between social-media and
citation counts?
• How do various social-media metrics differ?
• Why are papers tweeted, bookmarked, liked…?
• Who tweets (bookmarks, likes…) scientific papers?
• How do these aspects differ across scientific disciplines?
Two case studies on Twitter• large-scale analysis of tweets of biomedical papers
• in-depth analysis of astrophysicists on Twitter
Aim of the study• large-scale analysis of tweets of biomedical papers
• Twitter coverage• Twitter citation rates (tweets per paper)• correlation with citations
• discovering differences between:• documents• journals• disciplines & specialties
providing empirical framework to understand the extent to which biomedical journal articles are tweeted
Study I: Tweeting biomedicine
Haustein, S., Peters, I., Sugimoto, C.R., Thelwall, M., & Larivière, V. (in press). Tweeting biomedicine: an analysis of tweets and citations in the biomedical literature. Journal of the American Society for Information Science and Technology, http://arxiv.org/abs/1308.1838.
Data sets & methods• 1.4 million PubMed papers covered by WoS
• publication years: 2010-2012• document types: articles & reviews• matching of WoS and PubMed
• tweet counts collected by Altmetric.com• collection based on PMID, DOI, URL• matching WoS via PMID
• journal-based matching of NSF classification
• tweets per article, Twitter coverage and correlationwith citations for:• journals• NSF disciplines and specialties
Study I: Tweeting biomedicine
Data sets & methods: framework
Study I: Tweeting biomedicine
Data sets & methods: correlations
Study I: Tweeting biomedicine
PY=2010 PY=2011 PY=2012
Results: documents
Study I: Tweeting biomedicine
Publication year
Twitter coverage
Papers (T≥1)
Spearman's ρ Mean Median Maximum
T2010
2.4% 13,763 .104** 2.1 1 237C2010 18.3 7 3,922
T2011
10.9% 63,801 .183** 2.8 1 963C2011 5.7 2 2,300
T2012
20.4% 57,365 .110** 2.3 1 477C2012 1.3 0 234
T2010-2012
9.4% 134,929 .114** 2.5 1 963C2010-2012 5.1 1 3,922
• Twitter coverage is quite low but increasing
• correlation between tweets and citations is very low
Results: documents
Study I: Tweeting biomedicine
Article Journal C T
Hess et al. (2011). Gain of chromosome band 7q11 in papillary thyroid carcinomas of young patients is associated with exposure to low-dose irradiation PNAS 9 963
Yasunari et al. (2011). Cesium-137 deposition and contamination of Japanese soils due to the Fukushima nuclear accident PNAS 30 639
Sparrow et al. (2011). Google Effects on Memory: Cognitive Consequences of Having Information at Our Fingertips Science 11 558
Onuma et al. (2011). Rebirth of a Dead Belousov–Zhabotinsky Oscillator Journal of Physical Chemistry A -- 549
Silverberg (2012). Whey protein precipitating moderate to severe acne flares in 5 teenaged athletes Cutis -- 477
Wen et al. (2011). Minimum amount of physical activity for reduced mortality and extended life expectancy: a prospective cohort study Lancet 51 419
Kramer (2011). Penile Fracture Seems More Likely During Sex Under Stressful Situations Journal of Sexual Medicine -- 392
Newman & Feldman (2011). Copyright and Open Access at the Bedside New England Journal of Medicine 3 332
Reaves et al. (2012). Absence of Detectable Arsenate in DNA from Arsenate-Grown GFAJ-1 Cells Science 5 323
Bravo et al. (2011). Ingestion of Lactobacillus strain regulates emotional behavior and central GABA receptor expression in a mouse via the vagus nerve PNAS 31 297
Top 10 tweeted documents: catastrophe & topical / web & social media / curious story scientific discovery / health implication / scholarly community
Results: journals• 97.7% of 3,812
journals at least tweeted once
• two-thirds of journals have coverage below 20% and Twitter citation rate < 2.0
• high Twitter citation rates often caused by few papers
• high coverage and Twitter citation rates for general journals
Study I: Tweeting biomedicine
Results: disciplines
Study I: Tweeting biomedicine
Results: specialties
Study I: Tweeting biomedicine
• specialties differ in terms of coverage, Twitter citation rate and correlations with citations
• 47 of 61 specialties show low positive, 3 negative and 13 no correlation
bubb
le s
ize
= T
witt
er c
itatio
n ra
te
Aim of the study• in-depth analysis of astrophysicists on Twitter
• number of tweets, followers, retweets• characteristics of tweets: RTs, @messages,
#hashtags, URLs
• comparison with scientific output• publications• citations
• comparison of tweet and publication content provide evidence in how far astrophysicists on Twitter
use Twitter for scholarly communiation
Study II: Astrophysicists on Twitter
Haustein, S., Bowman, T.D., Holmberg, K., Larivière, V., & Peters, I., (submitted). Astrophysicists on Twitter: An in-depth analysis of tweeting and scientific publication behavior. Aslib Proceedings.
Data sets & methods• 37 astrophysicists on Twitter identified by
Holmberg & Thelwall (2013)
• web searches to identify person behind account
• publications in WoS journals• publication years: 2008-2012• author disambiguation
• Twitter account information
• 68,232 of 289,368 tweets downloaded and analyzed:• number of RTs per tweet• % of tweets that are RTs• % of tweets containing #hashtags, @usernames, URLs
Study II: Astrophysicists on Twitter
Holmberg, K., & Thelwall, M. (2013). Disciplinary differences in Twitter scholarly communication. In: Proceedings of ISSI 2013 – 14th International Conference of the International Society for Scientometrics and Informetrics, Vienna, Austria (Vol. 1, pp. 567-582).
Data sets & methods• grouping astrophysicists according to tweeting and
publication behavior
• analyzing differences of tweeting characteristics between user groups
Study II: Astrophysicists on Twitter
Selected astrophysicists(N=37)
tweet rarely(0.0-0.1 tweets per day)
tweet occasionally(0.1-0.9)
tweet regularly(1.2-2.9)
tweet frequently(3.7-58.2)
total(publishing activity)
do not publish(0 publications 2008-2012) -- -- 1 5 6
publish occasionally(1-9) 4 3 4 2 13
publish regularly(14-37) -- 5 5 3 13
publish frequently(46-112) 1 3 1 -- 5
total (tweeting activity) 5 11 11 10 37
Data sets & methods• comparison of tweet and publication content
• extraction of noun phrases from tweets and abstracts• limited to 18 most frequently publishing astrophysicists
to ensure certain number of abstracts• analyzing overlap of character strings• calculating similarity with cosine per person and overall
Study II: Astrophysicists on Twitter
Selected astrophysicists(N=37)
tweet rarely(0.0-0.1 tweets per day)
tweet occasionally(0.1-0.9)
tweet regularly(1.2-2.9)
tweet frequently(3.7-58.2)
total(publishing activity)
publish regularly(14-37) -- 5 5 3 13
publish frequently(46-112) 1 3 1 -- 5
total (tweeting activity) 1 8 6 3 18
Results: correlations• comparison of Twitter and publication activity and impact
• publications and tweets per day: ρ=−0.339*• citation rate and tweets per day: ρ=−0.457**• citation rate and RT rate: ρ=0.077
Study II: Astrophysicists on Twitter
Results: characteristics
Study II: Astrophysicists on Twitter
Mean share of tweets containing at least one user name orURL per person per group
Results: content similarity
Study II: Astrophysicists on Twitter
• overall similarity between abstracts and tweets is low• cosine=0.081• 4.1% of 50,854 tweet NPs in abstracts• 16.0% of 12,970 abstract NPs in tweets
• Twitter coverage among most frequent abstract terms is high, although this differs between users• 97,1% of 104 most frequent noun phrases on Twitter
Conclusions• Twitter coverage of biomedical papers is low but increasing
• number of tweets per paper varies between journals, disciplines, specialties and from year to year tweet counts need to be normalized accordingly
• correlations between tweet and citation counts are low (biomedical papers) or even moderately negative (astrophysicists) tweets cannot replace citations as measures of
scientific impact challenge is to differentiate between high tweet counts
because of value (to scientists and/or the general public)and curiosity
Outlook• user surveys and qualitative research to investigate who is
using scholarly content on social media and why
• empirical large-scale studies on other metrics
Haustein, S., Peters, I., Sugimoto, C.R., Thelwall, M., & Larivière, V. ( in press). Tweeting biomedicine: an analysis of tweets and citations in the biomedical literature. Journal of the American Society for Information Science and Technology.Haustein, S., Bowman, T.D., Holmberg, K., Larivière, V., & Peters, I., (submitted). Astrophysicists on Twitter: An in-depth analysis of tweeting and scientific publication behavior. Aslib Proceedings.
Stefanie Haustein
Thank you for your attention!
[email protected]@stefhaustein