We need to solve more than our access problems

Björn BrembsUniversität Regensburg - Neurogenetics

http://brembs.net - @brembs

Scholarly Infrastructurean obscenely expensive anachronism

antiquated tax-waste counter-productive

I II III

SCHOLARSHIP

Scientists produce publications, data and code

PROBLEM I.1

Dysfunctional scholarly literature

Antiquated Functionality• Limited access• Link-rot• No scientific impact analysis• Lousy peer-review • No global search• No functional hyperlinks• Useless data visualization• No submission standards• (Almost) no statistics• No content-mining• No effective way to sort,

filter and discover• No semantic enrichment• No networking feature• etc.

…it’s like the web in 1995!

Literature

Provided by:

Literature

Provided by:

Literature

Provided by:

PROBLEM I.2

Scientific data in peril

Small Data – Long Tail

Report on Integration of Data and Publications, ODE Report 2011http://www.alliancepermanentaccess.org/wp-content/plugins/download-monitor/download.php?id=ODE+Report+on+Integration+of+Data+and+Publications

PROBLEM I.3

Non-existent software archives

UNIVAC (A-2) 1953

DIGITALIZATION

I. Antiquated and missing functionality

Publishers parasitize public funds

Cost

s [th

ousa

nd U

S$/a

rticle

]

Legacy Modern(Sources: Van Noorden, R. (2013). Open access: The true cost of science publishing. Nature 495, 426–9; Packer, A. L. (2010). The SciELO Open Access: A Gold Way from the South. Can. J. High. Educ. 39, 111–126)

(SciELOUbiquityScholasticaScienceOpenPeerJF1000ResearchFrontiersetc.)

MONEY FOR NOTHING

II. Wasting billions on a parasitic industry

More and Better!

Application Instructions

Application Instructions

Publikationstätigkeit(vollständige Publikationsliste, darunter Originalarbeiten als Erstautor/in, Seniorautor/in, Impact-Punkte insgesamt und in den letzten 5 Jahren, darunter jeweils gesondert ausgewiesen als Erst- und Seniorautor/in, persönlicher Scientific Citations Index (SCI, h-Index nach Web of Science) über alle Arbeiten)

Publications:Complete list of publications, including original research papers as first author, senior author, impact points total and in the last 5 years, with marked first and last-authorships, personal Scientific Citations Index (SCI, h-Index according to Web of Science) for all publications.

Main Problems with the IF• Negotiable • Irreproducible • Mathematically

unsound

Methodology

Journal Rank

Qua

lity

sources: 10.1093/annonc/mdg203, 10.1371/journal.pone.0056180, 10.3389/fnhum.2013.00291, 10.1371/journal.pone.0109019, 10.1371/journal.pbio.1002273, 10.1101/071530, 10.1186/s13059-016-1044-7

Misconduct

Fang et al. (2012): Misconduct accounts for the majority of retracted scientific publications. doi: 10.1073/pnas.1212247109

Journal Rank

Frau

d

QUALITY?

“High-Impact” journals attract the most unreliable research

Statistical Power and Sample Size

Source: Daniel Lakens DOI: 10.1177/1745691614528520

Statistical Power and Sample Size

Source: Daniel Lakens DOI: 10.1177/1745691614528520

Statistical Power and Sample Size

Source: Daniel Lakens DOI: 10.1177/1745691614528520

Statistical Power and Sample Size

Productivity

Research questions:True:False:Significant:

20010010040

88444437

DOI: 10.1098/rsos.160384

PRODUCTIVITY?

“Publish-or-Perish” disadvantages meticulous scientists

Freedman LP, Cockburn IM, Simcoe TS (2015) http://journals.plos.org/plosbiology/article?id=info:doi/10.1371/journal.pbio.1002165

Irreproducibility

61%(n=100)

Open Science Collaboration

INCENTIVES

III. Counting Quality & Productivity => Selecting the sloppy scientists

SO MUCH FOR THAT

The disaster that is our scholarly infrastructure

OPEN ACCESS ACTIVISM“Pretty please be open!”

575+ such solutions and counting…

Institutions are trying…

The Department of Psychology embraces the values of open science and strives for replicable and reproducible research. For this goal we support transparent research with open data, open material, and pre-registrations. Candidates are asked to describe in what way they already pursued and plan to pursue these goals.

Complete list of publications, including original research papers as first author, senior author, impact points total and in the last 5 years, with marked first and last-authorships, personal Scientific Citations Index (SCI, h-Index according to Web of Science) for all publications.

versus

WHAT NOW?

Save time and money (and make science open by default as an added benefit)

Potential for Innovation

(Sources: Van Noorden, R. (2013). Open access: The true cost of science publishing. doi:10.1038/495426a, Packer, A. L. (2010). The SciELO Open Access: A Gold Way from the South. Can. J. High. Educ. 39, 111–126)

Potential for innovation: 9.8b p.a.

Cost

s [th

ousa

nd U

S$/a

rticle

]

Legacy SciELO

1. International Coordination

LEGAL

2. Cancel all subscriptions

3. Implement current technology

The square traversal process has been the foundation of scholarly

communication for nearly 400 years!

Negotiable• PLoS Medicine, IF 2-11 (8.4)

(The PLoS Medicine Editors (2006) The Impact Factor Game. PLoS Med 3(6): e291. http://www.plosmedicine.org/article/info:doi/10.1371%2Fjournal.pmed.0030291)

• Current Biology IF from 7 to 11 in 2003– Bought by Cell Press (Elsevier) in 2001…

Negotiable

https://quantixed.wordpress.com/2016/01/05/the-great-curve-ii-citation-distributions-and-reverse-engineering-the-jif/

June, 2014 (19 months)

Not Reproducible• Rockefeller University Press bought their data from Thomson Reuters• Up to 19% deviation from published records• Second dataset still not correct

Rossner M, van Epps H, Hill E (2007): Show me the data. The Journal of Cell Biology, Vol. 179, No. 6, 1091-1092 http://jcb.rupress.org/cgi/content/full/179/6/1091

Not Mathematically Sound• Left-skewed distributions• Weak correlation of individual

article citation rate with journal IF

Seglen PO (1997): Why the impact factor of journals should not be used for evaluating research. BMJ 1997;314(7079):497http://www.bmj.com/cgi/content/full/314/7079/497

Not Mathematically Sound

https://quantixed.wordpress.com/2016/01/05/the-great-curve-ii-citation-distributions-and-reverse-engineering-the-jif/

‘QUALITY’

Is journal rank like astrology?

Methodology I

Macleod MR, et al. (2015) Risk of Bias in Reports of In Vivo Research: A Focus for Improvement. doi:10.1371/journal.pbio.1002273

Methodology II

Brembs, B., Button, K., & Munafò, M. (2013). Deep impact: unintended consequences of journal rank. Frontiers in Human Neuroscience, 7. doi:10.3389/fnhum.2013.00291

Methodology III

Munafò, M., Stothart, G., & Flint, J. (2009). Bias in genetic association studies and impact factor Molecular Psychiatry, 14 (2), 119-120 DOI: 10.1038/mp.2008.77

‘Quality’

Brown, E. N., & Ramaswamy, S. (2007). Quality of protein crystal structures. Acta Crystallographica Section D Biological Crystallography, 63(9), 941–950. doi:10.1107/S0907444907033847

Excel Errors

DOI: 10.1186/s13059-016-1044-7 -omics studies

p-Value Errors

Cog. Neurosci & PsychDOI: 10.1101/071530

Journal Rank and Fraud/Error

Fang et al. (2012): Misconduct accounts for the majority of retracted scientific publications. PNAS 109 no. 42 17028-17033

Journal Rank and Fraud/Error

Fang et al. (2012): Misconduct accounts for the majority of retracted scientific publications. PNAS 109 no. 42 17028-17033

Replication in Psychology

Open Science Collaboration Science 2015;349:aac4716

Replication in Psychology

Open Science Collaboration Science 2015;349:aac4716

Software to control the experiment and save the data

Software to analyze and visualize the data

GitHub

Scientific Code with Persistent Identifiers

They parasitize public funds

+70%

They parasitize public funds

“The decision, based on market and competitor analysis, will bring Emerald’s APC pricing in line with the wider market, taking a mid-point position amongst its competitors.”

Emerald spokesperson