Peer Reviewing Data: experiences from a data journal

Peer Reviewing Data: experiences from a data journal

Varsha Khodiyar, PhD

Data Curation Editor, Scientific Data

15.09.2016

1

The data journal

2

Nature Research PowerPoint presentation title / 00.00.2016

Scientific Data, a Nature Research journal

Data Descriptor

Primary article type; sound science and facilitates data reuse

Analysis

New analyses or meta-analyses of existing data

Article

Original reports on advances in data sharing & reuse

Comment

Announcements of broad interest; usually invited

www.nature.com/scientificdata

3


Under the hood of a Data Descriptor

• Synthesis

• Analysis

• Conclusions

• Context in which the data was generated (background)

• How was the data generated?

• How was the data processed?

• Where is the data?

4


Data Descriptors have human and machine understandable components

Human readable representation of

studyi.e. article (HTML &

PDF)

Human readable representation of

studyi.e. article (HTML &

PDF)

5


Data Descriptors have human and machine understandable components

Machine accessible representation of

studyi.e. metadata

6

Guidance for authors

7


Templates and detailed guidance available online

8


Recommended repositories

Browse our recommended data repositories online.

• We currently list more than 80 repositories, across biological, medical, physical and social sciences

• When required, we provide guidance to authors on the best place to store their data

nature.com/sdata/policies/repositories

9

The review process

10


Editorial office

Susanna-Assunta SansoneHonorary Academic Editor

Andrew L. HuftonManaging Editor

Varsha K. KhodiyarData Curation Editor

11


Selection of Editorial Board members

Experts in their discipline

AND

Demonstrable experience of data standards, data reuse or data

analysis in their discipline

www.nature.com/sdata/about/editorial-board#eb

12


Data peer review

www.nature.com/sdata/policies/for-referees

Experimental Rigor and Technical Data Quality

Were data produced in a sound manner?

Technical quality of data – appropriate statistical analyses?

Experimental rigor - appropriate depth, coverage?

Completeness of the Description

Sufficient detail to allow others to reproduce these steps?

Sufficient detail to allow others to reuse this data?

Consistent with relevant minimum reporting standards?

Integrity of the Data Files and Repository Record

Do data files appear complete and match manuscript descriptions?

Are data archived to the most appropriate repository?

13


We capture metadata about the dataset being described in each Data Descriptor.

During the metadata curation process• Manuscript re-read• Data archive checked• Minor issues with the data and/or manuscript often identified

Metadata curation and final data checking

14


Why a Data Descriptor may be rejected

Reject without review

• Out of scope or no data present

Reject after review

• Serious flaws in the study design, e.g. lack of crucial controls

• Serious issues identified in the data files by the peer reviewers

After rejection

• Address concerns and resubmit to Scientific Data

• Resubmit to another data journal

• Withdraw data from Scientific Data integrated repositories

Data should be technically reliable and suitable for use by others

15

Summary

16


Data peer review at Scientific Data

Data Archive

• Checked multiple times

• Scientific reasoning underlying data reviewed by active researchers

• Technical validity reviewed by discipline experts

Data Citations

• Citation accuracy confirmed by specialist editor

• Citation format checked by editorial team

• Data linkage tested by production team

Data peer review does not have to be onerousReviewing data can save overall reviewing time

17


17

Thank you

Visit nature.com/scientificdata

Email [email protected]

Tweet @ScientificData

Science

Peer Reviewing Data: experiences from a data journal