Open materials & methods

• Increases the ability to run replications

Open data and code

• Increases the ability to check for computational reproducibility

• Increases the ability to determine the robustness of the statistical inference

Open research workflow

What is an open workflow?

• Research inputs and outputs are open:– Data, Code, Materials, Protocols, etc.

• The evolutions of these inputs and outputs over the research process– How and when versions of files changed

Open research workflow

• Increases ability to run replications

• Increases the ability to check computational & statistical reproducibility

• Increases the ability to detect researcher degrees of freedom

What is required?

• Increase open sharing

• Increase documentation

• Incentives

Enabling Open Workflows

Open Science Frameworkhttps://osf.io

Project Overview Page

Versioning

Registering Your Work

Sharing Your Work

Incentivizing Open Practices

• Badges for open practices

Badges to Acknowledge Open Practices

Badges to Acknowledge Open Practices

Incentivizing Open Practices

• Badges for open practices

• Additional metrics

Accessing Statistics

Search and

discover

Develop idea

Design study

Acquire materials

Collect dataStore data

Analyze data

Interpret findings

Write report

Publish report

Questions?

courtney@cos.iocontact@cos.io

Reproducibility Projects

Many Labs Project

Crowdsourcing Science