Common WorkflowLanguage Viewerhttps://view.commonwl.org

Mark RobinsonStian Soiland-Reyes Michael CrusoeCarole Goble

http://slides.com/markrobinson/cwlviewer

http://orcid.org/0000-0002-8184-7507

http://orcid.org/0000-0001-9842-9718

http://orcid.org/0000-0002-2961-9670

http://orcid.org/0000-0003-1219-2137

@soilandreyes

These slides are licensed under aCreative Commons Attribution 4.0 International License

2017-07-22 BOSC 2017ISMB/ECCB 2017, Prague

Difficulties in sharing CWLWorkflow spread over multiple text files

https://github.com/common-workflow-language/workflows/tree/master/workflows/lobSTR/lobSTR-workflow.cwl

Difficulties in sharing CWL

Often missing:

Detailed description (what, how)Provenance (who, where, when)

Versioning (which)

#!/usr/bin/env cwl-runner

cwlVersion: v1.0class: Workflow

inputs: #SCWRL scw_script: File scw_input_pdb_path: File scw_output_pdb_path: string scw_mutation: string scw_scwrl_path: string scw_log_path: string scw_error_path: string #PDB2GMX p2g_script: File p2g_output_gro_path: string p2g_output_top_path: string p2g_output_itp_path: string p2g_output_top_tar_path: string p2g_water_type: string p2g_force_field: string p2g_ignh: string p2g_gmx_path: string p2g_log_path: string p2g_error_path: string # ...

We need better incentives!

#!/usr/bin/env cwl-runnercwlVersion: "cwl:draft-3"

class: Workflow

inputs: - id: p1 type: - "null" - type: array items: File description: list of files containing the first end of paired end reads in fasta or fastq format

- id: p2 type: - "null" - type: array items: File description: list of files containing the second end of paired end reads in fasta or fastq format

- id: output_prefix type: string description: prefix for output files. will output prefix.aligned.bam and prefix.aligned.stats

- id: reference type: File description: "lobSTR's bwa reference files"

- id: rg-sample type: string description: Use this in the read group SM tag

- id: rg-lib type: string description: Use this in the read group LB tag

- id: strinfo type: File description: File containing statistics for each STR.

- id: noise_model type: File description: File to read noise model parameters from (.stepmodel) secondaryFiles: - "^.stuttermodel"

outputs: - id: bam type: File source: "#samindex/bam_with_bai"

- id: bam_stats type: File source: "#lobSTR/bam_stats"

- id: vcf type: File source: "#allelotype/vcf"

- id: vcf_stats type: File source: "#allelotype/vcf_stats"

hints: - class: DockerRequirement dockerLoad: https://workbench.qr1hi.arvadosapi.com/collections/download/qr1hi-4zz18-x2ae13tsx5jqg8d/1nduktd8dpvhdpgsva82lje0i710kgzb6rttks5jldx7s2y7k9/7e0c0ae3bf4e70442f9b8eee816ec23426d9e1169a2925316e5c932745e21613.tar dockerImageId: 7e0c0ae3bf4e70442f9b8eee816ec23426d9e1169a2925316e5c932745e21613

steps: - id: lobSTR run: lobSTR-tool.cwl inputs: - { id: p1, source: "#p1" } - { id: p2, source: "#p2" } - { id: output_prefix, source: "#output_prefix" } - { id: reference, source: "#reference" } - { id: rg-sample, source: "#rg-sample" } - { id: rg-lib, source: "#rg-lib" } outputs: - { id: bam } - { id: bam_stats }

- id: samsort run: samtools-sort.cwl inputs: - { id: input, source: "#lobSTR/bam" } - { id: output_name, default: "aligned.sorted.bam" } outputs: - { id: output_file }

- id: samindex run: samtools-index.cwl inputs: - { id: input, source: "#samsort/output_file" } outputs: - { id: bam_with_bai }

- id: allelotype run: allelotype.cwl inputs: - { id: bam, source: "#samindex/bam_with_bai" } - { id: reference, source: "#reference" } - { id: output_prefix, source: "#output_prefix" } - { id: noise_model, source: "#noise_model" } - { id: strinfo, source: "#strinfo" } outputs: - { id: vcf } - { id: vcf_stats }

Workflow Inputs

Workflow Outputs

reference

allelotype

lobSTR

rg-samplep1p2 output_prefixrg-lib strinfonoise_model

vcfvcf_statsbam_stats bam

samsort

samindex

"aligned.sorted.bam"

Visualisation = quick and easy understanding

CWL Viewerhttps://view.commonwl.org

View a Workflowhttps://view.commonwl.org/workflows/github.com/common-workflow-language/

workflows/tree/master/workflows/lobSTR/lobSTR-workflow.cwl

Interactivityhttps://view.commonwl.org/workflows/github.com/ProteinsWebTeam/ebi-

metagenomics-cwl/tree/master/tools/rRNA_selection.cwl

Download and Share

Visualisation Images(various formats)

GraphViz DOT source

Research Object Bundle/Origin Information

COMMONWORKFLOWLANGUAGE

A Research Object bundles and relates digitalresources of a scientific experiment orinvestigation:

Data used and results produced in experimentalstudyMethods employed to produce and analyse thatdataProvenance and settings for the experimentsPeople involved in the investigationAnnotations about these resources, to improveunderstanding and interpretation

bundle.zip├── mimetype├── .ro│ └── manifest.json└── workflow ├── completeWorkflow.cwl ├── concatFasta.cwl ├── fastqc.cwl ├── generateRReports.cwl ├── qiimeAddMetadata.cwl ├── readPair.yml ├── uparseChimeraCheck.cwl ├── uparseDerepWorkAround.cwl ├── uparseFastqMerge.cwl ├── uparseFilter.cwl └── uparseTruncateLength.cwl

{ "@context" : [ "https://w3id.org/bundle/context" ], "id" : "/", "manifest" : [ "manifest.json" ], "createdOn" : "2017-02-27T11:48:54.986Z", "createdBy" : { "uri" : "http://view.commonwl.org", "name" : "Common Workflow Language Viewer" }, "authoredBy" : [ { "uri" : "https://github.com/grbot", "name" : "Gerrit Botha" }, { "uri" : "https://github.com/mr-c", "name" : "Michael R. Crusoe" }, { "name" : "Ubuntu" }, { "uri" : "https://github.com/mepstein-gh", "name" : "Milt Epstein" }, { "uri" : "https://github.com/ShakunBaichoo", "name" : "ShakunBaichoo" } ], "importedFrom" : "https://github.com/h3abionet/h3abionet16S/tree/70498847fd6bfc63f44c59c748233cac7f8dd85d/workflows" "importedOn" : "2017-02-27T11:48:54.986Z", "importedBy" : { "uri" : "http://view.commonwl.org", "name" : "Common Workflow Language Viewer" }, "aggregates" : [ { "uri" : "/workflow/completeWorkflow.cwl", "mediatype" : "text/x-yaml", "createdOn" : "2017-02-27T11:48:55.21Z", "authoredBy" : [ { "uri" : "https://github.com/grbot", "name" : "Gerrit Botha" }, { "uri" : "https://github.com/mr-c", "name" : "Michael R. Crusoe" }, { "uri" : "https://github.com/mepstein-gh", "name" : "Milt Epstein" }, { "uri" : "https://github.com/ShakunBaichoo", "name" : "ShakunBaichoo" } ], "retrievedFrom" : "https://github.com/h3abionet/h3abionet16S/blob/70498847fd6bfc63f44c59c748233cac7f8dd85d/workflows/completeWorkflow.cwl" "conformsTo" : "https://w3id.org/cwl/v1.0", "bundledAs" : { "uri" : "urn:uuid:fbc4f48a-b111-4e1b-b6b9-d6a97c7c7f90", "folder" : "/workflow/" } }, { "uri" : "/workflow/concatFasta.cwl", "mediatype" : "text/x-yaml",

Provenance, attribution

Who made which part of the workflowwhen

CWL Viewer Research Objects

BOSC 2017Birds of aFeather

Today 12:20-13:40

BoF #2: Common Workflow Languagecommunity meetup

Farah Zaib Khan & Stian Soiland-Reyes:

ResearchObject.org & CWL integration

COMMONWORKFLOWLANGUAGE

Discovery of Workflowshttps://view.commonwl.org/workflows

In the Works!

JSON API Use of the cwltool referenceimplementation for parsing

Input labelling

More import options

And More!

Workflow Inputs

Workflow Outputs

p2

lobSTR

p2

rg-lib

rg-lib

p1

p1

reference

reference

allelotype

reference

output_prefix

output_prefix

output_prefix

noise_model

noise_model

rg-sample

rg-sample

strinfo

strinfo

vcf_stats vcfbambam_stats

samsort

input

samindex

bam

input

"aligned.sorted.bam"

output_name

cwl-svg

https://view.commonwl.org/

Join us!

https://github.com/common-workflow-language/cwlviewer

Poster (A-124):https://doi.org/10.7490/f1000research.1114375.1