SciWIn
As the volume and complexity of scientific data grows, researchers increasingly rely on custom scripts to analyze and extract findings from these data. While these scripts can be highly efficient, they often do not adhere to the FAIR principles [1], making it difficult for others to find, access, and reuse the data and the results. The lack of provenance and of a defined execution environment makes it difficult for others to reproduce results. A recent study analyzing Jupyter Notebooks from biomedical publications revealed that only 1,203 notebooks out of 22,578 Python notebooks defined all software dependencies in a manner that allowed error‑free installation and re‑execution [2].
The Common Workflow Language (CWL), a community‑driven standard for describing computational workflows, was created precisely to fill this gap [3]. In CWL, each step of a pipeline is represented as a CWL CommandLineTool that specifies the inputs, the outputs, the runtime environment and the command to be run. These CWL CommandLineTools can be linked together to form CWL Workflows. Because the description is standardized, the workflows are portable across different execution platforms and independent steps can be parallelized automatically. Despite its potential, adoption of CWL remains limited because most scientists are reluctant to learn a new syntax and to write CWL specifications by hand.
In response to this, SciWIn-Client (s4n) [4] has been developed to integrate seamlessly into the researchers’ regular work routine and assist researchers in the semi-automatic creation of CWL CommandLineTools, without requiring them to learn the intricacies of the language. To generate a CWL CommandLineTool a user simply prefixes a regular script or tool call. This approach ensures that researchers can continue to work with their familiar scripts and tools while still benefiting from the standardized CWL description of workflows. SciWIn parses the command invocation to infer inputs and the base command. Output detection leverages Git in the background to record the repository state before execution and automatically registers any newly created files as CWL outputs. Users may also complementary declare inputs and outputs explicitly when needed. Through the usage of Git, changes to both data and code remain traceable. Individual CWL CommandLineTools can be assembled into CWL workflows, either by defining connections between the tool’s inputs and outputs using SciWIn-Client or visually using SciWIn-Studio, a graphical user interface (GUI). In the GUI, users can drag‑and‑drop CWL CommandLineTools, draw edges to link outputs to downstream inputs, and visualize data flow, thus lowering the burden associated with pipeline construction. Researchers can also reuse workflows that were created by others through simple install and uninstall commands.
The completed workflow can be run locally on the researcher’s workstation supporting all major operation systems or dispatched to a remote execution engine such as REANA [5]. REANA was developed at CERN and is also deployed in PUNCH4NFDI. FAIRagro operates their own REANA instance in the de.NBI Cloud. REANA’s container‑based orchestration ensures that the workflow runs with the exact software stack captured in the CWL description, eliminating environment‑related failures. SciWIn can export (Provenance Run) RO-Crates [6, 7] that package workflows with comprehensive metadata, inputs, outputs, commands, software environments, and workflow structure ensuring reproducibility and making long-term preservation possible. These RO-Crates are designed to be useful for both humans and machines, and can be deposited in workflowhub.eu to make workflows findable and accessible to a broad research community.
DataPLANT is also utilizing Workflow RO-Crates to describe the computational aspects of their Annotated Research Context (ARC) [8] and CWL to define their computational workflows while leveraging Git as a version control system. This convergence of technologies enables collaboration between the two consortia on this topic. Researchers working with ARCs and workflows can gain from SciWIn to create and manage CWL workflows, while SciWIn can benefit from the ARC concept and DataPLANT’s infrastructure, including the PLANTdataHUB, which can facilitate collaboration among researchers. Future plans include integrating validation packages for CWL workflows within the PLANTdataHUB and enabling the validation and potentially execution of workflows using workflow executors like REANA directly from the hub. Additionally, SciWIn will be extended to implement the Task Execution Service (TES), allowing seamless integration with various computational environments, such as Kubernetes and Slurm, and further enhancing the interoperability and flexibility of the platform.
In summary, SciWIn‑Client and SciWIn‑Studio address the FAIR-workflow gap by automatically generating CWL CommandLineTools and Workflows from researchers’ everyday scripts, thereby bridging the divide between ad-hoc scripting and reproducible workflow management. This facilitates collaboration by making it possible to combine scripts written in different languages within a single workflow and empowers researchers to produce reproducible, portable, and reusable analyses.