Convert an abstract step into a concrete gxformat2 step using a tool summary.
/plugin marketplace add galaxyproject/foundry
/plugin install foundry-skills@galaxy-workflow-foundryThen invoke as:
/foundry-skills:implement-galaxy-tool-stepProduces: 2 artifacts.
Consumes: 3 artifacts.
gxformat2 skeleton with one more abstract step replaced by a concrete tool step (loop iteration output).
{
"id": "galaxy-workflow-draft",
"kind": "yaml",
"default_filename": "galaxy-workflow-draft.gxwf.yml",
"schema": "[[galaxy-workflow-draft]]",
"description": "gxformat2 skeleton with one more abstract step replaced by a concrete tool step (loop iteration output)."
}Ledger ([[open-requirements-ledger]]) with a blocking entry appended when a step's declared output can't be computed from its wired inputs; drives the fall-through to topology repair.
{
"id": "open-requirements-ledger",
"kind": "yaml",
"default_filename": "open-requirements.ledger.yml",
"description": "Ledger ([[open-requirements-ledger]]) with a blocking entry appended when a step's declared output can't be computed from its wired inputs; drives the fall-through to topology repair."
}Galaxy tool summary manifest from [[summarize-galaxy-tool]] conforming to [[galaxy-tool-summary]]; binds the abstract step to a concrete tool's ports via the embedded `parsed_tool` and generated `input_schemas`.
{
"id": "galaxy-tool-summary",
"description": "Galaxy tool summary manifest from [[summarize-galaxy-tool]] conforming to [[galaxy-tool-summary]]; binds the abstract step to a concrete tool's ports via the embedded `parsed_tool` and generated `input_schemas`.",
"inherited_schema": "[[galaxy-tool-summary]]",
"producers": [
"summarize-galaxy-tool"
]
}gxformat2 skeleton being filled in step by step; the step replaces a placeholder in this draft.
{
"id": "galaxy-workflow-draft",
"description": "gxformat2 skeleton being filled in step by step; the step replaces a placeholder in this draft.",
"inherited_schema": "[[galaxy-workflow-draft]]",
"producers": [
"advance-galaxy-draft-step",
"cwl-summary-to-galaxy-template",
"freeform-summary-to-galaxy-template",
"implement-galaxy-tool-step",
"nextflow-summary-to-galaxy-template",
"repair-galaxy-draft-topology"
]
}Carried obligations ledger [[open-requirements-ledger]]: read entries the design tier recorded against this step before implementing; append a blocking entry if its output can't be computed.
{
"id": "open-requirements-ledger",
"description": "Carried obligations ledger [[open-requirements-ledger]]: read entries the design tier recorded against this step before implementing; append a blocking entry if its output can't be computed.",
"producers": [
"advance-galaxy-draft-step",
"compare-against-iwc-exemplar",
"cwl-summary-to-galaxy-data-flow",
"cwl-summary-to-galaxy-interface",
"cwl-summary-to-galaxy-template",
"freeform-summary-to-galaxy-data-flow",
"freeform-summary-to-galaxy-interface",
"freeform-summary-to-galaxy-template",
"implement-galaxy-tool-step",
"nextflow-summary-to-galaxy-data-flow",
"nextflow-summary-to-galaxy-interface",
"nextflow-summary-to-galaxy-reference-data",
"nextflow-summary-to-galaxy-template",
"repair-galaxy-draft-topology"
]
}Carry the open-requirements ledger: read open entries bearing on this step's decisions, mark resolved the ones it closes, and append any new unmet need it surfaces.
references/notes/open-requirements-ledger.mdcontent/research/open-requirements-ledger.md---
type: research
subtype: design-spec
title: "Open-requirements ledger"
tags:
- research/design-spec
- target/galaxy
status: draft
created: 2026-06-16
revised: 2026-06-16
revision: 1
ai_generated: true
related_notes:
- "[[galaxy-workflow-draft-format]]"
related_molds:
- "[[advance-galaxy-draft-step]]"
- "[[repair-galaxy-draft-topology]]"
- "[[implement-galaxy-tool-step]]"
summary: "Carried unresolved-requirements artifact the source→Galaxy pipeline discharges or explicitly surrenders, autonomously."
---
# Open-requirements ledger
The `open-requirements-ledger` is a single artifact threaded through the source→Galaxy pipeline that records **obligations the pipeline has taken on but not yet met** — a declared output with no producer, a parameter whose value the source never pinned, a tool with no corpus exemplar. Each Mold that surfaces one **appends** it; each Mold whose decision closes one **marks it resolved**; the terminal path **surrenders** whatever remains open, explicitly, into the final artifact.
## Framing: obligations the pipeline discharges, not questions a human answers
This is deliberately *not* an "open questions for the user" list. The pipeline is autonomous — no human-in-the-loop gate is assumed. The ledger's consumers are **Molds and the loop's convergence gate**, with human readout a secondary affordance. An entry is closed by a downstream Mold doing work (wiring a producer, picking a wrapper, settling a value), or — when nothing can close it — surrendered: written into the final draft as a known, labelled gap rather than silently dropped or fabricated around.
The distinction matters because a "questions for a human" framing leaks an operator's personal interaction style into a tool meant to run inside anyone's harness. The ledger must
...Bind the abstract step against the deterministic tool summary manifest emitted upstream — read `parsed_tool` for ports/datatypes and `input_schemas.workflow_step_linked` for valid step `tool_state` shape.
references/schemas/galaxy-tool-summary.schema.jsonpackage://@galaxy-foundry/foundry#galaxyToolSummarySchema{
"$schema": "http://json-schema.org/draft-07/schema#",
"$id": "https://galaxyproject.org/foundry/schemas/galaxy-tool-summary.schema.json",
"$comment": "Canonical source: packages/foundry/src/schemas/galaxy-tool-summary/galaxy-tool-summary.schema.json in galaxyproject/foundry. Mold frontmatter cites this as `content/schemas/galaxy-tool-summary.schema.json` for stability; the cast pipeline and Astro site resolve that ref back to this file at build time. The `$defs.ParsedTool` definition is a placeholder that the AJV validator replaces at compile time with the upstream `parsedToolSchema` from `@galaxy-tool-util/schema`. Static consumers that do not use the validator should consult `[[parsed-tool]]` for the upstream definition.",
"title": "Galaxy Tool Summary",
"description": "Deterministic per-tool manifest emitted by `galaxy-tool-cache summarize`. Bundles cache provenance, the upstream `ParsedTool` payload, and per-representation input JSON Schemas so downstream Molds can bind a single workflow step against a Galaxy tool wrapper without re-parsing XML.",
"$ref": "#/$defs/GalaxyToolSummary",
"$defs": {
"GalaxyToolSummary": {
"title": "GalaxyToolSummary",
"description": "Top-level shape. Every Galaxy tool summary is exactly this object.",
"type": "object",
"additionalProperties": false,
"required": [
"schema_version",
"tool_id",
"tool_version",
"cache_key",
"source",
"artifacts",
"parsed_tool",
"input_schemas",
"warnings"
],
"properties": {
"schema_version": {
"type": "integer",
"const": 1,
"description": "Manifest schema version. v1 is the shape emitted by `galaxy-tool-cache summarize` in `@galaxy-tool-util/cli@1.3.x`."
},
"tool_id": {
"type": "string",
"minLength": 1,
"description": "Tool id as it appears in the cached `ParsedTool`. For Tool Shed wrappers this is the short XML id, not the fully qualified `toolshed.g2.bx.psu.edu/repos/...` form."
},
"tool_version": {
"type": [
"string",
"null"
],
"description": "Tool version from the cached `ParsedTool`. Null only when the wrapper itself omits a version."
},
"cache_key": {
"type": "string",
"mi
...In/out contract: the draft this Mold reads and mutates in place conforms to [[galaxy-workflow-draft]]. Cast bundles the JSON Schema alongside the [[draft-validate]] CLI checks.
references/schemas/galaxy-workflow-draft.schema.jsonpackage://@galaxy-tool-util/schema#galaxyWorkflowDraftJsonSchema{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$defs": {
"WorkflowStepSchema": {
"type": "object",
"required": [],
"properties": {
"id": {
"anyOf": [
{
"type": "null"
},
{
"type": "string"
}
]
},
"label": {
"anyOf": [
{
"type": "null"
},
{
"type": "string"
}
]
},
"doc": {
"anyOf": [
{
"type": "null"
},
{
"type": "string"
},
{
"type": "array",
"items": {
"type": "string"
}
}
]
},
"position": {
"anyOf": [
{
"type": "null"
},
{
"type": "object",
"required": [
"top",
"left"
],
"properties": {
"top": {
"type": "number"
},
"left": {
"type": "number"
}
},
"additionalProperties": false
}
]
},
"tool_id": {
"anyOf": [
{
"type": "null"
},
{
"type": "string"
}
]
},
"tool_shed_repository": {
"$ref": "#/$defs/Shared15"
},
"tool_version": {
"anyOf": [
{
"type": "null"
},
{
"type": "string"
}
]
},
"errors": {
"anyOf": [
{
"type": "null"
},
{
"type": "string"
}
]
},
"uuid": {
"anyOf": [
{
"type": "null"
},
{
"type": "string"
}
]
},
"in": {
"$ref": "#/$defs/Shared16"
},
"out": {
"anyOf": [
{
"type": "array",
"items": {
"$ref": "#/$defs/Shared4"
...Validate the mutated draft against draft-contract rules; with --concrete, also gate the extracted concrete subset (including the step just implemented) against full gxformat2.
Trigger: After implementing or modifying a concrete tool step in the draft.
references/cli/draft-validate.jsoncontent/cli/gxwf/draft-validate.md{
"type": "cli-command",
"tool": "gxwf",
"command": "draft-validate",
"summary": "Validate a `class: GalaxyWorkflowDraft` workflow against draft-contract rules; with --concrete, also validate the extracted concrete subset.",
"source_path": "content/cli/gxwf/draft-validate.md",
"source_revision": 1,
"package": "@galaxy-tool-util/cli",
"description": "Validate a draft Galaxy workflow (class: GalaxyWorkflowDraft)",
"synopsis": "gxwf draft-validate [options] <file>",
"args": [
{
"raw": "file",
"name": "file",
"required": true,
"variadic": false,
"description": "Draft workflow file (.gxwf.yml)"
}
],
"options": [
{
"flags": "--format <fmt>",
"name": "format",
"description": "Force format: format2 (native is rejected for drafts)",
"takesArgument": true,
"argumentPlaceholder": "<fmt>",
"optionalArgument": false,
"negatable": false
},
{
"flags": "--json",
"name": "json",
"description": "Output structured JSON report",
"takesArgument": false,
"optionalArgument": false,
"negatable": false
},
{
"flags": "--report-html [file]",
"name": "reportHtml",
"description": "Write HTML report to file (or stdout if omitted)",
"takesArgument": true,
"argumentPlaceholder": "[file]",
"optionalArgument": true,
"negatable": false
},
{
"flags": "--report-markdown [file]",
"name": "reportMarkdown",
"description": "Write Markdown report to file (or stdout if omitted)",
"takesArgument": true,
"argumentPlaceholder": "[file]",
"optionalArgument": true,
"negatable": false
},
{
"flags": "--concrete",
"name": "concrete",
"description": "Additionally extract the concrete subset and run the regular `gxwf validate` checks on it",
"takesArgument": false,
"optionalArgument": false,
"negatable": false
},
{
"flags": "--cache-dir <dir>",
"name": "cacheDir",
"description": "Tool cache directory (for --concrete tool-state validation)",
"takesArgument": true,
"argumentPlaceholder": "<dir>",
"optionalArgument": false,
"negatable": false
},
{
"flags": "--no-tool-state",
"name": "toolState",
"description": "Skip tool-state validation in the --c
...Implement identifier-derived collection reshaping via Apply Rules.
Trigger: When collection element identifiers need regex parsing, nesting-level swaps, regrouping, or paired identifier assignment.
references/notes/galaxy-apply-rules-dsl.mdcontent/research/galaxy-apply-rules-dsl.md---
type: research
subtype: component
title: "Galaxy Apply Rules DSL"
tags:
- research/component
- target/galaxy
status: draft
created: 2026-04-30
revised: 2026-05-02
revision: 2
ai_generated: true
related_notes:
- "[[galaxy-collection-tools]]"
- "[[galaxy-collection-semantics]]"
- "[[nextflow-to-galaxy-channel-shape-mapping]]"
- "[[nextflow-operators-to-galaxy-collection-recipes]]"
- "[[collection-build-list-paired-with-apply-rules]]"
- "[[collection-build-named-bundle]]"
- "[[collection-cleanup-after-mapover-failure]]"
- "[[collection-flatten-after-fanout]]"
- "[[collection-split-identifier-via-rules]]"
- "[[collection-swap-nesting-with-apply-rules]]"
- "[[collection-unbox-singleton]]"
- "[[relabel-via-rules-and-find-replace]]"
- "[[iwc-transformations-survey]]"
sources:
- "https://github.com/jmchilton/galaxy-agentic-collection-transform (initial research seed)"
- "https://github.com/galaxyproject/galaxy/blob/main/lib/galaxy/util/rules_dsl.py"
- "https://github.com/galaxyproject/galaxy/blob/main/lib/galaxy/util/rules_dsl_spec.yml"
summary: "Reference for Galaxy's Apply Rules DSL: rule operations, mapping operations, composition patterns, pitfalls."
---
Reference for Galaxy's Apply Rules DSL — the rule grammar consumed by `__APPLY_RULES__` (see [[galaxy-collection-tools]] for the surrounding tool catalog and [[galaxy-collection-semantics]] for collection mapping/reduction semantics).
**Key principle:** rules transform collection metadata (identifiers, indices, tags) as tabular data; mapping operations turn the resulting columns back into collection structure.
**Sources of truth in Galaxy:**
- `lib/galaxy/util/rules_dsl.py` — rule implementation
- `lib/galaxy/util/rules_dsl_spec.yml` — test spec covering every rule type
- `lib/gala
...Connect concrete Galaxy tool inputs/outputs while preserving collection mapping and reduction semantics.
Trigger: When implementing a step with data_collection inputs, mapped outputs, reductions, or nested collection wiring.
references/notes/galaxy-collection-semantics.mdcontent/research/galaxy-collection-semantics.md---
type: research
subtype: component
title: "Galaxy collection semantics"
tags:
- research/component
- target/galaxy
status: draft
created: 2026-04-30
revised: 2026-05-05
revision: 3
ai_generated: false
related_notes:
- "[[galaxy-xsd]]"
- "[[galaxy-collection-tools]]"
- "[[galaxy-apply-rules-dsl]]"
- "[[nextflow-to-galaxy-channel-shape-mapping]]"
- "[[nextflow-operators-to-galaxy-collection-recipes]]"
- "[[galaxy-tool-job-failure-reference]]"
- "[[galaxy-workflow-invocation-failure-reference]]"
- "[[iwc-transformations-survey]]"
- "[[galaxy-discover-datasets]]"
sources:
- "https://github.com/galaxyproject/galaxy/blob/7765fae934fbfdee77e3be5f5b235e43735273ae/lib/galaxy/model/dataset_collections/types/collection_semantics.yml"
companions:
- "galaxy-collection-semantics.yml"
- "galaxy-collection-semantics.upstream.myst"
summary: "Vendored formal spec of Galaxy dataset-collection mapping/reduction semantics, with labeled examples and pinned test references."
---
> **Vendored from upstream**, pinned at SHA `7765fae`. Two files live next to this note:
>
> - `galaxy-collection-semantics.yml` — the structured source. **Agents and casting should consume this.** It carries the `tests:` blocks that pin concrete Galaxy test names; the rendered upstream view drops them.
> - `galaxy-collection-semantics.upstream.myst` — Galaxy's auto-generated MyST/LaTeX rendering of the YAML, vendored only so the human view below has something to render. Sync is manual.
>
> **When to consult:** authoring or reasoning about Molds and patterns that touch `data_collection` inputs, map-over / reduction shape changes, sub-collection mapping, `paired_or_unpaired`, or `sample_sheet`.
```vendored-myst
file: galaxy-collection-semantics.upstream.myst
source: https://github.com/g
...Connect concrete Galaxy tool inputs/outputs while preserving collection mapping and reduction semantics.
Trigger: When implementing a step with data_collection inputs, mapped outputs, reductions, or nested collection wiring.
references/notes/galaxy-collection-semantics.upstream.mystcontent/research/galaxy-collection-semantics.upstream.myst# Collection Semantics
This document describes the semantics around working with Galaxy dataset collections.
In particular it describes how they operate within Galaxy tools and workflows.
:::{admonition} You Probably Don't Need to Read This
:class: caution
Any significantly sophisticated workflow language will have ways to collect data
into arrays or vectors or dictionaries and apply operations across this data (mapping)
or reduce the dimensionality of this data (reductions). Typically, this is explicitly
annotated with map functions or for loops. Galaxy however is designed to be a point
and click interface for connecting steps and running tools. It is important that steps
just connect and just do the most natural thing - and this is what Galaxy does.
This document just provides a mathematical formalism to that "what should just
intuitively work" that can be used to document test cases and help with implementation.
This is reference documentation not user documentation, Galaxy should just work.
:::
## Mapping
If a tool consumes a simple dataset parameter and produces a simple dataset parameter,
then any collection type may be "mapped over" the data input to that tool. The result of
that is the tool being applied to each element of the collection and "implicit collections"
being created from the outputs that are produced from those operations. Those implicit
collections have the same element identifiers in the same order as the input collection that is
mapped over. Each element of the implicit collections correspond to their own job and
Galaxy very naturally and intuitively parallelizes jobs without extra work from the user
and without any knowledge of the tool.
(BASIC_MAPPING_PAIRED)=
(BASIC_MAPPING_PAIRED_OR_UNPAIRED_PAIRED)=
(BASIC_MAPPING_PAIRED_OR_UNPAIRED_UN
...Connect concrete Galaxy tool inputs/outputs while preserving collection mapping and reduction semantics.
Trigger: When implementing a step with data_collection inputs, mapped outputs, reductions, or nested collection wiring.
references/notes/galaxy-collection-semantics.ymlcontent/research/galaxy-collection-semantics.yml- doc: |
# Collection Semantics
This document describes the semantics around working with Galaxy dataset collections.
In particular it describes how they operate within Galaxy tools and workflows.
:::{admonition} You Probably Don't Need to Read This
:class: caution
Any significantly sophisticated workflow language will have ways to collect data
into arrays or vectors or dictionaries and apply operations across this data (mapping)
or reduce the dimensionality of this data (reductions). Typically, this is explicitly
annotated with map functions or for loops. Galaxy however is designed to be a point
and click interface for connecting steps and running tools. It is important that steps
just connect and just do the most natural thing - and this is what Galaxy does.
This document just provides a mathematical formalism to that "what should just
intuitively work" that can be used to document test cases and help with implementation.
This is reference documentation not user documentation, Galaxy should just work.
:::
## Mapping
If a tool consumes a simple dataset parameter and produces a simple dataset parameter,
then any collection type may be "mapped over" the data input to that tool. The result of
that is the tool being applied to each element of the collection and "implicit collections"
being created from the outputs that are produced from those operations. Those implicit
collections have the same element identifiers in the same order as the input collection that is
mapped over. Each element of the implicit collections correspond to their own job and
Galaxy very naturally and intuitively parallelizes jobs without extra work from the user
and without any knowledge of the tool.
...Insert built-in Galaxy collection-operation steps when a direct tool connection cannot express the needed shape.
Trigger: When a step needs collection construction, filtering, extraction, zipping, unzipping, flattening, merging, or relabeling.
references/notes/galaxy-collection-tools.mdcontent/research/galaxy-collection-tools.md---
type: research
subtype: component
title: "Galaxy collection-operation tools"
tags:
- research/component
- target/galaxy
status: draft
created: 2026-04-30
revised: 2026-05-02
revision: 2
ai_generated: true
related_notes:
- "[[galaxy-collection-semantics]]"
- "[[galaxy-apply-rules-dsl]]"
- "[[nextflow-to-galaxy-channel-shape-mapping]]"
- "[[nextflow-operators-to-galaxy-collection-recipes]]"
- "[[iwc-transformations-survey]]"
sources:
- "https://github.com/jmchilton/galaxy-agentic-collection-transform (initial research seed)"
- "https://github.com/galaxyproject/galaxy/tree/main/lib/galaxy/tools (XML wrappers; source of truth)"
summary: "Catalog of Galaxy's collection-operation tools — purpose, IO, parameters, selection guide. Companion to galaxy-collection-semantics."
---
Catalog of Galaxy's built-in collection-operation tools (the `__BUILD_LIST__`, `__FILTER_FROM_FILE__`, `__APPLY_RULES__`, … family) — what each tool does, its inputs and outputs, and when to reach for it. Source of truth is the Galaxy XML wrappers under `lib/galaxy/tools/`; this is the high-level catalog. Pairs with [[galaxy-collection-semantics]], which describes the underlying mapping and reduction semantics rather than the user-facing tools.
These are **model operations** — they manipulate collection structure without processing file contents, so they're fast and don't grow storage.
## Tool Categories
### 1. Collection Creation Tools
#### Build List (`__BUILD_LIST__`)
**Version:** 1.2.0
**Purpose:** Build a new list collection from individual datasets or collections.
**Inputs:**
- `datasets` (repeat): Input datasets or collections
- `input`: Data input (optional)
- `id_cond/id_select`: Label selection method
- `idx`: Use index (0, 1, 2...)
- `identifier`: Use da
...Preserve concrete tool/job failure evidence while implementing step labels, tool ids, output labels, and collection wiring.
Trigger: When a selected wrapper has explicit failure semantics, dynamic outputs, non-default stdio rules, strict-shell behavior, or runtime-only failure risk.
references/notes/galaxy-tool-job-failure-reference.mdcontent/research/galaxy-tool-job-failure-reference.md---
type: research
subtype: component
title: "Galaxy tool and job failure reference"
tags:
- research/component
- target/galaxy
status: draft
created: 2026-05-02
revised: 2026-05-02
revision: 1
ai_generated: true
related_notes:
- "[[galaxy-workflow-invocation-failure-reference]]"
- "[[planemo-workflow-test-architecture]]"
- "[[galaxy-collection-semantics]]"
related_molds:
- "[[implement-galaxy-tool-step]]"
- "[[debug-galaxy-workflow-output]]"
sources:
- "~/projects/repositories/galaxy/lib/galaxy/tools/__init__.py"
- "~/projects/repositories/galaxy/lib/galaxy/tool_util/parser/xml.py"
- "~/projects/repositories/galaxy/lib/galaxy/tool_util/output_checker.py"
- "~/projects/repositories/galaxy/lib/galaxy/jobs"
- "~/projects/repositories/galaxy/lib/galaxy/webapps/galaxy/api/jobs.py"
summary: "Reference for Galaxy tool stdio rules, job failure detection, job states, and job API failure surfaces."
---
# Galaxy Tool And Job Failure Reference
This is reference material, not a debug recipe. Use it to understand what Galaxy can know about a failed tool job and which API surfaces preserve that evidence.
## Model
Galaxy tool failure handling is layered:
- The tool wrapper defines expected failure semantics through `detect_errors`, `<stdio>`, exit-code checks, regex checks, and command strictness.
- The job runner executes the command and captures exit code plus tool/job stdout and stderr streams.
- Galaxy evaluates configured failure rules and records structured `job_messages`.
- The job reaches a terminal state, output datasets may become `error`, and dependent jobs may pause or fail later.
- Workflow invocation APIs summarize those jobs, but job APIs preserve the most detailed tool-level evidence.
## Tool Wrapper Failure Controls
Important wrapper con
...Preserve testable output labels and collection element identifiers while replacing abstract steps with concrete gxformat2 steps.
Trigger: When a concrete step changes output labels, emits collection outputs, creates a diagnostic checkpoint, or makes a final output too weakly assertable.
references/notes/galaxy-workflow-testability-design.mdcontent/research/galaxy-workflow-testability-design.md---
type: research
subtype: component
tags:
- research/component
- target/galaxy
status: draft
created: 2026-05-03
revised: 2026-05-06
revision: 2
ai_generated: true
related_notes:
- "[[iwc-workflow-testability-survey]]"
- "[[iwc-test-data-conventions]]"
- "[[planemo-asserts-idioms]]"
- "[[iwc-shortcuts-anti-patterns]]"
- "[[planemo-workflow-test-architecture]]"
- "[[implement-galaxy-workflow-test]]"
- "[[gxformat2-schema]]"
- "[[gxformat2-workflow-inputs]]"
- "[[galaxy-datatypes-conf]]"
summary: "Design guidance for Galaxy workflow inputs, outputs, and checkpoints that make IWC-style workflow tests possible."
---
# Galaxy workflow testability design
Use this note when authoring or translating a Galaxy workflow **before** the `-tests.yml` file exists. It covers workflow structure choices that make later IWC-style tests meaningful: labels, promoted checkpoints, collection identifiers, and fixture-compatible inputs.
This is not a `content/patterns/` page. It is cross-cutting design guidance for Molds that need testable Galaxy workflows. Assertion syntax lives in [[planemo-asserts-idioms]]. Test YAML fixture shapes live in [[iwc-test-data-conventions]]. Accepted shortcut vs smell calls live in [[iwc-shortcuts-anti-patterns]]. Corpus evidence trail lives in [[iwc-workflow-testability-survey]].
## 1. Treat labels as API
Workflow input and output labels are not cosmetic. Planemo and IWC tests address workflow inputs and outputs by label, and the survey found exact label matches for every asserted output across 114 matched workflow/test pairs. A generated workflow should therefore pick stable, descriptive labels before test authoring starts.
Rules:
- Label every output that may need a test assertion.
- Treat input/output renames as breaking changes
...Turn operator-derived abstract transforms into concrete Galaxy wiring, collection operations, or review requests.
Trigger: When a concrete step implements behavior traced to map, join, groupTuple, branch, mix, combine, or multiMap.
references/notes/nextflow-operators-to-galaxy-collection-recipes.mdcontent/research/nextflow-operators-to-galaxy-collection-recipes.md---
type: research
subtype: component
title: "Nextflow operators to Galaxy collection recipes"
tags:
- research/component
- source/nextflow
- target/galaxy
status: draft
created: 2026-05-02
revised: 2026-05-02
revision: 1
ai_generated: true
related_notes:
- "[[nextflow-to-galaxy-channel-shape-mapping]]"
- "[[galaxy-collection-semantics]]"
- "[[galaxy-collection-tools]]"
- "[[galaxy-apply-rules-dsl]]"
- "[[iwc-transformations-survey]]"
- "[[iwc-tabular-operations-survey]]"
- "[[galaxy-data-flow-draft-contract]]"
- "[[iwc-map-over-lifecycle-survey]]"
- "[[nextflow-patterns]]"
related_molds:
- "[[nextflow-summary-to-galaxy-data-flow]]"
- "[[implement-galaxy-tool-step]]"
- "[[debug-galaxy-workflow-output]]"
sources:
- "https://github.com/galaxyproject/foundry/issues/53"
summary: "Classifies common Nextflow operators as Galaxy wiring, collection semantics, explicit steps, or review triggers."
---
# Nextflow Operators To Galaxy Collection Recipes
Most Nextflow operators are not Galaxy tools. Translate them first as source-side data-flow intent, then decide whether the Galaxy representation is simple wiring, collection semantics, an explicit Galaxy step, or a user-review checkpoint.
## Decision Vocabulary
| Label | Meaning |
|---|---|
| `channel-only rewiring` | The operator disappears into Galaxy connections, labels, branch wiring, or output selection. |
| `Galaxy collection semantics` | Translation relies on collection identifiers, collection type, map-over, reduction, or nesting behavior. |
| `explicit Galaxy step` | Add a collection-operation, tabular, text-processing, or domain tool step. |
| `user review` | Translation is likely lossy or semantically ambiguous. |
## Operator Recipes
| Nextflow operator | Galaxy recipe | Class | Confi
...Check whether a concrete tool input/output can preserve the intended source-derived Galaxy collection shape.
Trigger: When implementing concrete steps for source-derived File/list/paired/list:paired/list:list inputs or outputs.
references/notes/nextflow-to-galaxy-channel-shape-mapping.mdcontent/research/nextflow-to-galaxy-channel-shape-mapping.md---
type: research
subtype: component
title: "Nextflow-to-Galaxy channel shape mapping"
tags:
- research/component
- source/nextflow
- target/galaxy
status: draft
created: 2026-05-02
revised: 2026-05-06
revision: 2
ai_generated: true
related_notes:
- "[[nextflow-workflow-io-semantics]]"
- "[[nextflow-params-to-galaxy-inputs]]"
- "[[nextflow-path-glob-to-galaxy-datatype]]"
- "[[galaxy-collection-semantics]]"
- "[[galaxy-collection-tools]]"
- "[[galaxy-apply-rules-dsl]]"
- "[[iwc-transformations-survey]]"
- "[[nextflow-operators-to-galaxy-collection-recipes]]"
- "[[galaxy-data-flow-draft-contract]]"
- "[[iwc-conditionals-survey]]"
- "[[manifest-to-mapped-collection-lifecycle]]"
- "[[map-workflow-enum-to-tool-parameter]]"
- "[[regex-relabel-via-tabular]]"
- "[[relabel-via-rules-and-find-replace]]"
- "[[reshape-relabel-remap-by-collection-axis]]"
- "[[sync-collections-by-identifier]]"
- "[[tabular-compute-new-column]]"
- "[[tabular-concatenate-collection-to-table]]"
- "[[tabular-cut-and-reorder-columns]]"
- "[[tabular-filter-by-column-value]]"
- "[[tabular-filter-by-regex]]"
- "[[tabular-group-and-aggregate-with-datamash]]"
- "[[tabular-join-on-key]]"
- "[[tabular-pivot-collection-to-wide]]"
- "[[tabular-prepend-header]]"
- "[[tabular-relabel-by-row-counter]]"
- "[[tabular-split-taxonomy-string]]"
- "[[tabular-sql-query]]"
- "[[tabular-synthesize-bed-from-3col]]"
- "[[tabular-to-collection-by-row]]"
- "[[iwc-map-over-lifecycle-survey]]"
- "[[nextflow-patterns]]"
related_molds:
- "[[nextflow-summary-to-galaxy-interface]]"
- "[[nextflow-summary-to-galaxy-data-flow]]"
- "[[nextflow-summary-to-galaxy-template]]"
- "[[implement-galaxy-tool-step]]"
sources:
- "https://github.com/galaxyproject/foundry/
...
# implement-galaxy-tool-step
Follow the procedure below and use the artifact/reference sections as the runtime contract.
## When To Use
- Convert an abstract step into a concrete gxformat2 step using a tool summary.
## Inputs
- Read artifact `galaxy-tool-summary`. Schema: galaxy-tool-summary. Produced by `summarize-galaxy-tool`. Galaxy tool summary manifest from summarize-galaxy-tool conforming to galaxy-tool-summary; binds the abstract step to a concrete tool's ports via the embedded `parsed_tool` and generated `input_schemas`.
- Read artifact `galaxy-workflow-draft`. Schema: galaxy-workflow-draft. Produced by `advance-galaxy-draft-step`, `cwl-summary-to-galaxy-template`, `freeform-summary-to-galaxy-template`, `implement-galaxy-tool-step`, `nextflow-summary-to-galaxy-template`, `repair-galaxy-draft-topology`. gxformat2 skeleton being filled in step by step; the step replaces a placeholder in this draft.
- Read artifact `open-requirements-ledger`. Produced by `advance-galaxy-draft-step`, `compare-against-iwc-exemplar`, `cwl-summary-to-galaxy-data-flow`, `cwl-summary-to-galaxy-interface`, `cwl-summary-to-galaxy-template`, `freeform-summary-to-galaxy-data-flow`, `freeform-summary-to-galaxy-interface`, `freeform-summary-to-galaxy-template`, `implement-galaxy-tool-step`, `nextflow-summary-to-galaxy-data-flow`, `nextflow-summary-to-galaxy-interface`, `nextflow-summary-to-galaxy-reference-data`, `nextflow-summary-to-galaxy-template`, `repair-galaxy-draft-topology`. Carried obligations ledger open-requirements-ledger: read entries the design tier recorded against this step before implementing; append a blocking entry if its output can't be computed.
## Outputs
- Write artifact `galaxy-workflow-draft` as `galaxy-workflow-draft.gxwf.yml`. Format: `yaml`. Schema: galaxy-workflow-draft. gxformat2 skeleton with one more abstract step replaced by a concrete tool step (loop iteration output).
- Write artifact `open-requirements-ledger` as `open-requirements.ledger.yml`. Format: `yaml`. Ledger (open-requirements-ledger) with a blocking entry appended when a step's declared output can't be computed from its wired inputs; drives the fall-through to topology repair.
## Required Tools
- **`gxwf`** (gxwf). `npm install -g @galaxy-tool-util/cli@^1.8.1`.
Ephemeral run: `npx --yes --package @galaxy-tool-util/cli@1.8.1 gxwf`.
Check: `gxwf --help | grep -q draft-validate`.
Docs: https://github.com/jmchilton/galaxy-tool-util-ts/tree/main/packages/cli
## Load Upfront
- `references/notes/open-requirements-ledger.md`: Research note copied verbatim into the bundle. Carry the open-requirements ledger: read open entries bearing on this step's decisions, mark resolved the ones it closes, and append any new unmet need it surfaces.
- `references/schemas/galaxy-tool-summary.schema.json`: Schema file copied verbatim into the bundle. Bind the abstract step against the deterministic tool summary manifest emitted upstream — read `parsed_tool` for ports/datatypes and `input_schemas.workflow_step_linked` for valid step `tool_state` shape.
- `references/schemas/galaxy-workflow-draft.schema.json`: Schema file copied verbatim into the bundle. In/out contract: the draft this Mold reads and mutates in place conforms to galaxy-workflow-draft. Cast bundles the JSON Schema alongside the draft-validate CLI checks.
## Load On Demand
- `references/cli/draft-validate.json`: CLI command reference packaged as a sidecar. Validate the mutated draft against draft-contract rules; with --concrete, also gate the extracted concrete subset (including the step just implemented) against full gxformat2. Use when: after implementing or modifying a concrete tool step in the draft.
- `references/notes/galaxy-apply-rules-dsl.md`: Research note copied verbatim into the bundle. Implement identifier-derived collection reshaping via Apply Rules. Use when: collection element identifiers need regex parsing, nesting-level swaps, regrouping, or paired identifier assignment.
- `references/notes/galaxy-collection-semantics.md`: Research note copied verbatim into the bundle. Connect concrete Galaxy tool inputs/outputs while preserving collection mapping and reduction semantics. Use when: implementing a step with data_collection inputs, mapped outputs, reductions, or nested collection wiring.
- `references/notes/galaxy-collection-semantics.upstream.myst`: Companion file copied verbatim into the bundle. Sibling of `references/notes/galaxy-collection-semantics.md`; read it where that note directs.
- `references/notes/galaxy-collection-semantics.yml`: Companion file copied verbatim into the bundle. Sibling of `references/notes/galaxy-collection-semantics.md`; read it where that note directs.
- `references/notes/galaxy-collection-tools.md`: Research note copied verbatim into the bundle. Insert built-in Galaxy collection-operation steps when a direct tool connection cannot express the needed shape. Use when: a step needs collection construction, filtering, extraction, zipping, unzipping, flattening, merging, or relabeling.
- `references/notes/galaxy-tool-job-failure-reference.md`: Research note copied verbatim into the bundle. Preserve concrete tool/job failure evidence while implementing step labels, tool ids, output labels, and collection wiring. Use when: a selected wrapper has explicit failure semantics, dynamic outputs, non-default stdio rules, strict-shell behavior, or runtime-only failure risk.
- `references/notes/galaxy-workflow-testability-design.md`: Research note copied verbatim into the bundle. Preserve testable output labels and collection element identifiers while replacing abstract steps with concrete gxformat2 steps. Use when: a concrete step changes output labels, emits collection outputs, creates a diagnostic checkpoint, or makes a final output too weakly assertable.
- `references/notes/nextflow-operators-to-galaxy-collection-recipes.md`: Research note copied verbatim into the bundle. Turn operator-derived abstract transforms into concrete Galaxy wiring, collection operations, or review requests. Use when: a concrete step implements behavior traced to map, join, groupTuple, branch, mix, combine, or multiMap.
- `references/notes/nextflow-to-galaxy-channel-shape-mapping.md`: Research note copied verbatim into the bundle. Check whether a concrete tool input/output can preserve the intended source-derived Galaxy collection shape. Use when: implementing concrete steps for source-derived File/list/paired/list:paired/list:list inputs or outputs.
## Validation
- Validate `galaxy-workflow-draft.gxwf.yml` for artifact `galaxy-workflow-draft` against the galaxy-workflow-draft schema when a validator is available.
## Procedure
Replace one abstract step in the gxformat2 draft with a concrete tool step, using the upstream tool summary. One invocation resolves exactly the chosen step's `TODO_*` / `_plan_*` slots into a concrete `tool_id`, `tool_version`, `tool_state`, and wrapper-determined port names, and returns the mutated draft. This is the "Implement" leaf of the per-step loop owned by advance-galaxy-draft-step.
Single step in scope. This skill owns the chosen step and the wiring that connects it to ports already in the draft. It does not redesign topology and does not unwind earlier iterations — cross-step rework is the orchestrator's call.
### Sequence
1. **Read the step's plan.** From the galaxy-workflow-draft, take the chosen step's deferred evidence: `_plan_state`, `_plan_context`, `_plan_in`, `_plan_out`, and any `TODO_*` slots the template or data-flow brief left for this phase.
2. **Bind to the tool summary.** Read the galaxy-tool-summary manifest: `parsed_tool` gives concrete input/output port names and datatypes; shape the step's `tool_state` against `input_schemas.workflow_step_linked`. Set `tool_version`, and set `tool_id` — confirming or correcting an identity-pinned id rather than re-deriving a good pin from scratch. For a built-in/stock tool the `tool_id` is the bare id (`Filter1`, `Cut1`, collection ops) and `tool_version` must come from the summary's cached pin — never invent a stock version; the summary already resolved it against the shed via summarize-galaxy-tool. If `input_schemas` is `null`, consult `warnings[]` for why before binding by hand.
3. **Wire ports.** Connect the step's inputs to their upstream producers and its outputs to downstream consumers per the `_plan_in` / `_plan_out` intent, using real wrapper port names. Preserve collection mapping and reduction semantics (galaxy-collection-semantics); for a source-derived shape, check the chosen input/output can actually carry the intended File / list / paired / list:paired shape (nextflow-to-galaxy-channel-shape-mapping).
4. **Close shape gaps.** When a direct tool connection cannot express the needed shape, insert a built-in collection-operation step (galaxy-collection-tools); for identifier-derived reshaping — regex parsing, nesting swaps, paired assignment — use Apply Rules (galaxy-apply-rules-dsl); for a transform traced to a Nextflow operator (map, join, groupTuple, branch, mix, combine, multiMap), turn it into concrete wiring or a review request via nextflow-operators-to-galaxy-collection-recipes. A built-in step inserted here is itself a stock tool: resolve its concrete `tool_version` through the galaxy-tool-cache flow (summarize-galaxy-tool on the bare id) rather than guessing — or leave it draft-tier for the next loop iteration to realize.
5. **Preserve testability.** Keep output labels and collection element identifiers stable and addressable (galaxy-workflow-testability-design). Do not rename a labeled output, drop a checkpoint, or make a final output too weakly assertable just to satisfy this step's wiring.
6. **Validate.** Run draft-validate `--concrete` over the mutated draft: it checks draft-contract rules and gates the extracted concrete subset — including the step just implemented — against full gxformat2. On green, return the draft for the next loop iteration; on red, route the diagnostic back to whichever decision above it implicates.
### Failure ownership
When the wrapper can't cleanly carry what the plan needs — wrong datatype, missing parameter, unsupported collection shape — record where a later failure should be investigated: tool/job failure, data-flow mistake, template wiring mistake, wrapper mismatch, or test/assertion issue. Consult galaxy-tool-job-failure-reference when the selected wrapper has explicit failure semantics (exit-code rules, stdio regex, strict-shell, dynamic outputs); implement the step's labels and wiring so that evidence survives to runtime rather than being erased by the concretization.
When the step's declared output cannot be computed from the inputs wired to it — the connection graph says the ports connect, but no wired input carries the evidence the output needs — do not fabricate it. Append a blocking entry to the open-requirements-ledger naming the step, the uncomputable output, and the missing evidence, and fall through to repair-galaxy-draft-topology rather than emitting a step that validates but can't run. A missing Tool Shed wrapper is not this case; that gap routes through the discover-or-author branch.
## Runtime Notes
- Do not read Foundry source files at runtime; use only files packaged in this skill bundle and user-supplied artifacts.
- Preserve declared artifact filenames unless the user or harness supplies explicit paths.
- Carry unresolved assumptions into the output artifact instead of silently inventing missing source evidence.