implement-galaxy-tool-step

---
type: research
title: "Open-requirements ledger"
tags:
  - target/galaxy
status: draft
created: 2026-06-16
revised: 2026-11-06
revision: 6
related_notes:
  - "[[galaxy-workflow-draft-format]]"
related_molds:
  - "[[advance-galaxy-draft-step]]"
  - "[[repair-galaxy-draft-topology]]"
  - "[[implement-galaxy-tool-step]]"
summary: "Carried workflow-run knowledge artifact for obligations, discoveries, decisions, brief-change recommendations, and convergence state."
---

# Open-requirements ledger

The `open-requirements-ledger` is the carried workflow-run knowledge artifact threaded through the source→Galaxy pipeline. It records **obligations the pipeline has taken on but not yet met** — a declared output with no producer, a parameter whose value the source never pinned, a tool with no corpus exemplar — and **source work it decided not to carry**. It also records evidence discovered during implementation, decisions made within the approved scope, and recommended changes to an immutable Workflow Brief. Each Mold that surfaces an obligation **appends** it; each Mold whose decision closes one **marks it resolved**; the terminal path **surrenders** whatever remains open, explicitly, into the final artifact.

This ledger is about the workflow being built. It is distinct from the opt-in `foundry-feedback-ledger`, which records actionable defects or friction in the Foundry assets building it. It is also not the harness's phase-progress tracker: execution status belongs to harness state, checkpoints, and reports rather than requirement entries.

## Framing: obligations the pipeline discharges, not questions a human answers

This is deliberately *not* an "open questions for the user" list. The pipeline is autonomous — no human-in-the-loop gate is assumed. The ledger's consumers a
...

---
type: research
title: "Galaxy Apply Rules DSL"
tags:
  - target/galaxy
status: draft
created: 2026-04-30
revised: 2026-09-26
revision: 3
related_notes:
  - "[[galaxy-collection-tools]]"
  - "[[galaxy-collection-semantics]]"
  - "[[nextflow-to-galaxy-channel-shape-mapping]]"
  - "[[nextflow-operators-to-galaxy-collection-recipes]]"
  - "[[collection-build-list-paired-with-apply-rules]]"
  - "[[collection-build-named-bundle]]"
  - "[[collection-cleanup-after-mapover-failure]]"
  - "[[collection-flatten-after-fanout]]"
  - "[[collection-split-identifier-via-rules]]"
  - "[[collection-swap-nesting-with-apply-rules]]"
  - "[[collection-unbox-singleton]]"
  - "[[relabel-via-rules-and-find-replace]]"
  - "[[iwc-transformations-survey]]"
sources:
  - "https://github.com/galaxyproject/galaxy/blob/a63da1dfd1960360f4aa2fddc6a75396954d750a/lib/galaxy/util/rules_dsl.py"
  - "https://github.com/galaxyproject/galaxy/blob/a63da1dfd1960360f4aa2fddc6a75396954d750a/lib/galaxy/util/rules_dsl_spec.yml"
  - "https://github.com/galaxyproject/galaxy/blob/a63da1dfd1960360f4aa2fddc6a75396954d750a/lib/galaxy/managers/collections.py"
  - "https://github.com/galaxyproject/galaxy/blob/a63da1dfd1960360f4aa2fddc6a75396954d750a/lib/galaxy/tools/apply_rules.xml"
summary: "How Apply Rules derives columns from collection metadata, filters rows, and maps them into new collection structure."
---

# Galaxy Apply Rules DSL

Galaxy's `__APPLY_RULES__` reorganizes a collection by deriving columns from element identifiers and tags, transforming those columns, and mapping them into a new collection. It can relabel, filter, sort, flatten, or change nesting without processing file contents. The output is named `output`.

Use a dedicated collection tool when it expresses the operation directly, such as flattenin
...

---
type: research
title: "Galaxy collection semantics"
tags:
  - target/galaxy
status: draft
created: 2026-04-30
revised: 2026-05-05
revision: 3
license: MIT
license_file: LICENSES/galaxy.LICENSE
related_notes:
  - "[[galaxy-xsd]]"
  - "[[galaxy-collection-tools]]"
  - "[[galaxy-apply-rules-dsl]]"
  - "[[nextflow-to-galaxy-channel-shape-mapping]]"
  - "[[nextflow-operators-to-galaxy-collection-recipes]]"
  - "[[galaxy-tool-job-failure-reference]]"
  - "[[galaxy-workflow-invocation-failure-reference]]"
  - "[[iwc-transformations-survey]]"
  - "[[galaxy-discover-datasets]]"
sources:
  - "https://github.com/galaxyproject/galaxy/blob/7765fae934fbfdee77e3be5f5b235e43735273ae/lib/galaxy/model/dataset_collections/types/collection_semantics.yml"
companions:
  - "galaxy-collection-semantics.yml"
summary: "Vendored formal spec of Galaxy dataset-collection mapping/reduction semantics, with labeled examples and pinned test references."
---

> **Vendored from upstream**, pinned at SHA `7765fae`. One file travels with this note:
>
> - `galaxy-collection-semantics.yml` — the structured source. **Agents and casting should consume this.** It carries the `tests:` blocks that pin concrete Galaxy test names; the rendered upstream view drops them.
>
> The Foundry also vendors Galaxy's auto-generated MyST/LaTeX rendering of the same YAML, which
> is what the human view below renders from. It is a site asset: casting does not carry it, so
> do not look for it beside this note in a skill bundle. Sync is manual.
>
> **When to consult:** authoring or reasoning about Molds and patterns that touch `data_collection` inputs, map-over / reduction shape changes, sub-collection mapping, `paired_or_unpaired`, or `sample_sheet`.

```vendored-myst
file: galaxy-collection-semantics.upstream.myst
source: h
...

- doc: |
    # Collection Semantics

    This document describes the semantics around working with Galaxy dataset collections.
    In particular it describes how they operate within Galaxy tools and workflows.

    :::{admonition} You Probably Don't Need to Read This
    :class: caution

    Any significantly sophisticated workflow language will have ways to collect data
    into arrays or vectors or dictionaries and apply operations across this data (mapping)
    or reduce the dimensionality of this data (reductions). Typically, this is explicitly
    annotated with map functions or for loops. Galaxy however is designed to be a point
    and click interface for connecting steps and running tools. It is important that steps
    just connect and just do the most natural thing - and this is what Galaxy does.
    This document just provides a mathematical formalism to that "what should just
    intuitively work" that can be used to document test cases and help with implementation.
    This is reference documentation not user documentation, Galaxy should just work.
    :::

    ## Mapping

    If a tool consumes a simple dataset parameter and produces a simple dataset parameter,
    then any collection type may be "mapped over" the data input to that tool. The result of
    that is the tool being applied to each element of the collection and "implicit collections"
    being created from the outputs that are produced from those operations. Those implicit
    collections have the same element identifiers in the same order as the input collection that is
    mapped over. Each element of the implicit collections correspond to their own job and
    Galaxy very naturally and intuitively parallelizes jobs without extra work from the user
    and without any knowledge of the tool.

...

---
type: research
title: "Galaxy collection-operation tools"
tags:
  - target/galaxy
status: draft
created: 2026-04-30
revised: 2026-09-26
revision: 3
related_notes:
  - "[[galaxy-collection-semantics]]"
  - "[[galaxy-apply-rules-dsl]]"
  - "[[nextflow-to-galaxy-channel-shape-mapping]]"
  - "[[nextflow-operators-to-galaxy-collection-recipes]]"
  - "[[iwc-transformations-survey]]"
sources:
  - "https://github.com/galaxyproject/galaxy/tree/a63da1dfd1960360f4aa2fddc6a75396954d750a/lib/galaxy/tools"
  - "https://github.com/galaxyproject/galaxy/blob/a63da1dfd1960360f4aa2fddc6a75396954d750a/lib/galaxy/tools/__init__.py"
  - "https://github.com/galaxyproject/galaxy/blob/a63da1dfd1960360f4aa2fddc6a75396954d750a/lib/galaxy/tools/model_operation_macros.xml"
  - "https://github.com/galaxyproject/galaxy/blob/a63da1dfd1960360f4aa2fddc6a75396954d750a/lib/galaxy/tools/actions/model_operations.py"
summary: "Catalog of Galaxy collection-operation tools, with versioned inputs, outputs, defaults, mapping behavior, and diagnostic qualifications."
---

Galaxy's built-in collection-operation tools assemble, select, reshape, and annotate collections. Their outputs usually copy history dataset associations while sharing the underlying dataset files. They do not rerun scientific analysis or duplicate those files. Some operations read file contents, including identifier tables, tag tables, null markers, and the emptiness check. Creating a large cross product still creates many history objects.

The catalog describes the wrappers and implementations at Galaxy commit `a63da1dfd1960360f4aa2fddc6a75396954d750a`. Versions below are the newest wrappers present at that commit, not a promise that every Galaxy server exposes them. Older wrappers can have different inputs. A workflow's concrete `tool_ver
...

---
type: research
title: "Galaxy tool and job failure reference"
tags:
  - target/galaxy
status: draft
created: 2026-05-02
revised: 2026-09-26
revision: 3
related_notes:
  - "[[galaxy-workflow-invocation-failure-reference]]"
  - "[[planemo-workflow-test-architecture]]"
  - "[[galaxy-collection-semantics]]"
related_molds:
  - "[[implement-galaxy-tool-step]]"
  - "[[debug-galaxy-workflow-output]]"
sources:
  - "https://github.com/galaxyproject/galaxy/blob/a63da1dfd1960360f4aa2fddc6a75396954d750a/lib/galaxy/tool_util/parser/xml.py"
  - "https://github.com/galaxyproject/galaxy/blob/a63da1dfd1960360f4aa2fddc6a75396954d750a/lib/galaxy/tool_util/parser/stdio.py"
  - "https://github.com/galaxyproject/galaxy/blob/a63da1dfd1960360f4aa2fddc6a75396954d750a/lib/galaxy/tool_util/output_checker.py"
  - "https://github.com/galaxyproject/galaxy/blob/a63da1dfd1960360f4aa2fddc6a75396954d750a/lib/galaxy/jobs/command_factory.py"
  - "https://github.com/galaxyproject/galaxy/blob/a63da1dfd1960360f4aa2fddc6a75396954d750a/lib/galaxy/jobs/__init__.py"
  - "https://github.com/galaxyproject/galaxy/blob/a63da1dfd1960360f4aa2fddc6a75396954d750a/lib/galaxy/jobs/runners/__init__.py"
  - "https://github.com/galaxyproject/galaxy/blob/a63da1dfd1960360f4aa2fddc6a75396954d750a/lib/galaxy/model/__init__.py"
  - "https://github.com/galaxyproject/galaxy/blob/a63da1dfd1960360f4aa2fddc6a75396954d750a/lib/galaxy/schema/schema.py"
  - "https://github.com/galaxyproject/galaxy/blob/a63da1dfd1960360f4aa2fddc6a75396954d750a/lib/galaxy/schema/jobs.py"
  - "https://github.com/galaxyproject/galaxy/blob/a63da1dfd1960360f4aa2fddc6a75396954d750a/lib/galaxy/webapps/galaxy/api/jobs.py"
  - "https://github.com/galaxyproject/galaxy/blob/a63da1dfd1960360f4aa2fddc6a75396954d750a/lib/galaxy/webapps/galaxy/services/jobs.py"
  - "
...

---
type: research
title: "Binding a workflow step to a user-defined tool"
tags:
  - target/galaxy
status: draft
created: 2026-09-24
revised: 2026-10-23
revision: 4
related_notes:
  - "[[galaxy-user-tool-authoring-help]]"
  - "[[galaxy-user-tool-authoring]]"
  - "[[galaxy-workflow-draft-format]]"
  - "[[gxformat2-schema]]"
related_molds:
  - "[[author-galaxy-tool-wrapper]]"
  - "[[implement-galaxy-tool-step]]"
  - "[[advance-galaxy-draft-step]]"
sources:
  - "https://github.com/galaxyproject/galaxy/blob/5af1d03c929748006dd53ad451e5cb1fd140399a/client/src/components/Tool/authoringHelp.yml"
  - "https://github.com/galaxyproject/galaxy/blob/5af1d03c929748006dd53ad451e5cb1fd140399a/lib/galaxy/managers/tools.py"
  - "https://github.com/galaxyproject/galaxy/blob/5af1d03c929748006dd53ad451e5cb1fd140399a/lib/galaxy/webapps/galaxy/api/dynamic_tools.py"
  - "https://github.com/galaxyproject/galaxy/blob/5af1d03c929748006dd53ad451e5cb1fd140399a/lib/galaxy_test/workflow/inline_user_defined_tool.gxwf.yml"
summary: "How a workflow step uses an authored GalaxyUserTool: embedded under run:, never by tool_uuid, tool_id or content_id; registering via /api/unprivileged_tools."
---

A gxformat2 step that uses an authored `GalaxyUserTool` carries the whole tool definition under its `run:` key. That is the only way the Foundry binds a step to a user-defined tool.

A step never names a user-defined tool by `tool_uuid`, `tool_id`, or `content_id` alone. The tool is not in a server's toolbox, so a `tool_id` does not resolve to it, even when the string matches the tool's `id`. A `uuid` identifies one registered copy of the tool, for its owner, on one server. Galaxy's workflow editor and its exports write any such references themselves, and Galaxy rejects a `tool_uuid` step without an embedded def
...

---
type: research
tags:
  - target/galaxy
status: draft
created: 2026-05-03
revised: 2026-09-23
revision: 3
related_notes:
  - "[[iwc-workflow-testability-survey]]"
  - "[[iwc-test-data-conventions]]"
  - "[[planemo-asserts-idioms]]"
  - "[[iwc-shortcuts-anti-patterns]]"
  - "[[planemo-workflow-test-architecture]]"
  - "[[implement-galaxy-workflow-test]]"
  - "[[gxformat2-schema]]"
  - "[[gxformat2-workflow-inputs]]"
  - "[[galaxy-datatypes-conf]]"
summary: "Design guidance for Galaxy workflow inputs, outputs, and checkpoints that make IWC-style workflow tests possible."
---

# Design a Galaxy workflow that can be tested

Choose the workflow's public inputs and outputs while designing the analysis, before writing its `-tests.yml`. A workflow test supplies `job:` values by input name and checks `outputs:` by output name. It can only check results the workflow exposes. A good interface gives the test a reproducible input, an addressable result, and at least one assertion that would catch a meaningful wrong result.

This note concerns workflow design. [[iwc-test-data-conventions]] covers the test file's input fixtures, [[planemo-asserts-idioms]] covers assertion syntax, and [[iwc-shortcuts-anti-patterns]] explains when a smoke check is enough. [[iwc-workflow-testability-survey]] holds the broader corpus evidence.

## Name the public interface before writing tests

Give each input and promoted output a stable, descriptive public name. In an exported gxformat2 workflow this is commonly the entry's `id`. The structural [[gxformat2-schema]] also permits `label`. Check the **effective name in the workflow being tested** instead of assuming a step label, generated dataset name, or array position will become the test key. In the pinned [Scanpy workflow](https://github.com/galaxy
...

---
type: research
title: "gxformat2 structural JSON Schema"
tags:
  - target/galaxy
status: draft
created: 2026-05-05
revised: 2026-11-06
revision: 5
license: MIT
license_file: LICENSES/galaxy-tool-util-ts.LICENSE
related_notes:
  - "[[galaxy-collection-semantics]]"
  - "[[galaxy-datatypes-conf]]"
  - "[[galaxy-workflow-testability-design]]"
  - "[[gxformat2-workflow-inputs]]"
  - "[[galaxy-workflow-draft]]"
sources:
  - "https://github.com/jmchilton/galaxy-tool-util-ts/blob/7ae4ecd0ba8d492225f58a6d455c4cc5317298f0/packages/schema/src/galaxy-workflow.ts"
  - "https://github.com/galaxyproject/galaxy/blob/v26.1.1/lib/galaxy/job_execution/actions/post.py"
  - "https://github.com/galaxyproject/galaxy/blob/v26.1.1/lib/galaxy/workflow/modules.py"
companions:
  - "gxformat2.schema.json"
summary: "Vendored gxformat2 structural schema, its input, output, and step vocabulary, and the limits of schema-only validation."
---

# gxformat2 structural schema

The companion `gxformat2.schema.json` is a vendored Draft-07 JSON Schema generated from the `@galaxy-tool-util/schema` Effect-TS model at `7ae4ecd`. Casting carries it with this note. Use it to inspect the structural vocabulary of a gxformat2 workflow, then check connections, tool references, and runtime behavior with gxwf and Galaxy validation. The schema does not establish that an `outputSource` exists or that a tool state is valid for its wrapper.

The schema is regenerated by `pnpm sync:vendored --update`, which builds the upstream package and runs `gxwf structural-schema --format format2` before copying it here. Do not edit the companion by hand.

## Root and input fields

The root requires `class: GalaxyWorkflow`, `inputs`, `outputs`, and `steps`. Those sections accept arrays or keyed maps. The root also permits `id`, `labe
...

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$defs": {
    "WorkflowStepSchema": {
      "type": "object",
      "required": [],
      "properties": {
        "id": {
          "anyOf": [
            {
              "type": "null"
            },
            {
              "type": "string"
            }
          ]
        },
        "label": {
          "anyOf": [
            {
              "type": "null"
            },
            {
              "type": "string"
            }
          ]
        },
        "doc": {
          "anyOf": [
            {
              "type": "null"
            },
            {
              "type": "string"
            },
            {
              "type": "array",
              "items": {
                "type": "string"
              }
            }
          ]
        },
        "position": {
          "anyOf": [
            {
              "type": "null"
            },
            {
              "type": "object",
              "required": [
                "top",
                "left"
              ],
              "properties": {
                "top": {
                  "type": "number"
                },
                "left": {
                  "type": "number"
                }
              },
              "additionalProperties": false
            }
          ]
        },
        "tool_id": {
          "anyOf": [
            {
              "type": "null"
            },
            {
              "type": "string"
            }
          ]
        },
        "tool_shed_repository": {
          "anyOf": [
            {
              "type": "null"
            },
            {
              "type": "object",
              "required": [
                "name",
                "changeset_revision",
                "owner",
                "tool_shed"
              ],
              "properties": {
                "name": {
                  "type": "string"
                },
                "changeset_revision": {
                  "type": "string"
                },
                "owner": {
                  "type": "string"
                },
                "tool_shed": {
                  "type": "string"
                }
              },
              "additionalProperties": false
            }
          ]
        },
        "tool_version": {
          "anyOf": [
            {
           
...

---
type: research
title: "Nextflow operators to Galaxy collection recipes"
tags:
  - source/nextflow
  - target/galaxy
status: draft
created: 2026-05-02
revised: 2026-05-02
revision: 1
related_notes:
  - "[[nextflow-to-galaxy-channel-shape-mapping]]"
  - "[[galaxy-collection-semantics]]"
  - "[[galaxy-collection-tools]]"
  - "[[galaxy-apply-rules-dsl]]"
  - "[[iwc-transformations-survey]]"
  - "[[iwc-tabular-operations-survey]]"
  - "[[galaxy-data-flow-draft-contract]]"
  - "[[iwc-map-over-lifecycle-survey]]"
  - "[[nextflow-patterns]]"
related_molds:
  - "[[nextflow-summary-to-galaxy-data-flow]]"
  - "[[implement-galaxy-tool-step]]"
  - "[[debug-galaxy-workflow-output]]"
sources:
  - "https://github.com/galaxyproject/foundry/issues/53"
summary: "Classifies common Nextflow operators as Galaxy wiring, collection semantics, explicit steps, or review triggers."
---

# Nextflow Operators To Galaxy Collection Recipes

Most Nextflow operators are not Galaxy tools. Translate them first as source-side data-flow intent, then decide whether the Galaxy representation is simple wiring, collection semantics, an explicit Galaxy step, or a user-review checkpoint.

## Decision Vocabulary

| Label | Meaning |
|---|---|
| `channel-only rewiring` | The operator disappears into Galaxy connections, labels, branch wiring, or output selection. |
| `Galaxy collection semantics` | Translation relies on collection identifiers, collection type, map-over, reduction, or nesting behavior. |
| `explicit Galaxy step` | Add a collection-operation, tabular, text-processing, or domain tool step. |
| `user review` | Translation is likely lossy or semantically ambiguous. |

## Operator Recipes

| Nextflow operator | Galaxy recipe | Class | Confidence |
|---|---|---|---|
| `map` | Treat metadata-only proje
...

---
type: research
title: "Nextflow-to-Galaxy channel shape mapping"
tags:
  - source/nextflow
  - target/galaxy
status: draft
created: 2026-05-02
revised: 2026-05-06
revision: 2
related_notes:
  - "[[nextflow-workflow-io-semantics]]"
  - "[[nextflow-params-to-galaxy-inputs]]"
  - "[[nextflow-path-glob-to-galaxy-datatype]]"
  - "[[galaxy-collection-semantics]]"
  - "[[galaxy-collection-tools]]"
  - "[[galaxy-apply-rules-dsl]]"
  - "[[iwc-transformations-survey]]"
  - "[[nextflow-operators-to-galaxy-collection-recipes]]"
  - "[[galaxy-data-flow-draft-contract]]"
  - "[[iwc-conditionals-survey]]"
  - "[[manifest-to-mapped-collection-lifecycle]]"
  - "[[map-workflow-enum-to-tool-parameter]]"
  - "[[regex-relabel-via-tabular]]"
  - "[[relabel-via-rules-and-find-replace]]"
  - "[[reshape-relabel-remap-by-collection-axis]]"
  - "[[sync-collections-by-identifier]]"
  - "[[tabular-compute-new-column]]"
  - "[[tabular-concatenate-collection-to-table]]"
  - "[[tabular-cut-and-reorder-columns]]"
  - "[[tabular-filter-by-column-value]]"
  - "[[tabular-filter-by-regex]]"
  - "[[tabular-group-and-aggregate-with-datamash]]"
  - "[[tabular-join-on-key]]"
  - "[[tabular-pivot-collection-to-wide]]"
  - "[[tabular-prepend-header]]"
  - "[[tabular-relabel-by-row-counter]]"
  - "[[tabular-split-taxonomy-string]]"
  - "[[tabular-sql-query]]"
  - "[[tabular-synthesize-bed-from-3col]]"
  - "[[tabular-to-collection-by-row]]"
  - "[[iwc-map-over-lifecycle-survey]]"
  - "[[nextflow-patterns]]"
related_molds:
  - "[[nextflow-summary-to-galaxy-interface]]"
  - "[[nextflow-summary-to-galaxy-data-flow]]"
  - "[[nextflow-summary-to-galaxy-template]]"
  - "[[implement-galaxy-tool-step]]"
sources:
  - "https://github.com/galaxyproject/foundry/issues/52"
summary: "Maps common Nextflow channel, tuple, and
...