nextflow-test-to-galaxy-test-plan

---
type: research
tags:
  - source/nextflow
component: "Nextflow Testing and Test Fixtures"
status: draft
created: 2026-05-01
revised: 2026-09-15
revision: 5
summary: "nf-test patterns mapped to Galaxy planemo asserts and CWL test equivalents — backs the nextflow test-plan Molds and summarize-nextflow §7."
companions:
  - "component-nextflow-testing.yml"
sources:
  - "https://www.nf-test.com/"
  - "https://www.nf-test.com/docs/assertions/"
  - "https://www.nf-test.com/docs/assertions/snapshots/"
  - "https://www.nf-test.com/docs/configuration/"
  - "https://nf-co.re/docs/contributing/nf-test/assertions"
  - "https://nf-co.re/docs/developing/testing/overview"
  - "https://github.com/nf-core/test-datasets"
  - "https://www.nextflow.io/docs/latest/config.html#config-profiles"
  - "https://nf-co.re/docs/contributing/pipelines#test-data"
related_molds:
  - "[[summarize-nextflow]]"
  - "[[nextflow-test-to-galaxy-test-plan]]"
  - "[[nextflow-test-to-cwl-test-plan]]"
  - "[[implement-galaxy-workflow-test]]"
related_notes:
  - "[[nextflow-test-case-selection]]"
  - "[[planemo-asserts-idioms]]"
  - "[[tests-format]]"
  - "[[iwc-test-data-conventions]]"
  - "[[component-nf-core-tools]]"
---

# Nextflow Testing and Test Fixtures

Operational grounding for three Molds:

- [[summarize-nextflow]] §7 — enumerate whole-pipeline `test_candidates[]` and make the default `test_selection` from a real nf-core or DSL2 pipeline.
- [[nextflow-test-to-galaxy-test-plan]] — translate nf-test fixtures + assertions into Galaxy equivalents.
- [[nextflow-test-to-cwl-test-plan]] — the same translation, toward CWL.

The summarize side is mostly *enumeration*: walk `tests/*.nf.test`, extract structured fields per the Mold §7 spec. The translation side is *mapping*: each nf-test assertion pattern has a (
...

# nf-test patterns → Galaxy / CWL test equivalents.
# Companion to component-nextflow-testing.md.
#
# Rows ordered by frequency in real nf-core pipelines (snapshot block first).

mappings:

  # ---- Snapshot block (nf-core canonical idiom) ----

  - id: snapshot.match
    nf_test_pattern: "assert snapshot(...).match()"
    description: "Top-level snapshot assertion comparing serialized form of N captured values to a stored .nf.test.snap file."
    galaxy_equivalent: "Per-output assertion block in <workflow>-tests.yml; no single-line analog. Decompose into per-capture assertions (rows below)."
    cwl_equivalent: "cwltest expected_outputs entries per output; no single-line analog. Decompose."
    target_link: "[[tests-format]]"
    notes: "The whole-snapshot idiom is lossy on translation; explode into N per-capture assertions and accept the surface-area increase."

  - id: snapshot.succeeded_task_count
    nf_test_pattern: "workflow.trace.succeeded().size()"
    description: "Count of successfully completed tasks. nf-core canonical first capture; brittle across pipeline versions but stable per release."
    galaxy_equivalent: "Implicit in workflow execution success. Galaxy workflow tests pass iff the workflow completes; per-step success surfaces via has_text on log outputs."
    cwl_equivalent: "Implicit in cwltest's `should_succeed` / `should_fail` boolean. No per-task count."
    target_link: "[[planemo-asserts-idioms]] §1 (Plain text reports / logs)"
    notes: "Drop on translation; targets verify success implicitly."

  - id: snapshot.versions_yml
    nf_test_pattern: "ch_versions or path('versions.yml')"
    description: "The `versions` topic channel collected per process and dumped to versions.yml at workflow end. nf-core canonical second capture."
    galaxy_equiv
...

---
type: research
tags:
  - target/galaxy
status: draft
created: 2026-04-30
revised: 2026-09-23
revision: 3
related_notes:
  - "[[galaxy-workflow-testability-design]]"
  - "[[iwc-test-data-conventions]]"
  - "[[planemo-asserts-idioms]]"
  - "[[implement-galaxy-workflow-test]]"
  - "[[tests-format]]"
  - "[[iwc-conditionals-survey]]"
  - "[[iwc-map-over-lifecycle-survey]]"
  - "[[iwc-tabular-operations-survey]]"
  - "[[iwc-transformations-survey]]"
summary: "Judgments for when weak Galaxy workflow-test assertions are useful smoke checks and when they conceal untested results."
---

# IWC test-suite shortcuts and anti-patterns

A workflow test should fail when a meaningful result is wrong. Start by naming that failure, then choose an assertion that would catch it. A size, shape, or existence check can establish that an output was produced, but it cannot establish that the analysis is correct. Keep such a **smoke check** when the result is genuinely variable or opaque, and add a stable content check or workflow checkpoint whenever one is available. Using a weak check merely because another IWC test uses it is an anti-pattern.

These authoring judgments guide [[implement-galaxy-workflow-test]]. [[planemo-asserts-idioms]] covers assertion forms, [[iwc-test-data-conventions]] covers fixtures, and [[galaxy-workflow-testability-design]] covers which intermediate results to expose as workflow outputs.

## Choose by the failure the test must catch

| Output | Useful check | Anti-pattern |
| --- | --- | --- |
| Small, byte-stable text or data | Compare with an expected `file:`. A `checksum:` can cover a large byte-stable output. | Replacing an exact comparison with size alone when exact content is stable. |
| Variable JSON or HDF5 | Assert a stable property, group, key, or count 
...

---
type: research
tags:
  - target/galaxy
status: draft
created: 2026-04-30
revised: 2026-09-23
revision: 4
related_notes:
  - "[[galaxy-workflow-testability-design]]"
  - "[[iwc-shortcuts-anti-patterns]]"
  - "[[planemo-asserts-idioms]]"
  - "[[implement-galaxy-workflow-test]]"
  - "[[tests-format]]"
  - "[[iwc-tabular-operations-survey]]"
summary: "IWC and Planemo workflow-test fixture shapes, file provenance, collection identifiers, input hashes, and built-in index values."
---

# IWC test data conventions

An IWC workflow test pairs each workflow with a sibling `<workflow>-tests.yml`. Each YAML test case supplies `job:` inputs under the workflow's input labels and `outputs:` assertions under its output labels. The optional `test-data/` directory holds local fixtures. These are the [IWC contribution layout](https://github.com/galaxyproject/iwc/blob/main/workflows/README.md) and [Planemo test-format](https://planemo.readthedocs.io/en/latest/test_format.html) contracts. For decisions about the workflow interface before writing fixtures, see [[galaxy-workflow-testability-design]]. For the exact supported YAML vocabulary and static validation, see [[tests-format]].

Examples below come from the cited IWC files or Planemo documentation. Corpus observations describe the workflows sampled for this note, not a rule that every IWC workflow follows.

## Fixture locations and provenance

A `class: File` input can use `path:` for a local fixture or `location:` for a URI. Planemo resolves a relative `path:` from the workflow and test directory. IWC's contribution guide recommends small inputs and suggests publishing a toy dataset to Zenodo for a permanent URL. The repository also permits an optional local `test-data/` directory. It gives no universal size cutoff or mandatory hos
...

---
type: research
title: "Nextflow nf-test snapshots to Galaxy/Planemo assertions"
tags:
  - source/nextflow
  - target/galaxy
status: draft
created: 2026-05-05
revised: 2026-05-05
revision: 2
related_notes:
  - "[[tests-format]]"
  - "[[summary-nextflow]]"
  - "[[planemo-asserts-idioms]]"
  - "[[iwc-test-data-conventions]]"
  - "[[iwc-shortcuts-anti-patterns]]"
  - "[[component-nextflow-testing]]"
related_molds:
  - "[[nextflow-test-to-galaxy-test-plan]]"
  - "[[implement-galaxy-workflow-test]]"
sources:
  - "https://github.com/galaxyproject/foundry/issues/65"
  - "https://github.com/galaxyproject/foundry/issues/142"
summary: "Translates nf-test snapshot assertions into Galaxy workflow test-format assertions, broken out by module-level vs pipeline-level test shape."
---

# Nextflow nf-test snapshots to Galaxy/Planemo assertions

Translation guide for the nf-test → Galaxy direction. Companion to [[tests-format]] (assertion vocabulary), [[planemo-asserts-idioms]] (output-type → assertion-family decision table), and [[summary-nextflow]] (source-side schema). This note maps the *snapshot* shape specifically; per-output assertion picking (size delta, image dimensions, etc.) defers to [[planemo-asserts-idioms]].

The corpus split is load-bearing: in the pinned `workflow-fixtures/pipelines/` fixture set, module-level, subworkflow-level, and pipeline-level `*.nf.test` files snapshot different things. Do not depend on exact counts from one fixture checkout; use the split as a shape distinction and re-sample the current fixture set when making inventory claims.

## 0. The framework primitive

nf-test's only assertion mechanism for content is `assert snapshot(<expr>).match()`. The framework serializes `<expr>` to JSON and writes it to a sibling `<name>.nf.test.snap` on first run
...

---
type: research
title: "Selecting the first Nextflow test case"
tags:
  - source/nextflow
status: draft
created: 2026-09-15
revised: 2026-09-15
revision: 2
summary: "A deterministic policy for choosing the first whole-pipeline Nextflow test case without mistaking profile names for coverage."
sources:
  - "https://github.com/galaxyproject/foundry/issues/67"
  - "https://www.nf-test.com/docs/configuration/"
  - "https://www.nf-test.com/docs/cli/list/"
  - "https://www.nextflow.io/docs/latest/config.html#config-profiles"
related_molds:
  - "[[summarize-nextflow]]"
  - "[[nextflow-test-to-galaxy-test-plan]]"
  - "[[nextflow-test-to-cwl-test-plan]]"
related_notes:
  - "[[component-nextflow-testing]]"
  - "[[component-nextflow-inspect]]"
  - "[[open-requirements-ledger]]"
---

# Selecting the first Nextflow test case

The first translated test should be small enough to run during workflow construction while still exercising the scientific path the translated workflow claims to implement. Profile names help find candidates, but they do not establish either property.

This note owns **selection**. [[component-nextflow-testing]] owns fixture and assertion interpretation after cases have been enumerated. [[summarize-nextflow]] owns applying the selection policy and emits both the candidate set and the decision. The two Nextflow test-plan Molds consume that decision; they must not silently choose a different case. [[run-workflow-test]] is downstream of translation and sees a Galaxy or CWL test artifact, not a Nextflow profile, so it does not use this note.

## Unit of selection

A candidate is a whole-pipeline execution, not merely a profile name:

- for nf-test, one `test(...)` case inside a `nextflow_pipeline { ... }` suite, identified by file path and test name;
- otherwise,
...

---
type: research
tags:
  - target/galaxy
status: draft
created: 2026-04-30
revised: 2026-09-23
revision: 7
related_notes:
  - "[[galaxy-workflow-testability-design]]"
  - "[[iwc-test-data-conventions]]"
  - "[[iwc-shortcuts-anti-patterns]]"
  - "[[implement-galaxy-workflow-test]]"
  - "[[tests-format]]"
  - "[[planemo-workflow-test-architecture]]"
  - "[[validate-tests]]"
  - "[[iwc-tabular-operations-survey]]"
  - "[[galaxy-discover-datasets]]"
summary: "Choose Galaxy workflow-test output assertions by the failure they should catch, the output's stability, and the available checkpoints."
---

# Choosing Planemo workflow-test assertions

Name the wrong result a test must catch before choosing YAML syntax. For a fixed input and byte-stable output, compare the result with an expected file or checksum. For a variable result, assert the stable properties that would fail if the analysis were wrong. A size, shape, or existence check is a useful smoke test when content is genuinely variable, ideally alongside a stable checkpoint. It is an anti-pattern when it replaces a more meaningful check that is available. [[iwc-shortcuts-anti-patterns]] develops that judgment, while [[iwc-test-data-conventions]] covers input fixtures and [[tests-format]] supplies the exact assertion vocabulary.

## 1. Choose by the property, then the output type

| Output | Meaningful check to try | If the result varies |
| --- | --- | --- |
| Text report or HTML | [[tests-format#has_text_model|has_text]] for a stable result, section, or value. Use [[tests-format#has_line_model|has_line]] for a complete stable line. | Use [[tests-format#has_text_matching_model|has_text_matching]] or [[tests-format#has_line_matching_model|has_line_matching]] for a bounded variable field. Line count or size can supplement
...