nextflow-test-to-galaxy-test-plan

---
type: research
tags:
  - source/nextflow
component: "Nextflow Testing and Test Fixtures"
status: draft
created: 2026-05-01
revised: 2026-08-03
revision: 3
summary: "nf-test patterns mapped to Galaxy planemo asserts and CWL test equivalents — backs the nextflow test-plan Molds and summarize-nextflow §7."
companions:
  - "component-nextflow-testing.yml"
sources:
  - "https://www.nf-test.com/"
  - "https://www.nf-test.com/docs/assertions/"
  - "https://www.nf-test.com/docs/assertions/snapshots/"
  - "https://www.nf-test.com/docs/configuration/"
  - "https://nf-co.re/docs/contributing/nf-test/assertions"
  - "https://nf-co.re/docs/developing/testing/overview"
  - "https://github.com/nf-core/test-datasets"
  - "https://www.nextflow.io/docs/latest/config.html#config-profiles"
  - "https://nf-co.re/docs/contributing/pipelines#test-data"
related_molds:
  - "[[summarize-nextflow]]"
  - "[[nextflow-test-to-galaxy-test-plan]]"
  - "[[nextflow-test-to-cwl-test-plan]]"
  - "[[implement-galaxy-workflow-test]]"
related_notes:
  - "[[planemo-asserts-idioms]]"
  - "[[tests-format]]"
  - "[[iwc-test-data-conventions]]"
  - "[[component-nf-core-tools]]"
---

# Nextflow Testing and Test Fixtures

Operational grounding for three Molds:

- [[summarize-nextflow]] §7 — extract `nf_tests[]` and `test_fixtures` from a real nf-core or DSL2 pipeline.
- [[nextflow-test-to-galaxy-test-plan]] — translate nf-test fixtures + assertions into Galaxy equivalents.
- [[nextflow-test-to-cwl-test-plan]] — the same translation, toward CWL.

The summarize side is mostly *enumeration*: walk `tests/*.nf.test`, extract structured fields per the Mold §7 spec. The translation side is *mapping*: each nf-test assertion pattern has a (sometimes lossy) Galaxy or CWL equivalent.

Companion structured form: `component
...

# nf-test patterns → Galaxy / CWL test equivalents.
# Companion to component-nextflow-testing.md.
#
# Rows ordered by frequency in real nf-core pipelines (snapshot block first).

mappings:

  # ---- Snapshot block (nf-core canonical idiom) ----

  - id: snapshot.match
    nf_test_pattern: "assert snapshot(...).match()"
    description: "Top-level snapshot assertion comparing serialized form of N captured values to a stored .nf.test.snap file."
    galaxy_equivalent: "Per-output assertion block in <workflow>-tests.yml; no single-line analog. Decompose into per-capture assertions (rows below)."
    cwl_equivalent: "cwltest expected_outputs entries per output; no single-line analog. Decompose."
    target_link: "[[tests-format]]"
    notes: "The whole-snapshot idiom is lossy on translation; explode into N per-capture assertions and accept the surface-area increase."

  - id: snapshot.succeeded_task_count
    nf_test_pattern: "workflow.trace.succeeded().size()"
    description: "Count of successfully completed tasks. nf-core canonical first capture; brittle across pipeline versions but stable per release."
    galaxy_equivalent: "Implicit in workflow execution success. Galaxy workflow tests pass iff the workflow completes; per-step success surfaces via has_text on log outputs."
    cwl_equivalent: "Implicit in cwltest's `should_succeed` / `should_fail` boolean. No per-task count."
    target_link: "[[planemo-asserts-idioms]] §1 (Plain text reports / logs)"
    notes: "Drop on translation; targets verify success implicitly."

  - id: snapshot.versions_yml
    nf_test_pattern: "ch_versions or path('versions.yml')"
    description: "The `versions` topic channel collected per process and dumped to versions.yml at workflow end. nf-core canonical second capture."
    galaxy_equiv
...

---
type: research
tags:
  - target/galaxy
status: draft
created: 2026-04-30
revised: 2026-05-03
revision: 2
related_notes:
  - "[[galaxy-workflow-testability-design]]"
  - "[[iwc-test-data-conventions]]"
  - "[[planemo-asserts-idioms]]"
  - "[[implement-galaxy-workflow-test]]"
  - "[[tests-format]]"
  - "[[iwc-conditionals-survey]]"
  - "[[iwc-map-over-lifecycle-survey]]"
  - "[[iwc-tabular-operations-survey]]"
  - "[[iwc-transformations-survey]]"
summary: "What IWC test suites cut corners on (accepted) vs what's a code smell — existence-only probes, sim_size deltas, image dim checks, label coupling."
---

# IWC test-suite shortcuts and anti-patterns

## Purpose

When an agent translates or authors a Galaxy workflow for IWC submission, the test suite it writes will be reviewed against IWC's *de facto* style — not against an idealized assertion ladder. That style routinely tolerates assertions that look weak in isolation. This note distinguishes the corner-cutting that is **normal and accepted** in the corpus from the patterns that an agent should treat as **smells** worth flagging.

This note owns accepted-vs-smell calls. For positive workflow-structure guidance behind label stability, checkpoint promotion, and collection identifier design, use [[galaxy-workflow-testability-design]].

Grounding: 115 `*-tests.yml` files under `workflow-fixtures/iwc-src/workflows/` (mirror of `galaxyproject/iwc`), prior synthesis in `galaxy-brain/vault/projects/workflow_state/skills/COMPONENT_GALAXY_WORKFLOW_TESTING.md`. Path citations below are relative to `iwc-src/workflows/` unless absolute.

## TL;DR rules of thumb

1. **Default to tolerant assertions.** `compare: sim_size` + `delta:`, `has_image_*` + `delta:`, `has_text` substring, `has_h5_keys`, `has_n_lines` + `delta:` are *the IW
...

---
type: research
tags:
  - target/galaxy
status: draft
created: 2026-04-30
revised: 2026-05-03
revision: 3
related_notes:
  - "[[galaxy-workflow-testability-design]]"
  - "[[iwc-shortcuts-anti-patterns]]"
  - "[[planemo-asserts-idioms]]"
  - "[[implement-galaxy-workflow-test]]"
  - "[[tests-format]]"
  - "[[iwc-tabular-operations-survey]]"
summary: "How IWC workflows organize and reference test data — Zenodo-first, SHA-1 integrity, collection shapes, CVMFS gotchas."
---

# IWC test data conventions

Reference for an agent implementing or editing a `<workflow>-tests.yml` in IWC style. All evidence cited from `/Users/jxc755/projects/repositories/workflow-fixtures/iwc-src/workflows/` (raw IWC clone) and `workflows/README.md`. Authoritative spec: [planemo.readthedocs.io/en/latest/test_format.html](https://planemo.readthedocs.io/en/latest/test_format.html). The companion analysis at `/Users/jxc755/projects/repositories/galaxy-brain/vault/projects/workflow_state/skills/COMPONENT_GALAXY_WORKFLOW_TESTING.md` is the synthesized source for several normative claims here.

This note owns **test YAML fixture shapes**. For workflow-structure choices that make those fixtures possible before the test file exists, use [[galaxy-workflow-testability-design]].

## 1. Where does test data live? Remote vs in-repo

Two storage patterns. They mix freely inside one job.

**Remote `location:` (default for any non-trivial input).** Strongly preferred for anything bigger than a toy fixture. Order of preference, observed in the corpus:

- **Zenodo** — overwhelming default, persistent DOI-backed URL.
  - `read-preprocessing/short-read-qc-trimming/short-read-quality-control-and-trimming-tests.yml:13,17` — `https://zenodo.org/records/11484215/files/paired_r1.fastq.gz` (note the modern `/records/` pl
...

---
type: research
title: "Nextflow nf-test snapshots to Galaxy/Planemo assertions"
tags:
  - source/nextflow
  - target/galaxy
status: draft
created: 2026-05-05
revised: 2026-05-05
revision: 2
related_notes:
  - "[[tests-format]]"
  - "[[summary-nextflow]]"
  - "[[planemo-asserts-idioms]]"
  - "[[iwc-test-data-conventions]]"
  - "[[iwc-shortcuts-anti-patterns]]"
  - "[[component-nextflow-testing]]"
related_molds:
  - "[[nextflow-test-to-galaxy-test-plan]]"
  - "[[implement-galaxy-workflow-test]]"
sources:
  - "https://github.com/galaxyproject/foundry/issues/65"
  - "https://github.com/galaxyproject/foundry/issues/142"
summary: "Translates nf-test snapshot assertions into Galaxy workflow test-format assertions, broken out by module-level vs pipeline-level test shape."
---

# Nextflow nf-test snapshots to Galaxy/Planemo assertions

Translation guide for the nf-test → Galaxy direction. Companion to [[tests-format]] (assertion vocabulary), [[planemo-asserts-idioms]] (output-type → assertion-family decision table), and [[summary-nextflow]] (source-side schema). This note maps the *snapshot* shape specifically; per-output assertion picking (size delta, image dimensions, etc.) defers to [[planemo-asserts-idioms]].

The corpus split is load-bearing: in the pinned `workflow-fixtures/pipelines/` fixture set, module-level, subworkflow-level, and pipeline-level `*.nf.test` files snapshot different things. Do not depend on exact counts from one fixture checkout; use the split as a shape distinction and re-sample the current fixture set when making inventory claims.

## 0. The framework primitive

nf-test's only assertion mechanism for content is `assert snapshot(<expr>).match()`. The framework serializes `<expr>` to JSON and writes it to a sibling `<name>.nf.test.snap` on first run
...

---
type: research
tags:
  - target/galaxy
status: draft
created: 2026-04-30
revised: 2026-05-11
revision: 6
related_notes:
  - "[[galaxy-workflow-testability-design]]"
  - "[[iwc-test-data-conventions]]"
  - "[[iwc-shortcuts-anti-patterns]]"
  - "[[implement-galaxy-workflow-test]]"
  - "[[tests-format]]"
  - "[[planemo-workflow-test-architecture]]"
  - "[[validate-tests]]"
  - "[[iwc-tabular-operations-survey]]"
  - "[[galaxy-discover-datasets]]"
summary: "Decision and idiom guide for picking planemo workflow-test assertions: which family per output type, how to size tolerances, when to validate."
---

# Planemo asserts: idiom and decision guide

Companion to [[iwc-test-data-conventions]] (input shapes), [[galaxy-workflow-testability-design]] (workflow structure before test YAML exists), and [[iwc-shortcuts-anti-patterns]] (what's accepted vs smell). This note is forward-looking: when authoring a new `<workflow>-tests.yml`, which assertion family fits which output, and what the recommended tolerances and operators are.

The **vocabulary itself is not restated here** — every assertion's parameter list, types, defaults, required fields, and Python docstring is rendered from the test-format JSON Schema at [[tests-format]]. Assertion names below deep-link into that page (e.g. [[tests-format#has_text_model|has_text]] jumps straight to that `$def`).

## 1. Choose by output type

The single most useful decision table. Pick the row that matches the file format the workflow emits; default to the recommended assertion family.

| Output type | Default assertion family | Why | Fallback |
|---|---|---|---|
| **Plain text reports / logs** (FastQC summary, MultiQC text section) | [[tests-format#has_text_model|has_text]] (substring on a known stable token) + [[tests-format#has_n_lines
...