new module: cellpheno/nis by Chrisa142857 · Pull Request #12179 · nf-core/modules

Chrisa142857 · 2026-06-26T20:18:51Z

Description

cellpheno/nis runs whole-brain 3D nuclei instance segmentation on a single lightsheet
microscopy tile: a 2D U-Net per slice, 2D→3D flow conversion, flow-following to
obtain instances per depth-chunk, and GNN gap-stitching across chunks. It wraps
the C++/LibTorch CellPheno NIS executable from
https://github.com/Chrisa142857/Lightsheet_microscopy_image_3D_nuclei_instance_segmentation.

This is a GPU, container-only module (no Conda package), following the
established parabricks/* and numorph/3dunet pattern: a single vendor container,
no environment.yml, an in-script guard that errors under -profile conda/mamba,
label 'process_gpu', and versions via the topic: versions + eval mechanism.

PR checklist

Notes for reviewers

No conda support is intentional and matches parabricks/*.
process_gpu emits a non-standard-label warning (as it does for every GPU
module); paired with process_high.
Versions come from eval("cat /usr/local/share/cellpheno-nis/VERSION") because the binary
has no --version; the Dockerfile writes that file from the same ARG as the
image tag to keep them in lockstep.
Container hosting: the image is published at
quay.io/wzq10101/cellpheno-nis:1.0.0 (my personal Quay namespace — I'm a new
nf-core member and don't have quay.io/nf-core push access yet). nf-core modules lint is clean (42 passed, 0 failed) under the default quay.io registry. Happy to
have the team re-host it under quay.io/nf-core/cellpheno-nis during review — I'll
update the container line to match.

mashehu · 2026-06-30T07:21:47Z

+    // It has no Conda/Bioconda package and requires a CUDA runtime, so it is
+    // distributed only as a dedicated GPU container image (cf. the parabricks and
+    // numorph/3dunet modules, which likewise ship a vendor GPU image with no conda).
+    container "quay.io/wzq10101/cellpheno-nis:1.0.0"


do you have the dockerfile for this image?
If so, please add it here and request nf-core to upload to their registry, similar to how numorhp did it https://github.com/nf-core/modules/blob/master/modules/nf-core/numorph/3dunet/main.nf

mashehu

for modules that don't support conda, please add them to the .github/skip_nf_test.json file

Chrisa142857 · 2026-06-30T16:25:20Z

Thanks @mashehu! Addressed both points:

Dockerfile + registry: added a self-contained Dockerfile (plus a README.md) to the module that clones and builds the C++/LibTorch/CUDA NIS binary from source — following the numorph/3dunet pattern — and pointed the container at quay.io/nf-core/cellpheno-nis:1.0.0. Could you trigger the nf-core build/upload to quay.io/nf-core? I built and validated the image locally (it's currently mirrored at quay.io/wzq10101/cellpheno-nis:1.0.0 for reference).
Conda: added modules/nf-core/cellpheno/nis to .github/skip_nf_test.json (the module is GPU/container-only and has no conda package, same as parabricks/*).

Chrisa142857 · 2026-06-30T17:00:32Z

Status after the latest push:

✅ nf-core lint modules (cellpheno/nis) passes (default quay.io registry)
✅ conda matrix passes now that the module is in skip_nf_test.json

❌ The GPU Test jobs fail on missing test data, not on the module:

ERROR ~ No such file or directory:
https://raw.githubusercontent.com/nf-core/test-datasets/.../imaging/segmentation/cellpheno_nis/tile

The gpu-tagged test references a tile + TorchScript models in nf-core/test-datasets
that aren't there yet. This is a planned follow-up (it's the unchecked "include test
data" box): contribute a small real tile to nf-core/test-datasets, host the .pt
weights on Zenodo (the numorph/3dunet precedent), and regenerate the snapshot on a GPU
runner. The stub test (CPU) passes and is snapshotted.

Happy to coordinate the test-datasets + Zenodo follow-up, or open the module without the
live GPU test for now (as discussed for vendor GPU modules). Let me know which you prefer.

The nf-core/modules submission (PR nf-core/modules#12179) publishes the NIS image to ghcr.io/chrisa142857/cellpheno-nis:1.0.0 (quay.io/nf-core push access pending), so point the module's container line there. PR_BODY.md holds the submission description. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjA3vZUv5qkJdZnmjpRpiY

Chrisa142857 · 2026-07-09T19:30:43Z

Update — the test-data follow-up is now complete, so this should be ready for another look:

Tile: cellpheno/nis: add test tile + models test-datasets#2136 is approved (single ~700 KB real P4 crop). Once it's merged the gpu-tagged test can resolve the tile.
Models: the TorchScript weights are hosted on Zenodo — 10.5281/zenodo.21284682 (https://zenodo.org/records/21284682/files/cellpheno_nis_models.zip), following the numorph/3dunet precedent. The test now fetches them from there (they're too big for test-datasets).
Device-agnostic models: I fixed the binary to map_location the flow/GNN modules onto whatever --device is passed, so a single model set runs on any GPU index (cuda:0/1/2/…), not just cuda:0.
Wider CUDA compat: the module Dockerfile now builds against CUDA 12.1 / LibTorch 2.5.1+cu121 (was 12.8/cu128) — no custom .cu kernels, so LibTorch's fat binary still covers sm_50..sm_90, and this lowers the driver floor to ≥530.

Recap of your two earlier asks, both done: added the self-contained Dockerfile (numorph pattern) and added the module to .github/skip_nf_test.json for conda.

Could you please, when you have a moment: (1) merge test-datasets#2136, and (2) trigger the nf-core build/upload of the Dockerfile to quay.io/nf-core/cellpheno-nis:1.0.0? I don't have push access to the quay.io/nf-core org (it's mirrored at quay.io/wzq10101/cellpheno-nis:1.0.0 for reference). Then the GPU test should pass on a runner. Thanks!

mashehu · 2026-07-09T19:46:13Z

since you are the author of the upstream software, could you not make a release and publish it on bioconda? Just building from main is not good practice for reproducibility

Chrisa142857 · 2026-07-09T21:03:22Z

Good point on reproducibility — agreed, building from main was wrong. I've done two things:

Cut a tagged release v1.0.0 of the upstream software and pinned the Dockerfile to it (NIS_SOURCE_REF=v1.0.0, commit 5420a1c) instead of main, so the image is now reproducible.
On bioconda: happy to pursue it. One caveat to flag — NIS is a C++/LibTorch/CUDA GPU executable (not a Python package), so a recipe has to compile against conda's libtorch/opencv/CUDA and the auto-built BioContainer is a GPU image. This is the same situation as the numorph/3dunet example you linked, which ships a pinned release + vendor container rather than a conda package. I'll work up a bioconda recipe and, if it's accepted, switch the module's container to the resulting biocontainer.

Would you be OK merging with the pinned-release container now (reproducible via v1.0.0), and I migrate the container line to the biocontainer once the bioconda recipe lands? That unblocks the module while the conda packaging goes through review. If you'd rather hold the module until bioconda is merged, that's fine too — just let me know your preference.

Draft a bioconda recipe (conda-recipe/) to package the NIS C++/LibTorch/CUDA binary, so the nf-core module can consume a versioned conda package / BioContainer instead of building from source (per nf-core/modules#12179 review). Validated locally against conda-forge libtorch 2.12 + libopencv 5.0: all sources compile cleanly (no LibTorch API drift). Recipe declares the CUDA compiler and nccl (libtorch_cuda references NCCL symbols) and installs the binary as `cellpheno-nis` (upstream `main` must not go on the global PATH). cpp/CMakeLists.txt: bump cmake_minimum_required 3.0 -> 3.18 (CMake 4 removed <3.5 compatibility) and drop the hardcoded personal TORCH_LIBRARIES path (find_package(Torch) sets it; located via CMAKE_PREFIX_PATH). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjA3vZUv5qkJdZnmjpRpiY

Chrisa142857 · 2026-07-09T22:34:15Z

Bioconda recipe submitted: bioconda/bioconda-recipes#67064. It builds the C++/LibTorch/CUDA binary against conda's libtorch/libopencv (pinned to the v1.0.1 release). If it's accepted I'll switch the module's container to the resulting BioContainer; in the meantime the Dockerfile is pinned to v1.0.1 so the current image is reproducible.

mashehu · 2026-07-10T04:35:33Z

+    output:
+    tuple val(meta), path("*_NIScpp_results_*.zip"), emit: nis
+    tuple val(meta), path("*_remap.zip")           , emit: remap, optional: true
+    tuple val("${task.process}"), val('cellpheno_nis'), eval("cat /usr/local/share/cellpheno-nis/VERSION"), topic: versions, emit: versions_cellpheno_nis


I don't think this works in all container systems

mashehu · 2026-07-10T04:36:48Z

+    // The GPU is selected by the executor (e.g. CUDA_VISIBLE_DEVICES); NIS defaults
+    // to `cuda:0`. Override with `--device cuda:N` via `task.ext.args` if needed.
+    """
+    main \\


if you want your people to use your software in wider circumstances, follow CLI standards, e.g. giving the using meaningful command names, (should be cellpheno nis here, for example)

Chrisa142857 · 2026-07-10T11:25:19Z

Thanks, both addressed in the latest push:

Version reporting: dropped the eval() topic (agreed, not portable across all container engines) and now emit a versions.yml from the script block via cat …/VERSION — the classic, universally-supported pattern.
CLI naming: the binary is now installed and called as cellpheno-nis instead of main (updated the module Dockerfile + script/stub). This also matches the bioconda package name in Add cellpheno-nis bioconda/bioconda-recipes#67064, so the tool has one consistent CLI name across conda and container. (Kept the hyphenated form to match bioconda; happy to move to a cellpheno nis sub-command later if you'd prefer a multi-tool CLI.)

The stub test now asserts output existence rather than snapshotting the version file. Note the quay.io/nf-core/cellpheno-nis image will need a (re)build from the updated Dockerfile for the rename to take effect.

@mashehu

…f-test Address review (@mashehu): - Add a self-contained Dockerfile that clones and builds the C++/LibTorch NIS binary from source (analogous to numorph/3dunet), plus a README, so nf-core can build and upload the image to quay.io/nf-core/cellpheno-nis. - Point the container at quay.io/nf-core/cellpheno-nis:1.0.0. - Add modules/nf-core/cellpheno/nis to .github/skip_nf_test.json (no conda support).

- Dockerfile: build against CUDA 12.1 / LibTorch 2.5.1+cu121 (was 12.8 / 2.7.0+cu128). No custom .cu kernels, so LibTorch's fat binary covers sm_50..sm_90; this lowers the NVIDIA driver floor to >=530 and matches the postproc image. Use the non-cuDNN CUDA base (LibTorch bundles cuDNN). - tests: fetch the (device-independent) TorchScript models from a stable Zenodo archive (10.5281/zenodo.21284682) instead of test-datasets, which only holds the small tile now. Stub test uses dummy dirs (no downloads).

Build the NIS binary from the tagged v1.0.0 release instead of the moving main branch, so the container is reproducible (per review feedback).

… CLI - Drop eval()-based version topic (not portable across container systems); emit versions.yml from the script block instead (classic nf-core pattern). - Install/call the binary as 'cellpheno-nis' rather than the collision-prone 'main', following CLI naming standards (matches the bioconda package). - Stub test asserts output existence (drops the version-file snapshot).

nf-core lint requires versions via the topic channel, but eval() at output time is not portable (per review). Resolve both: emit the versions topic with a literal version string ('1.0.0', kept in lockstep with the container tag) instead of eval(cat VERSION). Lint passes (0 failures).

Match the nf-core convention for tools without a CLI version (cf. vt/decompose): inline '// WARN: Version information not provided by tool on CLI...' comment on the literal version, and canonical meta.yml description. Lint: 38 passed, 0 failed.

The stub test staged both inputs from the same path (basename 'nis'), causing a Nextflow 'input file name collision' that would fail CI. Point the models dummy input at a distinct dir. Restore snapshot-based stub assertion (matches numorph/3dunet); output is now deterministic (literal version topic), so the snapshot is stable. Lint: 0 failures.

Container tag, versions-topic string and Dockerfile VERSION were still 1.0.0 while the pinned source and bioconda package are 1.0.2 -> align all to 1.0.2. Fix README (binary is installed as cellpheno-nis, not main; accurate release steps). Regenerate stub snapshot.

mashehu · 2026-07-14T09:23:18Z

I would recommend to also add a --version output to your tool. it's another standard thing helping to make things reproducible.

Also please don't just copy-paste output from LLMs as comments, it feels nicer to read things from an actual person 🙂

Use eval(cellpheno-nis --version) for the versions topic now that the binary has a --version flag, instead of a hardcoded string. Drop the now-unused baked VERSION file. Pin source build to v1.0.3.

mashehu reviewed Jun 30, 2026

View reviewed changes

mashehu requested changes Jun 30, 2026

View reviewed changes

Chrisa142857 requested review from a team, edmundmiller and maxulysse as code owners June 30, 2026 16:24

atrigila added the awaiting-changes will be closed after 30 days label Jul 2, 2026

Chrisa142857 mentioned this pull request Jul 9, 2026

cellpheno/nis: add test tile + models nf-core/test-datasets#2136

Open

mashehu requested changes Jul 9, 2026

View reviewed changes

Chrisa142857 mentioned this pull request Jul 9, 2026

Add cellpheno-nis bioconda/bioconda-recipes#67064

Open

mashehu reviewed Jul 10, 2026

View reviewed changes

Chrisa142857 added 10 commits July 14, 2026 11:21

new module: cellpheno/nis

5ed0639

cellpheno/nis: host container on quay.io (registry-prefix lint)

5e8fbd5

cellpheno/nis: real test data via UNZIP + existence assertions

3bf095d

cellpheno/nis: pin Dockerfile build to release v1.0.0 (not main)

88271da

Build the NIS binary from the tagged v1.0.0 release instead of the moving main branch, so the container is reproducible (per review feedback).

cellpheno/nis: bump pinned source to v1.0.1 (cmake compat fix)

ffe93d4

Chrisa142857 added 3 commits July 14, 2026 11:21

cellpheno/nis: pin source build to release v1.0.2

4846c62

mashehu force-pushed the cellpheno-nis branch from 1e243f3 to 51f2bd0 Compare July 14, 2026 09:21

mashehu reviewed Jul 14, 2026

View reviewed changes

Comment thread modules/nf-core/cellpheno/nis/tests/main.nf.test Outdated

Apply suggestion from @mashehu

0651073

mashehu reviewed Jul 14, 2026

View reviewed changes

Comment thread modules/nf-core/cellpheno/nis/tests/main.nf.test Outdated

mashehu and others added 2 commits July 14, 2026 11:31

Apply suggestion from @mashehu

2f256bc

cellpheno/nis: report version via 'cellpheno-nis --version' (v1.0.3)

6fb261b

Use eval(cellpheno-nis --version) for the versions topic now that the binary has a --version flag, instead of a hardcoded string. Drop the now-unused baked VERSION file. Pin source build to v1.0.3.

Uh oh!

Conversation

Chrisa142857 commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

PR checklist

Notes for reviewers

Uh oh!

mashehu Jun 30, 2026

Choose a reason for hiding this comment

Uh oh!

mashehu left a comment

Choose a reason for hiding this comment

Uh oh!

Chrisa142857 commented Jun 30, 2026

Uh oh!

Chrisa142857 commented Jun 30, 2026

Uh oh!

Chrisa142857 commented Jul 9, 2026

Uh oh!

mashehu Jul 9, 2026

Choose a reason for hiding this comment

Uh oh!

Chrisa142857 commented Jul 9, 2026

Uh oh!

Chrisa142857 commented Jul 9, 2026

Uh oh!

mashehu Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

mashehu Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

Chrisa142857 commented Jul 10, 2026

Uh oh!

mashehu commented Jul 14, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Chrisa142857 commented Jun 26, 2026 •

edited

Loading