Skip to content

Add verkko-fillet 0.1.25 to bio compiled software - #185

Merged
kennedydane merged 1 commit into
masterfrom
dane/verkko-fillet-0.1.25
Jul 21, 2026
Merged

Add verkko-fillet 0.1.25 to bio compiled software#185
kennedydane merged 1 commit into
masterfrom
dane/verkko-fillet-0.1.25

Conversation

@kennedydane

Copy link
Copy Markdown
Collaborator

What does this PR do?

Adds verkko-fillet 0.1.25 (bio, compiled) — a toolkit for cleaning Verkko genome assemblies — installed via a uv virtual environment, plus the external tools it needs.

Deliberately not conda: verkko-fillet is a pure-Python py3-none-any wheel on PyPI as verkkofillet, and there is no bioconda package at all (bioconda-recipes/recipes/verkkofillet → 404). Upstream's own conda instructions only build a dependency environment and then pip install the package, so the uv path costs nothing. Follows the existing cnvpytor.yaml pattern.

Also newly packaged, because verkko-fillet shells out to them:

Package Version Notes
seqtk 1.5 Plain make — upstream has no configure and no install target, so the binary is copied out by hand
mashmap 3.1.3 The only cmake build here. Needs C++17, CMake ≥3.21, GSL and htslib
minimap2 2.28 Version entry only (repo had 2.24; 2.28 is what upstream validated against)

bgzip comes from the existing htslib 1.22.1 module, not the samtools tarball. bedtools 2.31.1 and meryl 1.4.1 already existed and happen to match upstream's validated versions exactly.

Two upstream packaging quirks worth knowing about

  1. Empty install_requirespip install verkkofillet pulls in nothing. Every runtime dependency is pinned explicitly in pyproject.toml, derived by grepping src/verkkofillet for third-party imports. Deliberately not from upstream's requirements.txt, which is a Sphinx docs list concatenated with a pip freeze of the author's whole environment (it contains twine, PyQt5, snakemake…).
  2. No console entry points — the wheel's dist-info has no entry_points.txt; the ~40 CLI scripts live inside site-packages/verkkofillet/bin. The module puts that directory on PATH, which is what lets getChrNames.sh resolve its which neighborhood lookup.

The README's external-tool list (mashmap, samtools/bgzip, seqtk) undersells the real requirements — bedtools, meryl and minimap2 are also used. All are wired up via depends_on() with pinned versions.

Checklist

  • Linters pass locally: cd ansible && uv run yamllint . && uv run ansible-lint — reaches Profile 'production' was required, and it passed.
  • CI is green on this PR.
  • Double tag wired up: name tag + never on each include_tasks entry in bio/main.yml; version tag + never on each leaf task file.
  • Checksum set on every download (sha256, each verified against the upstream tarball while planning).
  • Module template(s) added: seqtk.lua, mashmap.lua, verkko-fillet.lua.
  • Symbolic file modes; mode: preserve when copying the built seqtk and mashmap binaries.
  • Uses the path variables from ansible/group_vars/all.
  • README software table updated (MashMap under Aligners, verkko-fillet under Assembly).

Notes for the maintainer

Upstream: https://github.com/marbl/verkko-fillet (v0.1.25, MIT) · https://github.com/marbl/MashMap (v3.1.3) · https://github.com/lh3/seqtk (v1.5)

Install order matters — each depends_on target must exist before the module referencing it:

ansible-playbook site.yaml -t seqtk,seqtk1.5
ansible-playbook site.yaml -t mashmap,mashmap3.1.3
ansible-playbook site.yaml -t minimap2,minimap22.28
ansible-playbook site.yaml -t verkko-fillet,verkko-fillet0.1.25

Not yet built on the cluster. mashmap is the step most likely to need attention on first run: it is the only cmake build, and I pinned the repo's cmake/3.27.0 rather than /usr/bin/cmake since upstream requires ≥3.21 and the build host's system cmake is unknown. GSL/htslib are injected via environment: following the seqwish.yaml precedent.

What was verified locally

  • yamllint silent; ansible-lint production profile passed (0 failures, 445 files)
  • ansible-playbook site.yaml --syntax-check clean
  • Built a throwaway venv from the committed lockfile: verkkofillet 0.1.25 imports on Python 3.12 and .venv/lib/python3.12/site-packages/verkkofillet/bin/ contains 38 scripts, confirming the PATH the module prepends
  • All three .lua templates rendered under Jinja StrictUndefined with real values — no undefined names
  • The YAML anchor in mashmap.yaml resolves to two identical environment mappings

Note that --list-tags cannot confirm the new tags, since the role's include_tasks are dynamic and are not expanded; the tags are only genuinely exercised by a real playbook run.

Known limitations (documented in the module's help())

  • screenAssembly() will not workbin/screen-assembly.sh resolves verkko's install layout off PATH (verkko=$(dirname $(which verkko)), then $verkko/../lib/verkko/scripts/) and needs the real lib/verkko/ tree plus its bundled contaminant FASTAs. verkko is not packaged in this repo. Everything else only reads a Verkko output directory — grepping for any verkko binary invocation across the source returns zero hits outside that one script. Packaging verkko 2.3.2 would be a separate change.
  • graphAlign() needs GraphAligner, not packaged here. Its path is a runtime parameter, so an external build can be passed in without repackaging.
  • Dot plots need gnuplot, container-only here. The call site is guarded by shutil.which("gnuplot") and degrades gracefully.
  • verkkofillet ships a prebuilt ~480 KB x86-64 binary (bin/neighborhood) inside the wheel — nothing to build, but we do redistribute an opaque binary.

Unrelated, not touched

mash appears to be registered twice in ansible/roles/compiled/tasks/bio/main.yml (~lines 307 and 321) — looks like an accidental duplicate block. Worth a separate cleanup commit.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GKjARUHkPUnhsRo6K1Faya

verkko-fillet is a toolkit for cleaning Verkko genome assemblies. Installed
via a uv virtual environment rather than conda: it is a pure-Python wheel on
PyPI as `verkkofillet`, and there is no bioconda package at all (upstream's
conda docs only build a dependency environment, then pip install the package).

Two upstream packaging quirks are compensated for here:

  * It declares an empty install_requires, so every runtime dependency is
    pinned explicitly in pyproject.toml. The list was derived by grepping
    src/verkkofillet for third-party imports, NOT from the upstream
    requirements.txt (which is a Sphinx list concatenated with a pip freeze
    of the author's whole environment).
  * It ships no console entry points; its ~40 CLI scripts live inside
    site-packages/verkkofillet/bin. The module puts that directory on PATH,
    which is what lets getChrNames.sh resolve `which neighborhood`.

The README lists only mashmap, samtools (with bgzip) and seqtk as external
requirements, but the source also shells out to bedtools, meryl and minimap2.
All three already existed in the repo. Newly packaged:

  * seqtk 1.5     - plain make, no configure or install target upstream
  * mashmap 3.1.3 - the only cmake build here; needs C++17, CMake >= 3.21,
                    GSL and htslib, so the repo's cmake/3.27.0, libgsl/2.7.1
                    and htslib/1.22.1 trees are injected into the build env
  * minimap2 2.28 - version entry only, matching what upstream validated
                    against (the repo previously had 2.24)

bgzip comes from the htslib module, not the samtools tarball.

Install (bottom-up, so each depends_on target exists first):
  ansible-playbook site.yaml -t seqtk,seqtk1.5
  ansible-playbook site.yaml -t mashmap,mashmap3.1.3
  ansible-playbook site.yaml -t minimap2,minimap22.28
  ansible-playbook site.yaml -t verkko-fillet,verkko-fillet0.1.25

Not yet built on the cluster; mashmap is the step most likely to need
attention on first run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GKjARUHkPUnhsRo6K1Faya
Copilot AI review requested due to automatic review settings July 20, 2026 08:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@kennedydane
kennedydane requested a review from MikeCTZA July 21, 2026 12:49
@kennedydane
kennedydane merged commit f2a5c2f into master Jul 21, 2026
1 check passed
@kennedydane
kennedydane deleted the dane/verkko-fillet-0.1.25 branch July 21, 2026 13:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants