Skip to content

QUANT: silent per-sample drop from order-dependent combine(by:) on groupKey #459

Description

@jonasscheid

Description of the bug

In QUANT (subworkflows/local/quant/main.nf), runs are matched to their group's FDR-filter file with:

ch_runs_score_switched
    .map { meta, idxml -> [groupKey([id: "${meta.sample}_${meta.condition}"], meta.group_count), meta, idxml] }
    .combine(filter_q_value, by:0)

The left key is a groupKey(...) object; filter_q_value's key is a plain [id: ...] map. groupKey's equals() is asymmetric (groupKey(v,n) == v is true, but v == groupKey(v,n) is false, see nextflow-io/nextflow#4104). So whether a pair matches depends on which operand is compared first, which depends on the arrival order of OPENMS_IDRIPPER vs OPENMS_IDFILTER_Q_VALUE.

When IDRipper arrives first, the pair is silently dropped — no error, pipeline exits 0. Under executor dispatch jitter (e.g. AWS Batch) this drops a large fraction of samples at QUANT:OPENMS_IDFILTER_QUANT.

Reproduced on Nextflow 25.10.4: forcing IDRipper-before-IDFILTER_Q_VALUE drops the sample 5/5 runs; reversing the order keeps it. Matches the field report in #458 (Failure A).

CI misses it: test is one group, test_single_quant is small, and test_full asserts only that the run finishes.

Fix: drop the groupKey from the match key and combine on a plain [id: ...] map (symmetric equality). The groupKey size isn't needed here — the key is discarded right after the combine.

Command used and terminal output

nextflow run nf-core/mhcquant -r 3.2.0 -profile docker --quantify true ...
# multi-sample, group_count=1 samplesheet on awsbatch
# QUANT:OPENMS_IDFILTER_QUANT receives far fewer tasks than QUANT:OPENMS_IDRIPPER, no error

Relevant files

subworkflows/local/quant/main.nf

System information

  • Nextflow: 25.10.4
  • Executor: awsbatch (also reproducible locally by forcing task arrival order)
  • Container: Docker
  • nf-core/mhcquant: 3.2.0

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions