Skip to content

Locate breaks and scatter-region genes per locus, not per gene field - #1178

Merged
etal merged 2 commits into
masterfrom
etal/fix/breaks-gene-loci
Aug 18, 2026
Merged

etal merged 2 commits into
masterfrom
etal/fix/breaks-gene-loci

Conversation

@etal

@etal etal commented Aug 18, 2026

Copy link
Copy Markdown
Owner

Two gene-name consumers keyed on a bin's whole gene field instead of the individual names it comma-joins, so a bin covering two genes became a third gene that does not exist, and both still used the first-to-last-occurrence span rule that #1173 and #1177 removed from by_gene and scatter --gene. cnvlib/reports.py::get_gene_intervals feeds the released breaks table; cnvlib/plots.py::gene_coords_by_range feeds scatter -c chr:start-end labels. Both now group bins into loci with cnvlib.cnary.gene_runs, so one rule serves genemetrics, the diagram, scatter --gene, scatter -c region and breaks.

Both functions date to the 2014 import, when target's shortest_name collapsed multi-accession labels to one name and CNVkit never emitted a comma-joined gene value. Whole-field keying was correct by construction and became wrong in 203ed6d (2016-12-10), when annotation started flowing through join_strings. Nothing defends it: the maintainer's answer in #15 (2015) is that a FOO,BAR bin "really represent[s] 2 or more genes", and #107 → aef8b78 (2016-05-13) chose one row per real gene, with overlapping spans, for the sibling gainloss report. The breaks path was simply never revisited; #220 (2017) is a user asking what a Y_RNA row means, answered in prose rather than in code.

Decisions

A co-binned breakpoint is reported once per gene, not once under a joined name. That is #107's rule, and the gene column's documented use is breaks Sample.cnr Sample.cns | cut -f1 | sort -u > gene-breaks.txt, whose stated purpose is a list of gene names and whose sort -u absorbs the duplication. Under the old rule a reader filtering that file for CDKN2A found nothing, because CDKN2A's breakpoints were filed under CDKN2A,RP11-145E5.5.

Plot labels do the opposite where spans coincide. Genes whose runs have identical bounds share one stripe under a joined label, because two identical axvspan stripes with two text calls at the same coordinates overprint. This is the mechanics gene_coords_by_name adopted in #1177 for #458, and it is the only point where the two surfaces diverge: a table row is filtered, a stripe is looked at.

probes_left and probes_right count every bin in the locus's run, including off-target bins and backbone baits it absorbs. A copy number alteration affects a genomic region regardless of the gene model, so a CGH backbone bin or an antitarget bin inside a gene supports a breakpoint there as well as a targeted bin does. genemetrics already counts exactly those bins in its probes column, and the two sides now sum to it; before, 19 of 58 amplicon rows disagreed with it, and 5 more disagreed because their gene did not exist. test/formats/regression/p2-9_2.cnr is the UCSF500-shaped case: 1120 CGH bins and 12451 Background, 37 of them inside a real gene's run.

A trailing comma in the gene field names nothing. 171 bins of test/formats/wgs-chr17.cnr are labeled MIR3186,MIR4740,, which splits to a third, empty name; if gene: skips it at both sites, mirroring reports.group_by_genes. Adding "" to the tuple handed to gene_runs was rejected: it would make such a bin absorbable rather than run-terminating, diverging from by_gene — a subtler disagreement than the one being fixed.

Measured

Baseline worktree at 8e025ea, both trees discriminated by cnvlib.__file__ under the exact invocation.

fixture rows before rows after comma-named rows
breaks amplicon.cnr amplicon.cns 49 58 9 → 0
the same at -m 4 30 38 6 → 0
breaks regression/p2-9_2.cnr (comma-free) 42 42 0 → 0
breaks wgs-chr17.cnr, 83 block-mean segments 93 218 48 → 0

On amplicon all five invented names disappear (APC,CTC-554D6.1, ATM,C11orf65, GOPC,ROS1, CDKN2A,RP11-145E5.5, KDR,RP11-530I17.1), the documented cut -f1 | sort -u list goes from 26 names to 29 real ones, and real genes recover the bins the phantoms held: APC chr5:112090792 (2, 10) → (2, 39), ATM chr11:108143413 (22, 18) → (22, 43), SMAD4 chr18:48592058 (2, 4) → (9, 4). On p2-9_2 the same 42 breakpoints are reported and two rows' counts change, PTPRD chr9:8733906 (35, 2) → (35, 10) and chr9:9003938 (36, 1) → (37, 8), both from absorbed CGH bins — that fixture isolates the counting decision from the comma fix. On wgs-chr17, Y_RNA had one 39.1 Mb interval (chr17:7,440,413-46,556,560) built from the 3 bins whose whole field was exactly Y_RNA, though 7706 bins name it; every segment boundary inside that span was a phantom row, 38 of them, and they sorted high because the ranking key is min(probes_left, probes_right). They become 6 real rows, and the widest interval on the chromosome falls to 1.18 Mb.

For labels, amplicon chr17 goes from [ERBB2, 'ERBB2,MIR4728'] to [ERBB2, MIR4728], and chr5 from two APC boxes split by a gap to one contiguous locus 112,043,331-112,179,950 beside CTC-554D6.1's own span; renderings were compared in both trees. Over all of wgs-chr17 the widest region falls from 39.1 Mb to 1.18 Mb, and -c chr17 now agrees with -c chr17 -g Y_RNA on all 13 loci in the window they share.

get_gene_intervals also got faster despite producing 76% more entries on wgs-chr17 (0.203 s → 0.144 s), the per-row itertuples loop being gone, and vectorizing the two probe-side counts takes get_breakpoints' scan from 0.055 s to 0.024 s there.

Review findings fixed

  • probes_left/probes_right kept a list-era idiom over what is now an ndarray view, and computed the second count as the complement of the first the long way; verified byte-identical on three fixtures across min_probes in {1, 2, 3, 5, 10}.
  • The label join was written twice; it is now one _join_labels helper, which states the overprint rule once.
  • The probes equality claimed in doc/reports.rst is true of genemetrics without -s and false with it, since -s splits a gene at each breakpoint. Qualified.
  • The locus rule now precedes the column list it is a premise for.
  • The new cross-check against genemetrics inherited min_probes=3, dropping 25 of 79 amplicon loci from its lookup table and reaching a KeyError rather than an assertion for any gene below it.
  • ignore += params.ANTITARGET_ALIASES grew a caller-supplied list in place, by two entries per call, at both sites. The same shape survives at cnvlib/segmentation/__init__.py:464, on the .cns gene-column path, and is left for its own change.

Tests

Nine new cases. Four in test/test_analysis.py pin the gene column against commas, one row per co-binned gene at one breakpoint, probes_left + probes_right against genemetrics, silence for a boundary in the gap between two loci of one name, and a placeholder bin inside a gene counting toward its support. Five in test/test_plots.py pin per-gene labeling, joint labeling for a shared locus, agreement with gene_coords_by_name over the same window, placeholder absorption, and the trailing-comma name. All four breaks cases and three of the five label cases fail against 8e025ea; the other two pin decisions rather than regressions, and were checked against wrong implementations instead — naming-bin-only counts fail on ATM (40 ≠ 65), unjoined per-gene regions fail the shared-locus case.

Full suite green, including the slow marker's frozen end-to-end pipeline; mypy and the doc build clean.

etal added 2 commits August 17, 2026 19:41
get_gene_intervals tallied each bin's whole gene field, so a bin labeled
with two overlapping genes became a third gene that does not exist, and
five such names reached the released breaks table on
test/formats/amplicon.cnr: APC,CTC-554D6.1, ATM,C11orf65, GOPC,ROS1,
CDKN2A,RP11-145E5.5 and KDR,RP11-530I17.1. Anyone filtering that output
for CDKN2A found nothing, though the documented use of the gene column is
exactly that pipeline. It also condensed every occurrence of a name on a
chromosome into one interval, so a segment boundary in the gap between two
loci of a repeat-family name was reported as a breakpoint within the gene,
counting one locus's bins on the left and another's on the right.

Both follow from the same 2014 tally, and both are fixed by grouping bins
into loci with cnvlib.cnary.gene_runs, as by_gene and genemetrics have done
since #1173 and scatter --gene since #1177. A bin naming several genes now
belongs to each of their loci and is reported once per gene, the rule
adopted for the sibling gainloss report in #107.

probes_left and probes_right now count every bin in the locus's run,
including off-target and backbone bins it absorbs: a copy number alteration
affects a genomic region regardless of the gene model, so those bins support
a breakpoint as well as a targeted bin does. This is what genemetrics
already counts in its probes column, and the two sides now sum to it
exactly, where 19 of 58 amplicon rows disagreed with it before.

On amplicon the table goes from 49 rows to 58 and no gene value contains a
comma; real genes recover the bins the invented names held, e.g. APC at
chr5:112090792 from (2, 10) to (2, 39). On a comma-free panel with a SNP
backbone, test/formats/regression/p2-9_2.cnr, the same 42 breakpoints are
reported and two rows' counts change, from absorbing bins labeled CGH.
Against 83 block-mean segments over test/formats/wgs-chr17.cnr, 93 rows
become 218, the 48 comma-joined names become none, and Y_RNA's 38 phantom
cross-locus rows become 6 real ones.
gene_coords_by_range keyed on a bin's whole gene field, so the label for a
region containing the co-binned pair ERBB2,MIR4728 named a gene that does
not exist, and ERBB2 was highlighted twice: once at its own locus and once
as a nested box under the joined name, with a gap where its co-binned bins
had been excluded. MIR4728 was never labeled at all. The function also
extended a name's region from its first occurrence in the window to its
last, so scatter -c chr17 on test/formats/wgs-chr17.cnr drew Y_RNA as a
single 39.1 Mb stripe while scatter -c chr17 -g Y_RNA, rewritten in #1177,
drew its 29 loci separately -- one command, two answers to where a gene is.

Both are fixed by grouping bins with cnvlib.cnary.gene_runs, so the region
form now locates genes as --gene and genemetrics do. Genes whose runs
coincide exactly, as overlapping genes labeled in one bin do, share a
stripe under a joined label rather than stacking identical stripes and
overprinting their labels; that consolidation, and the placeholder-name
guard, are now the one _join_labels helper both entry points call.

On amplicon chr17 the labels go from [ERBB2, 'ERBB2,MIR4728'] to
[ERBB2, MIR4728]; on chr5 APC becomes one contiguous locus,
112,043,331-112,179,950, alongside CTC-554D6.1's own span. Over all of
wgs-chr17 the widest region falls from 39.1 Mb to 1.18 Mb.

The ignore parameter is also no longer widened in place: passing a list
grew the caller's own list by two entries per call.
@codecov

codecov Bot commented Aug 18, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.45455% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 76.39%. Comparing base (8e025ea) to head (b7fb828).

Files with missing lines Patch % Lines
cnvlib/reports.py 90.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1178      +/-   ##
==========================================
+ Coverage   76.25%   76.39%   +0.13%     
==========================================
  Files          75       75              
  Lines        8428     8418      -10     
  Branches     1492     1487       -5     
==========================================
+ Hits         6427     6431       +4     
+ Misses       1568     1556      -12     
+ Partials      433      431       -2     
Flag Coverage Δ
unittests 76.39% <95.45%> (+0.13%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@etal
etal merged commit b4509b4 into master Aug 18, 2026
15 checks passed
@etal
etal deleted the etal/fix/breaks-gene-loci branch August 18, 2026 03:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant