Repository navigation
Locate breaks and scatter-region genes per locus, not per gene field - #1178
Merged
Merged
Conversation
get_gene_intervals tallied each bin's whole gene field, so a bin labeled with two overlapping genes became a third gene that does not exist, and five such names reached the released breaks table on test/formats/amplicon.cnr: APC,CTC-554D6.1, ATM,C11orf65, GOPC,ROS1, CDKN2A,RP11-145E5.5 and KDR,RP11-530I17.1. Anyone filtering that output for CDKN2A found nothing, though the documented use of the gene column is exactly that pipeline. It also condensed every occurrence of a name on a chromosome into one interval, so a segment boundary in the gap between two loci of a repeat-family name was reported as a breakpoint within the gene, counting one locus's bins on the left and another's on the right. Both follow from the same 2014 tally, and both are fixed by grouping bins into loci with cnvlib.cnary.gene_runs, as by_gene and genemetrics have done since #1173 and scatter --gene since #1177. A bin naming several genes now belongs to each of their loci and is reported once per gene, the rule adopted for the sibling gainloss report in #107. probes_left and probes_right now count every bin in the locus's run, including off-target and backbone bins it absorbs: a copy number alteration affects a genomic region regardless of the gene model, so those bins support a breakpoint as well as a targeted bin does. This is what genemetrics already counts in its probes column, and the two sides now sum to it exactly, where 19 of 58 amplicon rows disagreed with it before. On amplicon the table goes from 49 rows to 58 and no gene value contains a comma; real genes recover the bins the invented names held, e.g. APC at chr5:112090792 from (2, 10) to (2, 39). On a comma-free panel with a SNP backbone, test/formats/regression/p2-9_2.cnr, the same 42 breakpoints are reported and two rows' counts change, from absorbing bins labeled CGH. Against 83 block-mean segments over test/formats/wgs-chr17.cnr, 93 rows become 218, the 48 comma-joined names become none, and Y_RNA's 38 phantom cross-locus rows become 6 real ones.
gene_coords_by_range keyed on a bin's whole gene field, so the label for a region containing the co-binned pair ERBB2,MIR4728 named a gene that does not exist, and ERBB2 was highlighted twice: once at its own locus and once as a nested box under the joined name, with a gap where its co-binned bins had been excluded. MIR4728 was never labeled at all. The function also extended a name's region from its first occurrence in the window to its last, so scatter -c chr17 on test/formats/wgs-chr17.cnr drew Y_RNA as a single 39.1 Mb stripe while scatter -c chr17 -g Y_RNA, rewritten in #1177, drew its 29 loci separately -- one command, two answers to where a gene is. Both are fixed by grouping bins with cnvlib.cnary.gene_runs, so the region form now locates genes as --gene and genemetrics do. Genes whose runs coincide exactly, as overlapping genes labeled in one bin do, share a stripe under a joined label rather than stacking identical stripes and overprinting their labels; that consolidation, and the placeholder-name guard, are now the one _join_labels helper both entry points call. On amplicon chr17 the labels go from [ERBB2, 'ERBB2,MIR4728'] to [ERBB2, MIR4728]; on chr5 APC becomes one contiguous locus, 112,043,331-112,179,950, alongside CTC-554D6.1's own span. Over all of wgs-chr17 the widest region falls from 39.1 Mb to 1.18 Mb. The ignore parameter is also no longer widened in place: passing a list grew the caller's own list by two entries per call.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1178 +/- ##
==========================================
+ Coverage 76.25% 76.39% +0.13%
==========================================
Files 75 75
Lines 8428 8418 -10
Branches 1492 1487 -5
==========================================
+ Hits 6427 6431 +4
+ Misses 1568 1556 -12
+ Partials 433 431 -2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two gene-name consumers keyed on a bin's whole
genefield instead of the individual names it comma-joins, so a bin covering two genes became a third gene that does not exist, and both still used the first-to-last-occurrence span rule that #1173 and #1177 removed fromby_geneandscatter --gene.cnvlib/reports.py::get_gene_intervalsfeeds the releasedbreakstable;cnvlib/plots.py::gene_coords_by_rangefeedsscatter -c chr:start-endlabels. Both now group bins into loci withcnvlib.cnary.gene_runs, so one rule servesgenemetrics, the diagram,scatter --gene,scatter -c regionandbreaks.Both functions date to the 2014 import, when
target'sshortest_namecollapsed multi-accession labels to one name and CNVkit never emitted a comma-joinedgenevalue. Whole-field keying was correct by construction and became wrong in 203ed6d (2016-12-10), when annotation started flowing throughjoin_strings. Nothing defends it: the maintainer's answer in #15 (2015) is that aFOO,BARbin "really represent[s] 2 or more genes", and #107 → aef8b78 (2016-05-13) chose one row per real gene, with overlapping spans, for the sibling gainloss report. Thebreakspath was simply never revisited; #220 (2017) is a user asking what aY_RNArow means, answered in prose rather than in code.Decisions
A co-binned breakpoint is reported once per gene, not once under a joined name. That is #107's rule, and the gene column's documented use is
breaks Sample.cnr Sample.cns | cut -f1 | sort -u > gene-breaks.txt, whose stated purpose is a list of gene names and whosesort -uabsorbs the duplication. Under the old rule a reader filtering that file for CDKN2A found nothing, because CDKN2A's breakpoints were filed underCDKN2A,RP11-145E5.5.Plot labels do the opposite where spans coincide. Genes whose runs have identical bounds share one stripe under a joined label, because two identical
axvspanstripes with twotextcalls at the same coordinates overprint. This is the mechanicsgene_coords_by_nameadopted in #1177 for #458, and it is the only point where the two surfaces diverge: a table row is filtered, a stripe is looked at.probes_leftandprobes_rightcount every bin in the locus's run, including off-target bins and backbone baits it absorbs. A copy number alteration affects a genomic region regardless of the gene model, so aCGHbackbone bin or an antitarget bin inside a gene supports a breakpoint there as well as a targeted bin does.genemetricsalready counts exactly those bins in itsprobescolumn, and the two sides now sum to it; before, 19 of 58 amplicon rows disagreed with it, and 5 more disagreed because their gene did not exist.test/formats/regression/p2-9_2.cnris the UCSF500-shaped case: 1120CGHbins and 12451Background, 37 of them inside a real gene's run.A trailing comma in the gene field names nothing. 171 bins of
test/formats/wgs-chr17.cnrare labeledMIR3186,MIR4740,, which splits to a third, empty name;if gene:skips it at both sites, mirroringreports.group_by_genes. Adding""to the tuple handed togene_runswas rejected: it would make such a bin absorbable rather than run-terminating, diverging fromby_gene— a subtler disagreement than the one being fixed.Measured
Baseline worktree at 8e025ea, both trees discriminated by
cnvlib.__file__under the exact invocation.breaks amplicon.cnr amplicon.cns-m 4breaks regression/p2-9_2.cnr(comma-free)breaks wgs-chr17.cnr, 83 block-mean segmentsOn amplicon all five invented names disappear (
APC,CTC-554D6.1,ATM,C11orf65,GOPC,ROS1,CDKN2A,RP11-145E5.5,KDR,RP11-530I17.1), the documentedcut -f1 | sort -ulist goes from 26 names to 29 real ones, and real genes recover the bins the phantoms held: APC chr5:112090792 (2, 10) → (2, 39), ATM chr11:108143413 (22, 18) → (22, 43), SMAD4 chr18:48592058 (2, 4) → (9, 4). On p2-9_2 the same 42 breakpoints are reported and two rows' counts change, PTPRD chr9:8733906 (35, 2) → (35, 10) and chr9:9003938 (36, 1) → (37, 8), both from absorbedCGHbins — that fixture isolates the counting decision from the comma fix. On wgs-chr17,Y_RNAhad one 39.1 Mb interval (chr17:7,440,413-46,556,560) built from the 3 bins whose whole field was exactlyY_RNA, though 7706 bins name it; every segment boundary inside that span was a phantom row, 38 of them, and they sorted high because the ranking key ismin(probes_left, probes_right). They become 6 real rows, and the widest interval on the chromosome falls to 1.18 Mb.For labels, amplicon chr17 goes from
[ERBB2, 'ERBB2,MIR4728']to[ERBB2, MIR4728], and chr5 from two APC boxes split by a gap to one contiguous locus 112,043,331-112,179,950 beside CTC-554D6.1's own span; renderings were compared in both trees. Over all of wgs-chr17 the widest region falls from 39.1 Mb to 1.18 Mb, and-c chr17now agrees with-c chr17 -g Y_RNAon all 13 loci in the window they share.get_gene_intervalsalso got faster despite producing 76% more entries on wgs-chr17 (0.203 s → 0.144 s), the per-rowitertuplesloop being gone, and vectorizing the two probe-side counts takesget_breakpoints' scan from 0.055 s to 0.024 s there.Review findings fixed
probes_left/probes_rightkept a list-era idiom over what is now an ndarray view, and computed the second count as the complement of the first the long way; verified byte-identical on three fixtures acrossmin_probesin {1, 2, 3, 5, 10}._join_labelshelper, which states the overprint rule once.probesequality claimed indoc/reports.rstis true ofgenemetricswithout-sand false with it, since-ssplits a gene at each breakpoint. Qualified.genemetricsinheritedmin_probes=3, dropping 25 of 79 amplicon loci from its lookup table and reaching aKeyErrorrather than an assertion for any gene below it.ignore += params.ANTITARGET_ALIASESgrew a caller-supplied list in place, by two entries per call, at both sites. The same shape survives atcnvlib/segmentation/__init__.py:464, on the.cnsgene-column path, and is left for its own change.Tests
Nine new cases. Four in
test/test_analysis.pypin the gene column against commas, one row per co-binned gene at one breakpoint,probes_left + probes_rightagainstgenemetrics, silence for a boundary in the gap between two loci of one name, and a placeholder bin inside a gene counting toward its support. Five intest/test_plots.pypin per-gene labeling, joint labeling for a shared locus, agreement withgene_coords_by_nameover the same window, placeholder absorption, and the trailing-comma name. All fourbreakscases and three of the five label cases fail against 8e025ea; the other two pin decisions rather than regressions, and were checked against wrong implementations instead — naming-bin-only counts fail on ATM (40 ≠ 65), unjoined per-gene regions fail the shared-locus case.Full suite green, including the
slowmarker's frozen end-to-end pipeline; mypy and the doc build clean.