feat(meth): emit Bismark-compatible XR/XG/XM tags - #15
Open
nh13 wants to merge 6 commits into
Open
Conversation
nh13
force-pushed
the
feat/issue-14-meth-sam-tags
branch
2 times, most recently
from
June 15, 2026 05:17
035da72 to
72e23c7
Compare
nh13
marked this pull request as ready for review
June 15, 2026 07:17
Under --meth, write Bismark-compatible read- and genome-conversion
tags on every aligned record. XR follows mate (R1=CT, R2=GA, SE=CT);
XG = "GA" iff (is_R2 XOR r->rev), reproducing the (XR,XG) ->
{OT,OB,CTOT,CTOB} encoding used by Bismark, BSBolt, and DRAGEN.
Unlocks correct strand classification in MethylDackel for
non-directional / PBAT libraries (its FLAG-based fallback miscalls
those silently).
Walk the alignment CIGAR over the unconverted reference window and emit a per-base methylation-call string in SAM SEQ orientation. Alphabet matches Bismark/DRAGEN: . non-cytosine; z/Z CpG un/methylated; x/X CHG; h/H CHH; u/U cytosine in unknown context. The walk works in top-strand-ref coordinates throughout: CIGAR is ref-ordered and SAM SEQ is already revcomp'd to align with the top strand for r->rev=1 reads, so a single forward walk over (seq_pos, ref_pos) produces a string aligned with SAM SEQ with no reversal. For XG=GA (bottom-strand cytosine source), context is read from top-strand ref[ref_pos-1] and ref[ref_pos-2] (CpG iff top[ref_pos-1]=C). Read base is taken in top-strand-ref frame: meth bottom C is preserved and shows G, unmeth bottom C->T BS-converts and shows A. Insertions in the read and soft clips at the read ends produce '.' in the corresponding XM positions.
End-to-end shell test: build a small synthetic CpG-rich reference, align a methylated OT-fragment PE pair with --meth, verify XR/XG follow Bismark's (R1, R2) of OT encoding (CT,CT and GA,CT) and that XM shows Z at every CpG cytosine position. Run from the repo root after make: ./test/test-meth-tags.sh
The XM methylation-call string is emitted in SAM SEQ orientation, where the leading soft-clip length is `qlen - r->qe` for reverse-strand reads (the SAM writer computes clip_len[0] the same way) but `r->qs` only for forward reads. write_meth_tags seeded the XM walk at r->qs unconditionally, so reverse reads with asymmetric soft-clips indexed shifted query bases and emitted mis-framed calls across the aligned region. Seed the walk at `qlen - r->qe` for reverse reads. On a 277,525-pair chr17 em-seq simulation this raises position-matched XR/XG/XM concordance against holodeck ground truth from 99.94% to 99.97% and removes 197 mis-tagged reads.
Adds a read whose 3' tail runs off the reference, so it maps to the reverse strand with a leading soft-clip. The read is the genomic reverse complement with no bisulfite conversion, so every cytosine is methylated and all XM calls must be uppercase. The previous r->qs-seeded walk emitted spurious lowercase calls; this case fails before the framing fix and passes after.
nh13
force-pushed
the
feat/issue-14-meth-sam-tags
branch
from
June 19, 2026 05:55
5a15366 to
f3fb49f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements #14. Emits Bismark-compatible
XR:Z/XG:Z/XM:Ztags under--meth.Commits
feat(meth): emit XR:Z and XG:Z SAM tags— write XR (read conversion) and XG (genome strand conversion) underMB_F_METH. XR follows mate (R1=CT, R2=GA, SE=CT); XG = "GA" iff (is_R2 XOR r->rev), reproducing Bismark's (XR,XG) -> {OT, OB, CTOT, CTOB} encoding.feat(meth): emit XM:Z methylation call string— walk the alignment CIGAR over the unconverted reference window and emit a per-base methylation-call string in SAM SEQ orientation. Alphabet matches Bismark/DRAGEN:.non-cytosine;z/ZCpG;x/XCHG;h/HCHH;u/Ucytosine in unknown context.test(meth): smoke test for XR/XG/XM tag emission—test/test-meth-tags.sh, an end-to-end shell test over a synthetic CpG-rich reference and a methylated OT-fragment PE pair.fix(meth): correct XM frame for reverse-strand soft-clipped reads— see below.test(meth): cover reverse-strand soft-clipped XM framing— regression test for the fix; fails before, passes after.NM:iis unchanged and remains bisulfite-aware: on the 4-base master the asymmetric matrix scores ref-C/read-T (and ref-G/read-A) as a match, so converted positions are not counted as edits (mean NM ≈ 0.57 on fully-converted reads).MD:Zis not emitted; no methylation tool reads it.Encoding under directional
--methXM walk
Walks (seq_pos, ref_pos) forward through the CIGAR. SAM SEQ is always in top-strand-ref orientation (
sam_write_sqrevcomps whenr->rev=1), so a single forward walk produces a string aligned with SAM SEQ in display order; no end-of-walk reversal.The walk is seeded at the SAM leading soft-clip offset:
r->qsfor forward reads,qlen - r->qefor reverse reads (mirroringclip_len[0]in the SAM writer, sincer->qs/r->qeare in original-read coordinates). Insertions (I) and soft clips at the read ends produce.at the corresponding XM positions.For XG=CT (top-strand source), context is read from
ref[ref_pos+1]andref[ref_pos+2]. Read base in top-strand frame: C -> methylated, T -> unmethylated.For XG=GA (bottom-strand source), the bottom-strand 5'->3' context base is
comp(top[ref_pos-1]), so CpG ifftop[ref_pos-1]=C, etc. Read base in top-strand frame: G -> methylated (bottom C preserved, comp = G), A -> unmethylated (bottom C->T BS, comp = A).Reverse-strand soft-clip fix
The XM string is in SAM SEQ orientation, where the leading soft-clip length is
qlen - r->qefor reverse-strand reads butr->qsonly for forward reads. The originalwrite_meth_tagsseeded the walk atr->qsunconditionally, so reverse reads with asymmetric soft-clips indexed shifted query bases and emitted mis-framed calls across the aligned region. The smoke test missed this because its fragment was full-length150M(no clip). Fixed by seeding the walk atqlen - r->qefor reverse reads; the new test maps a read whose 3' tail runs off the reference (forcing a reverse alignment with a leading soft-clip) and asserts no spurious calls.Validation
Smoke test (synthetic, in-tree)
holodeck ground-truth concordance, chr17 1x em-seq (rerun on 4-base master)
Simulated 277,525 PE fragments with holodeck
0.2.1(methylate-> annotated VCF, thensimulate --methylation-mode em-seq --golden-bam, seed 42), which writes a golden BAM with Bismark-compatible XR/XG/XM tags from the simulator's per-haplotype methylation truth. Aligned withminibwa map --meth, then compared XR+XG+XM per (QNAME, mate). All 555,050 reads mapped (0 unmapped).The ~23k position differences are segdup/homolog misplacements (minibwa placed the read at a different chr17 position than the simulator's source); the XM at minibwa's chosen position is internally consistent with the reference window it aligned to. Of the 143 remaining position-matched differences, all are cases where minibwa chose a different soft-clip/indel boundary than the simulator's truth CIGAR (so XM legitimately shifts); only 5 share an identical CIGAR, and those are conservative
u(unknown-context) calls next to referenceNbases. XR and XG are 100% concordant on all position-matched reads.The reverse-clip fix above raised position-matched concordance from 99.94% to 99.97% (removing 197 mis-tagged reverse reads).
Example head-to-head from a position-matched fragment:
Open follow-ups (not in this PR)
--methoutput through MethylDackel and compare per-CpG bedGraph against holodeck's--cpg-truth-bedgraphand against bwameth/Bismark on the same reads.build.ymlcurrently builds api-test only).