Skip to content

feat(meth): emit Bismark-compatible XR/XG/XM tags - #15

Open
nh13 wants to merge 6 commits into
masterfrom
feat/issue-14-meth-sam-tags
Open

feat(meth): emit Bismark-compatible XR/XG/XM tags#15
nh13 wants to merge 6 commits into
masterfrom
feat/issue-14-meth-sam-tags

Conversation

@nh13

@nh13 nh13 commented May 9, 2026

Copy link
Copy Markdown
Collaborator

Implements #14. Emits Bismark-compatible XR:Z / XG:Z / XM:Z tags under --meth.

Rebased onto the 4-base --meth master. This branch originally sat on the older 3-letter meth lineage. It has been rebased onto current master, where seeding is 3-letter but base alignment and mate rescue run in 4-base space against the original reference with an asymmetric scoring matrix (the BISCUIT-style strategy). The tag code is decoupled from that change — it consumes the final CIGAR, coordinates, original read, and original reference (l2b_getseq) — so it rebased cleanly, and the revalidation below was rerun on the 4-base aligner.

Commits

  1. feat(meth): emit XR:Z and XG:Z SAM tags — write XR (read conversion) and XG (genome strand conversion) under MB_F_METH. XR follows mate (R1=CT, R2=GA, SE=CT); XG = "GA" iff (is_R2 XOR r->rev), reproducing Bismark's (XR,XG) -> {OT, OB, CTOT, CTOB} encoding.
  2. feat(meth): emit XM:Z methylation call string — walk the alignment CIGAR over the unconverted reference window and emit a per-base methylation-call string in SAM SEQ orientation. Alphabet matches Bismark/DRAGEN: . non-cytosine; z/Z CpG; x/X CHG; h/H CHH; u/U cytosine in unknown context.
  3. test(meth): smoke test for XR/XG/XM tag emissiontest/test-meth-tags.sh, an end-to-end shell test over a synthetic CpG-rich reference and a methylated OT-fragment PE pair.
  4. fix(meth): correct XM frame for reverse-strand soft-clipped reads — see below.
  5. test(meth): cover reverse-strand soft-clipped XM framing — regression test for the fix; fails before, passes after.

NM:i is unchanged and remains bisulfite-aware: on the 4-base master the asymmetric matrix scores ref-C/read-T (and ref-G/read-A) as a match, so converted positions are not counted as edits (mean NM ≈ 0.57 on fully-converted reads). MD:Z is not emitted; no methylation tool reads it.

Encoding under directional --meth

Mate, r->rev XR:Z XG:Z Bismark strand
R1, rev=0 CT CT OT
R1, rev=1 CT GA OB
R2, rev=0 GA GA CTOB
R2, rev=1 GA CT CTOT

XM walk

Walks (seq_pos, ref_pos) forward through the CIGAR. SAM SEQ is always in top-strand-ref orientation (sam_write_sq revcomps when r->rev=1), so a single forward walk produces a string aligned with SAM SEQ in display order; no end-of-walk reversal.

The walk is seeded at the SAM leading soft-clip offset: r->qs for forward reads, qlen - r->qe for reverse reads (mirroring clip_len[0] in the SAM writer, since r->qs/r->qe are in original-read coordinates). Insertions (I) and soft clips at the read ends produce . at the corresponding XM positions.

For XG=CT (top-strand source), context is read from ref[ref_pos+1] and ref[ref_pos+2]. Read base in top-strand frame: C -> methylated, T -> unmethylated.

For XG=GA (bottom-strand source), the bottom-strand 5'->3' context base is comp(top[ref_pos-1]), so CpG iff top[ref_pos-1]=C, etc. Read base in top-strand frame: G -> methylated (bottom C preserved, comp = G), A -> unmethylated (bottom C->T BS, comp = A).

Reverse-strand soft-clip fix

The XM string is in SAM SEQ orientation, where the leading soft-clip length is qlen - r->qe for reverse-strand reads but r->qs only for forward reads. The original write_meth_tags seeded the walk at r->qs unconditionally, so reverse reads with asymmetric soft-clips indexed shifted query bases and emitted mis-framed calls across the aligned region. The smoke test missed this because its fragment was full-length 150M (no clip). Fixed by seeding the walk at qlen - r->qe for reverse reads; the new test maps a read whose 3' tail runs off the reference (forcing a reverse alignment with a leading soft-clip) and asserts no spurious calls.

Validation

Smoke test (synthetic, in-tree)

$ ./test/test-meth-tags.sh
PASS: --meth XR/XG/XM tags emit correctly for a methylated OT fragment
PASS: --meth XM is framed correctly for reverse-strand soft-clipped reads

holodeck ground-truth concordance, chr17 1x em-seq (rerun on 4-base master)

Simulated 277,525 PE fragments with holodeck 0.2.1 (methylate -> annotated VCF, then simulate --methylation-mode em-seq --golden-bam, seed 42), which writes a golden BAM with Bismark-compatible XR/XG/XM tags from the simulator's per-haplotype methylation truth. Aligned with minibwa map --meth, then compared XR+XG+XM per (QNAME, mate). All 555,050 reads mapped (0 unmapped).

XR+XG+XM byte-identical
overall (555,050 reads) 548,125 (98.75%)
position+strand matched (531,946) 531,803 (99.97%)

The ~23k position differences are segdup/homolog misplacements (minibwa placed the read at a different chr17 position than the simulator's source); the XM at minibwa's chosen position is internally consistent with the reference window it aligned to. Of the 143 remaining position-matched differences, all are cases where minibwa chose a different soft-clip/indel boundary than the simulator's truth CIGAR (so XM legitimately shifts); only 5 share an identical CIGAR, and those are conservative u (unknown-context) calls next to reference N bases. XR and XG are 100% concordant on all position-matched reads.

The reverse-clip fix above raised position-matched concordance from 99.94% to 99.97% (removing 197 mis-tagged reverse reads).

Example head-to-head from a position-matched fragment:

                     XR:Z  XG:Z  XM:Z
golden  R1 (rev=0):  CT    CT    .......x..........z...h...x........x..h...............hh.h.x.........z........Z........xZ.h.x....h..
minibwa R1 (rev=0):  CT    CT    .......x..........z...h...x........x..h...............hh.h.x.........z........Z........xZ.h.x....h..
golden  R2 (rev=1):  GA    CT    .....z........xZ.h.x....h....x.....h...h.x.....h..h.............h...h....h.x.........h...hhh.....x..
minibwa R2 (rev=1):  GA    CT    .....z........xZ.h.x....h....x.....h...h.x.....h..h.............h...h....h.x.........h...hhh.....x..

Open follow-ups (not in this PR)

  • Pipe minibwa --meth output through MethylDackel and compare per-CpG bedGraph against holodeck's --cpg-truth-bedgraph and against bwameth/Bismark on the same reads.
  • Add the meth smoke test to CI (build.yml currently builds api-test only).

@nh13
nh13 force-pushed the feat/issue-14-meth-sam-tags branch 2 times, most recently from 035da72 to 72e23c7 Compare June 15, 2026 05:17
@nh13
nh13 marked this pull request as ready for review June 15, 2026 07:17
nh13 added 6 commits June 18, 2026 22:54
Under --meth, write Bismark-compatible read- and genome-conversion
tags on every aligned record. XR follows mate (R1=CT, R2=GA, SE=CT);
XG = "GA" iff (is_R2 XOR r->rev), reproducing the (XR,XG) ->
{OT,OB,CTOT,CTOB} encoding used by Bismark, BSBolt, and DRAGEN.

Unlocks correct strand classification in MethylDackel for
non-directional / PBAT libraries (its FLAG-based fallback miscalls
those silently).
Walk the alignment CIGAR over the unconverted reference window and
emit a per-base methylation-call string in SAM SEQ orientation.
Alphabet matches Bismark/DRAGEN: . non-cytosine; z/Z CpG un/methylated;
x/X CHG; h/H CHH; u/U cytosine in unknown context.

The walk works in top-strand-ref coordinates throughout: CIGAR is
ref-ordered and SAM SEQ is already revcomp'd to align with the top
strand for r->rev=1 reads, so a single forward walk over (seq_pos,
ref_pos) produces a string aligned with SAM SEQ with no reversal.

For XG=GA (bottom-strand cytosine source), context is read from
top-strand ref[ref_pos-1] and ref[ref_pos-2] (CpG iff top[ref_pos-1]=C).
Read base is taken in top-strand-ref frame: meth bottom C is preserved
and shows G, unmeth bottom C->T BS-converts and shows A.

Insertions in the read and soft clips at the read ends produce '.'
in the corresponding XM positions.
End-to-end shell test: build a small synthetic CpG-rich reference,
align a methylated OT-fragment PE pair with --meth, verify XR/XG
follow Bismark's (R1, R2) of OT encoding (CT,CT and GA,CT) and that
XM shows Z at every CpG cytosine position.

Run from the repo root after make: ./test/test-meth-tags.sh
The XM methylation-call string is emitted in SAM SEQ orientation, where the
leading soft-clip length is `qlen - r->qe` for reverse-strand reads (the SAM
writer computes clip_len[0] the same way) but `r->qs` only for forward reads.
write_meth_tags seeded the XM walk at r->qs unconditionally, so reverse reads
with asymmetric soft-clips indexed shifted query bases and emitted mis-framed
calls across the aligned region.

Seed the walk at `qlen - r->qe` for reverse reads. On a 277,525-pair chr17
em-seq simulation this raises position-matched XR/XG/XM concordance against
holodeck ground truth from 99.94% to 99.97% and removes 197 mis-tagged reads.
Adds a read whose 3' tail runs off the reference, so it maps to the reverse
strand with a leading soft-clip. The read is the genomic reverse complement
with no bisulfite conversion, so every cytosine is methylated and all XM calls
must be uppercase. The previous r->qs-seeded walk emitted spurious lowercase
calls; this case fails before the framing fix and passes after.
@nh13
nh13 force-pushed the feat/issue-14-meth-sam-tags branch from 5a15366 to f3fb49f Compare June 19, 2026 05:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant