Skip to content

Speed up coverage_mask with str.find and a difference array - #229

Merged
levitsky merged 1 commit into
levitsky:masterfrom
TheNetherWatcher:perf/coverage-mask-find
Nov 4, 2026
Merged

levitsky merged 1 commit into
levitsky:masterfrom
TheNetherWatcher:perf/coverage-mask-find

Conversation

@TheNetherWatcher

Copy link
Copy Markdown
Contributor

coverage_mask located each peptide with re.finditer(f'(?={peptide})', protein). A lookahead pattern can't use the regex engine's literal prefix search, so every peptide costs a full regex scan of the protein, and each occurrence then adds 1 over its slice of the mask.

This finds the same overlapping occurrences with protein.find(peptide, i + 1), records +1 where an occurrence starts and -1 where it ends, and returns the running sum. Empty peptides are skipped (they matched everywhere before but added nothing). The result is the same array with the same dtype.

On a random 35,000-residue protein with tryptic peptides:

Peptides Before After
2,000 0.42 s 0.023 s
4,606 0.98 s 0.054 s

coverage() calls coverage_mask(), so it gets the same speedup.

Checks:

  • New tests for overlapping and repeated occurrences, empty and missing peptides and an empty protein. They pass on the old code too.
  • 200,000 random cases (2-, 3- and 20-letter alphabets) give identical masks and dtype to the old implementation.
  • A Lean 4 proof that the find loop yields exactly the lookahead matches and that the running sum equals the old mask at every residue: https://github.com/TheNetherWatcher/lean-perf-proofs/tree/main/pyteomics-coverage-mask

One difference, outside the documented input: a peptide containing regex metacharacters was matched as a pattern before and is now matched literally. Peptides are documented as plain one-letter sequences, and fast=False strips them to that.

Each peptide was located with a lookahead regex, which cannot use a
literal prefix search, and each occurrence added 1 over its span.
Find the same overlapping occurrences with str.find and add them as
+1/-1 at their ends, then take the running sum. The mask is the same.
@levitsky
levitsky merged commit f4f5029 into levitsky:master Nov 4, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants