Skip to content

Fail clearly on MS-Numpress arrays when pynumpress is missing - #225

Merged
levitsky merged 1 commit into
levitsky:masterfrom
aaronkanzer:fix/numpress-without-pynumpress
Oct 26, 2026
Merged

levitsky merged 1 commit into
levitsky:masterfrom
aaronkanzer:fix/numpress-without-pynumpress

Conversation

@aaronkanzer

Copy link
Copy Markdown
Contributor

pynumpress is optional, and neither pip install pyteomics nor packages that depend on pyteomics (matchms, for example) install it. Without it, pyteomics 5.0.1 reads a numpress-encoded array as if it were uncompressed:

from pyteomics import mzml
s = next(mzml.read("lin_mult8.mzML"))   # m/z array: MS-Numpress linear prediction compression, 72 bytes
s["m/z array"][:4]
# array([ 2.04753471e-312, nan, 9.76797317e-190, -6.03570075e+030])   no warning

When the encoded length is not a multiple of the dtype size, you get ValueError: buffer size must be a multiple of element size instead. All five real numpress files I tried (msconvert and OpenMS outputs) raise that error, which does not mention pynumpress.

Why: the numpress names are only added to _default_compression_map when pynumpress imports. Without it, _determine_compression finds no known compression term, _decompress gets None, and it returns the raw bytes unchanged.

Fix: when pynumpress is missing, still register the six numpress terms, mapped to a decoder that raises PyteomicsError("Decoding '<term>' requires pynumpress."). Nothing changes when pynumpress is installed. #198 uses the same pattern for the zstd codecs.

Tested:

  • New test_numpress_without_pynumpress runs in a subprocess with sys.modules['pynumpress'] = None and checks that all six terms raise PyteomicsError. It fails on master (TypeError: 'NoneType' object is not callable) and passes with the fix.
  • Full suite (all tests/test_*.py except test_usi and test_unimod, which need the network): 467 passed, 10 failed before. 468 passed, 10 failed after. The 10 failures are the same both times: the process-based map() tests fail with OSError: [Errno 22] in my sandbox.
  • Without pynumpress, the non-numpress mzML files (pyteomics, pymzml, psims and matchms test data) still read in full.

Side note for the maintainers, not part of this PR: in draft #198, the "MS-Numpress linear prediction compression followed by zstd compression" entry is mapped to _zlibNumpress(...) instead of _zstdNumpress(...).

Without pynumpress the numpress compression terms were not in the
compression map, so the array was decoded as if it were uncompressed.
Depending on the encoded length that either raised 'buffer size must be
a multiple of element size' or silently returned garbage values (NaN,
1e-312, ...). Register the six terms with a decoder that raises
PyteomicsError naming pynumpress.
@levitsky
levitsky merged commit 6b57fc4 into levitsky:master Oct 26, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants