Skip to content

test(qjl): pin the exact norm ratio and the shrinkage MSE - #100

Open
devYRPauli wants to merge 2 commits into
TheTom:mainfrom
devYRPauli:test/qjl-scale-and-shrinkage
Open

devYRPauli wants to merge 2 commits into
TheTom:mainfrom
devYRPauli:test/qjl-scale-and-shrinkage

Conversation

@devYRPauli

@devYRPauli devYRPauli commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Follow-up to #93. This settles the open question about test_dequantized_has_correct_scale.

Problem

test_dequantized_has_correct_scale accepts any average norm ratio between 0.5 and 2.0, and its comment says that the ratio should be near 1.0. With an orthogonal S, the ratio is exactly sqrt(pi/2) (about 1.2533) for every vector. If QJL_CONST is removed from the scale in dequantize, the ratio becomes 1.0, and all 13 tests in tests/test_qjl.py still pass. No test checks the 2/pi shrinkage derivation in the dequantize docstring.

Change

  • test_dequantized_has_correct_scale checks that the norm ratio of each vector equals QJL_CONST (rtol 1e-9).
  • The new test_shrinkage_mse_matches_closed_form checks the mean relative MSE at d in {64, 128, 256, 512}: pi/2 - 1 at shrinkage 1.0 and 1 - 2/pi at shrinkage 2/pi, with a tolerance of 0.02.

The second commit fixes CI. 91ad2fe moved numpy and scipy into a turboquant extra, and 8bf48f6 removed that extra. CI installs only the dev extra, so every run on main since 91ad2fe fails with ModuleNotFoundError: No module named 'numpy'. .github/workflows/ci.yml now installs both packages with the bounds the old pyproject.toml declared.

Testing

  • pytest tests/ --cov=turboquant --cov-fail-under=95: 559 passed, 6 skipped, coverage 95.29% (Python 3.12, numpy 2.5.3, scipy 1.18.1).
  • With QJL_CONST removed from the scale, the old tests pass and 8 of the new cases fail.
  • With shrinkage ignored in dequantize, 4 of the new cases fail.
  • The CI install steps with the new line, replayed with pip in a fresh Python 3.12 venv: 565 passed, coverage 95.29%.

With an orthogonal S, ||x_hat|| / ||x|| is exactly sqrt(pi/2) for
every vector. The old check accepted any average between 0.5 and 2.0,
so it passed with QJL_CONST removed from the scale.

Add a test for the closed forms in the dequantize docstring: relative
MSE is pi/2 - 1 at shrinkage 1.0 and 1 - 2/pi at 2/pi.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@devYRPauli

Copy link
Copy Markdown
Contributor Author

The CI failure is not from this change. CI on main fails the same way: since 91ad2fe, pyproject.toml has no runtime dependencies, so the install step does not install numpy or scipy and every test module fails at import. With numpy and scipy installed, the full suite passes locally on Python 3.12: 559 passed, 6 skipped, 95.29% coverage.

91ad2fe moved numpy and scipy into a turboquant extra, and 8bf48f6
dropped that extra when pyproject.toml became the refract-llm package.
CI installs only the dev extra, so every test module that imports
turboquant fails with ModuleNotFoundError. CI on main has failed on
every push since 91ad2fe.

Install the two packages in the workflow, with the bounds the old
pyproject.toml declared.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@devYRPauli

Copy link
Copy Markdown
Contributor Author

I added a second commit that fixes CI. .github/workflows/ci.yml now installs numpy and scipy, because pyproject.toml no longer declares them. All 3 test jobs pass on this PR now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant