Repository navigation
test(qjl): pin the exact norm ratio and the shrinkage MSE - #100
Open
devYRPauli wants to merge 2 commits into
Open
devYRPauli wants to merge 2 commits into
devYRPauli wants to merge 2 commits into
Conversation
With an orthogonal S, ||x_hat|| / ||x|| is exactly sqrt(pi/2) for every vector. The old check accepted any average between 0.5 and 2.0, so it passed with QJL_CONST removed from the scale. Add a test for the closed forms in the dequantize docstring: relative MSE is pi/2 - 1 at shrinkage 1.0 and 1 - 2/pi at 2/pi. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
The CI failure is not from this change. CI on main fails the same way: since 91ad2fe, pyproject.toml has no runtime dependencies, so the install step does not install numpy or scipy and every test module fails at import. With numpy and scipy installed, the full suite passes locally on Python 3.12: 559 passed, 6 skipped, 95.29% coverage. |
91ad2fe moved numpy and scipy into a turboquant extra, and 8bf48f6 dropped that extra when pyproject.toml became the refract-llm package. CI installs only the dev extra, so every test module that imports turboquant fails with ModuleNotFoundError. CI on main has failed on every push since 91ad2fe. Install the two packages in the workflow, with the bounds the old pyproject.toml declared. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
I added a second commit that fixes CI. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #93. This settles the open question about
test_dequantized_has_correct_scale.Problem
test_dequantized_has_correct_scaleaccepts any average norm ratio between 0.5 and 2.0, and its comment says that the ratio should be near 1.0. With an orthogonal S, the ratio is exactly sqrt(pi/2) (about 1.2533) for every vector. IfQJL_CONSTis removed from the scale indequantize, the ratio becomes 1.0, and all 13 tests intests/test_qjl.pystill pass. No test checks the 2/pi shrinkage derivation in thedequantizedocstring.Change
test_dequantized_has_correct_scalechecks that the norm ratio of each vector equalsQJL_CONST(rtol 1e-9).test_shrinkage_mse_matches_closed_formchecks the mean relative MSE at d in {64, 128, 256, 512}:pi/2 - 1at shrinkage 1.0 and1 - 2/piat shrinkage 2/pi, with a tolerance of 0.02.The second commit fixes CI. 91ad2fe moved numpy and scipy into a
turboquantextra, and 8bf48f6 removed that extra. CI installs only thedevextra, so every run onmainsince 91ad2fe fails withModuleNotFoundError: No module named 'numpy'..github/workflows/ci.ymlnow installs both packages with the bounds the oldpyproject.tomldeclared.Testing
pytest tests/ --cov=turboquant --cov-fail-under=95: 559 passed, 6 skipped, coverage 95.29% (Python 3.12, numpy 2.5.3, scipy 1.18.1).QJL_CONSTremoved from the scale, the old tests pass and 8 of the new cases fail.shrinkageignored indequantize, 4 of the new cases fail.