docs(embeddinggemma): quantization is bf16, not Q4_1 - #693
Open
Aitschend3251 wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
docs/docs/models/embeddinggemma.mdline 14 declaresQ4_1. The shipped model is not quantised.What the repo itself says.
src/model_list.jsongivesembed-gemma/300m:It is the only entry in that file carrying
none, and the docs page is the only placeQ4_1appears for this model.What the artefact says.
model.q4nxis 615,197,168 bytes for a ~308M-parameter model — 2.0 bytes per parameter, i.e. bf16 despite the.q4nxextension. Genuine 4-bit weights would be near 0.5 bytes/parameter. Verified independently on two machines and two FLM builds (v0.9.45 and v1.0.3), and a header parse reports 316 of 316 tensors as BF16; the same parser finds 197 quantised tensors in theQwen3-1.7B-NPU2container, so that is a finding rather than a parser artefact.Why
bf16and notnone.docs/docs/models/smolvla.mdalready uses- **Quantization:** bf16, so this follows existing usage on the docs side while agreeing withmodel_list.json.Why it is worth fixing. This line sent #661 down a dead end: the opening report named coarse quantisation as the likely cause of an embedding defect, on the strength of this line, and two of us spent time ruling it out. The likely origin looks innocent — the v1.0.3 release notes upgrade the Qwen3.5 family and Qwen3.6-MoE "from Q4_1 to Q4_K", so
Q4_1was a real value for the LLM containers; it just does not describe this one.One-line change, docs only.