Repository navigation
QE2E-INV-001 asserts Qwen3.5-9B ∈ [9.0B, 9.2B] but the repo's own 9b descriptor sums to 8.209B — the descriptor declares no DeltaNet tensors #3346
Description
QE2E-INV-001 asserts 9.0–9.2 B, but the repo's own 9B descriptor sums to 8.209 B
Found while writing the missing QE2E bodies for #3091/#3114. Nothing was widened or hardcoded to make a test pass — the equation is proved, the range instantiation is not.
contracts/qwen35-e2e-verification-v1.yaml:
QE2E-INV-001 invariant "Parameter count matches architecture"
formal: P(Qwen3.5-9B) ∈ [9.0B, 9.2B]
Evaluating that contract's own model_parameter_count formula on the repo's own constants — contracts/model-families/qwen3_5.yaml, size_variants.9b: V=248320, d=4096, L=32, n_h=16, d_k=256, intermediate=12288, untied lm_head — gives
8,208,535,552 = 8.209 B (outside [9.0B, 9.2B] by ~0.8 B)
The same descriptor's tensor_template declares no Gated DeltaNet tensors, so the missing ~0.8 B has no shape in the tree to come from. Two possibilities, and the repo cannot currently tell them apart:
- the range is wrong (copied from a model card rather than derived), or
- the descriptor is incomplete — it is missing the DeltaNet projections (
ssm_*, conv, the gates), which is exactly the ~0.8 B shape.
(2) is the more likely, and it would mean every consumer of size_variants.9b under-counts Qwen3.5-9B.
What is proved meanwhile
The new test qe2e_inv_001_parameter_sum_matches_tensor_inventory proves the equation: the config-derived sum equals the measured tensor inventory, over both layer types and two schedules. Dropping ffn_down from the config side turns it red. The ∈ [9.0B, 9.2B] instantiation is deliberately not asserted, because asserting it today would require widening the range until it passed.
Ask
Decide which side is wrong, with a measurement: sum the tensor shapes of a real Qwen3.5-9B GGUF and compare against both the descriptor and the range. Then fix the one that disagrees, and let QE2E-INV-001 assert the range.
Activity
noahgift commentedon Sep 16, 2026
Measured, and the answer is neither side of the question as posed — it was
ModelConstraints, which declared none of the five shape keys the descriptor carries (inner_size,state_size,conv_kernel,group_count,full_attention_interval). 18 of every 24 Qwen3.5 layers were counted with dense/GQA accounting, so their conv, gates, state norm and mixer projections were invisible.Ground truth, against a real file:
~/models/Qwen3.5-0.8B-Q4_K_M.gguf(sha256bd258782…dc517), header parsed directly — 320 tensors, 752,393,024 parameters. With the keys carried,model_parameter_countreproduces it exactly, delta 0, both layer kinds matching tensor for tensor. Dropping theattn_gateterm turns that test RED, short by exactlyd·inner_size.Two shapes there contradict dense accounting and are now modelled from the tensors:
attn_qis[1024, 4096]= 2·n_h·d_k (the q projection emits the output gate;attn_output [2048,1024]shows only q is doubled), and the file is tied — nooutput.weight.The 9B question is still open, and narrower. P(9B) moves 8,208,519,168 → 8,344,907,136, still 0.655 B below [9.0B, 9.2B]. The residual is not arithmetic: the 9b descriptor contradicts itself —
inner_size: 2048(the 0.8B's value, at four times the hidden dim) andgroup_count: 8, which failsgroup_count · state_size == inner_size(8·128 ≠ 2048) where the measured 0.8B satisfies it at 16·128. So the descriptor's mixer dims look copied from a smaller variant.The range was not widened and QE2E-INV-001 is not asserted. Settling it needs a real Qwen3.5-9B GGUF; the same header-parse oracle used here decides it in minutes once one is on the fleet.
Correction to this issue's own body: the "before" figure should be 8,208,519,168, not 8,208,535,552 — a 16,384 difference (
4·hidden_dim). The repo's measured value is the former.Fix in the PR linked above.
noahgift commentedon Sep 19, 2026
Root cause measured 2026-09-19 on the kernel branch (PMAT-3477-gdn-gpu @ b2bd0c5, lambda, by the 0.68.2 session): this is not a 9B descriptor bug. 4B, 9B and 27B fail identically on CPU and GPU at
forward_qwen35.rs:132(assert q.len() == num_v_heads*head_v_dim, 2048 vs 4096/4096/6144): those sizes havelinear_num_value_heads32/32/48 withlinear_num_key_heads16 — GQA inside the Gated DeltaNet block (each k/q head serves nv/nk value heads, repeat-interleaved as in Qwen3-Next), which the CPU reference never modelled (it assumes nk == nv). 0.8B and 2B (nk == nv) are fully green on CPU and GPU (apr qaALL GATES PASSED, 54.8 / 21.7 tok/s GPU). The GPU path refuses loudly before allocation.Fix = one mechanism for CPU and CUDA in the dense-sizes PR (CPU half first, 4B as the falsifier); the ladder rungs qwen35-4b/9b/27b-q4km (#3511) stay RED-by-construction until it lands and are expected to flip together. Kernel PR for the 0.8B path: #3517. Refs #3510, #3090, EPIC #3477.
PMAT-3477: Gated DeltaNet with num_v_heads > num_k_heads (GQA in the …
Revert "PMAT-3477: Gated DeltaNet with num_v_heads > num_k_heads (GQA…
test(PMAT-3346): the measured Qwen3.5-0.8B inventory the dense path c…
fix(PMAT-3346): ModelConstraints carries the gated-DeltaNet shape, so…
PMAT-3346 (adoption): the QE2E-INV-001 note lost its closing quote, a…
PMAT-3346: the roadmap row, so the AD-04 quorum for #3350 has a ticke…
PMAT-3346 (adoption): a bias vector is as wide as the projection it b…
PMAT-3346: quorum verdict 3/3 on e060c7e (AD-04)
PMAT-3477: Qwen3.5 (Gated DeltaNet hybrid) runs on CUDA — nine device…
evidence(ladder): 0.68.1 receipts re-measured at main a6f54e8 on both…
PMAT-3477: Qwen3.5 4B/9B/27B on CPU and CUDA — block_count loader fix…
release(0.69): batch — 15 receipted PRs + fleet-pv andon fix, one CI …
PMAT-3685: apr parity REFUSES (exit 12) a model the CUDA capability g…