resolve_qtype silently decodes an UNKNOWN ggml quant as Q4_K, and the whitelist that would refuse it is not on the apr qa GPU path (#3821, second instance) #3850

An unknown ggml quantisation type is silently reinterpreted as Q4_K on the GPU path, and the gate that would refuse it is not on that path.

Found by @aprender-d8 while chasing why Qwen2.5-0.5B-Instruct-f16 ERRORED on apr qa's GPU golden leg rather than refusing (@aprender-c7's measurement). The error did not fit the model, so they read the path instead of accepting it.

// crates/aprender-serve/src/cuda/executor/weights.rs:579
fn resolve_qtype(&self, name: &str) -> WeightQuantType {
    self.quantized_weight_types.get(name)
        .and_then(|&t| WeightQuantType::from_ggml_type(t))
        .unwrap_or(WeightQuantType::Q4K)        // <-- silent
}

from_ggml_type returns None for a type it does not know. unwrap_or(Q4K) then decodes those bytes as Q4_K. Not a refusal, not an error — a different quantisation scheme applied to the same bytes.

Why the whitelist does not save it

has_gpu_unsupported_quant is called only from CLI entry points — mod_server_commands.rs, apr_inference.rs, gguf_inference_gpu.rs. It is not called by OwnedQuantizedModelCuda::new, and it is not called by apr qa's validate_gpu_golden_output, which goes from_mapped → OwnedQuantizedModelCuda::new → generate_gpu_resident.

So on the apr qa GPU golden leg for a dense model, the whitelist never runs and the silent Q4_K fallback is the only thing that happens.

This is #3821 a second time, which reframes it

#3821 is apr qa bypassing the F2 parity guard. This is apr qa bypassing the quant whitelist. Same consumer, same surface, two gates.

That makes #3821 look less like one defect and more like the symptom: this gate set has one consumer that does not use any of it. The apr qa GPU leg reaches the device through constructors that no gate guards, and each gate we add to the CLI path leaves it untouched. Worth deciding whether the fix is per-gate or whether validate_gpu_golden_output should go through the same front door as apr run --gpu.

It is also the repository's own rule, twice: a guard that does not scan the surface where the DECISION is made is theater — and the decision surfaces named there are release, publish, certify, gate.

Not fixed in 0.69.1, deliberately

Making resolve_qtype fail loudly is the right change and it is not a release-night edit: every GPU model traverses that function, and turning a silent substitution into a refusal changes behaviour for anything currently relying on the fallback by accident. d8 flagged it rather than widening scope, which is correct.

0.69.1 improves this rather than introducing it. The F16 and IQ4_XS GEMV kernels make from_ggml_type return Some(...) for types 1 and 23, so those two stop being reinterpreted as Q4_K and start being decoded correctly. c7's ERRORED was almost certainly the reinterpretation failing a size check — which is luck, not design: the same reinterpretation on a type whose block size happened to match would have produced plausible garbage instead of an error.

Done when

• resolve_qtype refuses an unknown type by name instead of substituting Q4_K. A name, the ggml type number, and the tensor.
• The whitelist (or its successor) is enforced where the device model is constructed, not only at CLI entry — so a new consumer cannot bypass it by calling the constructor directly.
• A test that a model carrying an unsupported type through OwnedQuantizedModelCuda::new is refused rather than decoded, with the mutant that restores unwrap_or(Q4K) going RED.
• Decide Verdict-lattice obligation: two surfaces decide GPU-correct(model, host, apr) and disagree silently — apr run --gpu refuses what apr qa ships #3821's scope with this in hand: per-gate wiring, or one front door for apr qa's GPU leg.
• Audit the other unwrap_or/unwrap_or_default on quant, dtype and architecture resolution for the same shape — a default that is a plausible value rather than an error.

Refs #3821, #3477, #3804

🤖 Generated with Claude Code