0.70.2 P0: Qwen3.5-0.8B-UD-IQ2_XXS.gguf golden_output: GPU forward falls back to CPU (known failure in 0.70.1)

Known failure shipped in 0.70.1 (aprender 0.70.1, candidate cc4463f) under C272 5b; P0 for 0.70.2. Not a regression: it fails the same way on v0.69.5-rc.1 (5660b08).

Model: Qwen3.5-0.8B-UD-IQ2_XXS.gguf
Host(s): lambda, gx10
Check: golden_output (GPU dispatch)
0.70.1 result: FAIL: GPU forward failed and fell back to CPU (chat rc=14)
v0.69.5-rc.1 result: FAIL: dispatch reported CPU (GPU fell back); rc.1 chat has no exit 14
Logs: 0.70.1 lambda: /mnt/nvme-raid0/tmp/evid-0701b/lambda-ladder.log; 0.70.1 gx10: gx10:~/.cache/aprender-0701/progress2.log; rc.1 lambda: /mnt/nvme-raid0/tmp/screen-0701/logs/inv_Qwen3.5-0.8B-UD-IQ2_XXS.gguf.log; rc.1 gx10: /mnt/nvme-raid0/tmp/gx10-0701/rc1/ladder-only-inv_Qwen3.5-0.8B-UD-IQ2_XXS.gguf.log
Reproduce: scripts/model_ladder.sh --host <lambda|gx10> --only inv:Qwen3.5-0.8B-UD-IQ2_XXS.gguf

Done when: the ladder row is OK on lambda and gx10.

C279 item 6 (operator ruling, 2026-10-03), for 0.70.1 ONLY: the inventory.red_model entry for Qwen3.5-0.8B-UD-IQ2_XXS.gguf (#4004) is swapped for a 0.70.1-only known_red entry that covers its wrong answer and its GPU-to-CPU fallback. It prints as known red, never green. It fell back on v0.69.5-rc.1 too, so it is an old failure.

P0 for 0.70.2:

Restore the red_model entry (Qwen3.5-0.8B-UD-IQ2_XXS fails '2+2' on BOTH apr and llama.cpp (official template) — RED-MODEL candidate pending the step-3 logit margin #4004) and remove the 0.70.1 known_red entry.
Fix the GPU-to-CPU fallback for this file, or refuse the file by name.

Agent: aprender-27

docs(BLD-001): C280 replaces C279 — P1–P3 leave the exit bar, P4 is C…

Measurement (aprender-w4664): the GPU fallback is fixed; golden_output stays red because of the model's own output, not because of apr.

Fix: branch fix/4664-iq2xxs-gpu-fallback @ 53d5bfa5ab. The qwen35 batched prefill now runs IQ2_XXS/IQ2_S/IQ3_XXS/Q2_K projections as per-row GEMVs instead of refusing them and falling back to CPU.

host	apr	backend	ladder `inv:Qwen3.5-0.8B-UD-IQ2_XXS.gguf`
lambda (RTX 4090)	0.70.0 (`53d5bfa`)	`ran=gpu fell_back=false`	golden_output RED
gx10 (GB10, cc 12.1)	0.70.0 (`53d5bfa`)	`ran=gpu fell_back=false`, F2 guard receipt matches	golden_output RED: `Expected one of ["4"], got: 'The user wants to know about the 2+2 question.\nThe user wants…'`

I varied the input over 4 prompts on lambda, running apr CPU, apr GPU, and llama.cpp CPU and GPU on the same file (sha256 a369165c…). None of the four backends answers, and all four loop on restating the question:

"What is 2+2?" — apr: "The user asked to 2+2? / The user wants to know the answer…" ×N. llama.cpp: "…The user wants to know about the…"
"What is 3+5?" — apr: "The user is asking about the 3+5?" ×N. llama.cpp: "…Wait, the que…"
"Capital of France?" — apr CPU/GPU and llama.cpp CPU restate it; only llama.cpp GPU produced "Paris", in 1 of 4 runs.
"Three primary colors" — apr CPU and GPU produce identical output; llama.cpp restates the question.

Control (lambda, same binary, same GPU path, --max-tokens 128): Qwen3.5-0.8B-Q4_K_M ends "$2 + 2 = 4$.", while IQ2_XXS still loops ("The user wants to know the answer to the question." ×N). So the failure comes from 2-bit quantization of a 0.8B model (see #4004), and apr is not the cause. Evidence: lambda:/mnt/nvme-raid0/tmp/fix-4664/cmp/, gx10:~/.cache/fix-4664/receipts/gx10.only-inv_Qwen3.5-0.8B-UD-IQ2_XXS.gguf.json. Whether to keep this file as a gate-named model is the operator's call (proposed to aprender-a7); I have made no gate change.

fix(qa): #4664 refusal is read in golden_gate_for, so hybrid and cpu_…

docs(contracts): #4664 FALSIFY-MCL-018 lists the golden_gate_for-call…

Re-measure on current main (aprender-w4578): the GPU-to-CPU fallback still reproduces on main.

Binary: apr 0.70.1 (6192aa9b88), which is main at 6192aa9. Release build with --features cuda, sha256 prefix d8208cdfc6d3686a, run under the GPU lock.
GPU: RTX 4090.
Command: apr run Qwen3.5-0.8B-UD-IQ2_XXS.gguf --prompt "What is 2+2? Answer with one number." --max-tokens 16 --trace

Trace lines (the backend comes from the run's own output, not from intent):

Backend: GPU (CUDA, NVIDIA GeForce RTX 4090, 24035 MB VRAM) [qwen35 hybrid forward, #3090]
warning: GPU forward failed at probe position 0/18: Operation 'qwen35_cuda_prefill_deltanet' not supported: Invalid parameter: qwen35 prefill: no f32 dequant kernel for IQ2XXS ...
warning: GPU (CUDA) qwen35 path rejected, falling back to CPU: the F2 CPU-parity guard rejected the GPU path

The answer was 2+2?, which is wrong.
A second prompt ("Name the capital of France.") gave the same probe failure and the same fallback, so this is not one bad input.

Why main still falls back: the fix measured above (fix/4664-iq2xxs-gpu-fallback @ 53d5bfa5ab, ran=gpu fell_back=false) is not an ancestor of main 6192aa9. Main still has no f32 dequant kernel for IQ2XXS in the qwen35 prefill.

Proposal: re-rule into two parts, keep open.

Land 53d5bfa5ab. That fixes the fallback half, and it is an apr defect.
The golden_output half is the 2-bit 0.8B model's own answer, which is wrong even on a GPU run that does not fall back. Re-rule that half to the restored red_model entry (Qwen3.5-0.8B-UD-IQ2_XXS fails '2+2' on BOTH apr and llama.cpp (official template) — RED-MODEL candidate pending the step-3 logit margin #4004) per C279 item 6, not to an apr fix.

"Done when" then becomes: ran=gpu fell_back=false on both GPU hosts, with golden_output listed as a model red rather than an apr red.

Agent: aprender-w4578

0.70.2 ships on a CRUX-smoke emergency scope (operator ruling C322, A1 and A2). This issue is refused for 0.70.2 and moves to milestone 0.71.0, with no per-issue contract edit. The 0.70.2 release notes name it as a known failure.

P0 to P1 (2026-10-14 P0 triage): P0 now means it blocks the next release or carries the operator's own words on it; this issue does neither, so it stays on the 0.71.0 milestone at P1. Its 0.70.2 label belonged to a release that has shipped.

Re-parent: former parent #4689 is closed. Moved under open epic #3998 (same 0.71 train).

Re-parent: grouped under #4701 (the model-ladder ticket, open, under E1 #3998) with the other ladder known-fail rows #4661-#4664; same 0.71 train.

Folded into #4701 as a checklist row (TRIAGE-100, 2026-10-14): the open-ticket cap is 100, so this ticket is now tracked on that master and closed as not planned. Nothing is lost: reopen it when its work resumes, and an open PR for it reopens it. Pushed branch: build-kaizen/4664-golden-refuse, fix/4664-iq2xxs-gpu-fallback.

Agent: aprender-clerk

Navigation Menu

Repository navigation

0.70.2 P0: Qwen3.5-0.8B-UD-IQ2_XXS.gguf golden_output: GPU forward falls back to CPU (known failure in 0.70.1) #4664

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

0.70.2 P0: Qwen3.5-0.8B-UD-IQ2_XXS.gguf golden_output: GPU forward falls back to CPU (known failure in 0.70.1) #4664

Description

Activity

noahgift commented on Oct 3, 2026

noahgift commented on Oct 4, 2026

noahgift commented on Oct 5, 2026

noahgift commented on Oct 7, 2026

noahgift commented on Oct 14, 2026

noahgift commented on Oct 14, 2026

noahgift commented on Oct 14, 2026

noahgift commented on Oct 14, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions