Repository navigation
apr on GPU: refuse by name before OOM, never fall back to CPU silently #5325
Description
Scope: What apr does when the GPU path cannot run: silent CPU fallback, refusing only after a full CPU rerun, OOM instead of a pre-flight refusal, session headroom that doubles context, effective-config misreporting residency, and ladder/receipt fields that hide which backend ran.
Why it matters: A user who asked for GPU must know whether the GPU ran. A silent fallback turns a GPU defect into a slow success that no gate sees.
Done when: Every GPU verb either runs on GPU or exits non-zero with a named refusal before allocating, and the JSON receipt names the backend that ran; tested on the model ladder.
Epic: #3994. Folded under triage rule 7b (2026-10-14): each row below was its own issue, closed as not planned and kept here as a sub-ticket. Reopen an issue when its work starts or its PR opens.
Sub-tickets
- apr run --gpu: a refused GPU attempt redoes the whole generation on CPU, then refuses it (exit 14) — refuse before the redo #4969 apr run --gpu: a refused GPU attempt redoes the whole generation on CPU, then refuses it (exit 14) — refuse before the redo (milestone none)
- accept: the outcome in the title, with a receipt
- dense GGUF CUDA-init fallback to CPU is silent in text mode #4977 dense GGUF CUDA-init fallback to CPU is silent in text mode (milestone 0.71.1)
- accept: Done when: a dense GGUF run on a cuda build whose CUDA init fails prints one stderr line naming the CPU fallback, without --verbose, as the Qwen3.5 path already
- Small unified-memory CUDA devices: 16 GiB headroom refuses every model, no refusal before OOM, wrong pull hint #4947 Small unified-memory CUDA devices: 16 GiB headroom refuses every model, no refusal before OOM, wrong pull hint (milestone 0.71.0); branch
fix/e3-4947-recut3- accept: the outcome in the title, with a receipt
- qwen35 session: #4274 headroom doubles a 32k turn to 65536 positions — 27B OOMs and silently falls back to CPU #4443 qwen35 session: serve: Qwen3.5 chat turn 2 re-prefills from position 0; the session extend never engages via /v1/chat/completions #4274 headroom doubles a 32k turn to 65536 positions — 27B OOMs and silently falls back to CPU (milestone 0.71.0); branch
fix/4443-on-main- accept: the outcome in the title, with a receipt
- effective-config reports Qwen3.5 CUDA residency as 'cpu residency' and drops the F2 parity verdict (v0.69.3) #4374 effective-config reports Qwen3.5 CUDA residency as 'cpu residency' and drops the F2 parity verdict (v0.69.3) (milestone backlog); branch
fix/4374-effective-config-qwen35- accept: the outcome in the title, with a receipt
- ladder producer: measure_code should read apr code JSON backend.ran and backend.fell_back like measure_chat #4986 ladder producer: measure_code should read apr code JSON backend.ran and backend.fell_back like measure_chat (milestone 0.71.1); branch
la-72/4986-code-backend- accept: the outcome in the title, with a receipt
- FALSIFY-GPU-010 prose: an unreported completion does not null the backend when another ran on the CPU #5045 FALSIFY-GPU-010 prose: an unreported completion does not null the backend when another ran on the CPU (milestone 0.71.1); branch
w71-e3a/5045-gpu010-prose- accept: the outcome in the title, with a receipt
-
apr codeon Qwen3.5 Q4_K_M (4B, 9B) on CUDA: does it complete a scripted edit-and-verify task on lambda and gx10? UNMEASURED, and its driver runs throughapr serve --gpu(#3571) #3719apr codeon Qwen3.5 Q4_K_M (4B, 9B) on CUDA: does it complete a scripted edit-and-verify task on lambda and gx10? UNMEASURED, and its driver runs throughapr serve --gpu(apr serve cannot load Qwen3.5 at all — 'Architecture qwen35 … runs through Qwen35Model'; goal A's floor and the P0 decide-lane design both name apr serve #3571) (milestone 0.71.0); branchw71-e3a/3719-code-on-main- accept: the outcome in the title, with a receipt
Activity
noahgift commentedon Oct 22, 2026
Tracked in #3999 (triage 2026-10-22).
apr codeon Qwen3.5 Q4_K_M (4B, 9B) on CUDA: does it complete a scripted edit-and-verify task on lambda and gx10? UNMEASURED, and its driver runs throughapr serve --gpu(#3571) #3719noahgift commentedon Oct 29, 2026
Shipped in aprender v0.72.2 (crates.io 2026-10-29T23:11Z, tag v0.72.2 = 350f0ad).
Child #4977 (dense GGUF CUDA-init fallback silent in text mode): fixed by #6506 (a44cafe, Refs #4977). The CPU-fallback line now prints in text mode, not only behind --verbose, with a source-scan test. #4977 is already closed.
Still open: the remaining 7 sub-issues of this refuse-by-name-before-OOM tracker. Progress: 1/8 done.