apr code on Qwen3.5 Q4_K_M (4B, 9B) on CUDA: does it complete a scripted edit-and-verify task on lambda and gx10? UNMEASURED, and its driver runs through apr serve --gpu (#3571)

Operator scope for 0.69.1, quoted from the #3710 comment: "we need all modalities working: run, chat, serve and code and enough context to use if for RAH, and arbiter and for rmedia work."

The #3710 release-bar row, quoted:

code: apr code (the coding-agent verb) completes a scripted edit-and-verify task with the model as its backend

#3710's owner list shows it as unowned (apr code on Qwen3.5). This issue is that row. Owner: aprender-c3, routed by the cop (aprender-04).

Known before measuring

apr code has no --gpu or --no-gpu flag. Its default driver starts an apr serve child with --gpu (crates/aprender-orchestrate/src/agent/driver/apr_serve.rs:102), so its CUDA path is apr serve's. apr serve cannot load Qwen3.5 at all — 'Architecture qwen35 … runs through Qwen35Model'; goal A's floor and the P0 decide-lane design both name apr serve #3571 says serve cannot load Qwen3.5 and that only layer 1 of that fix has landed. If that still holds, apr code on Qwen3.5 fails in the serve child before its first tool call. Nobody has measured it.
0.69 verb set V: every one of the 111 registered apr verbs is receipted or refuses by name on Qwen3.5 #3597 asks that every verb either has a receipt or refuses by name on Qwen3.5. This row is stricter: the task must succeed, on CUDA.

done_when

Gap first, before any code. Commit a fixture: a small project with one failing test whose fix is a one-line edit, and one task prompt that asks the agent to make the fix and run the test. Run apr code -p --model <M> --project <fixture> --output-format json --emit-trace <t> for M in {Qwen3.5-4B-Q4_K_M, Qwen3.5-9B-Q4_K_M} × {lambda, gx10}, with the 0.69.1 release-candidate binary (version and SHA cited). For each of the 4 cells, post on this issue: PASS or FAIL, the first failing mechanism named (serve child did not load / fell back to CPU / tool call not parsed / wrong edit / test not run / wrong final answer), and the stderr or trace line that shows the backend.
CUDA engaged, proven. Every PASS cell cites a line from the serve child or the trace showing the forward ran on CUDA. The requested flag does not count.
Every FAIL cell either gets its fix in this issue's PR, with a falsifier that goes RED on the pre-fix binary, or points to the issue that carries the fix (for example apr serve cannot load Qwen3.5 at all — 'Architecture qwen35 … runs through Qwen35Model'; goal A's floor and the P0 decide-lane design both name apr serve #3571) under Refs.
Enforced, not promised. The four cells become rows (verb=code) of the release matrix that the pv SHACL release-readiness shape: the tag is REFUSED unless every (Q4_K model in host inventory × {lambda,gx10} × {run,chat,serve,code} × consumer-context) cell has a Pass receipt — a missing cell is a violation, never a skip #3715 shape refuses the tag on whenever a cell lacks a Pass receipt.

Out of scope here: the consumer-maximum context axis. That is #3693's, and the code cell at consumer context follows #3693 and #3716 once they are measured.

Refs #3710, #3571, #3597, #3715

Owner: aprender-f8 (routed by the cop, aprender-04). It moved from aprender-c3, which never started it and is carrying #3726/#3693/#3742. The PASS cells depend on #3571 step (2), owned by aprender-c7; the fixture and gap measurement (done_when 1) start now.

roadmap: PMAT-3719 leaves this branch; the cop moved #3719 to another…

done_when 1: the gap, measured (aprender-f8)

Binary: apr 0.69.0 (856009cc9), --features cuda, on both hosts. It is a local merge, never pushed: origin/main 52f43da71 + PMAT-3726-canonical-bpe c57c260bc (#3726 included, per the cop) + the harness. No 0.69.1 tag existed, so this is the closest release candidate.
Harness: scripts/apr_code_edit_verify.sh, fixture tests/fixtures/apr-code-edit-verify (branch PMAT-3719-apr-code-qwen35-cuda). It runs apr code -p --model M --project <fixture copy> --output-format json --emit-trace <t> under the fleet GPU lock with choom 1000.
How cells are judged: from artifacts only (the working-copy diff, an independent test re-run, the serve child's own output), never from what the agent says.
Artifacts: evidence/apr-code-3719/.

Cell	Verdict	First failing mechanism	Backend line (serve child)
lambda · Qwen3.5-4B-Q4_K_M	FAIL	serve child did not load	`Model ready: 0 layers, vocab_size=248320, hidden_dim=2560` · `gpu-layers: requested=all resolved=0 total=0 (backend=cuda)`
lambda · Qwen3.5-9B-Q4_K_M	FAIL	serve child did not load	`Model ready: 0 layers, vocab_size=248320, hidden_dim=4096` · `gpu-layers: requested=all resolved=0 total=0 (backend=cuda)`
gx10 · Qwen3.5-4B-Q4_K_M	FAIL	serve child did not load	`Model ready: 0 layers, vocab_size=248320, hidden_dim=2560` · `gpu-layers: requested=all resolved=0 total=0 (backend=cuda)`
gx10 · Qwen3.5-9B-Q4_K_M	FAIL	serve child did not load	`Model ready: 0 layers, vocab_size=248320, hidden_dim=4096` · `gpu-layers: requested=all resolved=0 total=0 (backend=cuda)`

In every cell the child then printed CUDA optimized model ready, which is a load-time line printed for a 0-layer model. The first completion came back HTTP 500: Operation 'generate_gpu_resident_streaming' not supported: Model architecture not supported for GPU-resident path. No CUDA forward ran in any cell. Refs #3571 (step 1 refuses the 0-layer load; step 2 is the Qwen3.5 route).

The next mechanism, found on the #3571 route (done_when 3: fixed in this PR)

On apr 0.69.0 (cc3892acd) = the baseline + aprender-c7's #3571 step (2) 5a4a8e102 (pre-receipt), run on gx10 · 4B:

Serve child: it loads and generates on CUDA, per its own lines Model ready: Qwen3.5 hybrid, 32 layers resident on the GPU, declared context 262144 tokens · chat template: Qwen3NoThink (thinking off) · gpu-layers: requested=all resolved=32 total=32 (backend=cuda).
Agent: the task still fails with "wrong edit: stats.py was not changed". The model answered "Without seeing the actual code, I'll assume…" with no tool call (num_turns 1, tokens_in 81).
Cause: apr code -p replaced its coding system prompt, for every model size, with COMPACT_SYSTEM_PROMPT = Answer the question. Be direct., which names no tool. A fake apr serve capturing the request body shows exactly that 31-character system message (evidence/apr-code-3719/route-cc3892acd/gx10-4B/fake-serve-request-capture.jsonl). This breaks apr code -p on every model, not only Qwen3.5.
Fix: b2b89d69e. -p keeps the size-scaled prompt interactive mode uses (PMAT-198). Falsifier falsify_3719_single_prompt_run_tells_the_model_about_its_tools is RED with the old override restored.

Found while building the harness

apr code --emit-trace never records a tool call. It writes 4 records, and the assistant turn is one text block whatever the agent did. PreToolUse/PostToolUse hooks are not wired into the loop either. So "the agent ran the test" is read from python shims on the agent's PATH.
apr code -p --output-format json prints nothing on stdout when the driver errors (rc 1, stderr only), with no error envelope.

Next: the same four cells on binary v3 0a58e2b7d (= cc3892acd + b2b89d69e) are queued on both hosts' GPU locks. Row keys for done_when 4 are agreed with aprender-62 (#3712) and aprender-97 (#3715).

PMAT-3719: harness --gpu-q PRIO queues the run through gpu-q (rule re…

PMAT-3719: the harness's apr call is proven locked and bounded (apren…

PMAT-3719: regenerate roadmap.yaml for the PMAT-3719 fragment; case t…

PMAT-3719: --gpu-q bounds the wait with GPUQ_WAIT (gpu-q v3, rule rev…

fix(apr code): a delimited <tool_call> missing only its closing brack…

PMAT-3719: judge keeps gpu-q queue lines out of evidence excerpts

fix(apr code): the system prompt teaches the tool calls the tools acc…

evidence(#3719): all four cells PASS on one binary (apr 0.69.0 7b161d…

All four cells PASS on one binary (aprender-f8)

Binary: apr 0.69.0 (7b161d743), --features cuda, the same SHA on lambda and gx10. It is a local merge: main 52f43da71 + #3726 + aprender-c7's #3571 step (2) 5a4a8e102 (pre-receipt) + branch PMAT-3719-apr-code-qwen35-cuda through ed714f055. The GPU was taken through gpu-q --prio 1. Evidence: evidence/apr-code-3719/v6-7b161d743/.

Cell	Verdict	Backend line (serve child)
lambda · Qwen3.5-4B-Q4_K_M	PASS	`Model ready: Qwen3.5 hybrid, 32 layers resident on the GPU` · `gpu-layers: requested=all resolved=32 total=32 (backend=cuda)`
lambda · Qwen3.5-9B-Q4_K_M	PASS	same
gx10 · Qwen3.5-4B-Q4_K_M	PASS	same
gx10 · Qwen3.5-9B-Q4_K_M	PASS	same

Every cell has chat template: Qwen3NoThink (thinking off) and zero [GPU->CPU FALLBACK] lines. Every cell made exactly the one-line fix (/ (len(values) - 1) → / len(values)); the agent ran python3 -m unittest test_stats -v itself, and the harness's independent re-run is OK.

Limit, stated plainly: the route prints no per-request CUDA line. So the CUDA evidence is full residency plus completions with no fallback marker, not a per-request trace.

done_when 3: the FAIL mechanisms met on the way, each fixed in this PR with a falsifier RED on the pre-fix code

-p never told any model it had tools. The system prompt was replaced with "Answer the question. Be direct." for every size (a fake serve captured the request). Fixed in b2b89d69e.
A tool call missing only its outer } was returned as text (gx10·4B). Fixed in a35f7b8a0.
The system prompt taught file_edit with old/new, but the tool requires old_string/new_string (and memory with key/value; it requires content). Qwen3.5-4B repeated the rejected call until the loop guard ended the turn; a logging HTTP proxy in front of the serve child showed it. Fixed in ed714f055. The falsifier checks every prompt example against the registered tool's schema.

The baseline gap (4/4 "serve child did not load", 0 layers) refs #3571.

done_when 4: the row keys are agreed with aprender-62 (#3712) and aprender-97 (#3715). The #3712 ladder calls this harness for each derived code cell (row B2, after #3745). The harness's locking is proven by scripts/check_apr_code_edit_verify.sh (30 rows, mutation-checked, dispatched by guard_tree).

Receipt: docs/audits/impl-PMAT-3719-receipt.md. Next: the quorum. The #3775 round could not seat three independent reviewers (gemini quota exhausted, gpt-oss 503), so a seat-fill has been requested from the cop.

PMAT-3719: the tool-call repair is split into small functions (comple…

P0 to P1 (2026-10-14 P0 triage): P0 now means it blocks the next release or carries the operator's own words on it; this issue does neither, so it stays on the 0.71.0 milestone at P1.

merged into #5312 (master ticket for epic #3994; triage rule 7b, 2026-10-14). It is a checklist row there; reopen this issue when work on it starts or its PR opens.

Moved: this sub-ticket is now a row in #5325 (apr on GPU: refuse by name before OOM, never fall back to CPU silently), split out of #5312 by theme.

fix(ladder): measure_code reads apr code's backend.ran and backend.fe…

feat(ci-tools): port model_ladder_cells_produce.py to ladder-cells-pr…

Navigation Menu

Repository navigation

`apr code` on Qwen3.5 Q4_K_M (4B, 9B) on CUDA: does it complete a scripted edit-and-verify task on lambda and gx10? UNMEASURED, and its driver runs through `apr serve --gpu` (#3571) #3719

Known before measuring

done_when

done_when 1: the gap, measured (aprender-f8)

The next mechanism, found on the #3571 route (done_when 3: fixed in this PR)

Found while building the harness

All four cells PASS on one binary (aprender-f8)

done_when 3: the FAIL mechanisms met on the way, each fixed in this PR with a falsifier RED on the pre-fix code

23 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

apr code on Qwen3.5 Q4_K_M (4B, 9B) on CUDA: does it complete a scripted edit-and-verify task on lambda and gx10? UNMEASURED, and its driver runs through apr serve --gpu (#3571) #3719

Description

Known before measuring

done_when

Activity

noahgift commented on Sep 21, 2026

noahgift commented on Sep 21, 2026

done_when 1: the gap, measured (aprender-f8)

The next mechanism, found on the #3571 route (done_when 3: fixed in this PR)

Found while building the harness

noahgift commented on Sep 21, 2026

All four cells PASS on one binary (aprender-f8)

done_when 3: the FAIL mechanisms met on the way, each fixed in this PR with a falsifier RED on the pre-fix code

23 remaining items

noahgift commented on Oct 14, 2026

noahgift commented on Oct 14, 2026

noahgift commented on Oct 14, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

`apr code` on Qwen3.5 Q4_K_M (4B, 9B) on CUDA: does it complete a scripted edit-and-verify task on lambda and gx10? UNMEASURED, and its driver runs through `apr serve --gpu` (#3571) #3719