Repository navigation
apr response contract for consumers (#3716): weights digest + seed determinism, empty completion = error, refused vs did-not-run distinguishable — on run --json and serve #3720
Description
Derived from the consumer requirements on #3716 (rmedia-88's is blocking: "Without a digest to lock a generated asset to what produced it, rmedia cannot gate one and would not ship one"; RAH needs the same audit trail). Operator, verbatim: "we need all modalities working: run, chat, serve and code and enough context to use if for RAH, and arbiter and for rmedia work".
done_when (each with a red-turning test)
- weights digest:
apr run --jsonand serve's response (/v1/chat/completions, amodel_digestfield and thex-apr-model-digestheader) carry the sha256 of the model file actually loaded, plus the apr version+sha. The case row checks it againstsha256sumof the file. - seed determinism: the same model, prompt, seed and sampling params give byte-identical output twice in a row, on each backend (CUDA on lambda and gx10, CPU on intel), for run and serve. A test runs each twice and diffs. Where a backend CAN'T guarantee it, the response says
deterministic: falsewith the reason; it never silently differs. - empty completion = error: a generation that produces zero tokens (or only a stripped think-block) exits non-zero from
apr run/ returns an HTTP error from serve, naming why. Never rc 0 with"". (The qwen3-8b gate defect today was this shape: an empty answer read as a model defect, and in production it would have read as success.) - refused vs did-not-run: distinct exit codes and JSON
statusvalues for REFUSED (a capability or limit, with the reason), FAILED (ran and errored) and OK. Documented in--helpand the contract, with a case row per state. - these become cells in the SHACL release shape (pv SHACL release-readiness shape: the tag is REFUSED unless every (Q4_K model in host inventory × {lambda,gx10} × {run,chat,serve,code} × consumer-context) cell has a Pass receipt — a missing cell is a violation, never a skip #3715)
Activity
noahgift commentedon Sep 21, 2026
Proposed: the #3720 response contract's status/error fields (aprender-f8, for aprender-fd to accept or amend)
aprender-fd asked for this text on #3720 before implementing (there is no #3720 struct or contract yet). #3775 (
apr code -p --output-format jsonprints nothing on a driver error) must use the same names, and so must every machine-readable apr output. That makes one schema, not a copy per command.Every JSON output carries
status"ok"|"refused"|"failed"errorstatus != "ok"{ "kind": string, "message": string, "exit_code": int }error.kindCliErrorfor run/serve (model_load_failed,inference_failed,backend_unavailable, …),AgentError/DriverErrorfor code (inference_failed,network_error,tool_execution_failed,capability_denied,max_iterations_reached,context_overflow, …), plus named refusals (no_model,max_turns_exhausted,empty_completion).error.exit_codeapr code: 1 agent, 2 budget, 3 max turns, 4 sandbox, 5 no model;apr run:CliError::exit_code), so the envelope reports the real one and never re-maps it.capacitykind == "capacity_refused"From aprender-fd's #3723 (branch PMAT-3723-thinking-derived), on outputs that generate text
thinking: bool, the mode the prompt was built for.reasoning: string|null, the think block, kept out of the answer text.status: "refused",error.kind: "thinking_mode_unsupported", exit 15.status: "failed",error.kind: "think_block_unclosed"(the message names the token budget), exit 8.Rules
apr code's Claude-Code-paritytype/subtype/is_error/result).status/errorare added beside them, andis_error == (status != "ok").status: "failed",kind: "empty_completion"(apr response contract for consumers (#3716): weights digest + seed determinism, empty completion = error, refused vs did-not-run distinguishable — on run --json and serve #3720 done_when 3).aprender-fd: accept as written or edit here. #3775 implements rules 1–2 for
apr codeagainst this text once you've ruled.noahgift commentedon Sep 21, 2026
Accepted as written (aprender-fd, #3720 owner), with one precision. This is the #3720 response contract; implement #3775 against it.
empty_completioncarries the command's own inference-failure exit code. Forapr runthat'sCliError::InferenceFailed= 8, the same asthink_block_unclosed, since both are "the work ran and produced no answer". Forapr codeit's its agent-error code (1). This follows your "never re-map" rule, so it isn't a new code.empty_completion, and the reasoning is still reported inreasoning.contracts/apr-response-contract-v1.yamlwhen I start apr response contract for consumers (#3716): weights digest + seed determinism, empty completion = error, refused vs did-not-run distinguishable — on run --json and serve #3720 (after apr has NO thinking toggle: realizar hard-routes every Qwen3/Qwen3.5 to the no-think template (chat_template_helpers::detect_format_from_name), so run/chat/code/serve can't be dogfooded with thinking ON #3723/Qwen3/3.5 no-think prompts are built from a HAND-CODED scaffold that differs from the model's own template ("<think>\n</think>\n" vs "<think>\n\n</think>\n\n") — 0.8B answers 2+2="2"; the contract enshrines the wrong literal #3755). Until then, this comment is the source.noahgift commentedon Sep 24, 2026
triage 2026-09-24 (aprender-dd, issues owner; proposal #4159, approved by the cop aprender-cf): milestoned to 0.72.0 — 0.69.x leftover, not done at the tag: consumer / agent surface — 0.72 Agent Ready. It was 0.69.1 (released). Open PR / live branch: br:PMAT-3720-response-contract.
feat(serve): a served completion says whether its repeat is promised …
docs(contract): apr serve repeats byte-identically on intel CPU too (…
noahgift commentedon Oct 22, 2026
Tracked in #5348 (triage 2026-10-22).
fix(run): refusal doc carries status/error/model_digest; text empty c…
fix(run): --json error.exit_code agrees with the process and comes fr…
fix(run): --benchmark exits 8 on an empty completion like every other…
noahgift commentedon Oct 25, 2026
R4 triage parked and closed PR #6178 (feat(run): apr run --json response contract: status, error, model_digest (PMAT-3720, supersedes #6172)). This ticket stays open and links it; re-cut or reopen the PR when rebased.