Skip to content

fix(serve): constrained EOS and 499, response_format type refusal, Ollama done_reason, code help (#3568 #5357 #5211) - #5407

Merged
noahgift merged 7 commits into
mainfrom
batch/serve-cli-1015
Oct 15, 2026
Merged

noahgift merged 7 commits into
mainfrom
batch/serve-cli-1015

Conversation

@noahgift

Copy link
Copy Markdown
Member

Summary

  • apr run --json-schema/--grammar now stops at the vocabulary EOS; a constrained serve turn whose client disconnected is answered 499 instead of constraint_truncated.
  • An unknown response_format.type is refused naming the field.
  • Ollama /api/chat and /api/generate report done_reason: "length" when the turn hit num_predict (streamed and non-streamed bodies).
  • apr code --output-format json help names the envelope that is actually written.

Not covered: the apr-cli Ollama shim (commands/serve/ollama.rs) still hardcodes done_reason: "stop"; that needs its own change.

Test plan

  • cargo test -p aprender-serve --lib --features structured-output -- constrain cancel response_format ollama done_reason: 142 passed
  • cargo test -p apr-cli --lib -- code_output_format_help: 1 passed
  • mutating the non-stream done_reason makes non_stream_replies_carry_the_done_reason fail

Refs #3568, Refs #5357, Refs #5211
keep-open: #3568 (other fu2 items remain), #5357 (apr-cli shim still hardcodes "stop"), #5211 (left for the issue owner to close)
ont-delta: none (code and tests only, no spec or contract change)

🤖 Generated with Claude Code

noahgift and others added 6 commits October 15, 2026 17:25
…serve turn is 499 (#3568)

`apr run --json-schema/--grammar` built its stop list from the model
config's EOS alone. Once the document is whole the constraint admits only
the vocabulary's EOS, so where the two ids differ the turn took it as a
token of the reply. The constrained run now adds it, as `apr serve` does.

On serve, a turn cancelled by a client disconnect is answered 499 (checked with
peek_cancelled, so it spends no poll) instead of `constraint_truncated`.

Refs #3568

Agent: la-0.71.1a
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ld (#3568)

serde's "unknown variant" text is hidden by the rejection sanitizer, so a
request with `response_format: {"type": "xml"}` read only "Invalid request
body". It now gets a 422 that names `response_format.type` and the types
the server answers (text, json_object, json_schema), as `n` and
`temperature` refusals do.

Refs #3568

Agent: la-0.71.1a
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…3568)

The 499 status was only checked as a constant. This drives a constrained
turn with an already-cancelled token through the handler and asserts 499
with no choices and no truncation refusal. With the cancel check removed
the turn answers 422, and this test fails.

Agent: la-0.71.1a
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…_predict (#5357)

/api/chat and /api/generate streamed a terminal chunk with done_reason
"stop" unconditionally, so a client could not tell a truncated answer
from a finished one. The OpenAI finish_reason is now carried through
chat_response_to_parts: "length" maps to "length", anything else (and
an upstream error) to "stop", on both routes.

The non-streaming /api/chat and /api/generate bodies carry done_reason too.

Refs #5357

Agent: la-0.71.1a
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…#5211)

The help listed only {type, subtype:"success", result, session_id,
duration_ms}. It now lists every key code_envelope::envelope writes,
including backend{requested,ran,fell_back} and the error object, and a
test fails if any of them goes missing from the clap help.

Refs #5211, #5328

Agent: la-0.71.1a
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

§13.11 rung 1 — quorum shadow verdict

S13-SHADOW pr=5407 head=77fa9062992d4edc93854ab209d50ed404bb3b17 verdict=REFUSE class=Q1 arm_rc=1

Shadow mode: this records a verdict and merges nothing. A refusal
to arm is not a block (§13 adds zero rows to §7) — the pull request is
exactly as green as it was.

… the done_reason change (#5357)

The two NDJSON stream rows cite the chat and generate handlers and the
framing helper by line; this branch moved them (handlers :654 and :742,
framing :397-437). The commands_enum.rs rows cite lines above the only
edit there, so they still hold.

Refs #5357

Agent: la-0.71.1a
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

@noahgift noahgift left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seat 1, aprender-seat-s11 @77fa906299 APPROVE (scope: Opus-authored commits 7fbc274, 6e74d03, 9d0486a, 0989285)

Read from the commit diffs. Not built or run here; I could not open receipts/c314-la711a-pr5407.md in my checkout, so I did not check it. No blocking finding.

  • 7fbc274: with_constraint_eos adds the vocabulary EOS to the stop list once, with no duplicate. The unit test covers add, no-duplicate and None. The structured-output test uses a caller stop list that lacks the EOS and asserts Complete with no EOS token in the reply. The 499 path runs peek_cancelled after the turn and before the verdict, so a cancelled turn is no longer reported as Truncated.
  • 6e74d03: response_format_of_known_type returns None for null. Anything whose type is not one of text, json_object, json_schema gets a 422 that names the field. Tests cover xml, a numeric type and an empty object, plus the known and null pass-through. Note: the refusal echoes the client's unknown type string back; this is a 422 JSON body, low risk.
  • 9d0486a: a test-only commit. It drives a pre-cancelled token through try_constrained_chat and asserts 499 with no choices and not constraint_truncated. It is not vacuous, because with the check removed the turn answers 422.
  • 0989285: ollama_done_reason maps "length" to "length" and everything else, including the upstream-error fallback, to "stop". It is wired through chat and generate, streaming and non-streaming. Tests cover the length case on both routes, the non-length case, and non-stream bodies for both finish values.

Non-blocking: the upstream-error fallback reports done_reason "stop" with the error text as content. That was already how it behaved.

@noahgift noahgift left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seat 2, aprender-seat-o10 @77fa906299 APPROVE (scope: Sonnet-authored commits)

Opus seat. The commits in scope are e790c1b and 77fa906 (Sonnet 5.5, la-0.71.1a); this session authored no commit on the PR. I read the full diff for consistency.

e790c1b (help text for apr code --output-format json): the help now matches what is written. I checked every key in the new help against aprender-orchestrate/src/agent/code_envelope.rs:132-151 at this head: type, subtype success|error, is_error, status, duration_ms, result, session_id, num_turns, tokens_in, tokens_out, total_cost_usd, backend, plus error{kind, message, exit_code} on error. The backend sub-keys requested, ran and fell_back match served_backend.rs:30-32. The test is gated cfg(all(test, feature = "training")). training is in apr-cli's default features, so the workspace-test lib run executes it, and it is green on this head.

77fa906 (surface_audit.csv line refs): I checked each ref against ollama_handlers.rs at this head. :654 = pub async fn ollama_chat_handler, :742 = pub async fn ollama_generate_handler, :397 = fn ndjson_response, :437 = its closing brace. All four are correct.

Consistency with the rest of the diff: the CSV refs follow the done_reason change in 0989285. The PR body's "Not covered" line (the apr-cli ollama.rs shim still hardcodes done_reason: "stop") is stated openly, not hidden.

CI rollup on 77fa906: SUCCESS (ci / gate, workspace-test, its 3 shards, x86-main, determinism).

Non-blocking (Rust test hygiene):

  1. ENVELOPE_FIELDS is a hand copy. It catches a key dropped from the help, but not a key added to envelope() that the help never names. Deriving the list from a fixture envelope's keys would close that.
  2. The help.contains(field) checks are substring tests, so short keys (ran, kind, type, status) also match other prose ("the device that ran"). Matching on word boundaries or on key,/key} would make the rows stricter.

@noahgift
noahgift added this pull request to the merge queue Oct 15, 2026
Merged via the queue into main with commit 0ab0cd9 Oct 15, 2026
13 checks passed
@noahgift
noahgift deleted the batch/serve-cli-1015 branch October 15, 2026 22:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant