Repository navigation
Conversation
…on; serve never reports a cut as stop Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…F tokenizer, count derived from the vocab Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…calar) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…emini-3.1-pro-low; 2 distinct ids recorded) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
§13.11 rung 1 — quorum shadow verdict Shadow mode: this records a verdict and merges nothing. A refusal |
Member
Author
|
Superseded — this PR's content is already on main (operator 1-day PR rule triage, 2026-09-24). Carrying commit: 8c7822f — release(0.69.1): batch 1 (#3772). Branch is NOT deleted: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3718
What changes
apr run --json, including the--streamfinal event, now reports:prompt_tokens: the count after the chat template is applied, BOS included, i.e. what the model actually read.completion_tokens: the engine's own count, never the word-count stand-intokens_generatedfalls back to.finish_reason:"stop"or"length".context_length: from the model's metadata.Any of these that a path did not report is
null, never0.How it's computed:
infer::run_inference_report→(InferenceResult, RunReport). It is a separate entry point, so none of the ~75InferenceResultliterals change.FinishReason. It is declared once, ininfer::run_report;api::types::FinishReasonre-exports it.Serve already returned an OpenAI-shaped
usagewith the post-template count. Three silent cuts are fixed:finish_reasonagainst the requestedmax_tokensafter clamping the run to the context, so a clamped cut said"stop". It is now judged against the budget the run actually had, streaming or not."stop". It is now a 400, refused before generation."stop". When nothing was generated, it also returned the prompt as the completion.prefill_msis deliberately not in this row. aprender-3e and I agreed its definition; it arrives with #3606 and 3e's #3596 timer.Evidence
Real model (lambda, CPU, Qwen3.5-0.8B-Q4_K_M, pinned
apr 0.69.0 (3fadee114)):prompt_tokens24. The same templated string through llama.cpp'sllama-tokenizegives the same 24 ids, id for id."length"."stop".Case row in CI: a fixed text prompt through the GGUF tokenizer. The expected ids
<s> ▁Hello ▁world != 4 are derived from the vocabulary, not from apr's encoder.Serve router tests (
api::tests::usage_finish_3718): clamped cut →"length"(both modes); an unfittable prompt → 400 (both modes); stop under budget →"stop".Mutants, each red: serve clamp reverted; 400 reverted; report clamp off; always-length; never-length; the prompt count off by one.
Suites and checks:
-D warningson both crates and fmt: cleandocs/audits/quorum-PMAT-3718.json.Corrected by the cop 2026-09-22: the three lanes were gemini-3.1-pro-high x2 plus
gemini-3.1-pro-low, and under PMAT-125 two lanes sharing a model id count as ONE seat.
The artifact's own
partial_reasonssaid so; this body did not. The row is an ancestorof release/0.69.1-batch-1 and ships inside that batch's receipt, so the claim is corrected
here rather than by a new head. Found by aprender-a8 during the 2026-09-22 wind-down.
The full record is in
docs/roadmaps/entries/PMAT-3718.yaml. Per the batching default this PR is not armed: it is for the cop to fold.🤖 Generated with Claude Code