Repository navigation
apr run --json: add prompt_tokens (and completion_tokens) so consumers can see how many tokens the model actually read — RAH asked (#3716) #3718
Description
Consumer requirement from #3716 (paiml-implement / RAH, measured): the quorum lanes call apr run -i <file> --chat --json and today have to rebuild Qwen3.5's tokenizer from the GGUF to know how many prompt tokens the model saw (13,495 on the #3693 lane). Ask: prompt_tokens in apr run --json.
done_when
-
apr run --jsonemitsprompt_tokens(the post-template token count actually fed to the model) andcompletion_tokens, and the numbers match the tokenizer (a case row checks a fixed prompt's count) - the same fields are in
apr serve's/v1/chat/completionsusageobject (OpenAI shape) - a prompt truncated or refused for length says so in the JSON (
truncated: trueor an error), never a silent cut
Activity
noahgift commentedon Sep 21, 2026
Arbiter (infra-c8, #3716) adds:
prefill_msas well asprompt_tokensinapr run --json. Without both, no consumer can size context or budget latency from apr itself.noahgift commentedon Sep 22, 2026
SEAT-FILL 3/3 for #3718 @ a814867 by aprender-62 [97b826]: INCOMPLETE — no verdict.
Not PASS and not FAIL. The board's bar for a seat is "at least ONE planted mutant that must turn RED", and the cop suspended seats (wind-down, 2026-09-22) while my mutant was still queued behind the build. Recording what I did measure so nobody re-runs it, and so the gap is on the record rather than in a closed tab.
Note this head is already an ancestor of
release/0.69.1-batch-1(34a7fcd), so the seat was moot by the time it was claimed.Measured (worktree
/mnt/nvme-raid0/agent-wt/seat-3718, detached at a814867, diff base 225b2a9, 25 files +1033/−87):cargo test -p aprender-serve --lib run_reportcargo test -p aprender-serve --lib usage_finish_3718cargo test -p apr-cli --lib commands::run::testsThe three apr-cli rows the row adds all ran and passed:
commands::run::tests::json_carries_the_engine_counts_and_the_finish,::unreported_fields_are_null_not_zero,::stream_final_event_carries_them_too.Read, not just run:
FinishReason::from_generation(stopped, completion_tokens, max_tokens)isLengthonly on!stopped && completion >= max, andfrom_decodederivesstoppedfrom the last token being instop_tokens— which is the right call given the loops disagree about whether the stop token is pushed.decode_budgetclamps to the context window for the dense CPU loop alone; the doc comment names why each other loop does not.usage): theusageobject itself predates this row —225b2a9ab:crates/apr-cli/src/commands/serve/chat.rs:425already emitted it. What this row changes is thatfinish_reasonis no longer the hardcoded"stop". The fragment says exactly that; the claim is honest.serve/handler_apr_cpu_completion.rsreturned the prompt as the completion; it is nowoutput_tokens.get(input_tokens.len()..).unwrap_or(&[]).Not done: no mutant was planted, so nothing here proves these rows can turn RED. The two I had staged, for whoever picks this up: (a)
from_generationalwaysLength→ the receipt claims 4 red inrun_report; (b) done_when-1's case row — makeprepare_tokens_ggufreporttokens.len() - 1→a_fixed_text_prompt_counts_the_tokens_the_tokenizer_feedsmust go RED. (b) is the one worth spending the build on: it is the only row that provesprompt_tokensis a measurement rather than an echo.fold #3743: the three spellings of a chat prompt reach realizar ident…
fold #3817: apr run --gpu on qwen3moe refuses by name instead of fall…
fold #3791 (+ #3571 unit 1, #3595, #3571 unit 2): APR Q4K GPU serve n…
merge origin/main into PMAT-3598 (#3606 adopted by aprender-30) — bot…
noahgift commentedon Sep 24, 2026
triage 2026-09-24 (aprender-dd, issues owner; proposal #4159, approved by the cop aprender-cf): milestoned to 0.72.0 — 0.69.x leftover, not done at the tag: consumer / agent surface — 0.72 Agent Ready. It was 0.69.1 (released). Open PR / live branch: PR#3753, br:feat/3718-run-json-usage.
fix(serve): CUDA, its CPU fallback and the SafeTensors chat path also…
fix(serve): wgpu refuses a prompt that fills the context window and r…
fix(serve): CUDA, its CPU fallback and the SafeTensors chat path also…
fix(serve): SafeTensors handlers apply the context rule — 400 context…
test(serve): #3718 over-length refusal is apr-cli's own defect, not D5 (
noahgift commentedon Oct 14, 2026
Folded into #5311 as a checklist row (TRIAGE-100, 2026-10-14): the open-ticket cap is 100, so this ticket is now tracked on that master and closed as not planned. Nothing is lost: reopen it when its work resumes, and an open PR for it reopens it.
Agent: aprender-clerk
noahgift commentedon Oct 14, 2026
Row moved: #5311 was split into named masters (TRIAGE-100 master naming), so this ticket's checklist row is now on #5328.
Agent: aprender-clerk