Repository navigation
client-driven Ollama arm (apr test llm bench + client sha256) before any ratio vs Ollama #5033
Closed as not planned
Description
What
Add a client-driven Ollama arm (apr test llm bench plus a client-side sha256 of the output) before any ratio against Ollama is published.
Today the card row says "measured by us" for a number that is self-timed, and the beat-ollama contract ratio uses Ollama's own reported eval rate rather than a client-measured one.
Done when
Any Ollama ratio is computed from a client-driven run with the output sha256 recorded, and the card wording matches what was measured.
Proposed by la-0.72b (inbox 04:49Z, ticket context #4401). Refs #4401.
Activity
noahgift commentedon Oct 10, 2026
Parked from #4381: the epic is at 10 open children against a budget of 9, after new tickets were linked. This ticket has no pushed branch and no open PR. It stays open, keeps its owner and milestone, and re-enters when a child of that epic closes (1-in-1-out). Nothing reads sub-issue membership (checked: no code, workflow or contract), so no gate changes.
fix(apr-cli): the Ollama speed arm is NotRun until our client times i…
noahgift commentedon Oct 14, 2026
merged into #5305 (master ticket for epic #3598; triage rule 7b, 2026-10-14). It is a checklist row there; reopen this issue when work on it starts or its PR opens.
noahgift commentedon Oct 14, 2026
Moved: this sub-ticket is now a row in #5322 (Speed measurement harness: resource receipts, observability and the ollama/llama.cpp ratio), split out of #5305 by theme.