Skip to content

fix(usage): OpenCode usage fidelity (reasoning, cache key, latency, aborts, local pricing) - #223

Merged
pacphi merged 6 commits into
mainfrom
fix/opencode-usage-fidelity
Sep 19, 2026
Merged

pacphi merged 6 commits into
mainfrom
fix/opencode-usage-fidelity

Conversation

@pacphi

@pacphi pacphi commented Sep 19, 2026

Copy link
Copy Markdown
Owner

Summary

PR 4 (last) of the usage-fidelity plan: OpenCode host. Every finding was re-verified at 8ebde36 before it was touched.

Finding Status What changed
O-1 reasoning missing from tokens and estimated cost Fixed A usage row's output now adds reasoning (reasoningOutput keeps it as detail). Guarded so it is never added twice.
O-2 cache key blind to in-place row rewrites Fixed Key gains upd = latest message/session time_updated.
O-3 latency measured to the row insert (~15 ms) Fixed Ends at assistant time.completed; no completed stamp = no sample. Completed stamp and each row's generation span join engaged time.
O-4 user stop counted as an exception Fixed MessageAbortedError goes to aborts. Per-host byHost[h].aborts; tile, session strip, CLI and tooltip corrected.
O-5 local model with no cost priced at the $3/$15 fallback Fixed Local providers (lmstudio, ollama, llama.cpp, registry billing: local) are left unpriced and counted as costEvidence.unpricedMessages; no cache saving quoted for them.
O-7 rows keyed (day, model), provider merged/last-wins Fixed at row level Rows keyed (day, model, provider). Session-level provider bucket still last-wins (deferred, below). Done before O-5 because O-5 needs the row's provider.
O-8 unreported tokens indistinguishable from zero Fixed Completed, error-free, all-zero-token responses are marked tokensUnreported on the row; one usage-not-reported:N warning on OpenCode source health (existing generic warnings, shown in the existing pill tooltip).

Also pinned (verified-correct behaviour): a message with a step-finish part carrying tokens/cost is counted once.

O-1: how additivity was verified

Not from memory and not from data. This machine has no reasoning-bearing message with a priced model (2 zero-token live-test sessions in opencode.db; the 0.9.10 legacy JSON messages are all lmstudio, all reasoning: 0, cost: 0), so the price cross-check is impossible here. Instead I read getUsage out of the installed OpenCode 1.18.31 binary (strings):

tokens = { input: max(0, inputTokens - cacheRead - cacheWrite),
           output: max(0, outputTokens - reasoningTokens),
           reasoning: reasoningTokens, cache: {write, read} }
cost   = input*p_in + output*p_out + cache.read*p_cr + cache.write*p_cw + reasoning*p_out

So output is stored net of reasoning and reasoning is a separate, additive field, priced at the output rate. Recorded cost already includes reasoning, so observed-cost rows are unchanged; only token totals and the estimated cost of missing-cost rows rise.

Risk I could not close: builds that predate that split stored reasoning inside output. The parser guards it with the message's own tokens.total: a total equal to input + cache + output (no reasoning add-on) means output already contained reasoning, so nothing is added. No total, or one that fits neither shape, falls to the verified current convention. Unit-tested both ways.

Before / after on real data (read-only)

Real data is nearly empty, said plainly:

  • ~/.local/share/opencode/opencode.db: 2 sessions, each 1 user + 1 assistant message, all zero tokens, cost 0, no time.completed. Before: each carried a fabricated latency sample (~10 ms created-gap) and rows with no provider. After: 0 latency samples (nothing completed), rows carry provider: lmstudio, nothing else changes (there is nothing to re-price). Both are the mid-turn/abandoned shape O-2 describes.
  • Legacy 0.9.10 JSON messages (8 sessions, 17 messages; only numeric/id fields copied into a scratch db, no text): latency histogram before [2,0,0,0,0,0], [2,0,1,0,0,0], [2,0,0,0,0,0] (all sub-2 s) and after [0,0,0,2,0,0], [0,0,0,2,1,0], [0,0,2,0,0,0] (10-30 s and 5-10 s, matching the 8-21 s the JSON shows); engaged time +19-20 s per session; 10 completed all-zero lmstudio responses flagged tokensUnreported (0 before, no signal). Tokens stay 0: those servers reported none.

Schema

SCHEMA_VERSION 24 to 25 with a v25: note. Test that a forged v24 cache is discarded. (The claude-window test's === 24 became >= 24.)

Docs / ADR

ADR-0038 dated correction note (no history rewrite), ADR index status, USAGE-SCORECARD-METRICS (cost, latency, reliability/aborts), TRANSCRIPTS, DASHBOARD, TROUBLESHOOTING, in current-state voice. ADR-0042/0052 do not describe OpenCode counting and are untouched. file:line citations remapped; doc-citations green.

Dropped / deferred

  • Session-level provider bucket (byProvider) is still the last observed provider for a session; attributing a mixed-provider session across provider buckets is a larger aggregation change. Rows themselves are now correct.
  • Custom-named local providers (an id that is not lmstudio/ollama/llama.cpp/local-openai) cannot be recognised: the transcript records the id, not the endpoint, and the base URL lives in a config file the reader must not open. They keep the fallback rate.
  • Out of scope, untouched: O-6, O-9, O-10, O-11, O-12, O-13, any Claude/Codex code, dashboard layout.

Verification

Node 22.22.3 / pnpm 11.17.0, local (macOS). Every fix has a test written before the code (a mutation check on O-1 confirmed the new tests bind it).

  • pnpm run check: typecheck, lint (0 errors; no warnings in touched files), lint:cc, lint:md, build all green. pnpm test node:test stage: 4092 tests, 4085 pass, 6 skipped, 1 fail. Coverage: 95.26% lines / 85.64% branches / 93.57% functions (floors 70%).
  • The 1 failure, on two full runs, is opencode-stock-ruflo-gateway "stock OpenCode keeps Ruflo and Agentic QE connected..." with spawnSync npm ETIMEDOUT (live npm install). It is the known slow-registry flake; rerun alone it passes (1/1).
  • Because that failure short-circuits the && chain, the remaining pnpm test scripts were run separately and pass: statusline-segments, statusline-window-ledger, statusline-brain, agentdb, health-history, harvest, dashboard, admin-model, admin.
  • pnpm run test:ui (dashboard client text/logic touched): 491 passed, 0 failed, then 8/8.

Windows pitfalls checked in the new tests: expected paths built with path.join, no bare absolute import(), no POSIX mode-bit or sh child commands, the CLI test sets HOME, USERPROFILE, XDG_CONFIG_HOME, APPDATA and XDG_DATA_HOME, every fixture db is closed before its tmp dir is removed, no CRLF-sensitive assertions, and no real ~/.local/share/opencode or ~/.config touched.

🤖 Generated with Claude Code

… is re-read

OpenCode inserts the assistant row when a turn starts and rewrites the same row
with tokens, cost and completed stamp later. The (MAX(time_created), COUNT)
key was blind to that, so a mid-turn scan stayed cached. Add the latest
message/session time_updated to the entry key and bump the schema to 25.
…ps as aborts

Latency was user.created to assistant.created (~15 ms, since OpenCode inserts
the assistant row immediately); it now ends at assistant time.completed, and no
sample is taken when a row never completed. The completed stamp and each row's
generation span join engaged time. MessageAbortedError routes to aborts, not
exceptions; the abort stat is kept per host and the reliability panel, session
strip and CLI read it over the codex and opencode hosts that record a stop.
OpenCode stores output net of reasoning (output = outputTokens - reasoning) and
prices reasoning at the output rate, so recording tokens.output alone
under-counted every reasoning model in tokens and estimated cost. Add reasoning
to the usage row (and the missing-cost portion), guarded against older builds
whose tokens.total shows output already contained it. Pin that a step-finish
part never double counts its message.
Rows were keyed (day, model), so the same modelID under two providers, or a
mid-session provider switch, merged into one row naming neither. addUsage now
takes an optional provider that joins the key; sources that do not pass one
keep their exact prior key and shape. The session-level inferenceProvider stays
the last observed one.
… unreported tokens

A local OpenCode model (lmstudio, ollama, llama.cpp) with no recorded cost fell
through to the Sonnet-class fallback (1M in + 1M out priced at $18) because the
pricer only ever saw the host id. Rows now price with their own provider; a
local provider's missing-cost portion is left unpriced and counted as
costEvidence.unpricedMessages, and no cache saving is quoted for it.

Completed responses that carried no token counts are marked tokensUnreported
on their row and raise one informational usage-not-reported:N warning on
OpenCode source health, so unreported usage is distinguishable from zero usage.
…rrection

Current-state docs for latency-to-completion, user stops as aborts, reasoning as
output, local models left unpriced, provider-keyed rows and the
usage-not-reported warning, plus a dated ADR-0038 correction note and remapped
file:line citations.
@pacphi
pacphi merged commit 01e8045 into main Sep 19, 2026
16 checks passed
@pacphi
pacphi deleted the fix/opencode-usage-fidelity branch September 19, 2026 18:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant