fix(usage): OpenCode usage fidelity (reasoning, cache key, latency, aborts, local pricing) - #223
Merged
Merged
Conversation
… is re-read OpenCode inserts the assistant row when a turn starts and rewrites the same row with tokens, cost and completed stamp later. The (MAX(time_created), COUNT) key was blind to that, so a mid-turn scan stayed cached. Add the latest message/session time_updated to the entry key and bump the schema to 25.
…ps as aborts Latency was user.created to assistant.created (~15 ms, since OpenCode inserts the assistant row immediately); it now ends at assistant time.completed, and no sample is taken when a row never completed. The completed stamp and each row's generation span join engaged time. MessageAbortedError routes to aborts, not exceptions; the abort stat is kept per host and the reliability panel, session strip and CLI read it over the codex and opencode hosts that record a stop.
OpenCode stores output net of reasoning (output = outputTokens - reasoning) and prices reasoning at the output rate, so recording tokens.output alone under-counted every reasoning model in tokens and estimated cost. Add reasoning to the usage row (and the missing-cost portion), guarded against older builds whose tokens.total shows output already contained it. Pin that a step-finish part never double counts its message.
Rows were keyed (day, model), so the same modelID under two providers, or a mid-session provider switch, merged into one row naming neither. addUsage now takes an optional provider that joins the key; sources that do not pass one keep their exact prior key and shape. The session-level inferenceProvider stays the last observed one.
… unreported tokens A local OpenCode model (lmstudio, ollama, llama.cpp) with no recorded cost fell through to the Sonnet-class fallback (1M in + 1M out priced at $18) because the pricer only ever saw the host id. Rows now price with their own provider; a local provider's missing-cost portion is left unpriced and counted as costEvidence.unpricedMessages, and no cache saving is quoted for it. Completed responses that carried no token counts are marked tokensUnreported on their row and raise one informational usage-not-reported:N warning on OpenCode source health, so unreported usage is distinguishable from zero usage.
…rrection Current-state docs for latency-to-completion, user stops as aborts, reasoning as output, local models left unpriced, provider-keyed rows and the usage-not-reported warning, plus a dated ADR-0038 correction note and remapped file:line citations.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PR 4 (last) of the usage-fidelity plan: OpenCode host. Every finding was re-verified at
8ebde36before it was touched.outputnow adds reasoning (reasoningOutputkeeps it as detail). Guarded so it is never added twice.upd= latest message/sessiontime_updated.time.completed; no completed stamp = no sample. Completed stamp and each row's generation span join engaged time.MessageAbortedErrorgoes toaborts. Per-hostbyHost[h].aborts; tile, session strip, CLI and tooltip corrected.lmstudio,ollama,llama.cpp, registrybilling: local) are left unpriced and counted ascostEvidence.unpricedMessages; no cache saving quoted for them.tokensUnreportedon the row; oneusage-not-reported:Nwarning on OpenCode source health (existing generic warnings, shown in the existing pill tooltip).Also pinned (verified-correct behaviour): a message with a
step-finishpart carrying tokens/cost is counted once.O-1: how additivity was verified
Not from memory and not from data. This machine has no reasoning-bearing message with a priced model (2 zero-token live-test sessions in
opencode.db; the 0.9.10 legacy JSON messages are all lmstudio, allreasoning: 0,cost: 0), so the price cross-check is impossible here. Instead I readgetUsageout of the installed OpenCode 1.18.31 binary (strings):So
outputis stored net of reasoning and reasoning is a separate, additive field, priced at the output rate. Recorded cost already includes reasoning, so observed-cost rows are unchanged; only token totals and the estimated cost of missing-cost rows rise.Risk I could not close: builds that predate that split stored reasoning inside
output. The parser guards it with the message's owntokens.total: a total equal to input + cache + output (no reasoning add-on) means output already contained reasoning, so nothing is added. No total, or one that fits neither shape, falls to the verified current convention. Unit-tested both ways.Before / after on real data (read-only)
Real data is nearly empty, said plainly:
~/.local/share/opencode/opencode.db: 2 sessions, each 1 user + 1 assistant message, all zero tokens, cost 0, notime.completed. Before: each carried a fabricated latency sample (~10 ms created-gap) and rows with no provider. After: 0 latency samples (nothing completed), rows carryprovider: lmstudio, nothing else changes (there is nothing to re-price). Both are the mid-turn/abandoned shape O-2 describes.[2,0,0,0,0,0],[2,0,1,0,0,0],[2,0,0,0,0,0](all sub-2 s) and after[0,0,0,2,0,0],[0,0,0,2,1,0],[0,0,2,0,0,0](10-30 s and 5-10 s, matching the 8-21 s the JSON shows); engaged time +19-20 s per session; 10 completed all-zero lmstudio responses flaggedtokensUnreported(0 before, no signal). Tokens stay 0: those servers reported none.Schema
SCHEMA_VERSION24 to 25 with av25:note. Test that a forged v24 cache is discarded. (The claude-window test's=== 24became>= 24.)Docs / ADR
ADR-0038 dated correction note (no history rewrite), ADR index status, USAGE-SCORECARD-METRICS (cost, latency, reliability/aborts), TRANSCRIPTS, DASHBOARD, TROUBLESHOOTING, in current-state voice. ADR-0042/0052 do not describe OpenCode counting and are untouched.
file:linecitations remapped;doc-citationsgreen.Dropped / deferred
byProvider) is still the last observed provider for a session; attributing a mixed-provider session across provider buckets is a larger aggregation change. Rows themselves are now correct.Verification
Node 22.22.3 / pnpm 11.17.0, local (macOS). Every fix has a test written before the code (a mutation check on O-1 confirmed the new tests bind it).
pnpm run check: typecheck, lint (0 errors; no warnings in touched files), lint:cc, lint:md, build all green.pnpm testnode:test stage: 4092 tests, 4085 pass, 6 skipped, 1 fail. Coverage: 95.26% lines / 85.64% branches / 93.57% functions (floors 70%).opencode-stock-ruflo-gateway"stock OpenCode keeps Ruflo and Agentic QE connected..." withspawnSync npm ETIMEDOUT(live npm install). It is the known slow-registry flake; rerun alone it passes (1/1).&&chain, the remainingpnpm testscripts were run separately and pass: statusline-segments, statusline-window-ledger, statusline-brain, agentdb, health-history, harvest, dashboard, admin-model, admin.pnpm run test:ui(dashboard client text/logic touched): 491 passed, 0 failed, then 8/8.Windows pitfalls checked in the new tests: expected paths built with
path.join, no bare absoluteimport(), no POSIX mode-bit orshchild commands, the CLI test setsHOME,USERPROFILE,XDG_CONFIG_HOME,APPDATAandXDG_DATA_HOME, every fixture db is closed before its tmp dir is removed, no CRLF-sensitive assertions, and no real~/.local/share/opencodeor~/.configtouched.🤖 Generated with Claude Code