Repository navigation
usage: count Codex reasoning tokens as thinking - #1794
Conversation
Every Codex row was recorded with thinkingTokens 0, so the thinking share from #1783 was blind on GPT lanes. The Responses usage block carries output_tokens_details.reasoning_tokens inside output_tokens, the same way Anthropic counts thinking; reasoningTokensFromResponses() reads it (rounded, capped at output, 0 when absent), splitResponsesUsage() and CodexForwardOutcome carry it from both GPT routes, and reportCodexOutcome records it. Accounting only; nothing on the wire changes. New test/ledger-thinking-codex.mjs (12 checks, 7 fail without the record change).
📦 This PR will not cut a releaseversion stays at 6.20.0 but 5 shipping file(s) changed: README.md, docs/pool-keys-and-analytics.md, site/index.html (+2 more)
If that is intended (revert, docs-only refactor, or a stacked PR whose
This comment is advisory and updates itself on every push; it never |
Live test: ✅ PASSHead Stages |
sprayberry-redline
left a comment
There was a problem hiding this comment.
Verdict: approve. Codex Responses usage now carries bounded reasoning-token counts through both forwarding routes into analytics and the ledger, with coverage for the reader, routes, summaries, and displayed share.
Minor:
test/ledger-thinking-codex.mjs:2: This introductory block narrates the motivation and structure of the patch rather than documenting a test-specific constraint. It adds a large maintenance burden without helping the assertions establish their behavior.
check-readme-line-count failed on this branch after the master merges: src/ measures 44,018 lines across 88 files against a stated ~43k (the site still said ~42k across 87).
sprayberry-redline
left a comment
There was a problem hiding this comment.
Verdict: approve. This carries Responses reasoning token usage through both Codex forwarding paths into analytics and ledger thinking totals. The reader validates and caps the value against output, and the new end-to-end coverage exercises translated and passthrough routes, absent details, and the ledger summaries.
What does this PR do?
The thinking share from #1783 now covers GPT traffic. Every Codex row was recorded with
thinkingTokens: 0(hardcoded inreportCodexOutcome), sodario usage,--by-keyand/analyticsshowed no reasoning on any ChatGPT-subscription lane, however much the model reasoned.reasoningTokensFromResponses()incodex-backend.tsreadsoutput_tokens_details.reasoning_tokensfrom the Responses usage block. OpenAI counts it insideoutput_tokens, the same way Anthropic counts thinking inside output, so it fits the existing field as is. Rounded, capped at the output it is part of, 0 when absent or malformed.splitResponsesUsage()carries it asreasoning;CodexForwardOutcomegainsreasoningTokens, set on both report sites (the translated Messages route and the Responses passthrough);reportCodexOutcomerecords it asthinkingTokens.Why: on one deployment's last 7 days, 22% of all output tokens were GPT through the Codex engine, and the thinking line could say nothing about any of it. With the per-key default effort from #1786 able to set
reasoning.efforton those lanes, the share is the number that says which of them need it.How to test
npm run buildnode test/ledger-thinking-codex.mjs(new, 12 checks: the reader's tolerance and cap,splitResponsesUsage, then a realstartProxywith a loopback Codex stub on both GPT routes, checked on the window record, the ledger cell, the consumer summary with its price, and theThinking:line; 7 of them fail against the proxy without the record change). It is its own file because the proxy reads the Codex base URL and the accounts folder once per process.codex-backend,ledger,ledger-wiring,ledger-thinking,ledger-per-key-tokens,analytics-recording,analytics-stream,responses-inbound-wiring,anthropic-responses-translate.test,anthropic-responses-stream.test,key-effort,keys-budget-proxy,metrics,tui-*.Checklist
npm run buildpassesnpm testpasses (CI; the suites above were run locally)src/? Then a bullet under## [Unreleased]inCHANGELOG.mdis in this PR## [x.y.z]heading addedproxy.ts: one field on the Codex analytics record;test/compat.mjsin CI