Skip to content

fix(ai): use cumulative input and cache tokens from Anthropic message_delta when streaming - #5352

Merged
marandaneto merged 1 commit into
PostHog:mainfrom
pcbeingused333:fix/ai-anthropic-stream-cumulative-input-usage
Oct 21, 2026
Merged

marandaneto merged 1 commit into
PostHog:mainfrom
pcbeingused333:fix/ai-anthropic-stream-cumulative-input-usage

Conversation

@pcbeingused333

Copy link
Copy Markdown
Contributor

Problem

@posthog/ai's Anthropic streaming wrapper only reads output_tokens (and the web search count) from message_delta. Anthropic's message_delta usage is cumulative: with server tools (web search, web fetch, code execution) its input_tokens, cache_creation_input_tokens and cache_read_input_tokens grow past the message_start values during the response. Streamed generations therefore under-report $ai_input_tokens and the cache counts.

This is the same bug that was fixed for posthog-python in PostHog/posthog-python#1074.

Changes

In packages/ai/src/anthropic/index.ts, the message_delta usage now also updates inputTokens, cacheCreationInputTokens and cacheReadInputTokens, keeping the larger of the message_start value and the delta value (totals never shrink). A field the delta omits or under-reports stays at the message_start value. rawUsage merging is unchanged.

A patch changeset for @posthog/ai is included.

Test strategy

  • Unit (packages/ai/tests/anthropic.test.ts, new PostHogAnthropic - streaming usage block, no API key needed): one test where message_delta reports larger cumulative input/cache counts (expects 120 / 30 / 70 instead of the message_start 50 / 10 / 20), and one where the delta omits or under-reports them (expects the message_start values to stay). The first fails on main; the second guards against regressions.
  • End-to-end smoke: built @posthog/ai, pointed the real @anthropic-ai/sdk through PostHogAnthropic at a local HTTP server that streams a real SSE sequence (message_start with input 50 / cache 10 / 20, final message_delta with input 120 / cache 30 / 70 and output 25), and read the captured event. On main the event reports input 50, cache creation 10, cache read 20, output 25. With this change it reports input 120, cache creation 30, cache read 70, output 25. No API keys were used.
  • Checks: tests/anthropic.test.ts passes (9 passed; the remaining tests in that file are skipped without ANTHROPIC_API_KEY), oxlint reports 0 errors and oxfmt --check is clean on the touched files.
  • Edge cases considered: delta omits a field, delta reports a smaller value, no web search (counts unchanged), message_start missing (previous ?? 0 behavior kept).

Release info Sub-libraries affected

Libraries affected

  • @posthog/ai

Checklist

  • Tests for new code
  • Accounted for the impact of any changes across different platforms
  • Accounted for backwards compatibility of any changes (no breaking changes!)
  • Took care not to unnecessarily increase the bundle size

If releasing new changes

  • Ran pnpm changeset to generate a changeset file

🤖 Agent context

Autonomy: Fully autonomous

  • Written with Claude Code (Claude Sonnet 5.5): the bug was found by comparing with the merged posthog-python fix, and the change, tests, smoke run and this description were produced by the agent. Agent-authored, needs human review.
  • The fix keeps the larger of the message_start and message_delta values rather than overwriting, since it also tolerates a delta that omits or under-reports a field.

🤖 Generated with Claude Code

@pcbeingused333
pcbeingused333 requested a review from a team as a code owner October 21, 2026 12:59

@marandaneto marandaneto left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks good; no qualifying findings. Verdict: correct.

…_delta when streaming

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@marandaneto
marandaneto force-pushed the fix/ai-anthropic-stream-cumulative-input-usage branch from 4256e04 to 5214612 Compare October 21, 2026 14:15
@marandaneto
marandaneto enabled auto-merge October 21, 2026 14:17
@marandaneto
marandaneto added this pull request to the merge queue Oct 21, 2026
Merged via the queue into PostHog:main with commit dd58d50 Oct 21, 2026
65 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants