Skip to content

Streamed Anthropic generations stack output tokens when a stream emits more than one message_delta #1099

Closed

Description

@Harsh23Kashyap

Version

7.75.0 (current master, includes #1074)

Steps to Reproduce

  1. Stream an Anthropic response whose stream emits more than one message_delta event. This happens with server tools such as web search or code execution, which pause and resume the turn inside one stream.
  2. Each message_delta carries cumulative usage, but the accumulator in posthog/ai/anthropic/_anthropic_stream.py merges output_tokens incrementally, so successive deltas stack.

Minimal repro:

from types import SimpleNamespace as NS
from posthog.ai.anthropic._anthropic_stream import _AnthropicStreamAccumulator

acc = _AnthropicStreamAccumulator()
acc.consume(NS(type="message_start", message=NS(usage=NS(input_tokens=100, output_tokens=1, cache_creation_input_tokens=0, cache_read_input_tokens=0))))
acc.consume(NS(type="message_delta", usage=NS(output_tokens=50)))
acc.consume(NS(type="message_delta", usage=NS(output_tokens=110)))
acc.consume(NS(type="message_stop"))
print(acc.usage_stats["output_tokens"])

Ran it twice, identical both times: 160

Expected Result

110, the final cumulative total. The Anthropic SDK accumulates stream usage by overwriting on each message_delta ('Usage counts on a message_delta are cumulative totals, so they overwrite rather than add', anthropic-sdk-python lib/streaming/_messages.py). #1074 fixed the same stacking for input and cache counts; output_tokens still stacks.

Actual Result

160 (50 + 110 stacked). Streamed generations over-report output tokens whenever a stream contains more than one message_delta.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions