Skip to content

fix(ai): keep the choices of an n>1 OpenAI chat stream apart - #1152

Merged
marandaneto merged 2 commits into
PostHog:mainfrom
jaeminbird:fix/openai-stream-n-choices
Oct 30, 2026
Merged

marandaneto merged 2 commits into
PostHog:mainfrom
jaeminbird:fix/openai-stream-n-choices

Conversation

@jaeminbird

Copy link
Copy Markdown
Contributor

Motivation and Context

Streaming an OpenAI chat completion with n>1 merges every choice into one. _ChatCompletionsStreamState (posthog/ai/openai/_streaming.py) reads only choices[0] of each chunk and appends to a single buffer, but each chunk carries one choice identified by choices[].index. The text interleaves ("HelloBonjour world le monde"), tool calls from different choices collide on index 0, and $ai_stop_reason comes from whichever choice finished last. The non-streamed path already keeps one item per choice.

How did you test it?

The stream state now collects content and tool calls per choice index and builds $ai_output_choices in the same shape as the non-streamed path; the stop reason is choice 0's, as in extract_openai_stop_reason. n=1 streams are unchanged.

  • New test_streaming_chat_keeps_n_choices_apart fails on main (interleaved text) and passes with the fix; 3 reruns passed.
  • The real openai SDK against a mock HTTP transport serving the documented n=2 SSE format: the streamed $ai_output_choices equals the non-streamed capture of the same answer.
  • OpenAI tests: 108 pass. ruff format/check, mypy baseline and python -W error -c "import posthog" clean. Full suite: no new failures (19 failures and 2 collection errors are pre-existing Windows/Anthropic environment problems).

Checklist

  • Tests added/updated. No public API change (make public_api_check not run).
  • No docs change needed.
  • Not run: a Linux run, a live OpenAI call.

Agent context

Autonomy: human-driven (agent-assisted). Kimi Code found the bug by comparing the streamed capture with the non-streamed one and wrote the fix and test; I reviewed and verified it.

@jaeminbird
jaeminbird requested a review from a team as a code owner October 30, 2026 05:56

@marandaneto marandaneto left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks good; no qualifying findings. Verdict: correct.

@marandaneto

Copy link
Copy Markdown
Member

thanks @jaeminbird

@marandaneto
marandaneto enabled auto-merge (squash) October 30, 2026 13:15
@marandaneto
marandaneto merged commit be77b83 into PostHog:main Oct 30, 2026
42 checks passed
@jaeminbird
jaeminbird deleted the fix/openai-stream-n-choices branch October 30, 2026 15:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants