Repository navigation
fix(ai): capture an error event when a streaming call fails to connect - #1146
pcbeingused333 wants to merge 3 commits into
Conversation
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Alex Castillo <alex.castillog33@gmail.com>
marandaneto
left a comment
There was a problem hiding this comment.
Automated advisory code review.
| response = super().create(**kwargs) | ||
| except Exception as exc: | ||
| accumulator = _AnthropicStreamAccumulator() | ||
| self._capture_streaming_event( |
There was a problem hiding this comment.
blocking: Preserve the provider exception when error telemetry cannot format rejected input — When Anthropic rejects malformed input such as messages=[None] with HTTP 400, the newly added capture call formats that same rejected input before reaching the bare raise. format_anthropic_input calls msg.get(...), raising AttributeError and replacing the original anthropic.BadRequestError; the async handler at posthog/ai/anthropic/anthropic_async.py:169 behaves identically. Previously callers received the actionable provider error and could handle BadRequestError normally. Ensure failures in this telemetry path cannot replace the original exception. Reproduction: reproduced — python -m pytest -p no:cacheprovider posthog/test/ai/test_review_connect.py -k invalid_anthropic --timeout=30 -vv fails both sync/async assertions on the reviewed head with AttributeError: 'NoneType' object has no attribute 'get', while both pass unchanged at the recorded merge base.
marandaneto
left a comment
There was a problem hiding this comment.
Automated advisory code review.
| kwargs, | ||
| usage_stats, | ||
| time.time() - start_time, | ||
| accumulated_content, |
There was a problem hiding this comment.
blocking: Keep the new Gemini handlers compatible with Mypy — Passing the initially empty, unannotated accumulated_content into this new exception handler makes Mypy require an explicit element type before the nested generator populates it. Both the sync and async additions introduce var-annotated errors and fail the required Mypy check; please annotate the accumulated content in both wrappers. Reproduction: reproduced — python -m mypy --no-site-packages --config-file mypy.ini posthog/ai/gemini/gemini.py posthog/ai/gemini/gemini_async.py reports exactly two annotation errors on the reviewed head and no issues on the recorded merge base.
…ror-capture # Conflicts: # posthog/ai/openai/openai.py # posthog/ai/openai/openai_async.py
💡 Motivation and Context
For streaming calls the OpenAI chat, OpenAI Responses, Anthropic and Gemini wrappers call the underlying
create/generate_content_streamoutside the try/finally that captures the event, and only wrap the iteration of the returned stream. A connection-time failure (401 bad key, 429 rate limit, invalid model) is raised before any generator exists, so no$ai_generationevent is sent; only errors raised mid-stream are recorded. Non-streaming calls already record these errors throughcall_llm_and_track_usage.The streaming call is now wrapped in
try/except Exception: on failure the existing_capture_streaming_eventis called with an empty stream state anderror=exc(which sets$ai_is_errorand$ai_http_status), and the exception is re-raised unchanged. Sync and async, all four providers. No issue is linked; the gap is visible from the code path and reproduced by the tests below.💚 How did you test it?
Eight tests in
posthog/test/ai/test_stream_midstream_error.py(OpenAI chat, OpenAI Responses, Anthropic and Gemini, sync and async) make the underlying call raise a 401 error and assert the captured event has$ai_is_errorand$ai_http_status == 401; all fail onmain(no event captured) and pass with the change.pytest posthog/test/ai: full suite green;ruff checkandruff format --checkclean. Not run against live provider APIs.📝 Checklist
If releasing new changes
.sampo/changesets/stream-connect-error.md(pypi/posthog: patch).🤖 Agent context
Autonomy: Human-driven (agent-assisted)
The change and the tests were written by Claude Code (Anthropic's agent) working for the PR author, who directs this contribution work and is the DRI. The agent compared the streaming and non-streaming paths of each wrapper, wrote the failing tests first, then wrapped only the initial call so the per-chunk handling is untouched. No manual testing against the providers was done.