Repository navigation
test(e2e): Add node-openai-send-to-sentry test app - #24824
Merged
Merged
Conversation
An e2e app that uses the OpenAI integration the way a user does and sends the data to a real Sentry project. A plain Sentry.init on an express app, no tunnel, and the stock openai client making real chat, streaming, tool-call, Responses API and embeddings requests through OpenRouter. The tests read the gen_ai spans back through the Sentry API, like the other *-send-to-sentry apps, and check op, name, origin, status, token usage, finish reasons and the recorded prompts and answers. A latest variant runs the same suite against openai@latest. The attribute checks use the stored names, which differ from what the SDK sends: sentry.origin is stored as origin, sentry.op as span.op, gen_ai.response.text as gen_ai.output.messages, and integers come back as strings. gen_ai.response.tool_calls is not stored under that name at all, so the tool-call test relies on the finish reason. Closes #24747 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Contributor
size-limit report 📦
|
Over the minutes a send-to-sentry test polls, the API closes the odd
connection ("TypeError: terminated, other side closed"). An error thrown
inside expect.poll fails the test instead of letting it retry, which is
how the tool-call test failed on the first CI run while the other four
passed. Treat a network error during a lookup like a non-2xx answer: log
it and report "not there yet".
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
RulaKhaled
added a commit
that referenced
this pull request
Oct 1, 2026
An e2e app that uses the Anthropic integration the way a user does and sends the data to a real Sentry project, per #24748. Same shape as the OpenAI one in #24824. **Stacked on #24855.** The `latest` variant needs that fix; this PR targets its branch and retargets to `develop` once it merges. **The app.** A plain `Sentry.init` on an express app, preloaded with `node --import`, no tunnel. The stock `@anthropic-ai/sdk` client talks to OpenRouter's Anthropic-compatible endpoint. Four routes make one real request each: a message, a streamed message, one through the `messages.stream()` helper, and a forced tool use. **The tests.** Each one reads the request's `gen_ai.chat` span back through the Sentry API, like the other `*-send-to-sentry` apps, and checks op, origin, status, model, token usage, and that the prompts and answers arrived. The helper test also checks there is exactly one `gen_ai.chat` span, which is what caught the duplicate fixed in #24855. `@anthropic-ai/sdk` is pinned to 0.63.0, the version the integration suite uses, and a `(latest)` variant runs the same tests against `@anthropic-ai/sdk@latest`. Both are optional, like the other send-to-sentry apps. Closes #24748 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Jan Peer Stöcklmair <jan.peer@sentry.io>
Contributor
|
👋 @nicohrubec, @andreiborza — Please review this PR when you get a chance! |
nicohrubec
reviewed
Oct 5, 2026
Contributor
|
👋 @andreiborza — Please review this PR when you get a chance! |
…AI app Review follow-ups. Each test now waits until the request span holds exactly one span with the expected op. A double-instrumented call used to pass, because the lookup took the first match and the parent check only looked for one id. The answer checks match the expected text instead of checking for a string: the SDK records a missing answer as `[""]`, which the typeof check accepted. Also adds a TODO to assert tool calls in gen_ai.output.messages once the integration records them there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replaces the app's own copy of the Sentry API helpers with @sentry-internal/test-utils/cli, the way the cloudflare send-to-sentry apps do it. The shared fetchSpanAttributes passes values through as the API returns them, and the API returns int attributes as strings, so the token count checks convert with Number(). Moving that conversion into fetchSpanAttributes is left for a separate PR, since it changes the helper for every app that uses it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
There are 3 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 74ed239. Configure here.
Every test now polls Sentry twice, once for the model span and once for the request span, and each poll can take up to 180s. The 210s test timeout could cut the second poll short, so raise it to 400s, the same as the Anthropic app. Also only require gen_ai.response.id on chat spans. The SDK records it only when the response has an id, and OpenAI embeddings responses have none. The embeddings test passed only because OpenRouter adds one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An e2e app that uses the OpenAI integration the way a user does and sends the data to a real Sentry project, per #24747.
The app. A plain
Sentry.initon an express app, preloaded withnode --import, no tunnel. The stockopenaiclient talks to OpenRouter. Five routes make one real request each: a chat completion, a streamed chat completion, a forced tool call, a Responses API call, and an embeddings call.The tests. Each one reads the request's
gen_aispan back through the Sentry API, like the other*-send-to-sentryapps, and checks op, origin, status, model, token usage, and that the prompts and answers arrived. The checks use the attribute names ingest stores, which differ from what the SDK sends (sentry.originis stored asorigin,gen_ai.response.textasgen_ai.output.messages). Ingest does not storegen_ai.response.tool_callsunder that name, so the tool-call test checks the finish reason instead.openaiis pinned to 7.5.0, the version the v7 integration suite uses, and a(latest)variant runs the same tests againstopenai@latest. Both are optional, like the other send-to-sentry apps.Closes #24747
🤖 Generated with Claude Code