Skip to content

ci: Run optional E2E tests in a separate workflow - #25548

Merged
andreiborza merged 6 commits into
developfrom
ab/optional-e2e-workflow
Oct 21, 2026
Merged

andreiborza merged 6 commits into
developfrom
ab/optional-e2e-workflow

Conversation

@andreiborza

Copy link
Copy Markdown
Member

What

Optional E2E tests now run in their own workflow. The build workflow starts it, waits for it to finish, and reports failures in one PR comment.

Why

Failed optional tests made the PR show a red cross, even though the PR could be merged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread .github/workflows/optional-e2e.yml
@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Optional E2E tests

Some optional checks were not successful. They do not block the merge. 1 failing, 63 successful. See the run.

Check Status
❌ CI: Optional E2E Tests / E2E cloudflare-flue-send-to-sentry Test (optional) Failing after 6m Details
Successful checks
Check Status
✅ CI: Optional E2E Tests / E2E cloudflare-pi-durable (latest) Test (optional) Successful in 58s Details
✅ CI: Optional E2E Tests / E2E nextjs-16-standalone (canary, turbopack) Test (optional) Successful in 21s Details
✅ CI: Optional E2E Tests / E2E node-openai-send-to-sentry Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E angular (canary) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E node-flue (latest) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E node-mastra (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E cloudflare-agent Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E cloudflare-mastra Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nuxt-5 Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nextjs-16 (canary, webpack) Test (optional) Successful in 4m Details
✅ CI: Optional E2E Tests / E2E astro-7 Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E node-mastra Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E cloudflare-pi-durable Test (optional) Successful in 33s Details
✅ CI: Optional E2E Tests / E2E node-pi-durable Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E cloudflare-think Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E deno (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E debug-id-sourcemaps Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nextjs-14 (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E tanstackstart-react-cloudflare Test (optional) Successful in 44s Details
✅ CI: Optional E2E Tests / E2E node-eve (latest) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E node-anthropic-send-to-sentry (latest) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E cloudflare-workers-send-to-sentry Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E nuxt-5-sourcemaps Test (optional) Successful in 22s Details
✅ CI: Optional E2E Tests / E2E astro-7-static Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E angular (canary) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E node-fastify-6 Test (optional) Successful in 22s Details
✅ CI: Optional E2E Tests / E2E vue-3 (no Options API) Test (optional) Successful in 53s Details
✅ CI: Optional E2E Tests / E2E nextjs-16-tunnel (cloudflare) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E nuxt-5-sourcemaps (client source maps kept) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E solidstart-2 Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-pi-durable (latest) Test (optional) Successful in 34s Details
✅ CI: Optional E2E Tests / E2E vue-3 (canary) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E gen-ai-libraries Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-anthropic-send-to-sentry Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E nextjs-orpc Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nextjs-14 (canary) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nextjs-15 (latest) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E gen-ai-libraries (cloudflare) Test (optional) Successful in 29s Details
✅ CI: Optional E2E Tests / E2E node-eve (orchestrion) Test (optional) Successful in 39s Details
✅ CI: Optional E2E Tests / E2E nextjs-16 (canary, turbopack) Test (optional) Successful in 4m Details
✅ CI: Optional E2E Tests / E2E supabase-nextjs Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E nuxt-4-static (canary) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E nuxt-3 (canary) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-express-send-to-sentry Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nextjs-16 (vinext, bun) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E nextjs-15 (turbo) Test (optional) Successful in 4m Details
✅ CI: Optional E2E Tests / E2E node-eve Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nuxt-4 (canary) Test (optional) Successful in 3m Details
✅ CI: Optional E2E Tests / E2E node-fastify-6 (tracer provider) Test (optional) Successful in 29s Details
✅ CI: Optional E2E Tests / E2E nextjs-16 (deno) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E node-openai-decisions-send-to-sentry (latest) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E node-openai-send-to-sentry (latest) Test (optional) Successful in 3m Details
✅ CI: Optional E2E Tests / E2E node-express-send-to-sentry-static Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E node-openai-decisions-send-to-sentry Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E react-router-7-framework-instrumentation Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E node-flue Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E nextjs-16 (cloudflare, latest) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E node-express-mcp-v2 Test (optional) Successful in 20s Details
✅ CI: Optional E2E Tests / E2E nextjs-16 (vinext) Test (optional) Successful in 3m Details
✅ CI: Optional E2E Tests / E2E nuxt-5-sourcemaps (static / nuxt generate) Test (optional) Successful in 2m Details
✅ CI: Optional E2E Tests / E2E node-express-mcp-v1 Test (optional) Successful in 15s Details
✅ CI: Optional E2E Tests / E2E cloudflare-think (ai v6) Test (optional) Successful in 1m Details
✅ CI: Optional E2E Tests / E2E react-send-to-sentry Test (optional) Successful in 2m Details

Comment thread .github/workflows/build.yml Outdated
run: ${{ matrix.build-command || 'pnpm test:build' }}

- wait: [install-playwright]
- name: Trigger optional E2E tests

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no strong feeling, but maybe this could be a JS-based in-repo action (like size limit etc)? would maybe be a bit easier to read/adjust in the future?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, moved the dispatch, wait and report logic into an in-repo optional-e2e-gh-action.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

size-limit report 📦

Compared: 76b0c7c (PR test merge using develop at 5e2b904)

Against: 5e2b904 (develop) · base run #62910

✅ Baseline matches: this diff contains only this PR's changes.

Path Size % Change Change
@sentry/browser 29.91 kB - -
@sentry/browser - with treeshaking flags 28.05 kB - -
@sentry/browser - with treeshaking flags tracing without tracing 27.95 kB - -
@sentry/browser (incl. Tracing) 52.1 kB - -
@sentry/browser (incl. Tracing + Span Streaming) 52.1 kB - -
@sentry/browser (incl. Tracing, Profiling) 55.05 kB - -
@sentry/browser (incl. Tracing, Replay) 91.85 kB - -
@sentry/browser (incl. Tracing, Replay) - with treeshaking flags 80.72 kB - -
@sentry/browser (incl. Tracing, Replay with Canvas) 96.58 kB - -
@sentry/browser (incl. Tracing, Replay, Feedback) 109.52 kB - -
@sentry/browser (incl. Feedback) 47.42 kB - -
@sentry/browser (incl. sendFeedback) 34.97 kB - -
@sentry/browser (incl. FeedbackAsync) 40.07 kB - -
@sentry/browser (incl. Metrics) 30.93 kB - -
@sentry/browser (incl. Logs) 31.21 kB - -
@sentry/browser (incl. Metrics & Logs) 31.84 kB - -
@sentry/react 31.73 kB - -
@sentry/react (incl. Tracing) 54.41 kB - -
@sentry/vue 38.07 kB - -
@sentry/vue (incl. Tracing) 55.03 kB - -
@sentry/svelte 29.93 kB - -
@sentry/remix (Remix 3 client bundle) 57.11 kB - -
CDN Bundle 31.67 kB - -
CDN Bundle (incl. Tracing) 52.61 kB - -
CDN Bundle (incl. Logs, Metrics) 33.87 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) 54.55 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) 74.79 kB - -
CDN Bundle (incl. Tracing, Replay) 90.28 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) 92.21 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) 96.42 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) 98.37 kB - -
CDN Bundle - uncompressed 93.29 kB - -
CDN Bundle (incl. Tracing) - uncompressed 156.14 kB - -
CDN Bundle (incl. Logs, Metrics) - uncompressed 99.84 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) - uncompressed 162.11 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) - uncompressed 229.88 kB - -
CDN Bundle (incl. Tracing, Replay) - uncompressed 276.31 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) - uncompressed 282.26 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) - uncompressed 290.01 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) - uncompressed 295.96 kB - -
@sentry/nextjs (client) 57.31 kB - -
@sentry/sveltekit (client) 52.48 kB - -
@sentry/core/server 41.8 kB - -
@sentry/core/browser 13.81 kB - -
@sentry/node 155.32 kB - -
@sentry/node/import (ESM hook with diagnostics-channel injection) 84.06 kB - -
@sentry/node - without tracing 96.95 kB - -
@sentry/node - without channel injection 133.3 kB - -
@sentry/aws-serverless 105.13 kB +0.01% +1 B 🔺
@sentry/cloudflare (withSentry) - minified 211.11 kB - -
@sentry/cloudflare (withSentry) 523.83 kB - -
@sentry/nextjs/cloudflare (withSentry) - minified 230.74 kB - -

View base workflow run

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 1983247. Configure here.

Comment thread dev-packages/optional-e2e-gh-action/index.mjs
Comment thread dev-packages/optional-e2e-gh-action/action.yml
@andreiborza
andreiborza marked this pull request as ready for review October 21, 2026 12:07

# This job only waits for optional-e2e.yml, which can take most of an hour. GitHub runners are free for public
# repos and Namespace runners are paid per minute, so the wait runs here and the tests run on Namespace.
runs-on: ubuntu-24.04

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 good call!

@mydea mydea left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

opened a stacked pr on top of this consolidating the code between e2e-optional e2e tests: #25616 but generally this is very nice, I like this!

@andreiborza
andreiborza merged commit 219312d into develop Oct 21, 2026
691 of 694 checks passed
@andreiborza
andreiborza deleted the ab/optional-e2e-workflow branch October 21, 2026 13:02
andreiborza pushed a commit that referenced this pull request Oct 21, 2026
…25616)

Stacked on #25548.

The optional E2E workflow copied most of \`job_e2e_tests\` from
build.yml, so the two could drift apart. The steps that prepare a test
application (runtimes, artifact downloads including the AWS ones,
prepare/validate, copy to temp, pnpm overrides) now live in
\`.github/actions/prepare-e2e-test\`. The action downloads from the
current run by default; optional-e2e.yml passes the build.yml run ID and
a token instead.

Installing Playwright, building and running the test stay in the
workflows: Playwright installs as a background step, and [a composite
action cannot declare background
steps](https://docs.github.com/actions/using-workflows/workflow-syntax-for-github-actions).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants