Skip to content

test(eve): expect a failed sign-in to stop re-prompting its connector - #4944

Merged
ruiconti merged 2 commits into
mainfrom
repro/4943-sign-in-reprompt-after-failure
Oct 18, 2026
Merged

ruiconti merged 2 commits into
mainfrom
repro/4943-sign-in-reprompt-after-failure

Conversation

@roboeve

@roboeve roboeve Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Repro for #4943: after a sign-in fails with retryable: false, the next tool on the same Vercel Connect connector starts a fresh sign-in in the same turn.

Why

list_issues and create_issue share one connector. The user signs in once; completeAuthorization throws ConnectionAuthorizationFailedError with retryable: false. list_issues fails as expected, but the session forgets the failure, so when the model calls create_issue the turn asks for a second sign-in instead of returning the same failure. docs/connections/overview.mdx says retryable: false stops re-prompts.

Approach

One integration test drives workflowEntry through the real session: sign-in, callback, failed exchange, then the second tool. It asserts no second authorization.required and that the turn ends. It's marked it.fails so CI stays green until the fix lands.

Validation

Without the marker, on main at 6210b1d7a (includes #4762, which reports the failed sign-in as failed):

pnpm exec vitest run --config vitest.integration.config.ts src/execution/session/entry-sign-in-failure.integration.test.ts

AssertionError: expected [ { data: { …(9) }, …(2) } ] to deeply equal []
  # the extra event is authorization.required for create_issue__oauth_issues

With it.fails, the file passes on this branch.

The same test (adapted to each release's test helpers) fails on eve@0.72.0, 0.74.0, 0.76.0, 0.78.0, and 0.80.0. Sharing one sign-in per connector arrived in eve@0.70.0; before that, each tool asked for its own sign-in anyway.

@vercel

vercel Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
eve-docs Ready Ready Preview, v0 Oct 18, 2026 2:58pm UTC
eve-pkg Ready Ready Preview, v0 Oct 18, 2026 2:58pm UTC

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Bundle + Package Summary: apps/fixtures/weather-agent

Key takeaways

  • No notable deltas vs main (6210b1d).

Delta vs main (6210b1d)

Area Metric Baseline Current Delta
Package Packed tarball 7.66 MB 7.66 MB +1.6 kB ⚠️
Package Unpacked publish size 26.97 MB 26.98 MB +6.5 kB ⚠️
Package Installed footprint 74.93 MB 74.94 MB +6.5 kB ⚠️
Package Published files 4171 4171 0
Package Installed files 8150 8150 0
Package Installed package instances 34 34 0
Package Distinct installed package names 32 32 0
Package Installed dependency edges 51 51 0
Package Installed optional peer edges 9 9 0
Runtime Unique function payloads 2 2 0
Runtime Total function bytes 20.77 MB 20.78 MB +7.0 kB ⚠️
Runtime Public routes 18 18 0
Changed function payloads vs main (6210b1d) (2)
Function Status Baseline Current Delta Route changes
functions/__server.func changed 10.38 MB 10.39 MB +3.5 kB ⚠️ none
functions/.well-known/workflow/v1/flow.func changed 10.38 MB 10.39 MB +3.5 kB ⚠️ none

eve init install

Metric Baseline Current Delta
Installed footprint 113.31 MB 113.32 MB +6.5 kB ⚠️
Installed packages 97 97 0
dependencies 4 4 0
devDependencies 2 2 0
Dependency package bytes 45.25 MB 45.26 MB +6.5 kB ⚠️
devDependency package bytes 5.11 MB 5.11 MB 0 B ➖
Build Metadata
  • Preset: vercel
  • Nitro: nitro@3.0.260903-beta
  • Output directory: apps/fixtures/weather-agent/.vercel/output
  • Build metadata timestamp: 2026-10-18T14:59:00.070Z
  • Route aliases: 18 public, 1 internal (19 total aliases)
  • Vercel routes in config: 21
  • Severity legend: 🔴 dominant/large, 🟠 notable, 🟡 watch, ⚪ small
Package Drill-Down

Package Details

  • Package: eve@0.80.1
  • Package directory: packages/eve
  • Tarball: 7.66 MB (eve-0.80.1.tgz)
  • Unpacked payload: 26.98 MB across 4171 published files
  • Installed footprint: 74.94 MB across 8150 installed files
  • Installed root package: 26.98 MB
  • Installed dependencies: 47.96 MB
  • Installed package instances: 34
  • Distinct installed package names: 32
  • Installed dependency edges: 51
  • Installed optional peer edges: 9
  • Runtime dependencies: 2
  • Peer dependencies: 7 (6 optional)

Installed footprint is measured from an isolated temporary npm install of the packed tarball.
Graph metrics read only package.json files in package directories directly beneath a node_modules boundary, including nested boundaries. Each directory is one package instance; distinct names come from those manifests. Dependency edges count each unique name in dependencies or optionalDependencies per instance; optional peer edges count peerDependencies marked optional.

Heavy installed dependencies

  • eve: 26.98 MB (36.0%)
  • @rolldown/binding-linux-x64-gnu: 19.81 MB (26.4%)
  • ai: 8.24 MB (11.0%)
  • zod: 6.15 MB (8.2%)
  • undici: 3.58 MB (4.8%)
Publish payload breakdown
Published file size
🟠 dist/src/compiled/shadcn-registry/index.js       [#####...................] 3.85 MB 14.3%
🟠 dist/src/compiled/@photon-ai/chat-adapter-ime... [###.....................] 2.29 MB 8.5%
🟠 dist/src/compiled/@ai-sdk/code-mode/index.js     [#.......................] 1.03 MB 3.8%
🟡 dist/src/compiled/@vercel/blob/index.js          [#.......................] 604.9 kB 2.2%
🟡 dist/src/compiled/_chunks/workflow/undici-D-I... [#.......................] 521.9 kB 1.9%
🔴 Other published files                            [########################] 18.69 MB 69.3%
Installed footprint breakdown
Installed package size
🔴 eve                             [########################] 26.98 MB 36.0%
🔴 @rolldown/binding-linux-x64-gnu [##################......] 19.81 MB 26.4%
🔴 ai                              [#######.................] 8.24 MB 11.0%
🔴 zod                             [#####...................] 6.15 MB 8.2%
🟠 undici                          [###.....................] 3.58 MB 4.8%
🟠 nitro                           [##......................] 1.89 MB 2.5%
🔴 Other installed packages        [#######.................] 8.28 MB 11.0%
Runtime dependencies (2)
Package Range Notes
nitro 3.0.260903-beta
undici 8.10.2
Peer dependencies (7)
Package Range Notes
@opentelemetry/api ^1.0.0 optional peer
ai catalog:
braintrust ^3.0.0 optional peer
chat ^4.42.0 optional peer
dd-trace ^6.13.0 optional peer
just-bash ^3.1.0 optional peer
microsandbox ^0.5.0 optional peer
eve init install drill-down

eve init install details

  • Command: eve init my-agent
  • Package manager: npm
  • Installed footprint: 113.32 MB across 10067 installed files
  • Installed packages: 97 total (91 transitive-only)
  • dependencies: 4 direct packages totaling 45.26 MB
  • devDependencies: 2 direct packages totaling 5.11 MB
  • Other transitive package files: 62.95 MB

Installed footprint is measured from an isolated temporary eve init my-agent using the current packed eve tarball.

Heavy installed dependencies

  • @typescript/typescript-linux-x64: 27.95 MB (24.7%)
  • eve: 26.98 MB (23.8%)
  • @rolldown/binding-linux-x64-gnu: 19.81 MB (17.5%)
  • zod: 9.76 MB (8.6%)
  • ai: 8.24 MB (7.3%)
Installed footprint breakdown
Installed package size
🔴 @typescript/typescript-linux-x64 [########################] 27.95 MB 24.7%
🔴 eve                              [#######################.] 26.98 MB 23.8%
🔴 @rolldown/binding-linux-x64-gnu  [#################.......] 19.81 MB 17.5%
🔴 zod                              [########................] 9.76 MB 8.6%
🔴 ai                               [#######.................] 8.24 MB 7.3%
🟠 undici                           [###.....................] 3.58 MB 3.2%
🔴 Other installed packages         [###############.........] 17.00 MB 15.0%
dependencies (4)
Package Range Installed size Share
@vercel/connect 2.5.2 276.3 kB 0.2%
ai ^7.0.137 8.24 MB 7.3%
eve file:eve-0.80.1.tgz 26.98 MB 23.8%
zod 4.5.4 9.76 MB 8.6%
devDependencies (2)
Package Range Installed size Share
@types/node 24.x 2.61 MB 2.3%
typescript 7.0.2 2.50 MB 2.2%
Function Drill-Down

Payload Size Graph

Unique function payload size and share of total
🔴 functions/.well-known/workflow/v1/flow.func     [########################] 10.39 MB 50.0%
🔴 functions/__server.func                         [########################] 10.39 MB 50.0%

Top Function Payloads

🟠 functions/.well-known/workflow/v1/flow.func • 1 public route • 10.39 MB
Metric Value
Public routes /.well-known/workflow/v1/flow
Runtime nodejs24.x
Handler index.mjs
Payload 10.39 MB
Function files 10.39 MB across 122 files
Traced dependencies 0 B
Signal 🟠 Bundled file _chunks/vercel.web.mjs is 2.38 MB (22.9%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 _chunks/vercel.web.mjs                         [############............] 2.38 MB 22.9%
🟠 _libs/undici.mjs                               [#####...................] 1.01 MB 9.7%
🟡 _chunks/sandbox3.mjs                           [####....................] 815.4 kB 7.8%
🟡 _chunks/compiled-artifacts-instrumentation.mjs [####....................] 734.8 kB 7.1%
🟡 _chunks/token-util-Bla5Z0G9.mjs                [####....................] 704.7 kB 6.8%
🔴 Other bundled files                            [########################] 4.74 MB 45.7%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x",
  "maxDuration": "max",
  "experimentalTriggers": [
    {
      "type": "queue/v2beta",
      "topic": "__eve776561746865722d6167656e74_wkf_workflow_*",
      "consumer": "default",
      "retryAfterSeconds": 5,
      "initialDelaySeconds": 0
    }
  ],
  "environment": {
    "WORKFLOW_PRECONDITION_GUARD": "1"
  }
}

🟠 functions/__server.func • 17 public routes, 1 internal alias • 10.39 MB
Metric Value
Public routes /
/.well-known/workflow/v1/webhook/[token]
/eve/v1/callback/[token]
/eve/v1/connections/[name]/callback/[attemptId]/[token]
/eve/v1/connections/[name]/callback/[token]
/eve/v1/health
/eve/v1/info
/eve/v1/session
/eve/v1/session/[parentSessionId]/subagents/[callId]/[childSessionId]/stream
/eve/v1/session/[sessionId]
/eve/v1/session/[sessionId]/cancel
/eve/v1/session/[sessionId]/clear
/eve/v1/session/[sessionId]/compact
/eve/v1/session/[sessionId]/reset
/eve/v1/session/[sessionId]/stream
/eve/v1/session/[sessionId]/stubs
/eve/v1/task-input/[token]
Internal aliases /__server
Runtime nodejs24.x
Handler index.mjs
Payload 10.39 MB
Function files 10.39 MB across 122 files
Traced dependencies 0 B
Signal 🟠 Bundled file _chunks/vercel.web.mjs is 2.38 MB (22.9%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 _chunks/vercel.web.mjs                         [############............] 2.38 MB 22.9%
🟠 _libs/undici.mjs                               [#####...................] 1.01 MB 9.7%
🟡 _chunks/sandbox3.mjs                           [####....................] 815.4 kB 7.8%
🟡 _chunks/compiled-artifacts-instrumentation.mjs [####....................] 734.8 kB 7.1%
🟡 _chunks/token-util-Bla5Z0G9.mjs                [####....................] 704.7 kB 6.8%
🔴 Other bundled files                            [########################] 4.74 MB 45.7%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x"
}

Build Timing: e2e/fixtures/agent-tools-sandbox

This is an informational timing measurement inside eve build, from preflight through publication. Output-size measurement and profile writing are excluded.

Build mode: deployable Vercel build with sandbox template prewarm included.

  • Build pipeline: 3.76 s -> 3.61 s (-153.0 ms) vs main (6210b1d).
  • Timing is informational: shared GitHub runners are too variable for a hard timing budget.
Detailed phase timings vs `main (6210b1d)`
Phase Baseline Current Delta
extension.check 17.1 ms 18.1 ms +1.0 ms
project.resolve 6.0 ms 0.6 ms -5.4 ms
workspace.create 1.0 ms 0.6 ms -0.4 ms
host.prepare 884.1 ms 659.6 ms -224.5 ms
vercel.service-prefix.resolve 1.5 ms 1.4 ms -0.1 ms
nitro.create 605.8 ms 612.7 ms +6.9 ms
sandbox.prewarm 434.5 ms 522.3 ms +87.8 ms
nitro.cache.prepare 0.2 ms 0.2 ms 0.0 ms
nitro.prepare 0.7 ms 0.6 ms -0.1 ms
nitro.public-assets 0.8 ms 0.7 ms -0.1 ms
nitro.prerender 0.5 ms 0.4 ms -0.1 ms
nitro.bundle 1.65 s 1.64 s -16.6 ms
nitro.cache.write 0.4 ms 0.3 ms -0.1 ms
vercel.workflow-function.materialize 82.5 ms 77.2 ms -5.3 ms
agent-summary.emit 0.7 ms 0.7 ms 0.0 ms
connect-manifest.emit 0.2 ms 0.3 ms +0.1 ms
nitro.close 0.1 ms 0.1 ms 0.0 ms
output.publish 3.7 ms 3.9 ms +0.2 ms
workspace.remove 2.8 ms 2.7 ms -0.1 ms

@roboeve
roboeve Bot changed the base branch from fix/4760-report-sign-in-after-completion to main October 16, 2026 11:19
@roboeve
roboeve Bot added this pull request to stack #4946 October 16, 2026 11:19
@roboeve
roboeve Bot removed this pull request from stack #4946 October 16, 2026 11:20
@roboeve
roboeve Bot changed the base branch from main to fix/4760-report-sign-in-after-completion October 16, 2026 11:20
@roboeve
roboeve Bot marked this pull request as ready for review October 16, 2026 11:29
@roboeve
roboeve Bot requested a review from a team as a code owner October 16, 2026 11:29
@ruiconti
ruiconti force-pushed the fix/4760-report-sign-in-after-completion branch 3 times, most recently from e22fabc to 0953132 Compare October 17, 2026 03:30
Base automatically changed from fix/4760-report-sign-in-after-completion to main October 17, 2026 03:48
Signed-off-by: Rui Conti <ruiconti@gmail.com>
@roboeve

roboeve Bot commented Oct 18, 2026

Copy link
Copy Markdown
Contributor Author

E2E Tests (Vercel) run 39138212622 is still red after one rerun, but each attempt failed on a different timeout, and none of the failures involve sign-in:

  • Attempt 1: agent-tools-hitl hitl/approval-response-settlement (reload resume did not settle within 20s), and agent-tools-sandbox redeploy (skill missing after the alias moved).
  • Attempt 2: both of those passed (approval-response-settlement passed 111/111 gates). The new failure is hitl/continuation/resolve-only-approval.control, which hit "The operation was aborted due to timeout". The rest of the suite passed: 69/70 evals, 602 gates, including all auth/* and hitl/authorizations/* evals.

This looks like deployed-eval timing flakiness, not a regression from this diff. I haven't rerun it again since it's already been retried once. If you want it green before merge, rerun the failed agent-tools-hitl job.

@ruiconti
ruiconti merged commit c1a6fb1 into main Oct 18, 2026
233 of 238 checks passed
@ruiconti
ruiconti deleted the repro/4943-sign-in-reprompt-after-failure branch October 18, 2026 16:43

This branch was successfully deployed

2 active deployments
Preview – eve-pkg — 535c204e Deployed Oct 18, 2026 by vercel[bot]
Preview – eve-docs — 535c204e Deployed Oct 18, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant