[prha]

Summary

Do not merge. This draft runs the nightly benchmark matrix from #4470 with one attempt instead of three. It checks run time, coverage, gateway rate limits, and cost before merge. The branch is #4470's head plus one empty commit, so it tests the same code. Results publish under preview/runs/ in the benchmark Blob store.

The matrix is 10 models × 3 fixtures × 1 attempt (30 live legs) plus 6 mock legs. The nightly run uses the same matrix with 3 attempts, on main only. The slots are high, mid, and low tiers per major lab, plus three open-weight models. It uses the registry concurrency: 8 live legs at a time, with one eval per leg. The estimated run time is about 25 minutes. Benchmark runs share one concurrency group, so this run queues behind any other benchmark run.

Validation

The run-eval-benchmark-full label starts the run. Results will be added here.

Checklist

• This change was requested or approved by a maintainer
• I ran the relevant checks from CONTRIBUTING.md
• I added tests and documentation where relevant
• I added a changeset if this touches the published eve package
• DCO sign-off passes for every commit (git commit --signoff)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
eve-docs	Ready	Preview, v0	Oct 8, 2026 5:24am UTC
eve-pkg	Ready	Preview, v0	Oct 8, 2026 5:24am UTC

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	github/​actions/​download-artifact@​3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c

View full report

[github-actions]

Bundle + Package Summary: apps/fixtures/weather-agent

Key takeaways

• No notable deltas vs main (89704cb).

Delta vs main (89704cb)

Area	Metric	Baseline	Current	Delta
Package	Packed tarball	7.49 MB	7.49 MB	-36 B ✅
Package	Unpacked publish size	26.39 MB	26.39 MB	+172 B ⚠️
Package	Installed footprint	74.06 MB	74.06 MB	+172 B ⚠️
Package	Published files	4159	4159	0
Package	Installed files	8160	8160	0
Package	Installed package instances	34	34	0
Package	Distinct installed package names	32	32	0
Package	Installed dependency edges	51	51	0
Package	Installed optional peer edges	9	9	0
Runtime	Unique function payloads	2	2	0
Runtime	Total function bytes	19.92 MB	19.92 MB	-120 B ✅
Runtime	Public routes	17	17	0

Changed function payloads vs main (89704cb) (2)

Function	Status	Baseline	Current	Delta	Route changes
functions/__server.func	changed	9.96 MB	9.96 MB	-60 B ✅	none
functions/.well-known/workflow/v1/flow.func	changed	9.96 MB	9.96 MB	-60 B ✅	none

eve init install

Metric	Baseline	Current	Delta
Installed footprint	112.35 MB	112.35 MB	+172 B ⚠️
Installed packages	103	103	0
dependencies	4	4	0
devDependencies	2	2	0
Dependency package bytes	44.04 MB	44.04 MB	+172 B ⚠️
devDependency package bytes	5.11 MB	5.11 MB	0 B ➖

Build Metadata

• Preset: vercel
• Nitro: nitro@3.0.260903-beta
• Output directory: apps/fixtures/weather-agent/.vercel/output
• Build metadata timestamp: 2026-10-08T05:24:31.653Z
• Route aliases: 17 public, 1 internal (18 total aliases)
• Vercel routes in config: 20
• Severity legend: 🔴 dominant/large, 🟠 notable, 🟡 watch, ⚪ small

Package Drill-Down

Package Details

• Package: eve@0.74.0
• Package directory: packages/eve
• Tarball: 7.49 MB (eve-0.74.0.tgz)
• Unpacked payload: 26.39 MB across 4159 published files
• Installed footprint: 74.06 MB across 8160 installed files
• Installed root package: 26.09 MB
• Installed dependencies: 47.97 MB
• Installed package instances: 34
• Distinct installed package names: 32
• Installed dependency edges: 51
• Installed optional peer edges: 9
• Runtime dependencies: 2
• Peer dependencies: 7 (6 optional)

Installed footprint is measured from an isolated temporary npm install of the packed tarball.
Graph metrics read only package.json files in package directories directly beneath a node_modules boundary, including nested boundaries. Each directory is one package instance; distinct names come from those manifests. Dependency edges count each unique name in dependencies or optionalDependencies per instance; optional peer edges count peerDependencies marked optional.

Heavy installed dependencies

• eve: 26.09 MB (35.2%)
• @rolldown/binding-linux-x64-gnu: 19.62 MB (26.5%)
• ai: 8.01 MB (10.8%)
• zod: 6.14 MB (8.3%)
• undici: 3.56 MB (4.8%)

Publish payload breakdown

Published file size
🟠 dist/src/compiled/shadcn-registry/index.js       [#####...................] 3.85 MB 14.6%
🟠 dist/src/compiled/@photon-ai/chat-adapter-ime... [###.....................] 2.29 MB 8.7%
🟠 dist/src/compiled/@ai-sdk/code-mode/index.js     [#.......................] 1.03 MB 3.9%
🟡 dist/src/compiled/@vercel/blob/index.js          [#.......................] 604.9 kB 2.3%
🟡 dist/src/compiled/_chunks/workflow/signal-exi... [#.......................] 521.9 kB 2.0%
🔴 Other published files                            [########################] 18.10 MB 68.6%

Installed footprint breakdown

Installed package size
🔴 eve                             [########################] 26.09 MB 35.2%
🔴 @rolldown/binding-linux-x64-gnu [##################......] 19.62 MB 26.5%
🔴 ai                              [#######.................] 8.01 MB 10.8%
🔴 zod                             [######..................] 6.14 MB 8.3%
🟠 undici                          [###.....................] 3.56 MB 4.8%
🟠 nitro                           [##......................] 1.89 MB 2.6%
🔴 Other installed packages        [########................] 8.75 MB 11.8%

Runtime dependencies (2)

Package	Range	Notes
nitro	3.0.260903-beta
undici	8.10.2

Peer dependencies (7)

Package	Range	Notes
@opentelemetry/api	^1.0.0	optional peer
ai	catalog:
braintrust	^3.0.0	optional peer
chat	^4.41.0	optional peer
dd-trace	^6.13.0	optional peer
just-bash	^3.1.0	optional peer
microsandbox	^0.5.0	optional peer

eve init install drill-down

eve init install details

• Command: eve init my-agent
• Package manager: npm
• Installed footprint: 112.35 MB across 10055 installed files
• Installed packages: 103 total (97 transitive-only)
• dependencies: 4 direct packages totaling 44.04 MB
• devDependencies: 2 direct packages totaling 5.11 MB
• Other transitive package files: 63.20 MB

Installed footprint is measured from an isolated temporary eve init my-agent using the current packed eve tarball.

Heavy installed dependencies

• @typescript/typescript-linux-x64: 27.95 MB (24.9%)
• eve: 26.09 MB (23.2%)
• @rolldown/binding-linux-x64-gnu: 19.62 MB (17.5%)
• zod: 9.76 MB (8.7%)
• ai: 8.01 MB (7.1%)

Installed footprint breakdown

Installed package size
🔴 @typescript/typescript-linux-x64 [########################] 27.95 MB 24.9%
🔴 eve                              [######################..] 26.09 MB 23.2%
🔴 @rolldown/binding-linux-x64-gnu  [#################.......] 19.62 MB 17.5%
🔴 zod                              [########................] 9.76 MB 8.7%
🔴 ai                               [#######.................] 8.01 MB 7.1%
🟠 undici                           [###.....................] 3.56 MB 3.2%
🔴 Other installed packages         [###############.........] 17.37 MB 15.5%

dependencies (4)

Package	Range	Installed size	Share
@vercel/connect	2.2.0	188.7 kB	0.2%
ai	^7.0.128	8.01 MB	7.1%
eve	file:eve-0.74.0.tgz	26.09 MB	23.2%
zod	4.5.4	9.76 MB	8.7%

devDependencies (2)

Package	Range	Installed size	Share
@types/node	24.x	2.61 MB	2.3%
typescript	7.0.2	2.50 MB	2.2%

Function Drill-Down

Payload Size Graph

Unique function payload size and share of total
🔴 functions/.well-known/workflow/v1/flow.func     [########################] 9.96 MB 50.0%
🔴 functions/__server.func                         [########################] 9.96 MB 50.0%

Top Function Payloads

🟠 functions/.well-known/workflow/v1/flow.func • 1 public route • 9.96 MB

Metric	Value
Public routes	/.well-known/workflow/v1/flow
Runtime	nodejs24.x
Handler	index.mjs
Payload	9.96 MB
Function files	9.96 MB across 127 files
Traced dependencies	0 B
Signal	🟠 Bundled file _chunks/vercel.web.mjs is 2.03 MB (20.4%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 _chunks/vercel.web.mjs                         [###########.............] 2.03 MB 20.4%
🟠 _libs/undici.mjs                               [#####...................] 1.01 MB 10.1%
🟡 _chunks/sandbox3.mjs                           [####....................] 815.4 kB 8.2%
🟡 _chunks/compiled-artifacts-instrumentation.mjs [####....................] 762.7 kB 7.7%
🟡 _chunks/token-util-CddTZ_ef.mjs                [####....................] 704.7 kB 7.1%
🔴 Other bundled files                            [########################] 4.63 MB 46.5%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x",
  "maxDuration": "max",
  "experimentalTriggers": [
    {
      "type": "queue/v2beta",
      "topic": "__eve776561746865722d6167656e74_wkf_workflow_*",
      "consumer": "default",
      "retryAfterSeconds": 5,
      "initialDelaySeconds": 0
    }
  ],
  "environment": {
    "WORKFLOW_PRECONDITION_GUARD": "1"
  }
}

---

🟠 functions/__server.func • 16 public routes, 1 internal alias • 9.96 MB

Metric	Value
Public routes	/ /.well-known/workflow/v1/webhook/[token] /eve/v1/callback/[token] /eve/v1/connections/[name]/callback/[attemptId]/[token] /eve/v1/connections/[name]/callback/[token] /eve/v1/health /eve/v1/info /eve/v1/session /eve/v1/session/[parentSessionId]/subagents/[callId]/[childSessionId]/stream /eve/v1/session/[sessionId] /eve/v1/session/[sessionId]/cancel /eve/v1/session/[sessionId]/clear /eve/v1/session/[sessionId]/compact /eve/v1/session/[sessionId]/reset /eve/v1/session/[sessionId]/stream /eve/v1/task-input/[token]
Internal aliases	/__server
Runtime	nodejs24.x
Handler	index.mjs
Payload	9.96 MB
Function files	9.96 MB across 127 files
Traced dependencies	0 B
Signal	🟠 Bundled file _chunks/vercel.web.mjs is 2.03 MB (20.4%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 _chunks/vercel.web.mjs                         [###########.............] 2.03 MB 20.4%
🟠 _libs/undici.mjs                               [#####...................] 1.01 MB 10.1%
🟡 _chunks/sandbox3.mjs                           [####....................] 815.4 kB 8.2%
🟡 _chunks/compiled-artifacts-instrumentation.mjs [####....................] 762.7 kB 7.7%
🟡 _chunks/token-util-CddTZ_ef.mjs                [####....................] 704.7 kB 7.1%
🔴 Other bundled files                            [########################] 4.63 MB 46.5%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x"
}

Build Timing: e2e/fixtures/agent-tools-sandbox

This is an informational timing measurement inside eve build, from preflight through publication. Output-size measurement and profile writing are excluded.

Build mode: deployable Vercel build with sandbox template prewarm included.

• Build pipeline: 3.44 s -> 3.42 s (-21.7 ms) vs main (89704cb).
• Timing is informational: shared GitHub runners are too variable for a hard timing budget.

Detailed phase timings vs `main (89704cb)`

Phase	Baseline	Current	Delta
extension.check	18.4 ms	18.2 ms	-0.2 ms
project.resolve	0.3 ms	0.3 ms	0.0 ms
workspace.create	0.6 ms	0.5 ms	-0.1 ms
host.prepare	735.1 ms	673.4 ms	-61.7 ms
vercel.service-prefix.resolve	1.4 ms	1.5 ms	+0.1 ms
nitro.create	561.9 ms	566.4 ms	+4.5 ms
sandbox.prewarm	330.1 ms	335.1 ms	+5.0 ms
nitro.cache.prepare	0.2 ms	0.3 ms	+0.1 ms
nitro.prepare	0.7 ms	0.7 ms	0.0 ms
nitro.public-assets	0.8 ms	0.8 ms	0.0 ms
nitro.prerender	0.4 ms	0.5 ms	+0.1 ms
nitro.bundle	1.63 s	1.64 s	+15.1 ms
nitro.cache.write	0.3 ms	0.5 ms	+0.2 ms
vercel.workflow-function.materialize	77.0 ms	90.1 ms	+13.1 ms
agent-summary.emit	0.7 ms	0.7 ms	0.0 ms
connect-manifest.emit	0.3 ms	0.3 ms	0.0 ms
nitro.close	0.1 ms	0.1 ms	0.0 ms
output.publish	6.7 ms	4.0 ms	-2.7 ms
workspace.remove	2.3 ms	2.7 ms	+0.4 ms