Skip to content

feat(vercel_ai): add VAI-015, tool writes to the filesystem - #70

Open
bradAGI wants to merge 1 commit into
trustabl:mainfrom
bradAGI:feat/vercel-ai-path-safety
Open

bradAGI wants to merge 1 commit into
trustabl:mainfrom
bradAGI:feat/vercel-ai-path-safety

Conversation

@bradAGI

@bradAGI bradAGI commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

The Vercel AI pack had no path-safety rule. Claude SDK (CSDK-004/012), OpenAI (OAI-006), ADK (ADK-004), and MCP (MCP-005) all ship one.

Mirrors CSDK-012 — the TypeScript half of the Claude SDK pair — including its coarse-signal caveat, stated in the explanation so the finding is honest about itself: it flags any filesystem write, not only unnormalized paths, because TS path-normalization analysis isn't wired yet. Confidence is 0.5 to match.

What makes it worth flagging in this pack specifically is the deployment shape. Vercel AI tools typically run inside the same server process as the request handler, not in a sandbox, so a model-steered write inherits the application's own filesystem permissions rather than a restricted set — a config file, a build artifact, source in the deployed bundle. Combined with the prompt-injection path (retrieved content or an earlier tool result carrying the filename), the model chooses both the path and the contents.

The fix names the stronger remedy for the common case: derive the filename server-side from an id rather than accepting a path from the model at all.

Verification — engine built at main:

$ trustabl rules validate .
OK: 86 rule pack(s), 207 rule(s) valid under rule schema version 14

Fire (writeFileSync(notePath, body) with notePath from the input schema): VAI-012, VAI-015
Silent (server-derived id, no filesystem write): VAI-012

(VAI-012 is the pre-existing repo-hygiene rule firing on the fixture having no AGENTS.md.)

No new predicates, so no schema_version bump.

The Vercel AI pack had no path-safety rule; Claude SDK (CSDK-004/012),
OpenAI (OAI-006), ADK (ADK-004), and MCP (MCP-005) all ship one. Mirrors
CSDK-012, the TypeScript half of that pair, including its coarse-signal
caveat: it flags any filesystem write rather than only unnormalized paths,
because TS path-normalization analysis is not yet wired.

The deployment shape is what makes it worth flagging here. Vercel AI
tools typically run inside the same server process as the request
handler rather than in a sandbox, so a model-steered write inherits the
application's own filesystem permissions.
@jhumel-code

Copy link
Copy Markdown
Collaborator

Thanks @bradAGI. The gap is real: claude_sdk, openai_sdk, google_adk and mcp all ship a path-safety rule and vercel_ai did not, and the deployment-shape argument for why it matters in this pack is the useful part of the writeup.

It checks out on my read. All required fields are set, has_write_call already exists at schema 14 so declining to bump schema_version is correct, vercel_ai_tool is a valid applies_to at scope: tool in the loader, language: typescript is stated explicitly, and severity low with confidence 0.5 matches CSDK-012, the rule this mirrors. VAI-015 is also clear of the ID contention around this pack (VAI-013, VAI-014 and VAI-016 are each claimed by other open PRs, 015 is not), so nothing here should need renumbering.

The engine half at trustabl/agent-reliability-analyzer#149 carries a byte-identical fixture copy plus fire and silent cases, so the pair is complete and neither side should land on its own.

One remark on the deployment paragraph: the argument is sound, though it makes the finding read heavier than a low. I would keep the severity for parity with CSDK-012 and leave the text as is.

Before this ships we need a rulebook rationale doc for VAI-015 in trustabl-rulebook. That is on us rather than a request to you, and we can pair on it if you want a hand.

sairenchristianbuerano added a commit to CodingWCal/trustabl-rules that referenced this pull request Sep 16, 2026
VAI-013 was claimed by two independent rules: this one (privileged tool
with no needsApproval) and the description-quality rule in trustabl#69, which
also takes VAI-014 as a contiguous pair. Only one can hold the id.

trustabl#69 was opened first in this repo, which is where rule ids are
allocated, and it claims 013/014 together, so the single rule here moves
instead. VAI-018 is the next free id: 001-012 and 016-017 are on main,
013/014 go to trustabl#69, and 015 is claimed by trustabl#70.

Rule text, severity, confidence and match are unchanged.
ivanpaghubasan pushed a commit to ivanpaghubasan/agent-reliability-analyzer that referenced this pull request Sep 22, 2026
Engine half of a coordinated pair with trustabl/agent-reliability-rules#70, on a
branch of the same name so the rules-sync job resolves the matching pack
rather than main. Neither half should merge alone — check-rules-sync.sh
fails if they do.

Mirrors vercel_ai/path_safety.yaml into testdata/rules-fixture and adds a
fire and a silent case to policyRuleCases, as
TestPolicyRules_AllRulesCovered requires.

The silent case applies the remediation VAI-015's fix text prescribes for
the common shape — derive the name server-side rather than accepting a
path from the model — instead of merely removing the write.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants