Conversation
Engine half of a coordinated pair with trustabl/agent-reliability-rules#70, on a branch of the same name so the rules-sync job resolves the matching pack rather than main. Neither half should merge alone — check-rules-sync.sh fails if they do. Mirrors vercel_ai/path_safety.yaml into testdata/rules-fixture and adds a fire and a silent case to policyRuleCases, as TestPolicyRules_AllRulesCovered requires. The silent case applies the remediation VAI-015's fix text prescribes for the common shape — derive the name server-side rather than accepting a path from the model — instead of merely removing the write.
|
Thanks @bradAGI, this is a clean engine half. The fixture copy of The test cases hold up on a read rather than passing vacuously. One nit, not blocking. The silent case keeps the description "Save a note to disk for later retrieval" while writing to an in-memory VAI-015 needs a rulebook rationale doc before it ships. That part is on us and we are happy to pair on it. I am glad to take this alongside the rules half. |
What the pair adds
The Vercel AI pack had no path-safety rule (CSDK-004/012, OAI-006, ADK-004, MCP-005 all exist). VAI-015 mirrors CSDK-012, including its coarse-signal caveat stated in the explanation: it flags any filesystem write, not only unnormalized paths, because TS path-normalization analysis isn't wired yet. Confidence 0.5 to match.
What makes it worth flagging in this pack is the deployment shape — Vercel AI tools typically run inside the same server process as the request handler rather than a sandbox, so a model-steered write inherits the application's own filesystem permissions.
What this PR does
vercel_ai/path_safety.yamlintotestdata/rules-fixture/.policyRuleCases, asTestPolicyRules_AllRulesCoveredrequires.The silent case applies the remediation the
fixtext prescribes for the common shape — derive the filename server-side rather than accepting a path from the model — instead of merely deleting the write. So it demonstrates the prescribed fix clears the finding, not just that the predicate is satisfiable.Verification