Conversation
Ports the ambiguous-name check to pydantic_ai. Claude SDK (CSDK-007), OpenAI (OAI-007), ADK (ADK-007), and MCP (MCP-003) all ship it; the five newer packs had none. The tool name sits directly beside the description in what the model sees, so it is half the selection signal, and a generic verb like process or handle spends that half on nothing.
|
Thanks @bradAGI, this is a clean port. Checking One question before this goes in. Discovery honors an explicit Small nit, the tail of the |
|
LGTM. This is helpful. Please fix the merge conflicts even in other PR. Thank you very much. |
Engine half of a coordinated pair with trustabl/agent-reliability-rules#107, on a branch of the same name so the rules-sync job resolves the matching pack rather than main. Neither half should merge alone — check-rules-sync.sh fails if they do. Mirrors pydantic_ai/tool_definition.yaml into testdata/rules-fixture and adds cases to policyRuleCases, as TestPolicyRules_AllRulesCovered requires. Three cases: the generic name, the verb-object remediation, and process_invoice_batch — a name that merely contains a listed word and must stay silent. The third pins that name_in matches the whole name rather than a substring, which is the plausible regression and the one that would otherwise flood well-named tools with findings.
Ports the ambiguous-name check to Pydantic AI. Claude SDK (CSDK-007), OpenAI (OAI-007), ADK (ADK-007), and MCP (MCP-003) all ship it; the newer packs had none.
The tool name sits directly beside the description in what the model sees, so it is half the selection signal — and a generic verb like
processorhandlespends that half on nothing. Pydantic AI derives the tool name from the function name unless one is passed explicitly, so a generically named helper function silently becomes a generically named tool — the defect arrives without anyone deciding to name a toolprocess.Same
name_inlist as CSDK-007, so behavior stays consistent across packs.Verification — engine built at
main:Fire (tool named
process): includesPYD-013Silent (same tool renamed
summarize_invoice):PYD-013absentA note on scope. I drafted a fifth rule in this batch for the Vercel AI SDK and dropped it after testing:
ToolDef.Nameis deliberately empty forvercel_ai_tool, because the SDK derives a tool's name from the key in thetools: { ... }record rather than from thetool({...})call. The engine asserts this directly —so every name-based predicate (
name_in,name_has_prefix) is structurally inert for that pack — it also blocks a Vercel idempotency rule, which is why the pack has none. Back-fillingNamefrom the record key at discovery time would unblock both. Happy to open that engine PR if it'd be useful.Paired engine PR follows, mirroring the rule into
testdata/rules-fixture/with fire/silent cases.