Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion internal/rules/policies_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1986,6 +1986,21 @@ def fetch_data(x: str) -> dict:
return {}
`,
toolConfig: nil, wantFires: false},
{name: "AG2-019 fires on a generic tool name", ruleID: "AG2-019", kind: models.KindAutoGenTool, src: `
def process(payload: str) -> str:
"""Handle an incoming payload and return the result of handling it."""
return payload
`, wantFires: true},
{name: "AG2-019 silent on a verb-object tool name", ruleID: "AG2-019", kind: models.KindAutoGenTool, src: `
def summarize_invoice(invoice_id: str) -> str:
"""Summarize one invoice by its identifier and return the summary text."""
return invoice_id
`, wantFires: false},
{name: "AG2-019 silent on a name merely containing a listed word", ruleID: "AG2-019", kind: models.KindAutoGenTool, src: `
def process_invoice_batch(batch_id: str) -> str:
"""Process one batch of invoices and return a per-invoice result summary."""
return batch_id
`, wantFires: false},
}

// policyRepoRuleCases covers repo-scoped rules.
Expand Down Expand Up @@ -2246,7 +2261,6 @@ var policyRepoRuleCases = []policyRepoCase{
},
models.RepoInventory{SDKsDetected: []models.SDK{models.SDKOpenAIAgents}},
false},

}

// optionsWithPermissionMode builds a ClaudeAgentOptionsDef whose captured
Expand Down
31 changes: 31 additions & 0 deletions testdata/rules-fixture/autogen/tool_definition.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,3 +54,34 @@ rules:
Annotate every parameter with a concrete type (str, int, list[str], a
typing.Annotated description, etc.). AutoGen propagates these annotations into
the schema the model sees, so the model emits correctly-shaped arguments.

- id: AG2-019
title: Ambiguous AutoGen tool name
severity: low
confidence: 0.9
language: python
applies_to:
- autogen_tool
scope: tool
match:
name_in:
- process
- handle
- run
- do
- execute
- perform
- work
- go
- thing
- stuff
explanation: >
The tool name is a generic verb that says nothing about what the tool
acts on. The name sits directly beside the description in what the model
sees, so it is half the selection signal, and a name like process or
handle spends that half on nothing — the model either calls the tool for
the wrong job or passes over it entirely. Each wrong pick costs a full round of the conversation — propose, execute, reply — so an ambiguous name spends the AG2-004 max_round budget rather than the chat reaching an answer.
fix: >
Rename to a verb-object form that names the thing acted on:
summarize_invoice, refund_charge, fetch_order_status. Keep the
description for the detail and let the name carry the subject.