Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion internal/rules/policies_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1986,6 +1986,21 @@ def fetch_data(x: str) -> dict:
return {}
`,
toolConfig: nil, wantFires: false},
{name: "LC-024 fires on a generic tool name", ruleID: "LC-024", kind: models.KindLangChainTool, src: `
def process(payload: str) -> str:
"""Handle an incoming payload and return the result of handling it."""
return payload
`, wantFires: true},
{name: "LC-024 silent on a verb-object tool name", ruleID: "LC-024", kind: models.KindLangChainTool, src: `
def summarize_invoice(invoice_id: str) -> str:
"""Summarize one invoice by its identifier and return the summary text."""
return invoice_id
`, wantFires: false},
{name: "LC-024 silent on a name merely containing a listed word", ruleID: "LC-024", kind: models.KindLangChainTool, src: `
def process_invoice_batch(batch_id: str) -> str:
"""Process one batch of invoices and return a per-invoice result summary."""
return batch_id
`, wantFires: false},
}

// policyRepoRuleCases covers repo-scoped rules.
Expand Down Expand Up @@ -2246,7 +2261,6 @@ var policyRepoRuleCases = []policyRepoCase{
},
models.RepoInventory{SDKsDetected: []models.SDK{models.SDKOpenAIAgents}},
false},

}

// optionsWithPermissionMode builds a ClaudeAgentOptionsDef whose captured
Expand Down
31 changes: 31 additions & 0 deletions testdata/rules-fixture/langchain/tool_definition.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -77,3 +77,34 @@ rules:
Pass a one-sentence description in the tool() config (or the
DynamicStructuredTool options) naming what the tool does, the inputs it
expects, and what it returns. Write it for the model, not a human reader.

- id: LC-024
title: Ambiguous LangChain tool name
severity: low
confidence: 0.9
language: python
applies_to:
- langchain_tool
scope: tool
match:
name_in:
- process
- handle
- run
- do
- execute
- perform
- work
- go
- thing
- stuff
explanation: >
The tool name is a generic verb that says nothing about what the tool
acts on. The name sits directly beside the description in what the model
sees, so it is half the selection signal, and a name like process or
handle spends that half on nothing — the model either calls the tool for
the wrong job or passes over it entirely. A LangChain agent is routinely handed a dozen or more tools at once, so a name that does not discriminate competes against every neighbor in the list, and each wrong pick spends an iteration against LC-102's max_iterations budget.
fix: >
Rename to a verb-object form that names the thing acted on:
summarize_invoice, refund_charge, fetch_order_status. Keep the
description for the detail and let the name carry the subject.