Conversation
Ports the CSDK-017/018 description-quality pair to MCP. MCP-001 only checks that a docstring exists, so a tool whose docstring reads "TODO" or "Gets data." passes today while giving connecting clients no selection signal. A placeholder costs more across a protocol boundary than inside an in-process SDK. The consumer has no other context to fall back on — it cannot read the source, the neighboring tools, or the project's docs the way a developer in the repo can. The server's authors also never see the consequence, since mis-selection surfaces in someone else's client session as a wrong answer rather than as an error on this side. And a client typically connects several servers at once, so the model chooses across all of them from these strings alone and a thin description competes badly against a well-described tool from an unrelated server.
|
Thanks @bradAGI. The port is faithful to CSDK-017/018 (same severity, same confidence, same predicates) and, more importantly, the explanation and fix text is genuinely rewritten for the protocol boundary instead of copied across with the wrong framing. That is exactly what I want in an SDK-specific pack. I read the fixture half in trustabl/agent-reliability-analyzer#165 and it is byte identical to this file, so One note rather than a change request: If you want a follow-up afterwards, the TypeScript side (the MCP-011 neighbourhood) has no placeholder or length rule yet. |
Engine half of a coordinated pair with trustabl/agent-reliability-rules#88, on a branch of the same name so the rules-sync job resolves the matching pack rather than main. Neither half should merge alone — check-rules-sync.sh fails if they do. Mirrors mcp/tool_definition.yaml into testdata/rules-fixture and adds cases to policyRuleCases, as TestPolicyRules_AllRulesCovered requires. Five cases rather than four. MCP-026 pairs description_length_lt with has_docstring so an absent docstring stays MCP-001's finding instead of double-reporting — an empty description is length 0, which is also under the threshold — and the fifth case is what pins that guard.
This branch has not been deployed
2 participants
Ports the CSDK-017/018 description-quality pair (merged in #40) to MCP. MCP-001 only checks that a docstring exists, so a tool whose docstring reads
"""TODO: describe this tool."""or"""Gets data."""passes today while giving connecting clients no selection signal.A placeholder costs more across a protocol boundary than inside an in-process SDK, for three reasons the rule text makes explicit:
tools/liststring is genuinely all it gets.That third point drives MCP-026's fix: name the domain the tool acts on rather than assuming the server name conveys it, because the model sees the tool alongside every other connected server's.
Verification — engine built at
main:Fire (one tool with
"""TODO: describe this tool.""", one with"""Gets data."""):MCP-025, MCP-026Silent (full description naming when to prefer the neighboring tool): no findings
MCP-026 pairs
description_length_lt: 40withhas_docstring: trueso it doesn't double-report against MCP-001 on an empty docstring, same as CSDK-018.No new predicates, so no
schema_versionbump.