Conversation
Ports OAI-010 to LangChain. The loss is larger here than in the OpenAI pack, and the rule text says so: LangChain already threads every tool start, end, and error through its callback system into LangSmith or whichever tracer is configured, so a bare print is the one diagnostic landing outside that — detached from the run and the step that produced it, and simply absent the moment anyone debugs from a trace rather than a terminal.
|
Thanks @bradAGI, this is a clean rule and a well-argued one. Checking it against The explanation is genuinely reframed around the callback system and the tracer rather than a straight port of the OpenAI text, and it names a concrete consequence, which is what I want from this pack. This pairs with trustabl/agent-reliability-analyzer#151, which mirrors the file byte for byte and adds the coverage cases, so the two repo contract is satisfied. The remaining piece is a rulebook rationale doc for |
Engine half of a coordinated pair with trustabl/agent-reliability-rules#72, on a branch of the same name so the rules-sync job resolves the matching pack rather than main. Neither half should merge alone — check-rules-sync.sh fails if they do. Mirrors langchain/observability.yaml into testdata/rules-fixture and adds cases to policyRuleCases, as TestPolicyRules_AllRulesCovered requires. Three cases: the print, the module-logger remediation the fix text prescribes, and a pprint call that must stay silent. The third pins has_print_call's bare-callee behavior, so the rule cannot regress into substring matching that sweeps in pprint and every other callee whose name contains "print".
Ports OAI-010 to LangChain — the observability dimension isn't covered in any of the five newer packs.
The loss is larger here than in the OpenAI pack, which is what the rule text leads with. LangChain already threads every tool start, end, and error through its callback system into LangSmith or whichever tracer is configured. So a bare
printis the one diagnostic that lands outside that — detached from the run and the step that produced it, and simply absent the moment anyone debugs from a trace rather than a terminal. The fix therefore names the callback system as well as a module logger.The stdio-transport corruption note from OAI-010 carries over, since a LangChain tool re-served over an MCP stdio transport has the same JSON-RPC interleaving problem.
Verification — engine built at
main:Fire (
print(f"looking up {order_id}")in the tool body):LC-020, LC-201Silent (
logger.info("looking up %s", order_id)):LC-201has_print_callmatches a bareprintcallee, sopprintand other attribute calls don't false-positive.No new predicates, so no
schema_versionbump.