Conversation
Engine half of a coordinated pair with trustabl/agent-reliability-rules#71, on a branch of the same name so the rules-sync job resolves the matching pack rather than main. Neither half should merge alone — check-rules-sync.sh fails if they do. Mirrors pydantic_ai/observability.yaml into testdata/rules-fixture and adds cases to policyRuleCases, as TestPolicyRules_AllRulesCovered requires. Three cases: the print, the module-logger remediation the fix text prescribes, and a pprint call that must stay silent. The third pins has_print_call's bare-callee behavior, so the rule cannot regress into substring matching that sweeps in pprint and every other callee whose name contains "print".
|
Thanks, @bradAGI. Pairing the fixture mirror with the tests in one PR, and saying plainly that neither half merges alone, is exactly the right shape. The fixture copy is byte identical to the file in trustabl/agent-reliability-rules#71, so I walked the three cases against The line removed from Only nit, and it is not blocking: the new cases land at the end of
|
What the pair adds
Pydantic AI had no observability rule; OpenAI ships OAI-010 and ADK ships ADK-009 for the same
print()pattern. The framing is Pydantic AI's own rather than a port: Pydantic AI emits OpenTelemetry spans for each run and tool call, so everything around this print is already correlated to a trace — the bare print is the one diagnostic landing outside it, unattached to the run that produced it.What this PR does
observability.yamlintotestdata/rules-fixture/.policyRuleCases, asTestPolicyRules_AllRulesCoveredrequires.print("looking up " + order_id)logger.info("looking up %s", order_id)pprint({...})Three cases rather than two. The
pprintcase pinshas_print_call's bare-callee behavior — the rule must not regress into substring matching that sweeps inpprintand every other callee whose name merely contains "print". The schema calls that out explicitly as the traphas_body_textfalls into, so it's worth a standing test rather than a comment.The silent case applies the remediation the
fixtext prescribes (a module logger) rather than deleting the call.Verification