fix(ai-hook): say when an MCP tool call was not checked against policy - #1391
Open
amascia-gg wants to merge 3 commits into
Open
fix(ai-hook): say when an MCP tool call was not checked against policy#1391amascia-gg wants to merge 3 commits into
amascia-gg wants to merge 3 commits into
Conversation
send_mcp_activity() returned MCPActivityResponse(allowed=True) on every failure path, which is byte-identical to a genuine policy approval. So a timeout, a 500, a malformed response, or a bug in parse_mcp_activity all permitted the MCP tool call with nothing to distinguish them from "the API said yes" -- an unreachable policy endpoint silently allowed everything. It now returns None for "no answer obtained", and the caller fails open *with a warning*, reusing the machinery the secret scan already uses when it cannot scan (HookResult.allow_with_warning -> systemMessage). Behaviour stays fail-open: a network blip must never block a tool call. Three related fixes on the way: - The "not an MCP pre-tool-use" early return stays a genuine, silent allow. Warning there would fire on every Read, Bash and Edit call. - refresh_and_maybe_submit_discovery() moves inside the try block. It walks the filesystem and calls the API, so it fails for the same reasons the request does; outside, its failures bypassed the MCP fail-open and surfaced as the generic "could not scan for secrets" warning, which was the wrong message -- the secret scan itself was fine. - The bare `except Exception` is split: expected RequestException logs at debug, anything else logs at warning, so a bug is findable instead of looking like a network blip. _scan_payloads() only returned a result for a *blocking* payload and otherwise built a fresh HookResult.allow(), which dropped the warning. It now collects non-blocking warnings and attaches them to the final allow. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1391 +/- ##
=======================================
Coverage 94.02% 94.03%
=======================================
Files 199 199
Lines 12286 12304 +18
=======================================
+ Hits 11552 11570 +18
Misses 734 734
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Cut the multi-paragraph narration around send_mcp_activity's fail-open down to the why, and shorten the user-facing warning to two sentences. The "(it was still scanned for secrets)" claim is dropped: the scan is skipped for an MCP call with no arguments, and yields no result when the API answers the scan with an error, so the claim was not always true. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The warning fired on any non-2xx, including the 404 an older self-hosted instance returns for the new /v1/agent-activity/mcp-activity route and the 4xx a workspace without the feature gets. Those users would see it on every MCP tool call, forever, so they would stop reading it. A 4xx is now a silent allow: the instance answered, it just has no policy to offer. Only an unreachable server or a 5xx warns. There is no capability flag in the cached instance metadata to gate on, hence the status split. The warning also survives a blocking secret scan, and no longer names a command to run: it reaches the model too (Cursor's agent_message), which would run it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
send_mcp_activity()returnedMCPActivityResponse(allowed=True, reason="")on every failure path — byte-identical to a genuine policy approval. A timeout, a 500, a malformed response, or a bug inparse_mcp_activityall permitted the MCP tool call, and the caller had no way to tell them from "the API said yes".So an unreachable policy endpoint silently allows every MCP tool call. Same class of invisible fail-open as #1375 (heredoc scans abandoned) and #1378 (keychain write-probe).
What has been done
send_mcp_activity()now returnsOptional[MCPActivityResponse], whereNonemeans "no answer obtained". The caller fails open with a warning, reusing what the secret scan already does when it cannot scan (HookResult.allow_with_warning→systemMessage).Behaviour stays fail-open — a network blip must never block a tool call. The change is visibility, not denial.
Three related fixes on the way through:
refresh_and_maybe_submit_discovery()moves inside the try block. It walks the filesystem and calls the API, so it fails for the same reasons the request does. Outside, its failures bypassed the MCP fail-open and surfaced as the generic "could not scan for secrets" warning — the wrong message, since the scan itself was fine.except Exceptionis split: an expectedRequestExceptionlogs at debug, anything else at warning, so a bug is findable instead of looking like a network blip.One bug found while wiring it up:
_scan_payloads()only returns a result for a blocking payload and otherwise builds a freshHookResult.allow(), which dropped the warning. It now collects non-blocking warnings and attaches them to the final allow.Validation
pytest tests/unit→ 2603 passed, 4 skipped_scan_payloadsblack/isort/flake8cleanNot covered: the Rust ai-hook port (#1381) does not implement MCP policy at all, so this widens the Python/Rust behavioural gap. The equivalence harness has no MCP cases yet — tracked separately.
PR check list
changelog.d/)