Skip to content

A budget pin over every MCP read tool (part of #4198) - #4224

Merged
erikdarlingdata merged 5 commits into
devfrom
fix/4198-mcp-budget-pin
Sep 25, 2026
Merged

erikdarlingdata merged 5 commits into
devfrom
fix/4198-mcp-budget-pin

Conversation

@erikdarlingdata

@erikdarlingdata erikdarlingdata commented Sep 25, 2026 •

Copy link
Copy Markdown
Owner

Part of #4198.

Why

#4198 measured every MCP read tool at default arguments on two busy production stores. It found 12 tools over 50 KB and 3 over 100 KB. The worst, get_query_store_regressions, returned 211 KB. The ruling: at default arguments, every read tool stays under one shared 32 KB budget, McpResponseBudget.DefaultBytes. The per-tool PRs fixed each offender's defaults.

This PR is the test that holds that line. It measures every MCP read tool on a seeded store and fails when a tool goes over the budget without a row on file.

What changes

The diff is one new test file, Darling/Darling.Tests/McpReadToolBudgetLiveTests.cs. It uses the McpResponseBudget.DefaultBytes constant that #4205 added.

  • It finds every [McpServerTool] method in the Darling service assembly by reflection. A new tool is measured with no change to the test.
  • It skips analyze_*, compare_* and audit_config by name. It skips the 16 write tools through an exact list, because no name pattern catches them all. A new write tool must be added to that list, or the test calls it.
  • It fills each tool's parameters the same way: the DI services, server_name set to the seeded server, and every other optional parameter at its default.
  • It seeds one busy server with 12 tenant databases. The seed holds regressed Query Store pairs (about 3 KB of query text each), 30 deadlock graphs and 30 blocked-process reports. It also holds collection-log history for 20 collectors, plan-correction rows and a page of long query completions. 50 more servers, one collection each, give get_fleet_overview a fleet-sized answer.
  • It measures each reply's UTF-8 bytes against the budget. A tool listed in ExemptOffenders must still be over the budget. When a fix brings a listed tool under, the test fails until its row is deleted.

The exempt list

The list began with seven tools. The per-tool PRs fixed five of them: get_query_store_regressions (#4264), get_collection_log (#4265), get_blocking (#4267), describe_custom_view_catalog (#4272) and get_query_store_top (#4273). get_fleet_overview already fit by the time this PR merged dev.

One row is left:

Tool Bytes Why it is still over
get_collection_health 35,145 #4268 compacts only healthy collectors with nothing to report. This fixture's collectors are not healthy, so nothing compacts.

It stays open under #4198. CI run 36142467742 measured the fixed tools: describe_custom_view_catalog 30,215 bytes, get_query_store_clutter 26,353, get_query_store_regressions 24,510, get_collection_log 21,929, get_query_store_top 16,911 and get_fleet_overview 2,087.

Known gaps

Some tools pass without a real test:

  • A required parameter with no usable default. The test passes a placeholder, so the tool answers empty or invalid. These are get_custom_alert_rule, get_custom_view, get_perfmon_trend, get_plan_xml, get_query_trend, get_wait_trend, run_custom_view_panel, validate_custom_alert_rule and validate_custom_view.
  • Seeded, but not enough to reach the budget. get_query_heatmap (395 bytes) needs many time buckets, and the fixture has two. get_deadlock_detail measured 19,261 bytes and get_plan_corrections 23,125.
  • Not seeded. get_analysis_findings, get_index_usage, get_object_locking, get_active_queries, and every PostgreSQL-target read (get_pg_*).

Left for #4198

  • get_collection_health on a server whose collectors are not healthy (the row above).
  • The Lite version of this test: the same check over Lite's MCP tools against the shared DuckDB fixture. This PR does not include it.

Test plan

CHANGELOG entry

None. This PR changes tests only.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FVjn4PBJN71NQXdFo6ZxNQ

McpReadToolBudgetLiveTests reflects over every [McpServerTool] method
on every [McpServerToolType] class in the Darling service assembly,
excludes write tools/analyze_*/compare_*/audit_config, binds each
tool's DI services and server_name generically, and asserts the reply
stays under McpResponseBudget.DefaultBytes unless the tool is named
in an explicit, byte-stamped exemption roster. A fix lands by
deleting its row.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TszxYhJJbTEh4LrZ56NYo3
@erikdarlingdata
erikdarlingdata changed the base branch from main to dev September 25, 2026 03:59
@erikdarlingdata
erikdarlingdata marked this pull request as ready for review September 25, 2026 05:23
@erikdarlingdata
erikdarlingdata enabled auto-merge (squash) September 25, 2026 05:23
#4205 merged first and owns McpResponseBudget.cs. Its version adds
CollectionLogFleetDefaultLimit alongside DefaultBytes. This PR's copy is
dropped in favor of that version, as the PR body noted would be required.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FVjn4PBJN71NQXdFo6ZxNQ
erikdarlingdata and others added 3 commits September 25, 2026 09:38
…as merged

After merging dev, the per-tool fixes for get_blocking (#4267),
get_collection_log (#4265), get_query_store_regressions (#4264),
get_collection_health (#4268), describe_custom_view_catalog (#4272) and
get_query_store_top (#4273) are all in, and get_fleet_overview already fit
(CI's stale-exemption message). Every row goes; CI's live run decides.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FVjn4PBJN71NQXdFo6ZxNQ
CI on 08e6e81 measured get_collection_health at 35,145 B, over the 32 KB
default. #4268 compacts only healthy collectors with nothing to report, and
this fixture's collectors are not healthy, so nothing compacts. The row goes
back, under #4198, which stays open for it. Every other exempt tool fits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FVjn4PBJN71NQXdFo6ZxNQ
@erikdarlingdata erikdarlingdata changed the title DO NOT MERGE (MCP response size): a budget pin over every MCP read tool (part of #4198) A budget pin over every MCP read tool (part of #4198) Sep 25, 2026
@erikdarlingdata
erikdarlingdata merged commit ceaa45f into dev Sep 25, 2026
15 of 16 checks passed
@erikdarlingdata
erikdarlingdata deleted the fix/4198-mcp-budget-pin branch September 25, 2026 14:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant