Skip to content

get_collection_health's default fits the MCP response budget (#4198) - #4319

Merged
erikdarlingdata merged 7 commits into
devfrom
fix/4198-collection-health-budget
Sep 25, 2026
Merged

erikdarlingdata merged 7 commits into
devfrom
fix/4198-collection-health-budget

Conversation

@erikdarlingdata

@erikdarlingdata erikdarlingdata commented Sep 25, 2026 •

Copy link
Copy Markdown
Owner

Closes #4198.

Why

get_collection_health was the last tool on both SKUs' ExemptOffenders lists (#4198's per-tool budget
backlog). #4268 already compacted HEALTHY collectors with nothing to report to a 7-field shape, but a row that
needs a look (failing, stale, stopped, erroring, denied, regressed, or an unexplained zero) still kept the full
~30-field shape by default, and on both fixtures that alone was over McpResponseBudget.DefaultBytes (32,768 B):
Darling 35,145 B (20 rows), Lite 43,115 B (42 rows). Lane M1 measured this in detail and designed the fix
(issue comments) but ran out of context before writing code; this lane implements it.

Lane M4b follow-up (2026-09-25): the leaner shape this PR shipped named errors and zero-rows as reasons a
row needs a look, but not the other three the predicate itself checks: session-missing runs, abandoned cycles,
or a permission denial cleared by a later success. A HEALTHY row whose only issue was one of those three carried
partial_detail: true with nothing in the shape saying why. Fixed below; see "What changes" and "Measured
bytes" for the updated field list and numbers.

What changes

A row that fails IsCollectionHealthCompactEligible now gets a new leaner shape instead of the full one,
on both SKUs, field-for-field identical:

collector, status, partial_detail, total_runs, errors, session_missing, abandoned, rows_stored, last_success, last_error, last_error_truncated, last_error_at, last_denied_at, denied_since_last_success, regressed_from_productive, output_finding, output_finding_truncated

  • partial_detail: true is the row marker — deliberately not compact, whose meaning is "healthy, nothing
    to report." CollectionHealthPayloadBudgetLiveTests (Darling) and CollectionHealthPayloadBudgetToolTests
    (Lite) assert a row that needs a look never carries compact, so a caller scanning for compact != true
    still never skips it.
  • last_error previews to the existing ErrorMessagePreviewLength (500 chars, shared with get_collection_log);
    output_finding (a template sentence from FormatOutputFinding, not raw free text, and the single dominant
    cost on Darling's fixture where every synthetic row hit the "needs a look" branch) gets its own shorter
    OutputFindingPreviewLength (200 chars). Both _truncated flags are always emitted (true/false), matching
    get_collection_log's error_message_truncated — that flag is not conditionally omitted, so lever (a) in the
    brief (omit the flag unless true) does not apply; McpHelpers.JsonOptions writes nulls and the row is an
    anonymous type, so a conditionally-omitted property isn't structurally available here anyway.
  • session_missing, abandoned and last_denied_at are now in the leaner shape (lane M4b, MCP read tools have no default response-size budget: at default arguments 12 tools return >50 KB and 3 return >100 KB for one server, more than an agent client's per-result cap #4198
    follow-up), same names and formats FullCollectionHealthRow uses, always emitted rather than conditional on
    being the reason a given row is partial. ExtensionMissingCount is also in
    IsCollectionHealthCompactEligible's predicate, but it surfaces under no field name anywhere in the full
    ~30-field row either — there is nothing for the leaner shape to mirror — so nothing was added for it.
    Remeasured after adding the three fields: still comfortably under budget on all four fixtures (numbers below).
  • full_detail=true is unchanged: every field on every row, full stop.
  • collector_detail_note now names both tiers (how many rows compacted, how many took the leaner shape) and the
    one restore sentence (full_detail=true) that applies to both.
  • Both [Description] texts' full_detail (#4198) paragraph is corrected — the old sentence claiming every
    other row "always carries every field" is no longer true — and kept byte-identical between the two SKUs,
    confirmed by CrossSkuSurfaceSourceTests.BothSkusToolDescriptions_StayByteIdentical (7/7 pass, including after
    the M4b field-list edit). The edit lives entirely after the <<GUIDE>> marker (the tail get_tool_guide
    serves separately), so McpToolsListBudgetTests' TotalCeilingBytes on both SKUs measured unchanged
    (92,222 Lite, 174,554 Darling) — ran both, no constant edit needed.
  • McpPayloadContractCensusTests (Darling) does not need new entries for the three added fields: its
    SecondBoundCutKeys / CutNoteKeys / SourceSideCutKeys / FieldPreviewCutKeys arrays census
    truncation/preview keys specifically (the *_truncated flags and their preview lengths), and
    session_missing/abandoned/last_denied_at are plain mirrored counts and timestamps, not previews — none
    of the four arrays applies to them. Confirmed by running the class: still green, no new failures asking for an
    entry.

Correction to M1's design notes: Lite already had a dedicated payload-budget test,
CollectionHealthPayloadBudgetToolTests.cs (added by #4268, before M1's session), not "no twin of Darling's
payload test" as M1's notes said. I extended the existing test with the 80% ceiling and marker assertions
instead of adding a new file.

Measured bytes

Fixture Before this PR After leaner shape (M1) After M4b's 3 fields Budget 80% ceiling
Darling McpReadToolBudgetLiveTests (mixed SUCCESS/ERROR, 20 collectors, ExemptOffenders) 35,145 B 21,027 B 22,147 B 32,768 B 26,214 B
Lite McpReadToolBudgetTests (mixed SUCCESS/ERROR, 42 collectors, ExemptOffenders) 43,115 B 28,017 B 30,369 B 32,768 B 26,214 B
Darling CollectionHealthPayloadBudgetLiveTests (realistic: mostly healthy + 6 pinned non-boring rows) n/a (new assert) 18,418 B 19,134 B 32,768 B 26,214 B
Lite CollectionHealthPayloadBudgetToolTests (same shape) n/a (new assert) 18,366 B 19,082 B 32,768 B 26,214 B

Both ExemptOffenders rows are still gone; both dictionaries are still empty (this PR still closes #4198, not
just advances it — the M4b fields did not put either fixture back over budget).

The realistic dedicated fixtures grew two more pinned rows each (six, not four): plan_cache_stats
(session-missing only) and tempdb_stats (abandoned cycles only, 1 in 250 runs to stay at a 0.4% abandon rate,
under the 0.5% WARNING cutoff so the row still bands HEALTHY). The third case — a permission denial cleared by a
later success — was already covered by the existing query_store_health row; it now also asserts
last_denied_at is non-null. All three new pin assertions were proven to fail against the pre-M4b shape (checked
out the parent commit's two production files, rebuilt, confirmed [FAIL] on both SKUs' dedicated tests, then
restored and rebuilt back to green) before being left in place.

One open point for the coordinator, updated: the ruling's item 6 says a default call must land at 80% of
budget "to leave room for a larger fleet." Both dedicated tests still clear the 26,214 B ceiling comfortably
(~27% headroom, down slightly from ~30%). Darling's ExemptOffenders fixture also still clears it (22,147 B).
Lite's ExemptOffenders fixture does not (30,369 B) and its margin under the hard 32,768 B budget has
thinned from 14% to 7.3% (2,399 B) after the M4b fields — worth watching if a future field gets added to
this shape, since that fixture is already the tightest of the four. Per the brief, I did not cut fields to buy
margin back; flagging the number for the coordinator instead.

Test plan

  • Lite build: dotnet build Lite/PerformanceMonitorLite.csproj — 0 warnings, 0 errors.
  • Darling service build: dotnet build Darling/PerformanceMonitor.Darling.Service/...csproj — 0 warnings, 0 errors.
  • Lite.Tests.CollectionHealthPayloadBudgetToolTests, Lite.Tests.McpReadToolBudgetTests — pass, bytes above.
  • Darling.Tests.CollectionHealthPayloadBudgetLiveTests, Darling.Tests.McpReadToolBudgetLiveTests (rig, port 55975) — pass, bytes above.
  • Lite.Tests.CrossSkuSurfaceSourceTests (description byte-identical pin) — 7/7 pass.
  • Lite.Tests.McpToolsListBudgetTests, Darling.Tests.McpToolsListBudgetTests — pass, totals unchanged.
  • Full Lite suite (Lite.Tests.exe, no filter): 5,389 total, 0 failed.
  • Full Darling suite (Darling.Tests.exe, no filter, fresh darlingtest, rig in UTC): first run found 2
    failures — DocCommentHygieneTests.NoMemberCarriesTwoStackedSummaryBlocks (an ExemptOffenders edit left
    the old <summary> block stacked above the new one instead of replacing it) and
    McpPayloadContractCensusTests.EveryCutKey_IsThePageDialect_OrClassified_OnBothSkus (the two new
    *_truncated keys needed a FieldPreviewCutKeys census entry). Both fixed, reconfirmed green (145/145)
    plus the collection-health-specific classes re-run clean. Did not re-run the full 14-minute suite a third
    time given context budget.
  • Lane M4b (this update), after git merge origin/dev: rebuilt both test projects (0 warnings, 0
    errors) and ran, by class name: Darling.Tests.CollectionHealthPayloadBudgetLiveTests,
    Darling.Tests.McpPayloadContractCensusTests, Darling.Tests.CrossSkuSurfaceSourceTests,
    Darling.Tests.McpToolsListBudgetTests, Darling.Tests.McpReadToolBudgetLiveTests (76/76 in one run,
    rig port 55975), then Darling.Tests.CollectionHealthAggregateTests +
    Darling.Tests.DocCommentHygieneTests (90/90) — this last pair is the subset most likely to catch a
    problem from this lane's own new doc comments and test edits. On Lite:
    PerformanceMonitorLite.Tests.CollectionHealthLatestNoteTests, CollectionHealthPayloadBudgetToolTests,
    CollectionHealthWindowTests, CrossSkuSurfaceSourceTests, McpReadToolBudgetTests,
    McpToolsListBudgetTests (12/12 in one run). All pass; measured bytes above.
  • Manual revert-and-rerun (lane M4b): checked out the parent commit's two production files over the working
    tree, rebuilt, confirmed the three new pin assertions [FAIL] on both SKUs' dedicated budget tests,
    then restored and rebuilt back to green — proves the pins actually test the new fields rather than
    passing vacuously.
  • Full Lite suite / full Darling suite: not re-run by lane M4b (brief: "No full suite: CI runs it"). This
    lane's changes are three new object-initializer fields, guide-text edits, and additive test assertions in
    the same classes already exercised above; no other subsystem touched.
  • Rig stopped and C:\GitHub\worktrees\rig-m4b deleted after lane M4b's runs.

CHANGELOG entry

SECTION: Fixed
ENTRY:

🤖 Generated with Claude Code

https://claude.ai/code/session_01FVjn4PBJN71NQXdFo6ZxNQ

erikdarlingdata and others added 6 commits September 25, 2026 11:56
…4198)

A row that fails IsCollectionHealthCompactEligible used to keep the full
~30-field shape regardless of full_detail, and measured on both #4198
fixtures that alone could not clear the response budget (30+ small fields
times every row that needs a look, which previewing free text cannot
touch). That row now gets a new leaner partial_detail shape: enough to say
what is wrong and since when, never reusing the compact marker so a caller
scanning for compact != true still never skips it.

CollectionHealthPayloadBudgetToolTests (already added by #4268, not new as
#4198's design notes assumed) gets the ruling's 80% ceiling and the new
marker assertions. McpReadToolBudgetTests' ExemptOffenders drops
get_collection_health, its last row.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TszxYhJJbTEh4LrZ56NYo3
…#4198)

Field-for-field mirror of Lite's fix: a row that fails
IsCollectionHealthCompactEligible now gets a new leaner partial_detail
shape instead of the full ~30-field fallback, with its own marker so a
caller scanning for compact != true still never skips a row that needs a
look. CollectionHealthPayloadBudgetLiveTests gets the ruling's 80% ceiling
and the new marker assertions; McpReadToolBudgetLiveTests' ExemptOffenders
drops get_collection_health, its last row. Live-test numbers to follow
once the rig confirms them.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TszxYhJJbTEh4LrZ56NYo3
DocCommentHygieneTests: my ExemptOffenders edit left the original
<summary> block stacked above the replacement one instead of overwriting
it. McpPayloadContractCensusTests: last_error_truncated and
output_finding_truncated are new *_truncated keys get_collection_health's
partial_detail shape introduces; classified as FieldPreviewCutKeys beside
error_message_truncated, the existing entry for the same two files.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TszxYhJJbTEh4LrZ56NYo3
…l shape (#4198)

PartialCollectionHealthRow dropped session_missing, abandoned and
last_denied_at, so a row failing IsCollectionHealthCompactEligible only on
one of those three carried partial_detail: true with nothing in the shape
saying why. Add all three, same names/formats as the full row, on both
SKUs, and update both get_collection_health guide texts' field list to
match (still byte-identical, CrossSkuSurfaceSourceTests). ExtensionMissingCount
is in the predicate but surfaces nowhere in the full row, so nothing was
added for it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TszxYhJJbTEh4LrZ56NYo3
@erikdarlingdata
erikdarlingdata marked this pull request as ready for review September 25, 2026 16:57
@erikdarlingdata
erikdarlingdata enabled auto-merge (squash) September 25, 2026 16:57
Picks up #4323's wall-clock fix for the PgTarget anomaly test that failed this PR's CI three times.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FVjn4PBJN71NQXdFo6ZxNQ
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant