Skip to content

Fix three analyzer findings: Rule 9 unused grants, Rule 33 CE guess labels, Rule 35 exchanges - #608

Merged
erikdarlingdata merged 3 commits into
devfrom
fix/analyzer-grant-ce-labels
Sep 28, 2026
Merged

erikdarlingdata merged 3 commits into
devfrom
fix/analyzer-grant-ce-labels

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

What does this PR do?

This PR fixes three analyzer findings from a review. Each fix is its own commit. I reproduced each finding before I changed the code.

Rule 9: a grant that used nothing never raised the excessive-grant warning (11aa46d)

The excessive-grant check needed MaxUsedMemoryKB > 0. A grant of 1 GB or more that used 0 KB is the worst waste, and the check skipped it.

The parser stored 0 when the MaxUsedMemory attribute was missing. So the model did not separate "used nothing" from "not reported". An estimated plan has no such attribute.

The fix adds MemoryGrantInfo.HasMaxUsedMemory. It is true when the XML has the attribute. Rule 9 now fires when all of these hold:

  • The grant is at least 1 GB.
  • HasMaxUsedMemory is true.
  • The query used 0 KB, or the grant is at least 10 times the use (the old test).

For 0 KB used, the message reads "Granted 10,998 MB but the query used none of it. The unused memory is reserved and unavailable to other queries." It has no ratio and no division. A plan with no MaxUsedMemory attribute does not fire.

Live check: on SQL Server 2025 (build 17.0.4045.5), a sort of dbo.Posts by Body that returned no rows in StackOverflow2013 got GrantedMemory="11261504" and MaxUsedMemory="0". So the engine does write a zero. The analyzer now reports the new message for that plan.

I checked the other code that reads MaxUsedMemoryKB. None of it is the same bug:

  • Nothing else divides by it. Every grant percentage divides by the granted memory and runs only when the grant is above 0. These are in TextFormatter.cs, HtmlExporter.cs, InsightsPanel.razor, AdviceContentBuilder.CardSections.cs, AdviceContentBuilder.WaitStats.cs and PlanViewerControl.RuntimeSummary.cs.
  • AdviceContentBuilder.WaitStats.cs tests MaxUsedKB > 0 before it computes the percentage. The result is 0 either way.
  • OperatorPropertiesPanel.razor (web statement panel) hides the "Max Used" row when it is 0. That panel hides every row that is 0, including Granted, Requested and Desired. It is a display style, not a missed finding.
  • PlanViewerControl.Properties.cs (desktop statement panel) always shows "Max Used". For an estimated plan the row reads 0 KB. I left that alone.
  • The MaxUsedMemoryKB on PlanNode is a different property. It only feeds the operator panels.
  • ShowPlanParser.Warnings.cs copies the value from the plan's own MemoryGrantWarning into a message. It only converts it to MB for that message.
  • BenefitScorer.cs does not use it. It only reads the grant wait time.

Two warnings can now appear for one grant. On the live plan the analyzer printed the plan's own warning ("Excessive Grant: Granted 10,998 MB, Used 0 MB") and the new Rule 9 message. The overlap was already possible for the ratio case. One line is SQL Server's record and the other is the analyzer's, so I left both.

Rule 33: the CE guess labels named the wrong predicates (1964013)

The labels in DetectCeGuess were wrong. I measured each guess again before I changed anything.

The setup:

  • SQL Server 2025 (build 17.0.4045.5). The database has AUTO_CREATE_STATISTICS and AUTO_UPDATE_STATISTICS off.
  • A heap with 100,000 rows. The columns a, b, c and d are integer, and s is varchar(50). None has statistics.
  • The number in each cell is EstimateRows on the scan, read from estimated plans (SET SHOWPLAN_XML ON).
  • CE 70 is USE HINT('FORCE_LEGACY_CARDINALITY_ESTIMATION') at compatibility level 170. CE 120 to 170 use the compatibility level itself. I ran all six levels.

Rows and percent of the table:

Predicate CE 70 CE 120 CE 130 to 170
a = 5 5,623 (5.62%) 316 (0.316%) 316 (0.316%)
a > 5, a < 5, a >= 5 30,000 (30%) 30,000 (30%) 30,000 (30%)
a BETWEEN 5 AND 10 9,000 (9%) 9,000 (9%) 9,000 (9%)
a >= 5 AND a <= 10 9,000 (9%) 9,000 (9%) 9,000 (9%)
s LIKE 'abc%' 9,657 (9.66%) 9,000 (9%) 9,000 (9%)
s LIKE '%abc%' 19,313 (19.3%) 9,000 (9%) 9,000 (9%)
a > 5 AND b > 5 9,000 (9%) 16,432 (16.43%) 16,432 (16.43%)
a BETWEEN @v AND @w (local variables) 9,000 (9%) 16,432 (16.43%) 16,432 (16.43%)
a = b 10,000 (10%) 10,000 (10%) 10,000 (10%)
ABS(a) = 5 5,623 (5.62%) 316 (0.316%) 10,000 (10%)
a = b AND c = d 1,000 (1%) 3,162 (3.16%) 3,162 (3.16%)
a > 5 AND b > 5 AND c > 5 AND d > 5 810 (0.81%) 10,462 (10.46%) 10,462 (10.46%)

Other results from the same runs:

  • a IS NULL on a nullable column gave the same numbers as a = 5 (CE 70 and CE 170 only).
  • a <> 5 gave 51,000 (51%) from CE 120 on and 94,377 under CE 70.
  • UPPER(s) = 'ABC', LEN(s) = 5, a + 1 = 5 and ISNULL(n, 0) = 5 behaved like ABS(a) = 5 (CE 70 and CE 170).
  • a = b AND c = d was run at CE 70, 120, 150 and 170 only.
  • Nothing gave 10% or 1% for an inequality.

I repeated seven of the predicates on a 400,000-row heap to check that the shares hold and that equality follows the row count:

Predicate CE 70 CE 170
a = 5 15,905 (3.98%) 632 (0.158%)
a > 5 120,000 (30%) 120,000 (30%)
a BETWEEN 5 AND 10 36,000 (9%) 36,000 (9%)
s LIKE 'abc%' 38,627 (9.66%) 36,000 (9%)
a > 5 AND b > 5 36,000 (9%) 65,727 (16.43%)
a = b 40,000 (10%) 40,000 (10%)
a = b AND c = d 4,000 (1%) 12,649 (3.16%)

What changed:

  • 30% is now the guess for an inequality. It was labeled equality.
  • 9% names BETWEEN or a two-sided range. From CE 120 it also names LIKE. Under CE 70 it also names two inequalities on different columns.
  • 16.4% is reported only for CE 120 and later. It is two inequalities on different columns, or a range on variables or on an expression. Under CE 70 the same predicates give 9%.
  • 10% stays, with a new label. My measurement found two producers. One is a column compared with another column (every CE). The other is an equality on an expression such as ABS(a) = 5 (from CE 130). It is not an inequality guess.
  • 1% stays for CE 70 only, with a new label. It is two 10% guesses multiplied, for example a = b AND c = d. No inequality predicate gives 1%.
  • The equality guess is now detected. It depends on the row count, not on a fixed share. It is the square root of the row count from CE 120, and the row count to the power 0.75 under CE 70. The match tolerance is 1%. The rule already skips tables under 100,000 rows. At 100,000 rows the equality guess is 0.3% (CE 120 and later) or 5.6% (CE 70). It falls as the table grows, so it stays clear of the fixed bands.
  • The rule now passes the statement's CardinalityEstimationModelVersion to DetectCeGuess. A band that does not exist in that estimator is not reported. A plan with no version can match any band, and its 9% label names only what both estimators agree on.

An equality estimate that comes from real statistics can also land within 1% of the square root of the row count. For example, 1,000 distinct values, evenly spread, in a 1,000,000-row table give exactly 1,000 rows. The warning text already says the optimizer "may be using" a default guess.

Rule 35: the expensive-operator warning named exchanges (d59ac4c)

The claim was that an exchange's own time is mostly waiting. So Rule 35 can name an exchange when the operator next to it is the real cost. The claim is real.

Committed plans already showed it in WarningBaseline.txt:

  • serially-parallel.sqlplan: the Sort has 17,111 ms and the Repartition Streams below it, which feeds the Sort, has the same 17,111 ms. Both are at 100.0% of the statement.
  • memory_grant_wait_plan.sqlplan and spill_plan.sqlplan: a Parallelism operator at 12,362 ms (46.1%).

I also reproduced it on a live actual plan. I ran the ROW_NUMBER() demo query over dbo.Posts joined to dbo.Users in StackOverflow2013 on SQL Server 2025. It ran at DOP 8 in 41,119 ms with SET STATISTICS XML ON. Rule 35 named node 12, a Repartition Streams that sits above the Posts scan and below a Sort that spilled: "Parallelism took 19,333ms (47.0% of statement elapsed)".

Node 12 was waiting, not working. Its slowest worker ran 21,051 ms, but the eight workers used only 19,398 ms of CPU together, about 2.4 s each. The statement had 312 s of CXSYNC_PORT and 210 s of CXPACKET wait across all threads. After the fix, node 12 has no Rule 35 warning and the Sort keeps its spill warning.

Two other live plans did not have an exchange named, before or after. One was an 8-way hash join with an aggregate. The other was a windowed sort over dbo.Comments. The problem needs the right plan shape.

The fix skips exchanges in Rule 35 with NodeTimeAttribution.IsExchangeOperator(node). The text report's "Expensive operators" list already skips exchanges for the same reason. Three Parallelism rows leave WarningBaseline.txt. Nothing else in that file changes.

Which component(s) does this affect?

  • Desktop App (PlanViewer.App)
  • Core Library (PlanViewer.Core)
  • CLI Tool (PlanViewer.Cli)
  • SSMS Extension (PlanViewer.Ssms)
  • Tests
  • Documentation

How was this tested?

For each fix I put the pre-fix source back, ran the test class, and then restored the fix. For Rule 9 I put back only PlanAnalyzer.Statement.cs. The new model property and the parser change do nothing on their own.

Test class Tests With the fix With the fix removed
ExcessiveMemoryGrantTests 9 9 pass 3 fail, 6 pass
CeGuessDetectionTests 45 45 pass 36 fail, 9 pass
ExpensiveOperatorExchangeTests and WarningCharacterizationTests 8 and 1 9 pass 8 fail, 1 pass
  • The 6 Rule 9 tests that pass without the fix are guards. They cover a missing attribute, a grant under 1 GB, the ratio path, two grants that must not fire, and the parser flag. I also broke the fix on purpose by dropping the HasMaxUsedMemory check. Then the missing-attribute test fails (1 of 9).
  • The 1 Rule 35 test that passes without the fix is the control. The same timing on a Sort is still flagged.
  • One of the 8 failures for Rule 35 is WarningCharacterizationTests. It fails because the baseline no longer matches the old behavior.
  • Full suite: dotnet test tests/PlanViewer.Core.Tests -c Debug ran 1,285 tests. 1,255 passed, 0 failed and 30 skipped. The 62 new tests are in that total. The build had no warnings.
  • Test data for Rule 9: the tests edit memory_grant_wait_plan.sqlplan in memory, so the parser and the analyzer are both in the path.
  • Test data for Rule 33: the tests build plans in code, as AnalyzerRuleGateTests does. Four tests parse one inline XML plan shaped like a real one. Every estimate in the tests is a value SQL Server wrote in the measurements above. I added no .sqlplan files, because each file in Plans adds rows to the warning baseline.
  • Test data for Rule 35: the tests use the three committed plans above, plus a control built in code.
  • Plans tested: the committed fixtures, estimated plans for the Rule 33 measurements, and four live actual plans from SQL Server 2025.
  • Platform: Windows only.
  • The scratch database I used for the measurements is dropped.

Checklist

  • I have read the contributing guide
  • My code builds with zero warnings (dotnet build -c Debug)
  • All tests pass (dotnet test)
  • I have not introduced any hardcoded credentials or server names

Not done

  • The other display sites for MaxUsedMemoryKB are unchanged. They are listed under Rule 9.
  • A plan captured while the query still runs might report MaxUsedMemory="0" before the query uses its grant. Rule 9 then reports the grant as unused. I did not test this.
  • The 16.4% band still spans 15.5% to 17.5%. I kept the old width.
  • Two more guesses appeared in the measurements and are not bands: a <> 5 (51%) and an OR of two inequalities (41.4%).
  • Under CE 70, LIKE has no fixed guess (9.66%, 19.3% and 4.83% in my runs), so it gets no label.

🤖 Generated with Claude Code

https://claude.ai/code/session_019n3G844aTidqrD6A6iMgza

erikdarlingdata and others added 3 commits September 28, 2026 18:06
The excessive-grant check required MaxUsedMemoryKB > 0, so the worst case
(MaxUsedMemory="0") never fired. It now fires when the plan reports
MaxUsedMemory="0" and the grant is at least 1 GB, with a message that says
the query used none of the grant (no ratio, no divide by zero).

The parser filled MaxUsedMemoryKB with 0 for a missing attribute, so an
estimated plan or a plan with no runtime grant info looked like "used
nothing". MemoryGrantInfo now has HasMaxUsedMemory, set from the attribute,
and the rule needs it before it treats 0 as real.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019n3G844aTidqrD6A6iMgza
DetectCeGuess had the wrong label on most of its bands. Measured on SQL
Server 2025 (100,000-row heap, no statistics, estimated plans, CE 70 through
FORCE_LEGACY_CARDINALITY_ESTIMATION and CE 120-170 through compat level):

- 30% is the guess for an inequality (a > 5), not for equality.
- 9% is BETWEEN or a two-sided range in every CE, LIKE in CE 120+, and two
  inequalities on different columns in CE 70.
- 16.43% is CE 120+ only: two inequalities on different columns, or a range
  on variables or on an expression. CE 70 gives 9% for the same predicates.
- 10% is one column compared with another (every CE), or an equality on an
  expression such as ABS(a) = 5 (CE 130+). It is not an inequality guess.
- 1% is CE 70 only: two 10% guesses multiplied (a = b AND c = d).
- Equality is not a fixed share: rows^0.5 in CE 120+, rows^0.75 in CE 70.
  It is now detected, to 1%, and the 100,000-row floor the rule already has
  keeps it clear of the fixed bands.

The call site passes the statement's CardinalityEstimationModelVersion, so a
band that does not exist in that estimator (16.43% or 1% in the wrong one) is
no longer reported as a guess.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019n3G844aTidqrD6A6iMgza
An exchange's elapsed time is mostly spent waiting on the operators that
feed it and drain it, so Rule 35 could name a Parallelism operator when the
operator beside it was the real cost. Three committed plans already showed
it (serially-parallel gave the Sort 17,111 ms and the Repartition Streams
below it, which feeds the Sort, the same 17,111 ms), and a live plan from
SQL Server 2025 did too: an exchange feeding a spilling Sort was named at
47% of the statement on about 2.4 s of CPU per thread.

Rule 35 now skips exchanges, as the text report's "Expensive operators" list
already does. The three Parallelism rows come out of the warning baseline;
nothing else in it changes.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019n3G844aTidqrD6A6iMgza
@erikdarlingdata
erikdarlingdata marked this pull request as ready for review September 28, 2026 22:25
@claude

claude Bot commented Sep 28, 2026

Copy link
Copy Markdown

Reviewed: no blocking findings. The HasMaxUsedMemory flag correctly separates a MaxUsedMemory of 0 from an absent attribute. The Rule 33 equality-guess checks run before the fixed bands, so the 100M-row / CE 70 case is labeled correctly. No new Core files need a Web csproj link, and the change bumps no version and adds no TRY_CONVERT or SQL generation. Tests cover the changed behavior. I only read the diff and did not build or run the tests.

Nit: DetectCeGuess is documented as measured on SQL Server 2025 only. A plan with no CE version (0) can still hit the 10% label's 'equality on an expression' wording under CE 120. The label covers that case with 0 or >= 130, so the text may be slightly off for it, but this is minor.

@erikdarlingdata

Copy link
Copy Markdown
Owner Author

On the nit: version 0 means the plan did not say which estimator it used, so it may be CE 130 or later, where an equality on an expression gives 10%. The label names both sources with "or" for that reason. When the plan says CE 120 or CE 70, the label names only the column comparison.

@erikdarlingdata
erikdarlingdata merged commit 85f9aca into dev Sep 28, 2026
4 checks passed
@erikdarlingdata
erikdarlingdata deleted the fix/analyzer-grant-ce-labels branch September 28, 2026 22:29
erikdarlingdata added a commit to erikdarlingdata/PerformanceMonitor that referenced this pull request Sep 29, 2026
…d exchange operators (#4686, #4687, #4690) (#4705)

Ports erikdarlingdata/PerformanceStudio#608 (85f9acab1) to PerformanceMonitor.PlanAnalysis, with the same conditions, thresholds and message text.

- Rule 9, Excessive Memory Grant (#4686): the new MemoryGrantInfo.HasMaxUsedMemory records whether the plan carried a MaxUsedMemory attribute. The rule fires for a grant of at least 1 GB when that attribute is present and either the query used 0 KB ("Granted N MB but the query used none of it.") or the grant is at least 10 times the use (the ratio message, as before). A plan with no MaxUsedMemory attribute does not fire.
- Rule 35, Expensive Operator (#4690): exchange operators (Parallelism, or Gather, Distribute or Repartition Streams) are skipped, because their elapsed time is mostly waiting on the operators around them. The operator that did the work keeps its warning.
- Rule 33, Estimated Plan CE Guess (#4687): DetectCeGuess takes the statement's CE model version and labels each guess for the predicate that produces it. 30% is the inequality guess, and 9%, 10%, 16.4% and 1% follow PerformanceStudio's measured table. It detects the equality guess (the square root of the row count from CE 120, the row count to the power 0.75 under CE 70, within 1%) and reports a band only when the plan's estimator has it. The 100,000-row minimum is the named constant CeGuessMinTableRows.
- Tests: PlanSync4686ExcessiveMemoryGrantTests, PlanSync4687CeGuessDetectionTests and PlanSync4690ExpensiveOperatorExchangeTests port PerformanceStudio's tests, with memory_grant_wait_plan.sqlplan, serially-parallel.sqlplan and spill_plan.sqlplan copied from its test plans. PlanSync4531Tests and PlanSync4535StatementRulesTests set HasMaxUsedMemory on their code-built actual-plan grants.
@erikdarlingdata erikdarlingdata mentioned this pull request Sep 29, 2026
2 of 8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant