Fix three analyzer findings: Rule 9 unused grants, Rule 33 CE guess labels, Rule 35 exchanges - #608
Conversation
The excessive-grant check required MaxUsedMemoryKB > 0, so the worst case (MaxUsedMemory="0") never fired. It now fires when the plan reports MaxUsedMemory="0" and the grant is at least 1 GB, with a message that says the query used none of the grant (no ratio, no divide by zero). The parser filled MaxUsedMemoryKB with 0 for a missing attribute, so an estimated plan or a plan with no runtime grant info looked like "used nothing". MemoryGrantInfo now has HasMaxUsedMemory, set from the attribute, and the rule needs it before it treats 0 as real. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019n3G844aTidqrD6A6iMgza
DetectCeGuess had the wrong label on most of its bands. Measured on SQL Server 2025 (100,000-row heap, no statistics, estimated plans, CE 70 through FORCE_LEGACY_CARDINALITY_ESTIMATION and CE 120-170 through compat level): - 30% is the guess for an inequality (a > 5), not for equality. - 9% is BETWEEN or a two-sided range in every CE, LIKE in CE 120+, and two inequalities on different columns in CE 70. - 16.43% is CE 120+ only: two inequalities on different columns, or a range on variables or on an expression. CE 70 gives 9% for the same predicates. - 10% is one column compared with another (every CE), or an equality on an expression such as ABS(a) = 5 (CE 130+). It is not an inequality guess. - 1% is CE 70 only: two 10% guesses multiplied (a = b AND c = d). - Equality is not a fixed share: rows^0.5 in CE 120+, rows^0.75 in CE 70. It is now detected, to 1%, and the 100,000-row floor the rule already has keeps it clear of the fixed bands. The call site passes the statement's CardinalityEstimationModelVersion, so a band that does not exist in that estimator (16.43% or 1% in the wrong one) is no longer reported as a guess. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019n3G844aTidqrD6A6iMgza
An exchange's elapsed time is mostly spent waiting on the operators that feed it and drain it, so Rule 35 could name a Parallelism operator when the operator beside it was the real cost. Three committed plans already showed it (serially-parallel gave the Sort 17,111 ms and the Repartition Streams below it, which feeds the Sort, the same 17,111 ms), and a live plan from SQL Server 2025 did too: an exchange feeding a spilling Sort was named at 47% of the statement on about 2.4 s of CPU per thread. Rule 35 now skips exchanges, as the text report's "Expensive operators" list already does. The three Parallelism rows come out of the warning baseline; nothing else in it changes. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019n3G844aTidqrD6A6iMgza
|
Reviewed: no blocking findings. The Nit: |
|
On the nit: version 0 means the plan did not say which estimator it used, so it may be CE 130 or later, where an equality on an expression gives 10%. The label names both sources with "or" for that reason. When the plan says CE 120 or CE 70, the label names only the column comparison. |
…d exchange operators (#4686, #4687, #4690) (#4705) Ports erikdarlingdata/PerformanceStudio#608 (85f9acab1) to PerformanceMonitor.PlanAnalysis, with the same conditions, thresholds and message text. - Rule 9, Excessive Memory Grant (#4686): the new MemoryGrantInfo.HasMaxUsedMemory records whether the plan carried a MaxUsedMemory attribute. The rule fires for a grant of at least 1 GB when that attribute is present and either the query used 0 KB ("Granted N MB but the query used none of it.") or the grant is at least 10 times the use (the ratio message, as before). A plan with no MaxUsedMemory attribute does not fire. - Rule 35, Expensive Operator (#4690): exchange operators (Parallelism, or Gather, Distribute or Repartition Streams) are skipped, because their elapsed time is mostly waiting on the operators around them. The operator that did the work keeps its warning. - Rule 33, Estimated Plan CE Guess (#4687): DetectCeGuess takes the statement's CE model version and labels each guess for the predicate that produces it. 30% is the inequality guess, and 9%, 10%, 16.4% and 1% follow PerformanceStudio's measured table. It detects the equality guess (the square root of the row count from CE 120, the row count to the power 0.75 under CE 70, within 1%) and reports a band only when the plan's estimator has it. The 100,000-row minimum is the named constant CeGuessMinTableRows. - Tests: PlanSync4686ExcessiveMemoryGrantTests, PlanSync4687CeGuessDetectionTests and PlanSync4690ExpensiveOperatorExchangeTests port PerformanceStudio's tests, with memory_grant_wait_plan.sqlplan, serially-parallel.sqlplan and spill_plan.sqlplan copied from its test plans. PlanSync4531Tests and PlanSync4535StatementRulesTests set HasMaxUsedMemory on their code-built actual-plan grants.
What does this PR do?
This PR fixes three analyzer findings from a review. Each fix is its own commit. I reproduced each finding before I changed the code.
Rule 9: a grant that used nothing never raised the excessive-grant warning (11aa46d)
The excessive-grant check needed
MaxUsedMemoryKB > 0. A grant of 1 GB or more that used 0 KB is the worst waste, and the check skipped it.The parser stored 0 when the
MaxUsedMemoryattribute was missing. So the model did not separate "used nothing" from "not reported". An estimated plan has no such attribute.The fix adds
MemoryGrantInfo.HasMaxUsedMemory. It is true when the XML has the attribute. Rule 9 now fires when all of these hold:HasMaxUsedMemoryis true.For 0 KB used, the message reads "Granted 10,998 MB but the query used none of it. The unused memory is reserved and unavailable to other queries." It has no ratio and no division. A plan with no
MaxUsedMemoryattribute does not fire.Live check: on SQL Server 2025 (build 17.0.4045.5), a sort of
dbo.PostsbyBodythat returned no rows in StackOverflow2013 gotGrantedMemory="11261504"andMaxUsedMemory="0". So the engine does write a zero. The analyzer now reports the new message for that plan.I checked the other code that reads
MaxUsedMemoryKB. None of it is the same bug:TextFormatter.cs,HtmlExporter.cs,InsightsPanel.razor,AdviceContentBuilder.CardSections.cs,AdviceContentBuilder.WaitStats.csandPlanViewerControl.RuntimeSummary.cs.AdviceContentBuilder.WaitStats.cstestsMaxUsedKB > 0before it computes the percentage. The result is 0 either way.OperatorPropertiesPanel.razor(web statement panel) hides the "Max Used" row when it is 0. That panel hides every row that is 0, including Granted, Requested and Desired. It is a display style, not a missed finding.PlanViewerControl.Properties.cs(desktop statement panel) always shows "Max Used". For an estimated plan the row reads 0 KB. I left that alone.MaxUsedMemoryKBonPlanNodeis a different property. It only feeds the operator panels.ShowPlanParser.Warnings.cscopies the value from the plan's ownMemoryGrantWarninginto a message. It only converts it to MB for that message.BenefitScorer.csdoes not use it. It only reads the grant wait time.Two warnings can now appear for one grant. On the live plan the analyzer printed the plan's own warning ("Excessive Grant: Granted 10,998 MB, Used 0 MB") and the new Rule 9 message. The overlap was already possible for the ratio case. One line is SQL Server's record and the other is the analyzer's, so I left both.
Rule 33: the CE guess labels named the wrong predicates (1964013)
The labels in
DetectCeGuesswere wrong. I measured each guess again before I changed anything.The setup:
AUTO_CREATE_STATISTICSandAUTO_UPDATE_STATISTICSoff.a,b,canddare integer, andsisvarchar(50). None has statistics.EstimateRowson the scan, read from estimated plans (SET SHOWPLAN_XML ON).USE HINT('FORCE_LEGACY_CARDINALITY_ESTIMATION')at compatibility level 170. CE 120 to 170 use the compatibility level itself. I ran all six levels.Rows and percent of the table:
a = 5a > 5,a < 5,a >= 5a BETWEEN 5 AND 10a >= 5 AND a <= 10s LIKE 'abc%'s LIKE '%abc%'a > 5 AND b > 5a BETWEEN @v AND @w(local variables)a = bABS(a) = 5a = b AND c = da > 5 AND b > 5 AND c > 5 AND d > 5Other results from the same runs:
a IS NULLon a nullable column gave the same numbers asa = 5(CE 70 and CE 170 only).a <> 5gave 51,000 (51%) from CE 120 on and 94,377 under CE 70.UPPER(s) = 'ABC',LEN(s) = 5,a + 1 = 5andISNULL(n, 0) = 5behaved likeABS(a) = 5(CE 70 and CE 170).a = b AND c = dwas run at CE 70, 120, 150 and 170 only.I repeated seven of the predicates on a 400,000-row heap to check that the shares hold and that equality follows the row count:
a = 5a > 5a BETWEEN 5 AND 10s LIKE 'abc%'a > 5 AND b > 5a = ba = b AND c = dWhat changed:
ABS(a) = 5(from CE 130). It is not an inequality guess.a = b AND c = d. No inequality predicate gives 1%.CardinalityEstimationModelVersiontoDetectCeGuess. A band that does not exist in that estimator is not reported. A plan with no version can match any band, and its 9% label names only what both estimators agree on.An equality estimate that comes from real statistics can also land within 1% of the square root of the row count. For example, 1,000 distinct values, evenly spread, in a 1,000,000-row table give exactly 1,000 rows. The warning text already says the optimizer "may be using" a default guess.
Rule 35: the expensive-operator warning named exchanges (d59ac4c)
The claim was that an exchange's own time is mostly waiting. So Rule 35 can name an exchange when the operator next to it is the real cost. The claim is real.
Committed plans already showed it in
WarningBaseline.txt:serially-parallel.sqlplan: the Sort has 17,111 ms and the Repartition Streams below it, which feeds the Sort, has the same 17,111 ms. Both are at 100.0% of the statement.memory_grant_wait_plan.sqlplanandspill_plan.sqlplan: a Parallelism operator at 12,362 ms (46.1%).I also reproduced it on a live actual plan. I ran the
ROW_NUMBER()demo query overdbo.Postsjoined todbo.Usersin StackOverflow2013 on SQL Server 2025. It ran at DOP 8 in 41,119 ms withSET STATISTICS XML ON. Rule 35 named node 12, a Repartition Streams that sits above the Posts scan and below a Sort that spilled: "Parallelism took 19,333ms (47.0% of statement elapsed)".Node 12 was waiting, not working. Its slowest worker ran 21,051 ms, but the eight workers used only 19,398 ms of CPU together, about 2.4 s each. The statement had 312 s of
CXSYNC_PORTand 210 s ofCXPACKETwait across all threads. After the fix, node 12 has no Rule 35 warning and the Sort keeps its spill warning.Two other live plans did not have an exchange named, before or after. One was an 8-way hash join with an aggregate. The other was a windowed sort over
dbo.Comments. The problem needs the right plan shape.The fix skips exchanges in Rule 35 with
NodeTimeAttribution.IsExchangeOperator(node). The text report's "Expensive operators" list already skips exchanges for the same reason. Three Parallelism rows leaveWarningBaseline.txt. Nothing else in that file changes.Which component(s) does this affect?
How was this tested?
For each fix I put the pre-fix source back, ran the test class, and then restored the fix. For Rule 9 I put back only
PlanAnalyzer.Statement.cs. The new model property and the parser change do nothing on their own.ExcessiveMemoryGrantTestsCeGuessDetectionTestsExpensiveOperatorExchangeTestsandWarningCharacterizationTestsHasMaxUsedMemorycheck. Then the missing-attribute test fails (1 of 9).WarningCharacterizationTests. It fails because the baseline no longer matches the old behavior.dotnet test tests/PlanViewer.Core.Tests -c Debugran 1,285 tests. 1,255 passed, 0 failed and 30 skipped. The 62 new tests are in that total. The build had no warnings.memory_grant_wait_plan.sqlplanin memory, so the parser and the analyzer are both in the path.AnalyzerRuleGateTestsdoes. Four tests parse one inline XML plan shaped like a real one. Every estimate in the tests is a value SQL Server wrote in the measurements above. I added no.sqlplanfiles, because each file inPlansadds rows to the warning baseline.Checklist
dotnet build -c Debug)dotnet test)Not done
MaxUsedMemoryKBare unchanged. They are listed under Rule 9.MaxUsedMemory="0"before the query uses its grant. Rule 9 then reports the grant as unused. I did not test this.a <> 5(51%) and an OR of two inequalities (41.4%).🤖 Generated with Claude Code
https://claude.ai/code/session_019n3G844aTidqrD6A6iMgza