Skip to content

Perfmon rate counters show a per-second rate in the web grid, its trend chart and both MCP tools - #4883

Merged
erikdarlingdata merged 9 commits into
devfrom
fix/perfmon-rate-counters-per-second
Oct 1, 2026
Merged

erikdarlingdata merged 9 commits into
devfrom
fix/perfmon-rate-counters-per-second

Conversation

@erikdarlingdata

@erikdarlingdata erikdarlingdata commented Oct 1, 2026 •

Copy link
Copy Markdown
Owner

Why

A rate counter, such as Batch Requests/sec, stores a running total that has grown since the counter started. The web Server > Activity > Perfmon Counters grid showed that total in its Value column. Batch Requests/sec showed 11,641 next to a delta of 66, so it read as 11,641 batches a second. The trend chart under the grid drew the same total as its Value line, and that line only ever climbs. The get_perfmon_stats MCP tool in both apps sent the total and the delta, but no rate.

The desktop charts already plot a rate counter per second through DeltaSeriesShaping (#3653). This change uses that same division everywhere else.

What changes

  • One division. DeltaSeriesShaping.PerSecond(delta, seconds) is now the only per-second division. It returns the delta over the seconds when the span is positive, and null otherwise. Shape rates every chart point through it, so the desktop charts plot the same numbers as before.
  • get_perfmon_stats, Lite and Darling. A rate row adds per_second: its delta over the seconds since the previous collection. When no delta was knowable, it is null, not 0. That happens on a first collection, a counter reset or a restart. A gauge row, or any row that is not a rate, has no per_second key. This follows the trend payload's peak_per_second. Both apps build each row with one shared builder, TrendPayloads.PerfmonLatestRow, so the two cannot drift apart. Both latest-snapshot reads now also select sample_interval_seconds, last, after cntr_type.
    • The tool description's head names the field in its rate clause. The head is byte-identical in both apps. Both budget pins were re-measured: get_perfmon_stats goes from 521 to 579 characters. Its guide text says the same at more length.
  • get_perfmon_trend, Lite and Darling. Each rate point adds per_second, using the same division over the bucket's delta and seconds. It is null where the interval is 0. The head is unchanged. The guide text names the field.
  • One rounding rule for the rates. Both tools round per_second and peak_per_second through TrendPayloads.RoundRate. It keeps four decimals. A rate under 0.001 keeps two significant digits instead. bucket_minutes can be as wide as a day, 86,400 s. One count in a bucket that wide is 0.0000116 a second. Four decimals published that as 0, so a nonzero delta read as no activity. It now publishes as 0.000012. A rate of 0.001 or more rounds as before. No tool description changes, so no budget pin moves.
  • Web grid. There are two new columns, Per second and Total since counter start. A rate row shows its rate under Per second and its running total under Total since counter start. The total never sits under a header that a reader takes for a rate. The total stays when no rate is known. For a counter that seldom fires, such as Number of Deadlocks/sec, it is the only count there is. When the rate is unknown, the row leaves out the 0 stored as its delta. That 0 is a stand-in, not a count. Gauges and every other kind of row keep their numbers as before, under Value and Delta. The Per second cell keeps two significant digits below 1, so one deadlock in 300 s reads 0.0033 and not 0.00.
  • Web trend chart. A rate counter draws one Per second line, with a /s unit. Its axis labels and its tooltip print through the new rate format. One deadlock in a 300 s collection is 0.0033 a second. Two decimals printed that as 0 on every axis label and in the tooltip. Now the axis runs from 0 to 0.004 and the tooltip reads 0.0033. Every other counter draws the same lines and prints the same numbers as before, so a gauge still plots its reading.
  • rate format. fmtRate in util.js prints a number from 1 up with up to two decimals, and a number below 1 with two significant digits. Only a true 0 prints as 0. It is registered as rate in FORMATTERS, and panels.js right-aligns a rate column. The custom view editor's format list mirrors the registry, so it offers rate too.
  • Desktop apps: no change. Neither Lite nor the Darling Viewer has a grid that lists the latest counters. Their Perfmon tab is a counter picker and a chart, and that chart already plots a rate per second.
  • Rate census, both apps. MeasurementContractCensusTests checks every payload key whose name says per second. Its key sweep matched only names with a prefix, such as peak_per_second. A key named just per_second was outside the sweep. It is inside it now, and both new keys pass as calls to the rate helper with the interval. A line in the census's own test file proves it sees a bare key, and that a bare key over a stored delta still fails.

Which component(s) does this affect?

  • Lite
  • Darling
  • Lite Tests
  • Darling Tests
  • SQL collection scripts
  • Documentation
  • Full Dashboard (deprecated)
  • CLI Installer (deprecated)

Test plan

  • Lite, end to end on DuckDB through the real tool bodies: PerfmonCounterTypeReadTests.ARateCounter_CarriesItsPerSecondFigure_AndNoOtherKindDoes. Batch Requests/sec gets per_second 0.22 (66 over 300 s) and keeps value 11,641. A rate with an interval of 0 gets null. A row with no stored type whose name ends in /sec gets 0.2. A gauge, an average numerator and a row with no type and no /sec name have no key. The trend's rate points carry null, then 0.22, and its gauge points carry no key.
  • Darling, without a store: PerfmonPerSecondPayloadTests (new, 6 tests). They run the tool's own row projection. The keys come out in the order they had before, with per_second last. Two of them cover the rounding: a counter that seldom fires, and a day-wide trend bucket through the shared trend builder.
  • Darling, the store read: PerfmonSql_LatestSnapshot_ValueAndDelta checks that the latest read selects sample_interval_seconds. EveryDarlingPerfmonRead_SelectsTheType_AndTheRowsKeepNullAsNull checks that it comes right after cntr_type in both apps' latest reads.
  • Darling, against a Postgres store: PerfmonCounterTypeLivePostgresTests now asserts per_second 0.2333 on the rate row (70 over 300 s). It also asserts no key on the gauge row, and per_second on each rate trend point. It needs DARLING_TEST_PG, so it runs in CI and was skipped here.
  • Web: WebPerfmonPerSecondBehaviourTests (new, 10 tests). They run the shipped server-tabs.js, util.js, panels.js and charts.js under Node through web-perfmon-harness.mjs, against scripted rows. The harness runs the real chart renderer, so the axis labels and the tooltip text are the ones a browser prints. Five grid tests check the headers, the number alignment and the cells. They cover a rate, a rate with no knowable delta, a gauge and an average row. They also cover a rate that seldom fires, at 37 since start and 0.0033 a second. A fifth test pins the digit rules of the rate format: 0.2333 a second prints as 0.23, and 1234.5678 a second prints as 1,234.57. Five chart tests check the lines, the axis labels and the tooltip. They cover a rate, one deadlock in 300 s, one deadlock in a day-wide bucket (86,400 s) and a gauge. A fifth test pins the same two digit rules in the tooltip.
  • Rounding: PerfmonPerSecondRoundingTests (new, 5 tests, Lite.Tests) pins RoundRate value by value. It publishes a one-count bucket at every width on the bucket ladder, including the day-wide maximum. No width reads 0, and each rate stays within the rounding of delta over seconds. The last two tests cover the peak rate and the latest-snapshot row.
  • DeltaSeriesShapingTests.PerSecond_IsTheDeltaOverTheStoredInterval_AndNullWhereTheChartBreaksItsLine covers the division's edge cases, and checks that it equals Shape's value at the same point.
  • Fails on the base commit 2b6578f99. These runs used the product files from that commit and the new tests:
Darling.Tests.PerfmonCounterTypeRungTests.EveryDarlingPerfmonRead_SelectsTheType_AndTheRowsKeepNullAsNull [FAIL]
Darling.Tests.DarlingMcpDataToolsSurfaceAndSqlTests.PerfmonSql_LatestSnapshot_ValueAndDelta [FAIL]
Darling.Tests.WebPerfmonPerSecondBehaviourTests.TheGrid_ShowsARateCounterPerSecond_AndItsRunningTotalUnderATotalHeader [FAIL]
Darling.Tests.WebPerfmonPerSecondBehaviourTests.TheTrendChart_ForARateThatSeldomFires_NeverReadsItsRateAsZero [FAIL]
Darling.Tests.WebPerfmonPerSecondBehaviourTests.TheTrendChart_ForALongRangeBucket_NeverReadsItsRateAsZero [FAIL]
Darling.Tests.WebPerfmonPerSecondBehaviourTests.ARateThatSeldomFires_ShowsItsRealRate_BesideItsTotal [FAIL]
Darling.Tests.WebPerfmonPerSecondBehaviourTests.ARateRowWithNoKnowableDelta_KeepsItsTotal_AndShowsNoRateAndNoDelta [FAIL]
Darling.Tests.WebPerfmonPerSecondBehaviourTests.AGaugeRow_AndAnAverageRow_KeepTheirNumbers [FAIL]
Darling.Tests.WebPerfmonPerSecondBehaviourTests.TheTrendChart_DrawsARateCounterPerSecond [FAIL]
Darling.Tests  Total: 79, Errors: 0, Failed: 9, Skipped: 0, Not Run: 0

PerformanceMonitorLite.Tests.PerfmonCounterTypeTests.BothMcpPerfmonTools_DescribeTheKinds_AndSpellThemThroughTheVocabulary [FAIL]
PerformanceMonitorLite.Tests.PerfmonCounterTypeReadTests.ARateCounter_CarriesItsPerSecondFigure_AndNoOtherKindDoes [FAIL]
Lite.Tests  Total: 20, Errors: 0, Failed: 2, Skipped: 0, Not Run: 0
  • PerfmonPerSecondPayloadTests, PerfmonPerSecondRoundingTests and the PerSecond test call code that is new in this change, so they do not build against the base commit. I set them aside for the run above.
  • Each new pin fails when its product line is reverted or changed, one line at a time. Every run was restored, and git status was clean after it:
server-tabs.js, perfmonRows without running_total       -> TheGrid_ShowsARateCounterPerSecond_AndItsRunningTotalUnderATotalHeader, ARateRowWithNoKnowableDelta_KeepsItsTotal_AndShowsNoRateAndNoDelta, ARateThatSeldomFires_ShowsItsRealRate_BesideItsTotal [FAIL]
server-tabs.js, delta_value kept when per_second is null -> ARateRowWithNoKnowableDelta_KeepsItsTotal_AndShowsNoRateAndNoDelta [FAIL]
server-tabs.js, Per second cell back to num2             -> ARateThatSeldomFires_ShowsItsRealRate_BesideItsTotal [FAIL]
server-tabs.js, rate chart back to two decimals          -> TheTrendChart_ForARateThatSeldomFires_NeverReadsItsRateAsZero, TheTrendChart_ForALongRangeBucket_NeverReadsItsRateAsZero [FAIL]
panels.js, rate missing from the number formats          -> TheGrid_ShowsARateCounterPerSecond_AndItsRunningTotalUnderATotalHeader [FAIL]
editor.js, rate missing from FORMAT_OPTIONS              -> ServerPageTabsTests.EveryColumnFormat_IsOneTheRendererKnows [FAIL] (an existing pin)
util.js, fmtRate keeps three significant digits below 1   -> TheGrid_PrintsARateBelowOneToTwoSignificantDigits_AndARateOfOneOrMoreToTwoDecimals, TheTrendChart_PrintsARateBelowOneToTwoSignificantDigits_AndARateOfOneOrMoreToTwoDecimals [FAIL]
util.js, fmtRate keeps three decimals from 1 up           -> TheGrid_PrintsARateBelowOneToTwoSignificantDigits_AndARateOfOneOrMoreToTwoDecimals, TheTrendChart_PrintsARateBelowOneToTwoSignificantDigits_AndARateOfOneOrMoreToTwoDecimals [FAIL]
TrendPayloads.cs, RoundRate rounds to 4 decimals         -> PerfmonPerSecondRoundingTests (all 5), PerfmonPerSecondPayloadTests.ARateThatSeldomFires_KeepsItsRate_BesideItsTotal, ADayWideTrendBucket_WithOneCount_PublishesItsRate_NotZero [FAIL]
TrendPayloads.cs, trend per_second back to 4 decimals    -> OneCountOverTheLargestBucket_PublishesItsRate_NotZero, OneCountOverEveryBucketWidth_PublishesANonzeroRate, ADayWideTrendBucket_WithOneCount_PublishesItsRate_NotZero [FAIL]
TrendPayloads.cs, peak_per_second back to 4 decimals     -> ThePeakRate_IsRoundedByTheSameRule, ADayWideTrendBucket_WithOneCount_PublishesItsRate_NotZero [FAIL]
TrendPayloads.cs, latest row back to 4 decimals          -> ALatestRow_IsRoundedByTheSameRule_AndKeepsItsTotal, ARateThatSeldomFires_KeepsItsRate_BesideItsTotal [FAIL]
  • Some pins guard behavior the base code already has. I proved each one against a wrong change:
    • The chart draws per_second for every counter: TheTrendChart_ForAGauge_IsUnchanged [FAIL].
    • The census key sweep goes back to the old pattern: ThePayloadKeyVerdicts_FireOnEveryShape_AndThePlantedPassthroughFails [FAIL].
    • Every row is treated as a rate row: AGaugeRow_AndAnAverageRow_KeepTheirNumbers [FAIL].
    • The gauge chart loses its digit grouping: TheTrendChart_ForAGauge_IsUnchanged [FAIL].
    • The rate chart prints whole numbers: TheTrendChart_DrawsARateCounterPerSecond [FAIL].
  • The first full Lite run failed MeasurementContractCensusTests.EveryRateHelper_DividesByTheIntervalItIsHanded. The first PerSecond called Shape and had no visible division. PerSecond now does the division, and Shape calls it.
  • A first draft of RoundRate sat in DeltaSeriesShaping. The census read the class name as a stored delta and failed EveryPerSecondPayloadKey_IsAReaderRateOrACSharpQuotient_OrIsNamed on peak_per_second. The helper lives in TrendPayloads now, and the census passes.
  • Both test projects build with 0 warnings.
  • Full suites on the head, with no live connection string:
Darling.Tests  Total: 19132, Errors: 0, Failed: 0, Skipped: 1197, Not Run: 1, Time: 148.531s
Lite.Tests  Total: 6497, Errors: 0, Failed: 0, Skipped: 0, Not Run: 0, Time: 371.482s
  • Not run: the page in a browser against a live store.

Worth checking

  • A rate row's Value cell in the web grid is blank on purpose. Its running total sits under Total since counter start, so the number is never under a header that a reader takes for a rate. The MCP payload still carries the total in value.
  • An average row (for example Average wait time (ms) under Lock waits) still shows the 0 stored as its delta after a restart. Its kind is not a rate, so this change does not touch it.
  • per_second and peak_per_second are rounded to 4 decimal places. A rate under 0.001 keeps two significant digits instead. A rate under 0.0001 now prints in exponent form in the JSON, for example 1.2E-05. That is valid JSON, and both the web page and the MCP clients read it as a number.
  • The custom view editor's format list now offers rate, because that list mirrors the formatters.

Checklist

  • I have read the contributing guide
  • My code builds with zero warnings (dotnet build -c Debug)
  • I have tested my changes against at least one SQL Server version
  • I have not introduced any hardcoded credentials or server names

CHANGELOG

SECTION: Fixed
ENTRY:

@erikdarlingdata
erikdarlingdata marked this pull request as ready for review October 1, 2026 00:15
…mall rates never read as 0

The web Perfmon Counters grid dropped a rate row's stored value to show its per-second figure. For a counter that
seldom fires that removed the only count there was: Number of Deadlocks/sec at 37 showed no total, and its rate
read 0.00. A rate row now keeps its running total under a Total since start column, whether or not a rate is known.
Only the stand-in 0 delta of an unknowable interval is blanked. Gauges and every other row keep their numbers.

A real low rate also printed as 0. One deadlock in a 300 s collection is 0.0033 a second, but the trend chart
formatted to two decimals, so every axis label read 0 and the tooltip read 0. A rate now prints through fmtRate
(two decimals from 1 up, two significant digits below it) on the chart axis, the tooltip and the grid's Per second
cell. A true 0 still reads 0, and a gauge's chart prints as it did.

get_perfmon_trend and get_perfmon_stats rounded per_second to 4 decimals, so one count in a day-wide bucket
(86,400 s) published as a rate of 0. Both apps build their payloads through TrendPayloads, which now rounds a rate
to 4 decimals, or to two significant digits where 4 decimals would keep fewer.

The web test harness now runs the shipped charts.js, so the pins read the real axis labels and tooltips.
@erikdarlingdata erikdarlingdata changed the title Perfmon rate counters show a rate, not their running total, in the web grid, its trend chart and both MCP tools Perfmon rate counters show a per-second rate in the web grid, its trend chart and both MCP tools Oct 1, 2026
@erikdarlingdata
erikdarlingdata marked this pull request as draft October 1, 2026 01:41
@erikdarlingdata
erikdarlingdata marked this pull request as ready for review October 1, 2026 01:43
@erikdarlingdata
erikdarlingdata marked this pull request as draft October 1, 2026 02:02
The cell is the raw counter value, which counts from the counter's own start:
an instance restart, or a database's own restart for a per-database counter.
"Since start" did not say whose start.
Every scripted rate had two significant digits, so changing fmtRate to keep
three left the suite green. Two new rows, 0.2333 and 1234.5678, pin a rate
below 1 at two significant digits and a rate of 1 or more at two decimals,
in the grid cell and in the chart tooltip.
@erikdarlingdata
erikdarlingdata marked this pull request as ready for review October 1, 2026 02:49
@erikdarlingdata
erikdarlingdata merged commit 65416cb into dev Oct 1, 2026
27 of 30 checks passed
@erikdarlingdata
erikdarlingdata deleted the fix/perfmon-rate-counters-per-second branch October 1, 2026 02:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant