Skip to content

Stores that expose the managed PostgreSQL on the network finish the settings-file migration - #4464

Merged
erikdarlingdata merged 2 commits into
devfrom
fix/managed-conf-verify-command-line-keys
Sep 27, 2026
Merged

erikdarlingdata merged 2 commits into
devfrom
fix/managed-conf-verify-command-line-keys

Conversation

@erikdarlingdata

@erikdarlingdata erikdarlingdata commented Sep 27, 2026 •

Copy link
Copy Markdown
Owner

The managed-conf migration's second step (Step B) never finished on a store that exposes its managed PostgreSQL on the network. This change lets it finish.

Classification: P (real bug)

Confirmed against a live PostgreSQL 18 instance (a fresh container, no operator conf), not just from code reading:

  • With -c listen_addresses=127.0.0.1,<private> on the command line and a listen_addresses = '127.0.0.1' line in postgresql.conf, after a reload, pg_file_settings reports that file row applied=false, error=setting could not be applied, and pg_settings.source = command line.
  • With -c listen_addresses=127.0.0.1 on the command line (loopback only — no network endpoint configured) and the same file line, the file row reports applied=true, error=(none) — the values happen to agree, so it verifies clean.

That matches what the installed stores showed: the store with no network endpoints (command line = 127.0.0.1, matching the file's rendered loopback value) verifies clean; the stores with a network endpoint (command line = 127.0.0.1,<private>) get an error row for listen_addresses, which ManagedConfMigrationRunner.VerifyStepB was counting as a mismatch.

Where: Darling/PerformanceMonitor.Darling.Service/ManagedConfMigrationRunner.cs, VerifyStepB (was comparing every key ParseConfText(renderedText) yields against a pg_file_settings-attributed row with no error). port never shows this symptom because darling.json's port and the rendered file's port line are normally the same value, so the file row still reports applied=true with no error even though it's not the one actually in force — only listen_addresses diverges in VALUE (the network IP) between what the file renders and what the command line carries.

DarlingStoreHostProfile.CommandLineOnlyKeys (["port", "listen_addresses"]) already exists for exactly this pair, used by the separate ComputeAndStoreManagedConfVerdictsAsync diagnostic path, but VerifyStepB never consulted it.

Effect on a real store: Step B restores the previous verified file every start it runs (the "restore" branch fires because previousText is passed) and reports Failed. It repeats every start — nothing in the retry path changes the outcome, since the same command-line/file mismatch recurs identically each time. This blocks the settings-file migration from ever completing on a store with a network endpoint (the store itself keeps running fine on the previous file; no connectivity impact — the command line is in force everywhere, confirmed by the field read).

Forward risk on a store that has only just done Step A (no listen_addresses row in its managed file yet): its next start runs Step B, which renders listen_addresses = '127.0.0.1' again and hits the identical command-line mismatch — it would log the same failure at its next start once it also has a network endpoint. Fixed by the same change.

Introduced by: the #4215/#4336 Step-B work (ManagedConfMigrationRunner.VerifyStepB, part of the #4215 conf-migration series). Network endpoints were not exercised by Step B's original test suite — every existing VerifyStepB_* pin builds its rows and rendered text with values that already agree.

The fix

VerifyStepB now skips DarlingStoreHostProfile.CommandLineOnlyKeys entirely — never looked up against pg_file_settings, on either side of the comparison — reusing the product's existing list rather than adding a second one. This is the smaller, correct change: the alternative (not rendering listen_addresses/port into darling-managed.conf at all) would touch ManagedConfFile.RenderBody, which every other Step-B/Step-A pin and the initial-migration Compare path also read from, for no gain — the file lines themselves are harmless; only comparing them against the command line was wrong. Network-exposure behavior (BuildServerRuntimeOptions, BuildListenAddresses) is untouched.

Pins

  • Mixed-case pin (VerifyStepB_ListenAddressesOverriddenByCommandLine_IsNotAMismatch_OtherKeyStillFails): a pg_file_settings-shaped row for listen_addresses with error = "setting could not be applied" is not a mismatch; a real mismatch on another key (work_mem) in the same batch still fails.
  • Field-case pin, listen_addresses (required) (VerifyStepB_OnlyListenAddressesOverriddenByCommandLine_Verifies): every OTHER rendered key matches and the only difference is the command-line-overridden listen_addresses row → Status == Verified, the verified stamp is written (ManagedConfMigrationSteps.IsVerified true against the rendered text), and the managed file is NOT restored to previousText. This is the pin that shows Step B now completes on a store with a network endpoint, instead of restoring the previous file on every start.
  • Field-case pin, port (VerifyStepB_OnlyPortOverriddenByCommandLine_Verifies): the same completion when the command-line-owned key that differs is port instead.
  • Row-fidelity capture: started a throwaway timescale/timescaledb:2.30.1-pg18 container with -c listen_addresses=127.0.0.1,192.0.2.10 (the RFC 5737 documentation address standing in for the container's own address — no real address recorded anywhere) and an included conf rendering the product's usual listen_addresses = '127.0.0.1' line, then ran the product's own ManagedConfFileSettings.SnapshotSql (SELECT sourcefile, sourceline, name, setting, applied, error FROM pg_file_settings) against it after a reload. Captured row for the managed file: name=listen_addresses, setting=127.0.0.1, applied=false, error="setting could not be applied"; the same capture with a port = '5555' line added showed name=port, setting=5555, applied=false, error="setting could not be applied". Both new facts' rows use exactly this shape (and comment says so, without the address).
  • RUNTIME RED on origin/dev (38f142f, a detached worktree, only the test file copied in — builds unchanged against the unmodified VerifyStepB): Darling.Tests.ManagedConfMigrationRunnerTests → Total: 21, Failed: 3 (the pre-existing mixed-case pin plus both new field-case pins). The required field-case pin's assertion: Assert.Equal() Failure: Values differ / Expected: Verified / Actual: Failed.
  • Mutation (temporary, reverted, never committed): disabled the CommandLineOnlyKeys skip in VerifyStepB (if (false && Array.IndexOf(...) >= 0)) → Total: 21, Failed: 3 (same three pins RED); restored the skip → Total: 21, Failed: 0.

Build: Darling.Tests.csproj 0 errors / 0 warnings (-p:EnableWindowsTargeting=true). ManagedConfMigrationTests (43/43), ManagedConfFileTests (33/33) and DocCommentHygieneTests (77/77) also pass on the branch. Lite.Tests untouched by this change.

How to check it

After install, a store with a configured network endpoint logs "conf migration verified (step B)" on startup instead of "failed verification … listen_addresses". pg_file_settings' file row for listen_addresses (and port, where the command line pins a different value) may still show applied=false, error="setting could not be applied" — that's expected: the command line is what's actually in force, and this check no longer treats that row as a mismatch.

CHANGELOG entry

SECTION: Fixed
ENTRY:

Refs

Closes #4215 field finding (a .479 report; see field notes for the affected stores' logs — no store or host names carried here).

A store with a configured network endpoint always starts PostgreSQL with listen_addresses forced onto the pg_ctl command line, which outranks postgresql.conf/darling-managed.conf unconditionally. The settings-file migration's Step B compared the rendered file's listen_addresses line against pg_file_settings and read a real mismatch, because PostgreSQL reports that file row with an error once the running value differs from the file's -- confirmed against a live PostgreSQL 18 instance. Step B now skips both command-line-owned keys (port, listen_addresses) using the same list the host profile already defines, instead of comparing them against a file position they can never win.
@erikdarlingdata
erikdarlingdata marked this pull request as ready for review September 27, 2026 13:05
@erikdarlingdata
erikdarlingdata merged commit a901c12 into dev Sep 27, 2026
22 of 24 checks passed
@erikdarlingdata
erikdarlingdata deleted the fix/managed-conf-verify-command-line-keys branch September 27, 2026 13:06
erikdarlingdata added a commit that referenced this pull request Sep 27, 2026
…that expose the managed PostgreSQL on the network (#4465)

Stores that expose the managed PostgreSQL on the network re-verify a settings file left unstamped by a hand edit or an interrupted start, instead of reporting a failure on every start.

This is the same cause as #4464, on the unstamped re-verification. An exposed store always starts PostgreSQL with port and listen_addresses on the pg_ctl command line, which outranks the file. So pg_file_settings reports darling-managed.conf's loopback-only listen_addresses row as an error on every start.

- The re-verification's scan moves into ManagedConfMigrationRunner.FindUnstampedManagedFileErrors. It is the previous inline loop, plus a skip for DarlingStoreHostProfile.CommandLineOnlyKeys, the same list VerifyStepB uses. DarlingManagedPostgres calls it and builds the Failed outcome as before. Nothing changes the address or port the store listens on.
- The other comparisons over pg_file_settings and pg_settings rows need no change:
  - Step A compares new error rows against the same server's earlier snapshot;
  - the verdicts already skip the command-line keys;
  - the startup heal and --check-settings do not read error rows;
  - VerifyStepB was fixed in #4464.
- Tests:
  - FindUnstampedManagedFileErrors: listen_addresses and port overridden by the command line are not errors, and a real error on another key still is;
  - ManagedConfFileTests.RenderBody_ListenAddresses_IsAlwaysLoopbackOnly pins the rendered listen_addresses to 127.0.0.1.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant