Skip to content

Make the Postgres writer pool minimum configurable (BUZZ_DB_MIN_POOL_SIZE, default 20) - #4627

Open
tlongwell-block wants to merge 1 commit into
mainfrom
sami/db-pool-min-size
Open

Make the Postgres writer pool minimum configurable (BUZZ_DB_MIN_POOL_SIZE, default 20)#4627
tlongwell-block wants to merge 1 commit into
mainfrom
sami/db-pool-min-size

Conversation

@tlongwell-block

Copy link
Copy Markdown
Collaborator

Problem

The relay could size the Postgres pool ceiling but not its floor.

BUZZ_DB_POOL_SIZE has been wired to max_connections for a while. DbConfig::min_connections also exists (crates/buzz-db/src/lib.rs:516) and is genuinely applied to the writer pool (lib.rs:681) — but the only value it ever took in production was the hardcoded 2 from DbConfig::default() (lib.rs:542), because relay main sets max/read-max explicitly and fills the rest from ..DbConfig::default() (main.rs:166-173). No env var reached it. The knob existed with no wire to the outside.

Change

Adds BUZZ_DB_MIN_POOL_SIZEConfig::db_pool_min_size (default 20), passed explicitly into the writer pool, mirroring how the maximum is already plumbed. Pre-warming keeps requests after a deploy or an idle period off the connect path, which against Aurora costs a TLS handshake per connection.

Three decisions worth a reviewer's attention, since each could reasonably have gone the other way:

1. The default lives at the relay env boundary, not in DbConfig::default().
buzz-admin also builds DbConfig with ..DbConfig::default() (crates/buzz-admin/src/main.rs:423) and is a short-lived CLI — migrate / add-member / list-members / reconcile-channels connect, do one operation, and exit (main.rs:130-153). Raising the library default would make every admin invocation eagerly open 20 sessions for zero throughput benefit. DbConfig::default().min_connections stays 2.

2. The value is clamped to db_pool_size.
sqlx obeys max_connections regardless — try_min_connections bails when the semaphore has no permit, commented "We must always obey max_connections" (sqlx-core-0.9.0/src/pool/inner.rs:400-418). So an unclamped pair is not a hang or a panic. But sqlx stores the requested minimum verbatim and explicitly puts validation on the caller:

"This value is clamped internally to not exceed max_connections. We've chosen not to assert min_connections <= max_connections anywhere because it shouldn't break anything internally... if the application allows either value to be dynamically set then it should be checking this condition itself" — pool/options.rs:194-200

Without the clamp, BUZZ_DB_POOL_SIZE=8 (already legal — the max parser accepts any positive value) plus the default minimum of 20 would leave pool.options() reporting an impossible min=20, max=8. Behaviourally inert, but it misreports the pool's own configuration to anyone introspecting it. @mari confirmed this against local Postgres: max=2, min=3 connected fine with live_size=2, while pool.options() still claimed min=3, max=2.

3. 0 is honoured rather than rejected — a deliberate divergence from the maxima.
Every neighbouring sizing parser has .filter(|&v| v > 0) so zero falls back to the default. That is right for a maximum and wrong for a minimum: zero is a meaningful minimum (a fully lazy pool — exactly what the read-replica pool pins on purpose), so copying the max parser would make the one value an operator would reach for to stop pre-warming silently mean 20 instead. Unparsable values still fall back to the default.

The reader pool is untouched. It stays pinned at min_connections(0) and lazy (lib.rs:726) so a replica that is down at boot cannot gate relay startup. Pre-warming it would reintroduce exactly that coupling.

Deliberately out of scope, for symmetry with BUZZ_DB_POOL_SIZE (which also governs only the Db pools): the audit pool's fixed max 5 / min 1 (buzz-relay/src/main.rs:351-352), the search pool's bare PgPoolOptions::new() (main.rs:406), and the push gateway's independent pools, which do not read relay Config at all.

Verification

At b380ba3e073919a487082fadcd15868e93a0c889, hermit toolchain 1.95.0:

  • cargo build -p buzz-relay --locked — clean
  • cargo test -p buzz-relay --locked --lib839 passed, 37 ignored, 1 failed
  • cargo clippy -p buzz-relay --locked --all-targets — no warnings, no errors
  • cargo fmt clean; pre-push rust-tests + desktop-tauri-checks hooks both green

The one failure is api::mesh_demo::tests::demo_join_forwarded_arm_round_trips_echo (504 vs 200). It is pre-existing and not caused by this change — I stashed the entire diff, confirmed git rev-parse HEAD = 027a74a61c8643a1d1086d3e8307fad89d7735f7 with a clean tree, and reproduced the identical left: 504, right: 200 failure, then restored.

Both new tests were mutation-tested rather than just observed green:

Mutant Result
Delete .min(db_pool_size) (the clamp) diesexplicit minimum above the maximum must clamp to the maximum
Add .filter(|&v| v > 0) (copy the max parser's shape) dieszero must be honoured, not treated as unset

The two mutants kill different tests, so the pair discriminates rather than overlapping. Control re-run green after each restore. One caveat worth stating: the clamp mutant also fails the two neighbouring db_pool_size/db_read_pool_size tests, but that is ENV_MUTEX poisoning collateral (both panic at .lock().unwrap()), not three independent kills — the real kill is the clamp assertion.

The clamp test covers all three arms: explicit min above explicit max, default min above a smaller max (fires with no min set at all), and min below max passing through unchanged.

The relay could size the pool ceiling via BUZZ_DB_POOL_SIZE but had no way
to set the floor: DbConfig::min_connections existed and was applied, yet the
only value it ever took in production was the hardcoded 2 from
DbConfig::default(), because relay main filled the rest of the struct from
that default.

Add BUZZ_DB_MIN_POOL_SIZE -> Config::db_pool_min_size (default 20) and wire it
into the writer pool explicitly, mirroring how the maximum is already
plumbed.

The default lives at the relay env boundary rather than in
DbConfig::default() on purpose. buzz-admin also constructs DbConfig with
..DbConfig::default() and is a short-lived CLI that connects, performs one
operation and exits, so raising the library default would make every admin
invocation eagerly open 20 sessions for no throughput benefit.

The value is clamped to db_pool_size. sqlx obeys max_connections regardless
(try_min_connections bails when the semaphore has no permit) so an unclamped
pair is not a hang, but sqlx stores the requested minimum verbatim and
documents that applications allowing either value to be set dynamically
should validate the pair themselves. Without the clamp, pool.options() would
report an impossible min > max.

Unlike the pool maxima, 0 is honoured rather than rejected: a minimum of zero
is a meaningful setting (a fully lazy pool, which is what the read-replica
pool pins deliberately), whereas a maximum of zero is not. Reusing the
maxima's filter(v > 0) would make the one value an operator would reach for
to stop pre-warming silently mean 20 instead.

The reader pool is untouched: it stays pinned at 0 and lazy so a replica that
is down at boot cannot gate relay startup.

Co-authored-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@buzz.block.builderlab.xyz>
Signed-off-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@buzz.block.builderlab.xyz>
@tlongwell-block
tlongwell-block requested a review from a team as a code owner August 3, 2026 22:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant