Make the Postgres writer pool minimum configurable (BUZZ_DB_MIN_POOL_SIZE, default 20) - #4627
Open
tlongwell-block wants to merge 1 commit into
Open
Make the Postgres writer pool minimum configurable (BUZZ_DB_MIN_POOL_SIZE, default 20)#4627tlongwell-block wants to merge 1 commit into
tlongwell-block wants to merge 1 commit into
Conversation
The relay could size the pool ceiling via BUZZ_DB_POOL_SIZE but had no way to set the floor: DbConfig::min_connections existed and was applied, yet the only value it ever took in production was the hardcoded 2 from DbConfig::default(), because relay main filled the rest of the struct from that default. Add BUZZ_DB_MIN_POOL_SIZE -> Config::db_pool_min_size (default 20) and wire it into the writer pool explicitly, mirroring how the maximum is already plumbed. The default lives at the relay env boundary rather than in DbConfig::default() on purpose. buzz-admin also constructs DbConfig with ..DbConfig::default() and is a short-lived CLI that connects, performs one operation and exits, so raising the library default would make every admin invocation eagerly open 20 sessions for no throughput benefit. The value is clamped to db_pool_size. sqlx obeys max_connections regardless (try_min_connections bails when the semaphore has no permit) so an unclamped pair is not a hang, but sqlx stores the requested minimum verbatim and documents that applications allowing either value to be set dynamically should validate the pair themselves. Without the clamp, pool.options() would report an impossible min > max. Unlike the pool maxima, 0 is honoured rather than rejected: a minimum of zero is a meaningful setting (a fully lazy pool, which is what the read-replica pool pins deliberately), whereas a maximum of zero is not. Reusing the maxima's filter(v > 0) would make the one value an operator would reach for to stop pre-warming silently mean 20 instead. The reader pool is untouched: it stays pinned at 0 and lazy so a replica that is down at boot cannot gate relay startup. Co-authored-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@buzz.block.builderlab.xyz> Signed-off-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@buzz.block.builderlab.xyz>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The relay could size the Postgres pool ceiling but not its floor.
BUZZ_DB_POOL_SIZEhas been wired tomax_connectionsfor a while.DbConfig::min_connectionsalso exists (crates/buzz-db/src/lib.rs:516) and is genuinely applied to the writer pool (lib.rs:681) — but the only value it ever took in production was the hardcoded2fromDbConfig::default()(lib.rs:542), because relaymainsets max/read-max explicitly and fills the rest from..DbConfig::default()(main.rs:166-173). No env var reached it. The knob existed with no wire to the outside.Change
Adds
BUZZ_DB_MIN_POOL_SIZE→Config::db_pool_min_size(default 20), passed explicitly into the writer pool, mirroring how the maximum is already plumbed. Pre-warming keeps requests after a deploy or an idle period off the connect path, which against Aurora costs a TLS handshake per connection.Three decisions worth a reviewer's attention, since each could reasonably have gone the other way:
1. The default lives at the relay env boundary, not in
DbConfig::default().buzz-adminalso buildsDbConfigwith..DbConfig::default()(crates/buzz-admin/src/main.rs:423) and is a short-lived CLI —migrate/add-member/list-members/reconcile-channelsconnect, do one operation, and exit (main.rs:130-153). Raising the library default would make every admin invocation eagerly open 20 sessions for zero throughput benefit.DbConfig::default().min_connectionsstays2.2. The value is clamped to
db_pool_size.sqlx obeys
max_connectionsregardless —try_min_connectionsbails when the semaphore has no permit, commented "We must always obeymax_connections" (sqlx-core-0.9.0/src/pool/inner.rs:400-418). So an unclamped pair is not a hang or a panic. But sqlx stores the requested minimum verbatim and explicitly puts validation on the caller:Without the clamp,
BUZZ_DB_POOL_SIZE=8(already legal — the max parser accepts any positive value) plus the default minimum of 20 would leavepool.options()reporting an impossiblemin=20, max=8. Behaviourally inert, but it misreports the pool's own configuration to anyone introspecting it. @mari confirmed this against local Postgres:max=2, min=3connected fine withlive_size=2, whilepool.options()still claimedmin=3, max=2.3.
0is honoured rather than rejected — a deliberate divergence from the maxima.Every neighbouring sizing parser has
.filter(|&v| v > 0)so zero falls back to the default. That is right for a maximum and wrong for a minimum: zero is a meaningful minimum (a fully lazy pool — exactly what the read-replica pool pins on purpose), so copying the max parser would make the one value an operator would reach for to stop pre-warming silently mean20instead. Unparsable values still fall back to the default.The reader pool is untouched. It stays pinned at
min_connections(0)and lazy (lib.rs:726) so a replica that is down at boot cannot gate relay startup. Pre-warming it would reintroduce exactly that coupling.Deliberately out of scope, for symmetry with
BUZZ_DB_POOL_SIZE(which also governs only theDbpools): the audit pool's fixedmax 5 / min 1(buzz-relay/src/main.rs:351-352), the search pool's barePgPoolOptions::new()(main.rs:406), and the push gateway's independent pools, which do not read relayConfigat all.Verification
At
b380ba3e073919a487082fadcd15868e93a0c889, hermit toolchain 1.95.0:cargo build -p buzz-relay --locked— cleancargo test -p buzz-relay --locked --lib— 839 passed, 37 ignored, 1 failedcargo clippy -p buzz-relay --locked --all-targets— no warnings, no errorscargo fmtclean; pre-pushrust-tests+desktop-tauri-checkshooks both greenThe one failure is
api::mesh_demo::tests::demo_join_forwarded_arm_round_trips_echo(504 vs 200). It is pre-existing and not caused by this change — I stashed the entire diff, confirmedgit rev-parse HEAD=027a74a61c8643a1d1086d3e8307fad89d7735f7with a clean tree, and reproduced the identicalleft: 504, right: 200failure, then restored.Both new tests were mutation-tested rather than just observed green:
.min(db_pool_size)(the clamp)explicit minimum above the maximum must clamp to the maximum.filter(|&v| v > 0)(copy the max parser's shape)zero must be honoured, not treated as unsetThe two mutants kill different tests, so the pair discriminates rather than overlapping. Control re-run green after each restore. One caveat worth stating: the clamp mutant also fails the two neighbouring
db_pool_size/db_read_pool_sizetests, but that isENV_MUTEXpoisoning collateral (both panic at.lock().unwrap()), not three independent kills — the real kill is the clamp assertion.The clamp test covers all three arms: explicit min above explicit max, default min above a smaller max (fires with no min set at all), and min below max passing through unchanged.