First incremental add after a load/search permanently doubles the codes buffer (Vec::reserve amortized growth on a tight buffer)

[RyanCodrai]

After a one-shot bulk add, a load/from_bytes, or a search/prepare, the codes buffer(s) are left at tight capacity (len == cap). The next add calls packed_out.reserve(n * bytes_per_row) at encode.rs:613 for just the new rows. Vec::reserve uses amortized growth, which doubles a tight buffer regardless of how small the request is (grow_amortized → max(len+additional, cap*2)). The index then holds a full extra dead copy of the codes as capacity slack, permanently — subsequent small adds fit inside the doubled capacity, so it is never released. The blocked cache doubles the same way (pack.rs:173 resize on the lazy-append path; lib.rs:932 extend_from_slice on the eager path).

This is the workflow the v7 sync() feature (#476) targets: load a large index, then add a small incremental delta. A 2.4GB loaded index balloons by +2.4GB (blocked-only) up to +4.8GB (packed + warm blocked) on its first incremental add and holds it for the index lifetime. It violates #333's stated oracle ("retained heap after add ≈ index size") on the incremental path #333 did not measure.

Distinct from #333 (encode_scratch/snap release), #475 (two layouts as live data), #480 (io_v7 read buffer), #483 (transient sync/load delta materialization).

Measured (dim 768, 2-bit, code 19.2MB; counting global allocator):

• Standalone: tight Vec<u8> cap 19_200_000 + reserve(192) → cap 38_400_000 (exactly 2x); reserve_exact(192) → cap 19_200_192.
• load then +1 row: over-index 34.1MB → 53.7MB (+19.6MB, one code copy), flat on every subsequent add.
• bulk add, search (warm blocked), then +1 row: over-index 19.7MB → 58.5MB (+38.8MB, both packed and blocked double).

Fix (conditional, must not regress growing multi-add): use reserve_exact / bounded headroom on both the codes and blocked grows only when the added rows are small relative to the existing buffer; keep amortized reserve when the add is a meaningful fraction of the buffer, since repeated large same-size adds rely on doubling for O(1) amortized cost.

---

Found by an overnight bug-hunt agent sweep (wave 19) against main @ c8d7ec0; the 2.0x doubling reproduced on the real allocator (tight 19.2MB + reserve(192) → 38.4MB) and the retained-forever behavior independently confirmed by a second agent before filing.

[jokrasno]

Happy to take this; can I get contributor access?

I built a local proof-of-concept while investigating #501, but I will not open a PR unless the proposed policy fits the project and contributor access is approved.

Proposed design:

• apply a shared growth helper only at the packed append, lazy blocked resize, and prepared blocked-cache patch sites
• if an append already fits, do nothing
• for a tiny growth request (no more than 25% of the existing length), reserve exactly the append plus 25% headroom
• retain normal Vec::reserve behavior for meaningful batches, preserving amortized growth

Deterministic baseline evidence: both a loaded and a prepared tight 32,768-byte code buffer grew to 65,536 bytes after one row on pristine 8202f194. The proposed policy bounds tiny-append slack to 25% instead of retaining a full dead copy.

Local validation includes four exact capacity/reallocation regressions, 10/10 caught mutants, pinned native and aarch64 Clippy, a standalone write/load/search/mutation consumer, the Python crate's Rust tests, and cargo test -p turbovec --release --locked (142 library tests plus all integration/doc tests). A rejected 12.5%-headroom iteration was discarded after an alternating-add benchmark exposed extra copies; the 25% policy stayed within measurement noise while retaining the exact memory bound.

If this policy matches your intended tradeoff, I can open a focused PR with the changelog entry and full test plan. If you prefer a different bound or growth rule, I can revise before submission.

[RyanCodrai]

/s w 50

[tq-bot]

Summary: A contributor volunteers to fix this and asks for contributor access. Their local proof-of-concept caps growth for tiny appends at 25% extra space, instead of a tight 32KB buffer doubling to 64KB after one row. It passed extensive testing; they'll open a pull request only if the maintainers approve this approach, and can adjust the bound first if preferred.