Skip to content

Vacuum on persisted tables does not reclaim on-disk space #163

Description

@pathscale

Context

From the 2026-07-27 concurrency-fix series (patch 0010, "vacuum: persist row moves through CDC"). Vacuum on persisted tables is now correct — row moves flow through the CDC stream via the VacuumPersistence sink, so the on-disk indexes follow the moves and the event-id stream stays gapless (tests/persistence/vacuum.rs proves reload works, including secondary indexes) — but it is consistency-only.

The gap

EmptyDataVacuum::defragment frees pages in memory (stats.pages_freed), and the moved rows' bytes are persisted at their new links. Nothing ever shrinks or truncates the on-disk .wt.data file:

  • Pages emptied by vacuum remain in the data file with stale bytes; the indexes simply stop pointing at them.
  • SpaceData::save_batch_data (src/persistence/space/data.rs) only ever writes pages; there is no free-page bookkeeping on disk beyond SpaceInfoPage.empty_links_list.
  • After many vacuum cycles the file is dominated by dead pages.

So "vacuum" on a persisted table today = in-memory compaction + disk consistency, not disk space reclamation.

Suggested direction

Options, roughly in order of ambition:

  1. Persist the freed-page set (extend SpaceInfoPage / TOC) so freed pages are reused by later save_batch_data page creation instead of appending — stops growth without shrinking.
  2. Physical compaction: rewrite the tail of the data file into freed pages and truncate — needs link rewrites, i.e. the same CDC move machinery vacuum already uses, so VacuumPersistence::apply_move is the natural hook.
  3. Offline compaction on load (cheapest to reason about; no concurrency).

Verification hints

tests/persistence/vacuum.rs::test_vacuum_on_persisted_table_survives_reload is the harness to extend: assert on the data file size (or page count) after vacuum + reload. Full suite runs take ~4s warm; 30-50 run loops are cheap for flake checks.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions