Skip to content

[memory]: Memory breakdown reporting and estimation functionality have to handle Hugepage allocations rounded to 2MB #386

Description

@rfsaliev

This is the fallout from IndexBuilder storing a live handle rather than a builder: the estimator has no way to know which allocator was configured, so it has to hardcode one. The result is that svs_index_builder_estimate_memory_dynamic is wrong for SVS_ALLOCATOR_KIND_HUGE_PAGE (2 MiB rounding is unaccounted for) and wrong for any custom allocator. dispatch_vamana_memory_estimate (~line 159 of dispatcher_vamana.cpp, unchanged here) is allocator-blind entirely — it still uses SimpleDataBuilder<index_type>{}.

Note that if the motivation for defaulting to lib::Allocator was to make the estimate line up with within_1pct(tracker.live_bytes, memory_usage) in the new tests, that's fixing the estimator's inaccuracy by degrading the runtime. Threading the kind through instead fixes both.

Originally posted by @ethanglaser in #380

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions