Skip to content

Bound file-controlled sizes in view(): levels region and dense vectors span (OOB read on crafted index) - #776

Open
professor-moody wants to merge 1 commit into
unum-cloud:mainfrom
professor-moody:bound-view-levels-and-dense-span
Open

Bound file-controlled sizes in view(): levels region and dense vectors span (OOB read on crafted index)#776
professor-moody wants to merge 1 commit into
unum-cloud:mainfrom
professor-moody:bound-view-levels-and-dense-span

Conversation

@professor-moody

Copy link
Copy Markdown

Summary

index_gt::view (and index_dense::view) read file-controlled sizes and dereference the memory mapping before bounding them, so a crafted .usearch index over-reads the mapping. Two sinks:

  1. Base view() levels region (index.hpp): view sizes a memory-mapped levels[] array from the file's header.size and walks it in the offset-precompute loop before the if (file.size() < total_bytes) check runs. A crafted header declaring more nodes than the file contains reads levels[] past the mapping. Fix: reject when the file does not extend to first_offset (the end of the levels array) before the loop.

  2. Dense view() vectors span (index_dense.hpp): vectors_buffer = {file.data() + offset, matrix_rows * matrix_cols} forms a span with no file-size check (and the product can overflow size_t), and the following if (file.size() - offset < sizeof(buffer)) guard is an unsigned subtraction that underflows once offset passes file.size(). Fix: bound the span with a checked multiply and make the header guard underflow-safe.

Impact

Memory-mapping a crafted or untrusted .usearch index via Index.view / Index.load(..., view=True) yields an out-of-bounds read (crash / adjacent-memory disclosure). CWE-125.

Validation

AddressSanitizer, control vs crafted, driving the real view() over a buffer-backed mapping: a well-formed index loads cleanly; a crafted one (header.size larger than the file / oversized matrix dims) is now rejected with a clear error and no OOB. Found with a structure-aware model/index parser fuzzer.

…s span

index_gt::view walked the memory-mapped levels[] array (sized from the
file's header.size) before the only file-size check, and index_dense::view
formed a matrix_rows*matrix_cols span into the mapping with no bound plus
an unsigned-underflow header guard. A crafted .usearch index over-read the
mapping. Bound the levels region before the offset-precompute loop, bound
the vectors span with a checked multiply, and make the header guard
underflow-safe. Validated with AddressSanitizer (crafted rejected cleanly,
valid indexes still load).
@professor-moody

Copy link
Copy Markdown
Author

Small correction to the Impact section of this PR: I named a Python entry point that does not exist.

I wrote Index.load(..., view=True). Index.load takes (self, path_or_buffer, progress) and has no view parameter and no **kwargs, so that call raises TypeError. Anyone reproducing from my text would hit that and reasonably wonder whether I had exercised it at all.

The entry points that actually memory-map, and therefore reach view(), are:

Index.view(path_or_buffer)
Index.restore(path_or_buffer, view=True)

Index.view alone is sufficient and is the one I should have written.

Nothing else changes. The two sinks and the fix are unaffected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant