Skip to content

dictBuilder: validate buffer bounds and check allocation in fastcover - #4794

Closed
filtede98 wants to merge 1 commit into
meta:devfrom
filtede98:fix-zdict-bounds-and-fastcover-null
Closed

filtede98 wants to merge 1 commit into
meta:devfrom
filtede98:fix-zdict-bounds-and-fastcover-null

Conversation

@filtede98

Copy link
Copy Markdown

Summary

This PR addresses several input validation and memory safety issues in lib/dictBuilder:

  1. ZDICT_trainFromBuffer_fastCover() (lib/dictBuilder/fastcover.c):

    • segmentFreqs = (U16 *)calloc(((U64)1 << parameters.f), sizeof(U16)); was not checked for allocation failure (unlike FASTCOVER_tryParameters(), which checks if (!segmentFreqs || !dict || !freqs)).
    • If calloc fails (e.g. OOM or large parameters.f), FASTCOVER_buildDictionary() -> FASTCOVER_selectSegment() dereferences segmentFreqs[idx] += 1; on a NULL pointer, causing an immediate segmentation fault and leaking ctx.
    • Now checks if (!segmentFreqs), destroying ctx and returning ERROR(memory_allocation).
  2. ZDICT_addEntropyTablesFromBuffer_advanced() (lib/dictBuilder/zdict.c):

    • Called by public API ZDICT_addEntropyTablesFromBuffer().
    • When dictBufferCapacity < dictContentSize, the pointer calculation (char*)dictBuffer + dictBufferCapacity - dictContentSize underflows size_t, producing an out-of-bounds pointer preceding dictBuffer that is subsequently passed to XXH64, ZDICT_analyzeEntropy, and memmove.
    • When dictBuffer == NULL or dictBufferCapacity < ZDICT_DICTSIZE_MIN, writing dictionary headers leads to null dereference or out-of-bounds writes.
    • Added explicit condition checks matching ZDICT_finalizeDictionary(), returning ERROR(dstSize_tooSmall).
  3. ZDICT_finalizeDictionary() (lib/dictBuilder/zdict.c):

    • Validates dictBuffer != NULL upfront before copying headers and content.
  4. Tests (tests/fuzzer.c):

    • Added unit tests verifying proper error returns on invalid buffer capacities and NULL buffer pointers without crashes or underflows.

- In ZDICT_trainFromBuffer_fastCover(), check that segmentFreqs allocation
succeeded before use. On allocation failure, destroy ctx and return
ERROR(memory_allocation) to prevent an immediate NULL pointer dereference
crash and memory leak.
- In ZDICT_addEntropyTablesFromBuffer_advanced(), validate dictBuffer != NULL,
dictBufferCapacity >= dictContentSize, and
dictBufferCapacity >= ZDICT_DICTSIZE_MIN to prevent size_t underflow in
(dictBufferCapacity - dictContentSize) and out-of-bounds reads in XXH64,
ZDICT_analyzeEntropy, and memmove.
- In ZDICT_finalizeDictionary(), validate dictBuffer != NULL before any
buffer copy operations.
- Add regression tests in tests/fuzzer.c exercising invalid capacities and
NULL buffer arguments.

Signed-off-by: Filippo Tedeschi <filippotedeschi98@gmail.com>
@meta-cla meta-cla Bot added the CLA Signed label Sep 25, 2026
@filtede98

Copy link
Copy Markdown
Author

Closing. The calloc of 2^f U16 in ZDICT_trainFromBuffer_fastCover() is half
the size of the 2^f U32 table that has already been allocated by then, so it
only matters on an allocation failure at exactly that point, and the rest are
checks for callers passing inconsistent arguments to the public API. Not worth
the change. Sorry for the noise.

@filtede98 filtede98 closed this Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant