Skip to content

pool, zstdmt: validate allocation bounds and NULL pointer inputs - #4805

Open
filtede98 wants to merge 1 commit into
meta:devfrom
filtede98:fix-pool-overflow-and-null-guards
Open

filtede98 wants to merge 1 commit into
meta:devfrom
filtede98:fix-pool-overflow-and-null-guards

Conversation

@filtede98

@filtede98 filtede98 commented Sep 29, 2026 •

Copy link
Copy Markdown

Summary

This patch strengthens input validation in the thread pool (lib/common/pool.c) and adds NULL checks in the multithreaded compression helpers (lib/compress/zstdmt_compress.c).

  1. Bounds in the thread pool:

    • POOL_create_advanced and POOL_resize_internal reject numThreads above POOL_MAX_THREADS (10000, defined in pool.h), as suggested in review.
    • POOL_create_advanced rejects queueSize >= (size_t)-1 / sizeof(POOL_job). Without it, queueSize + 1 can wrap around, and the worker and dispatch routines then do % ctx->queueSize with a zero queue size.
  2. Pointer checks in the thread pool:

    • POOL_joinJobs returns early when ctx is NULL, like POOL_free and POOL_sizeof.
    • POOL_add and POOL_tryAdd now assert that function is non-NULL, in both the multithreaded and single-threaded implementations. The assert on ctx was already there.
  3. NULL checks in the multithreaded compression helpers: ZSTDMT_sizeof_bufferPool, ZSTDMT_setBufferSize, ZSTDMT_sizeof_CCtxPool, ZSTDMT_getCCtx and ZSTDMT_releaseCCtx.

  4. Tests: tests/poolTests.c checks that POOL_create rejects an overflowing queueSize and numThreads and values above POOL_MAX_THREADS, that POOL_resize rejects (size_t)-1 and POOL_MAX_THREADS + 1, and that POOL_joinJobs(NULL) is safe. The new asserts and the NULL checks in zstdmt_compress.c have no dedicated tests.

Signed-off-by: Filippo Tedeschi filippotedeschi98@gmail.com

@meta-cla meta-cla Bot added the CLA Signed label Sep 29, 2026

@terrelln terrelln left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Lets put a much smaller limit on the number of threads, define it as a macro in the header. I'm thinking like 10K. No reason to approach 2^32.
  2. It is a programming error to pass a NULL ctx or function. So these could be asserts.

Validate parameters and context pointers in thread pool and multithreaded
compression helper routines:

1. Bound numThreads against POOL_MAX_THREADS (10,000) in POOL_create_advanced
and POOL_resize_internal, avoiding unbounded allocations. Prevent queueSize
overflow where queueSize plus one wraps to zero and causes modulo zero
operations in worker threads.

2. Enforce invariant asserts on ctx and function callback arguments in
POOL_add and POOL_tryAdd.

3. Add defensive NULL pointer checks in ZSTDMT buffer and context pool helper
functions including ZSTDMT_sizeof_bufferPool, ZSTDMT_setBufferSize,
ZSTDMT_sizeof_CCtxPool, ZSTDMT_getCCtx, and ZSTDMT_releaseCCtx.

4. Add unit test coverage in poolTests verifying invalid queue sizes and
thread counts exceeding POOL_MAX_THREADS fail safely.

Signed-off-by: Filippo Tedeschi <filippotedeschi98@gmail.com>
@filtede98
filtede98 force-pushed the fix-pool-overflow-and-null-guards branch from f6b173f to fa9cf08 Compare September 30, 2026 17:07
@filtede98

Copy link
Copy Markdown
Author

Thank you @terrelln for the review! Updated as requested:

  1. Defined POOL_MAX_THREADS as 10000 in pool.h and bound numThreads against it in POOL_create_advanced and POOL_resize_internal.
  2. Converted the checks on ctx and function in POOL_add and POOL_tryAdd into asserts (assert(ctx != NULL); assert(function != NULL);).
  3. Updated unit tests in poolTests.c to test the POOL_MAX_THREADS limit.

@ceedot-rock

Copy link
Copy Markdown

@Cyan4973 — the author addressed terrelln's review (POOL_MAX_THREADS 10000 cap, asserts in POOL_add/POOL_tryAdd) back on Sep 30 and it's been waiting on re-review since. terrelln's out for October — could you take the re-review?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants