Repository navigation
Conversation
terrelln
requested changes
Sep 30, 2026
terrelln
left a comment
Contributor
There was a problem hiding this comment.
- Lets put a much smaller limit on the number of threads, define it as a macro in the header. I'm thinking like 10K. No reason to approach 2^32.
- It is a programming error to pass a
NULLctxorfunction. So these could be asserts.
Validate parameters and context pointers in thread pool and multithreaded compression helper routines: 1. Bound numThreads against POOL_MAX_THREADS (10,000) in POOL_create_advanced and POOL_resize_internal, avoiding unbounded allocations. Prevent queueSize overflow where queueSize plus one wraps to zero and causes modulo zero operations in worker threads. 2. Enforce invariant asserts on ctx and function callback arguments in POOL_add and POOL_tryAdd. 3. Add defensive NULL pointer checks in ZSTDMT buffer and context pool helper functions including ZSTDMT_sizeof_bufferPool, ZSTDMT_setBufferSize, ZSTDMT_sizeof_CCtxPool, ZSTDMT_getCCtx, and ZSTDMT_releaseCCtx. 4. Add unit test coverage in poolTests verifying invalid queue sizes and thread counts exceeding POOL_MAX_THREADS fail safely. Signed-off-by: Filippo Tedeschi <filippotedeschi98@gmail.com>
filtede98
force-pushed
the
fix-pool-overflow-and-null-guards
branch
from
September 30, 2026 17:07
f6b173f to
fa9cf08
Compare
Author
|
Thank you @terrelln for the review! Updated as requested:
|
|
@Cyan4973 — the author addressed terrelln's review (POOL_MAX_THREADS 10000 cap, asserts in POOL_add/POOL_tryAdd) back on Sep 30 and it's been waiting on re-review since. terrelln's out for October — could you take the re-review? |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This patch strengthens input validation in the thread pool (lib/common/pool.c) and adds NULL checks in the multithreaded compression helpers (lib/compress/zstdmt_compress.c).
Bounds in the thread pool:
POOL_create_advancedandPOOL_resize_internalrejectnumThreadsabovePOOL_MAX_THREADS(10000, defined inpool.h), as suggested in review.POOL_create_advancedrejectsqueueSize >= (size_t)-1 / sizeof(POOL_job). Without it,queueSize + 1can wrap around, and the worker and dispatch routines then do% ctx->queueSizewith a zero queue size.Pointer checks in the thread pool:
POOL_joinJobsreturns early whenctxis NULL, likePOOL_freeandPOOL_sizeof.POOL_addandPOOL_tryAddnow assert thatfunctionis non-NULL, in both the multithreaded and single-threaded implementations. The assert onctxwas already there.NULL checks in the multithreaded compression helpers:
ZSTDMT_sizeof_bufferPool,ZSTDMT_setBufferSize,ZSTDMT_sizeof_CCtxPool,ZSTDMT_getCCtxandZSTDMT_releaseCCtx.Tests:
tests/poolTests.cchecks thatPOOL_createrejects an overflowingqueueSizeandnumThreadsand values abovePOOL_MAX_THREADS, thatPOOL_resizerejects(size_t)-1andPOOL_MAX_THREADS + 1, and thatPOOL_joinJobs(NULL)is safe. The new asserts and the NULL checks inzstdmt_compress.chave no dedicated tests.Signed-off-by: Filippo Tedeschi filippotedeschi98@gmail.com