Add internal locks for I/O primitives #825
Open
alexcrichton wants to merge 2 commits into
Open
Conversation
This commit changes the `descriptor_table.h` internals, headers, etc, to reference-count the entries that are inserted. The goal of this work is to lay the foundation for WebAssembly#807. At this time there's no synchronization around the descriptor table for WASIp2/WASIp3 because there need not be, but with coop threads this will no longer be optional. For example it'll be possible for a thread to call `close` while another thread is blocked in `read` on that, and the result shouldn't be UB such as a trap. At the very least code needs to execute at the start of I/O operations to flag that certain descriptors are in-use, and code needs to run at the end of I/O operations to indicate that the descriptor is no longer in-use. Today that's not possible because descriptors are borrowed raw from the table with no modifications. Reference-counting descriptors is intended to match the underlying implementation on Unix-style operating systems which is intended to enable more natural implementations of eventual semantics for concurrent I/O operations. After this commit concurrent I/O ops are still not safe (e.g. may trap, cause UB, etc), and work will be done in subsequent commits to fix this for I/O subsystems. This commit starts out by adding a reference counter to the start of all I/O descriptions that are allocated. This is managed by `descriptor_table_*` functions with incs/decs and a deallocation when the count reaches 0. The table itself holds a strong reference count to all items, and then whenever something is looked up within the table it additionally gets a strong reference count returned. This is radically different from today's APIs where the `descriptor_table_entry_t*` points directly within the global table. The problem, then, is that all previous callers of `descriptor_table_get_ref` need to get updated to `descriptor_table_get` plus a deallocation routine. Personally I do not have faith in myself manually placing deallocation routines. What I've opted to do here is to raise the Clang baseline for WASIp2/WASIp3 targets to Clang 22. With Clang 22 the `-fdefer-ts` flag is now available and the `defer` keyword can be used, massively simplifying the decrement of the reference count path (and resource cleanup in general). I've not touched WASIp1 here mostly because I just don't want to ruffle feathers. If someone else wants to use `defer` for WASIp1 I think that's fine, but I'm not going to be the one to introduce it. The end result of this commit is that all I/O descriptions have a reference count and nothing borrows directly from the global I/O table itself. All I/O operations have a local strongly-owned copy of the description that they're working on. This now simplifies the problem of concurrent I/O to "figure it out within this object" as opposed to "is your object even valid?". I plan to do further work after this PR to audit various subsystems for modifications to I/O descriptions, add locks where appropriate, etc. Notably the descriptor table itself won't need locks in coop threading mode, just the I/O objects themselves will need internal synchronization/assertions/etc. One note to call out is that testing this behavior is kind of hard. The consequence of a leak (forgotten dec ref) is that nothing happens. Everything generally proceeds as expected, so there's no easy way to write a test that everything's actually cleaned up. I've opted to add a debug-mode-only assertion to libc itself on program exit (hooked via `crt1-command.c`) that closes all descriptors and then asserts that the only live allocations are the stdio descriptors. This is intended to at least work with wasi-libc's test suite for detecting leaks (and caught one during development). A final note is that this additionally opens up the door to implement `dup` on WASIp2/WASIp3 targets. I think this'll be a useful tidbit for POSIX compat (see WebAssembly#802) and will have a mostly trivial implementation after this PR lands.
This commit goes through the I/O subsystems of wasi-libc for WASIp3 (TCP, UDP, files, and stdio) and adds necessary internal locks and synchronization. None of this has been previously required because threading was not possible so it wasn't possible to have parallel operations issued, but with WASIp3 coop threads on the horizon this will soon be possible. The goal of this commit is to ensure that I/O objects all behave "reasonably" for operations issued in parallel. The general strategy here is to add a libc-internal lock-per-object. All operations on the object start by acquiring the lock and then end by releasing the lock. The main question to deal with is what to do with blocking operations where a thread blocked in `read` shouldn't block a sibling thread doing a `write`, for example. To handle this each blocking operation drops the object's `lock` before issuing the blocking operation. State is left behind to indicate that a blocking operation is in-progress, however. Another part to handle here is that calls to `poll` leave objects in "blocking I/O in progress" state while the call to `poll` is active as well. The general problems that need to be protected against are: 1. Component model primitives can only reside in one waitable-set at a time. This means that if a thread is blocked on a waitable-set with stream and another thread puts that stream into a different waitable-set then the first thread won't operate correctly any more. 2. Component model streams only support at most one active operation on a stream at a time. Once issued another thread can't start a new operation. 3. Blocking operations in the component model cannot be cancelled. For example a thread blocked in `read` can't get cancelled by a thread calling `close`. For wasi-libc this effectively gives rise to the constraint that once a thread is blocked in an I/O operation the object in question can't handle any other requests for I/O. For example one thread blocked in `read` means that another thread doing a nonblocking `read` basically can't be supported. This also happens for other issues like another thread doing `poll`, for example. For now these sorts of concurrent I/O requests return `EOPNOTSUPP` to at least indicate that wasi-libc doesn't support what's happening. I'm not sure if this is going to be a viable strategy long-term, but my hope is that this covers the vast majority of real-world use cases of concurrent operations on file descriptors. Note that parallel reads/writes on the same file descriptors are intentionally supported. This is a known use case in the wild for many libraries so this is explicitly allowed and handled within wasi-libc, but mostly just because it falls out that these operations are happening on separate streams.
Collaborator
Author
|
r? @TartanLlama |
This was referenced Jul 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This commit goes through the I/O subsystems of wasi-libc for WASIp3
(TCP, UDP, files, and stdio) and adds necessary internal locks and
synchronization. None of this has been previously required because
threading was not possible so it wasn't possible to have parallel
operations issued, but with WASIp3 coop threads on the horizon this will
soon be possible. The goal of this commit is to ensure that I/O objects
all behave "reasonably" for operations issued in parallel.
The general strategy here is to add a libc-internal lock-per-object. All
operations on the object start by acquiring the lock and then end by
releasing the lock. The main question to deal with is what to do with
blocking operations where a thread blocked in
readshouldn't block asibling thread doing a
write, for example. To handle this eachblocking operation drops the object's
lockbefore issuing the blockingoperation. State is left behind to indicate that a blocking operation is
in-progress, however. Another part to handle here is that calls to
pollleave objects in "blocking I/O in progress" state while the callto
pollis active as well.The general problems that need to be protected against are:
time. This means that if a thread is blocked on a waitable-set with
stream and another thread puts that stream into a different
waitable-set then the first thread won't operate correctly any more.
a stream at a time. Once issued another thread can't start a new
operation.
example a thread blocked in
readcan't get cancelled by a threadcalling
close.For wasi-libc this effectively gives rise to the constraint that once a
thread is blocked in an I/O operation the object in question can't
handle any other requests for I/O. For example one thread blocked in
readmeans that another thread doing a nonblockingreadbasicallycan't be supported. This also happens for other issues like another
thread doing
poll, for example. For now these sorts of concurrent I/Orequests return
EOPNOTSUPPto at least indicate that wasi-libc doesn'tsupport what's happening. I'm not sure if this is going to be a viable
strategy long-term, but my hope is that this covers the vast majority of
real-world use cases of concurrent operations on file descriptors.
Note that parallel reads/writes on the same file descriptors are
intentionally supported. This is a known use case in the wild for many
libraries so this is explicitly allowed and handled within wasi-libc,
but mostly just because it falls out that these operations are happening
on separate streams.