Implement pipe and pipe2 on WASIp3#826
Open
alexcrichton wants to merge 3 commits into
Open
Conversation
This commit goes through the I/O subsystems of wasi-libc for WASIp3 (TCP, UDP, files, and stdio) and adds necessary internal locks and synchronization. None of this has been previously required because threading was not possible so it wasn't possible to have parallel operations issued, but with WASIp3 coop threads on the horizon this will soon be possible. The goal of this commit is to ensure that I/O objects all behave "reasonably" for operations issued in parallel. The general strategy here is to add a libc-internal lock-per-object. All operations on the object start by acquiring the lock and then end by releasing the lock. The main question to deal with is what to do with blocking operations where a thread blocked in `read` shouldn't block a sibling thread doing a `write`, for example. To handle this each blocking operation drops the object's `lock` before issuing the blocking operation. State is left behind to indicate that a blocking operation is in-progress, however. Another part to handle here is that calls to `poll` leave objects in "blocking I/O in progress" state while the call to `poll` is active as well. The general problems that need to be protected against are: 1. Component model primitives can only reside in one waitable-set at a time. This means that if a thread is blocked on a waitable-set with stream and another thread puts that stream into a different waitable-set then the first thread won't operate correctly any more. 2. Component model streams only support at most one active operation on a stream at a time. Once issued another thread can't start a new operation. 3. Blocking operations in the component model cannot be cancelled. For example a thread blocked in `read` can't get cancelled by a thread calling `close`. For wasi-libc this effectively gives rise to the constraint that once a thread is blocked in an I/O operation the object in question can't handle any other requests for I/O. For example one thread blocked in `read` means that another thread doing a nonblocking `read` basically can't be supported. This also happens for other issues like another thread doing `poll`, for example. For now these sorts of concurrent I/O requests return `EOPNOTSUPP` to at least indicate that wasi-libc doesn't support what's happening. I'm not sure if this is going to be a viable strategy long-term, but my hope is that this covers the vast majority of real-world use cases of concurrent operations on file descriptors. Note that parallel reads/writes on the same file descriptors are intentionally supported. This is a known use case in the wild for many libraries so this is explicitly allowed and handled within wasi-libc, but mostly just because it falls out that these operations are happening on separate streams.
alexcrichton
force-pushed
the
implement-pipe
branch
from
July 17, 2026 15:30
c833bd5 to
1aac071
Compare
This is expected to assist with porting applications to WASI as these are quite common syscalls. With WASIp3 these are also now able to be natively implemented. This commit fills out a simple-ish implementation which isn't focused on speed but is intended to at least be correct. One important note is that pipes aren't backed directly by `stream<u8>` as might otherwise be expected. This is to be able to implement buffering behavior which the component model does not support. Specifically applications frequently expect `write` on a pipe that's known to be empty to generally not block, and this implementation preserves that assumption. Internally, however, `stream` is still used. This enables `pipe` to get connected to `poll`, for example. Closes WebAssembly#726
alexcrichton
force-pushed
the
implement-pipe
branch
from
July 17, 2026 15:34
1aac071 to
8682d4b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is expected to assist with porting applications to WASI as these
are quite common syscalls. With WASIp3 these are also now able to be
natively implemented. This commit fills out a simple-ish implementation
which isn't focused on speed but is intended to at least be correct.
One important note is that pipes aren't backed directly by
stream<u8>as might otherwise be expected. This is to be able to implement
buffering behavior which the component model does not support.
Specifically applications frequently expect
writeon a pipe that'sknown to be empty to generally not block, and this implementation
preserves that assumption.
Internally, however,
streamis still used. This enablespipeto getconnected to
poll, for example.Closes #726