Skip to content

chore(curl): rebase Moli bindings onto upstream 0.4.51 and libcurl 8.23 - #2

Closed
ldm0 wants to merge 36 commits into
feat/prerequest-callbackfrom
followup/upstream-0.4.51
Closed

ldm0 wants to merge 36 commits into
feat/prerequest-callbackfrom
followup/upstream-0.4.51

Conversation

@ldm0

@ldm0 ldm0 commented Oct 18, 2026 •

Copy link
Copy Markdown
Member

Rebase the Moli fork patches onto curl-rust upstream 2fb602bb1d3a34ded4548cbe8a7c7202b1ab4ba2 (curl 0.4.51 / curl-sys 0.4.91), then bundle the released libcurl 8.23.0.

This is a separate follow-up to #1. The existing prereq branch and PR are unchanged. The rebased history retains Moli's Brotli, native WebSocket, proxy-header, byte-header, AWS-LC and pre-request support while preserving upstream's Multi handle lifetime and error-buffer fixes. Upstream now supplies proxy-header ownership and the apple-sectrust feature, so the overlapping fork code is removed.

libcurl 8.23 includes curl/curl#22902 (9e6673fb7d55b40d18383f49f465f1bd1cfce1ee); compile its conncache.c directly and delete the generated eviction patch. Port the build file list and IPv6 define. The HTTP/1.0 WebSocket upgrade and bounded receive patches remain necessary; the receive adaptation flushes deferred PONGs before awaiting more input. Existing regressions cover these paths.

Use the upstream Apple SecTrust feature and retain explicit SSL_CERT_FILE / SSL_CERT_DIR overrides, including reset. Add an isolated macOS invalid-CA override regression and include the feature in the Linux/macOS/Windows workflow.

Validation on Linux:

  • cargo fmt --all
  • curl unit/integration tests: 80 passed, 1 ignored; includes 16 WebSocket and 7 prereq tests
  • Native ABI: 3,199 checks passed
  • Doctests: 15 passed; atexit executable passed
  • curl Clippy passed with existing upstream warnings (not a warnings-as-errors check)

GitHub CI passed on Linux, macOS and Windows, including the Apple-specific regressions and the native ABI step: https://github.com/lexmount/curl-rust/actions/runs/39154343129

After rebasing, an ancestry-only merge retains the original prereq branch as an ancestor so this follow-up is directly mergeable. The source tree is identical to implementation commit 9d40ca9.

bjorn3 and others added 30 commits May 1, 2026 09:22
* Allow using windows-sys 0.61

This version no longer requires using big import libraries and instead
uses raw-dylib.

* Add missing advapi32 and secur32 dependencies

These import libraries previously got implicitly included through the
windows-targets crate.
* Update curl to 8.20.0

Changelog: https://curl.se/ch/8.20.0.html
Release blog: https://daniel.haxx.se/blog/2026/04/29/curl-8-20-0/

This fixes 8 CVEs.

Note that the next release will likely remove NTLM, so we'll need to
figure out how to deal with that when that happens.

* Update MSRV to 1.85

Latest rustls requires a newer version.
Seeing if linking another library helps...
Implements proxy_headers method to add headers only to the proxy.
* Consolidate some dependency versions into workspace

Consolidate some repeated dependency versions into workspace
dependencies so that they can be more easily kept in sync across `curl`
and `curl-sys` and with less repetition.

* Workspace membership
The curl crate needs a change for CURLOPT_PROXYHEADER.
Changelog: https://curl.se/ch/8.21.0.html
Release blog: https://daniel.haxx.se/blog/2026/06/24/curl-8-21-0/

This fixes 18 CVEs.

Note that 8.22 is being deferred until September.
Alex has transferred primary maintainership of curl-rust to me, which
changed the project URL in GitHub. Update the links in the README and
manifests to the new location. Also add myself to the manifest authors
so that it doesn't seem too weird whenever I publish the next version
(see sagebind#672).
Resolve issues in the `bump` branch created by @alexcrichton.

Resolves sagebind#662.

---------

Co-authored-by: Alex Crichton <alex@alexcrichton.com>
Co-authored-by: Stephen M. Coakley <me@stephencoakley.com>
Resolves sagebind#669.

```sh
$ cargo test --features apple-sectrust
   Compiling curl-sys v0.4.90+curl-8.21.0 (/Users/khiettamnguyen/nktnet/forks/curl-rust/curl-sys)
   Compiling curl v0.4.50 (/Users/khiettamnguyen/nktnet/forks/curl-rust)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 7.67s
     Running unittests src/lib.rs (target/debug/deps/curl-6e29ce0a7645e901)

running 4 tests
test easy::handler::double_seconds_to_duration_sub_second1 ... ok
test easy::handler::double_seconds_to_duration_sub_second2 ... ok
test easy::handler::double_seconds_to_duration_whole_second ... ok
test tests::is_initialized_before_main ... ok

test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/apple_sectrust.rs (target/debug/deps/apple_sectrust-c919398c06f98bc5)

running 1 test
test bundled_build_enables_apple_sectrust ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/atexit.rs (target/debug/deps/atexit-3a96f026124f4a9f)
Finishing...
     Running tests/easy.rs (target/debug/deps/easy-fce254703c9a5ff9)

running 37 tests
test dns_servers ... ok
test check_unix_socket ... ok
test download_zero_size ... ok
test misc ... ok
test noproxy ... ignored
test abort_read ... ok
test custom_headers ... ok
test get_smoke ... ok
test accept_encoding ... ok
test get_path ... ok
test cookie ... ok
test download_nonzero_size ... ok
test getters ... ok
test headers ... ok
test follow_location ... ok
test pause_write_then_resume ... ok
test path_as_is ... ok
test perform_in_perform_is_bad ... ok
test post2 ... ok
test post1 ... ok
test fail_on_error ... ok
test post3 ... ok
test port ... ok
test test_connect_timeout ... ok
test panic_in_callback - should panic ... ok
test progress ... ok
test test_timeout ... ok
test proxy ... ok
test url_encoding ... ok
test resolve ... ok
test put ... ok
test referer ... ok
test upload_nonzero_size ... ok
test upload_zero_size ... ok
test useragent ... ok
test userpass ... ok
test write_callback ... ok

test result: ok. 36 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.01s

     Running tests/multi.rs (target/debug/deps/multi-d9ab970d83882b9d)

running 3 tests
test smoke ... ok
test smoke2 ... ok
test upload_lots ... ok

test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s

     Running tests/post.rs (target/debug/deps/post-8d66756d76e214c9)

running 3 tests
test file ... ok
test custom ... ok
test buffer ... ok

test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s

     Running tests/protocols.rs (target/debug/deps/protocols-6f686c2e1927b2ef)

running 1 test
test static_with_ftp_disabled ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests curl

running 15 tests
test src/easy/handler.rs - easy::handler::Easy2<H>::resolve (line 2083) - compile ... ok
test src/easy/handle.rs - easy::handle::Easy::read_function (line 303) - compile ... ok
test src/easy/handle.rs - easy::handle::Easy::read_function (line 288) - compile ... ok
test src/lib.rs - (line 18) - compile ... ok
test src/easy/handle.rs - easy::handle::Easy (line 41) ... ok
test src/easy/handler.rs - easy::handler::Easy2<H>::http_headers (line 1580) ... ok
test src/easy/handle.rs - easy::handle::Easy::write_function (line 231) ... ok
test src/easy/handle.rs - easy::handle::Easy (line 33) ... ok
test src/easy/handle.rs - easy::handle::Easy::header_function (line 501) ... ok
test src/easy/handle.rs - easy::handle::Easy (line 57) ... ok
test src/easy/handle.rs - easy::handle::Easy::header_function (line 517) ... ok
test src/easy/handle.rs - easy::handle::Easy::write_function (line 217) ... ok
test src/multi.rs - multi::Multi::wait (line 598) ... ok
test src/easy/handler.rs - easy::handler::Easy2 (line 357) ... ok
test src/easy/handler.rs - easy::handler::Handler (line 32) ... ok

test result: ok. 15 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.72s

curl-rust(apple-sectrust-tls)$ gs
On branch apple-sectrust-tls
nothing to commit, working tree clean
curl-rust(apple-sectrust-tls)$ gp
Enumerating objects: 26, done.
Counting objects: 100% (26/26), done.
Delta compression using up to 12 threads
Compressing objects: 100% (16/16), done.
Writing objects: 100% (16/16), 2.60 KiB | 2.60 MiB/s, done.
Total 16 (delta 12), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (12/12), completed with 9 local objects.
remote: 
remote: Create a pull request for 'apple-sectrust-tls' on GitHub by visiting:
remote:      https://github.com/nktnet1/curl-rust/pull/new/apple-sectrust-tls
remote: 
To github.com:nktnet1/curl-rust.git
 * [new branch]      apple-sectrust-tls -> apple-sectrust-tls
curl-rust(apple-sectrust-tls)$ 
curl-rust(apple-sectrust-tls)$ cargo test --features apple-sectrust
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.13s
     Running unittests src/lib.rs (target/debug/deps/curl-6e29ce0a7645e901)

running 4 tests
test easy::handler::double_seconds_to_duration_sub_second2 ... ok
test easy::handler::double_seconds_to_duration_sub_second1 ... ok
test tests::is_initialized_before_main ... ok
test easy::handler::double_seconds_to_duration_whole_second ... ok

test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/apple_sectrust.rs (target/debug/deps/apple_sectrust-c919398c06f98bc5)

running 1 test
test bundled_build_enables_apple_sectrust ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/atexit.rs (target/debug/deps/atexit-3a96f026124f4a9f)
Finishing...
     Running tests/easy.rs (target/debug/deps/easy-fce254703c9a5ff9)

running 37 tests
test dns_servers ... ok
test check_unix_socket ... ok
test abort_read ... ok
test custom_headers ... ok
test noproxy ... ignored
test misc ... ok
test download_nonzero_size ... ok
test get_smoke ... ok
test download_zero_size ... ok
test accept_encoding ... ok
test cookie ... ok
test getters ... ok
test get_path ... ok
test headers ... ok
test follow_location ... ok
test path_as_is ... ok
test pause_write_then_resume ... ok
test fail_on_error ... ok
test perform_in_perform_is_bad ... ok
test test_connect_timeout ... ok
test test_timeout ... ok
test panic_in_callback - should panic ... ok
test url_encoding ... ok
test post2 ... ok
test progress ... ok
test port ... ok
test post3 ... ok
test post1 ... ok
test proxy ... ok
test resolve ... ok
test put ... ok
test referer ... ok
test upload_nonzero_size ... ok
test upload_zero_size ... ok
test useragent ... ok
test userpass ... ok
test write_callback ... ok

test result: ok. 36 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.01s

     Running tests/multi.rs (target/debug/deps/multi-d9ab970d83882b9d)

running 3 tests
test smoke ... ok
test smoke2 ... ok
test upload_lots ... ok

test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s

     Running tests/post.rs (target/debug/deps/post-8d66756d76e214c9)

running 3 tests
test buffer ... ok
test custom ... ok
test file ... ok

test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s

     Running tests/protocols.rs (target/debug/deps/protocols-6f686c2e1927b2ef)

running 1 test
test static_with_ftp_disabled ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests curl

running 15 tests
test src/easy/handler.rs - easy::handler::Easy2<H>::resolve (line 2083) - compile ... ok
test src/easy/handle.rs - easy::handle::Easy::read_function (line 303) - compile ... ok
test src/easy/handle.rs - easy::handle::Easy::read_function (line 288) - compile ... ok
test src/lib.rs - (line 18) - compile ... ok
test src/easy/handle.rs - easy::handle::Easy (line 57) ... ok
test src/easy/handler.rs - easy::handler::Easy2<H>::http_headers (line 1580) ... ok
test src/easy/handle.rs - easy::handle::Easy (line 41) ... ok
test src/easy/handle.rs - easy::handle::Easy::write_function (line 231) ... ok
test src/easy/handle.rs - easy::handle::Easy (line 33) ... ok
test src/multi.rs - multi::Multi::wait (line 598) ... ok
test src/easy/handle.rs - easy::handle::Easy::write_function (line 217) ... ok
test src/easy/handle.rs - easy::handle::Easy::header_function (line 501) ... ok
test src/easy/handler.rs - easy::handler::Handler (line 32) ... ok
test src/easy/handle.rs - easy::handle::Easy::header_function (line 517) ... ok
test src/easy/handler.rs - easy::handler::Easy2 (line 357) ... ok

test result: ok. 15 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.91s
```
…agebind#595)

Fixes sagebind#589. `RefCell`'s interface cannot be soundly used for this
purpose because libcurl needs a long-lived mutable pointer to the error
buffer.

For users, this PR should not have any visible effect except for slight
changes in performance perhaps. The auto trait implementations
(`Send`/`!Sync`/`Freeze`) of `Easy2` do not change.

Co-authored-by: Stephen M. Coakley <me@stephencoakley.com>
…nd#645)

Fix a potential segfault that can occur when dropping a `Multi` before
dropping an `Easy2Handle` that is attached to the multi handle and has
an active transfer. This can happen because dropping a `Multi` will also
always drop its inner `MultiData`, which holds socket and timer callback
functions supplied by the user, but the underlying `CURLM` handle is
_not_ dropped until both the `Multi` and all of its attached
`Easy2Handle`s are dropped. If the `Multi` is dropped first, then the
socket and timer callback functions will be dropped while the `CURLM`
handle is still active, which can lead to a segfault if the callbacks
are invoked after they have been dropped.

The libcurl docs have [this to say about
`CURLMOPT_SOCKETFUNCTION`](https://curl.se/libcurl/c/CURLMOPT_SOCKETFUNCTION.html):

> This callback may get invoked at any time when interacting with
libcurl. This may even happen after all transfers are done and is
*likely* to happen *during* a call to curl_multi_cleanup when cached
connections are shut down.

This paragraph is clear that libcurl _may_ invoke our
`CURLMOPT_SOCKETFUNCTION` callback when `curl_multi_cleanup` is invoked,
which in our case, is done by `RawMulti::drop` which might be called
after we've already freed our callback function.

To fix this, use `Rc` and `RefCell` to extend the life of the configured
callbacks until both the `Multi` and all attached `Easy2Handle`s are
dropped, avoiding the need to unset the callbacks until all socket
activity is done. (`MultiWaker` still theoretically may cause the
callbacks to be invoked, so we still unset them.)

Generally I would avoid `RefCell` as a way of solving lifetime problems,
but in this case it seems like `RefCell` models libcurl's actual
mutability contract decently well, so I think it works.

---

This issue can be easily reproduced with the following program, which
ends in a `SIGSEGV` 100% of the time on my machine:

```rust
use curl::{easy::Easy, multi::Multi};
use std::time::Duration;

fn main() {
    // Set up basically any request.
    let mut easy = Easy::new();
    easy.url("https://example.org").unwrap();

    // Set up a multi with any socket callback.
    let mut multi = Multi::new();
    multi.socket_function(|_, _, _| {}).unwrap();

    // Attach the easy handle to the multi handle.
    let handle = multi.add(easy).unwrap();

    // Execute the initial steps of establishing a socket, but without waiting
    // for the request to fully finish.
    multi.perform().unwrap();
    multi.wait(&mut [], Duration::from_secs(1)).unwrap();
    multi.perform().unwrap();

    // Drop the multi handle. This will free the callback functions, leaving
    // curl with dangling pointers.
    drop(multi);

    // Segfault when curl tries to call the socket callback.
    drop(handle);
}
```

Thanks to @joelposti for reporting this issue on
sagebind/isahc#459 and supplying a larger
reproduction of the issue with detailed steps, which made it much easier
for me to find the root cause here.
Set up an Actions workflow to automate the process of creating an
initial PR to bump the submodule whenever a new version of curl is
released.

This isn't going to do the work for us if there are build changes, but
at least it is a starting point and also notifies us when a new release
is ready.
`timeout_i32` has an off-by-one in its clamp check. It compares the
whole
seconds against `i32::MAX / 1000` using `>`, so a duration of exactly
2147483 seconds skips the clamp and goes into the arithmetic branch. If
the
sub-second part is 648ms or more, `secs * 1000 + millis` ends up past
`i32::MAX`.

In debug builds this panics with "attempt to add with overflow". In
release
builds it wraps around to a negative number instead, and libcurl's
`multi_wait()` rejects negative timeouts, so `Multi::wait()` and
`Multi::poll()` return `CURLM_BAD_FUNCTION_ARGUMENT` instead of waiting.

This changes it to work out the timeout in milliseconds first and then
clamp
it to `i32::MAX`, so there's no intermediate value that can overflow. I
also
added a unit test for the boundary values and for `Duration::MAX`.

Fixes sagebind#659

Co-authored-by: Stephen M. Coakley <me@stephencoakley.com>
Decode after one bounded socket read instead of slurping into the soft-limit receive queue. This lets callers apply backpressure before libcurl buffers an entire incoming stream.

Add a socket-level regression for one-byte receives, and document the bundled-libcurl patch. WebSocket tests passed 20 stress iterations; local tests, doctests, atexit and 3133 ABI checks passed.
ldm0 added 6 commits October 19, 2026 02:18
… patch

Use the released curl-8_23_0 source, including curl/curl#22902, instead of generating conncache.c. Port the build source list and IPv6 define. Keep the HTTP/1.0 upgrade and bounded receive fixes, flushing deferred PONGs before awaiting more input.
Use upstream apple-sectrust feature selection, preserving explicit SSL_CERT_FILE and SSL_CERT_DIR handling without probing automatic certificate paths. Exercise the native feature in the platform matrix and check invalid certificate overrides across initialization and reset in an isolated macOS test process.
Keep the existing mock-server check compatible with current Clippy and report all missing headers together.
…stream

The branch tree already includes the old fork patches rebased onto curl-rust 0.4.51, with duplicate implementations replaced by upstream code. Retain the previous branch as an ancestor so this separate follow-up is mergeable and reviews show only the upgrade delta.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants