Feature Proposal: Resumable Downloads (ResumableTask)

[o-nnerb]

Status

Early draft — posted here mainly so it's tracked against 4.5.0, not a finished design. The contract below is still being argued about internally; expect it to change before implementation starts. Feedback and alternative shapes welcome.

Overview

Goal: let a caller pause and resume a streaming download (DownloadTask's TaskResult<AsyncBytes>) using HTTP Range/If-Range, without adding a new concrete task type (BackgroundDownloadTask already covers OS-level background transfers on Darwin — this is a same-process, any-platform complement, not a replacement).

The mechanism piggybacks on infrastructure that already exists:

• RawTask._result(environment:) (Sources/RequestDL/Tasks/Sources/Raw Task/Raw/RawTask.swift) is the one place that resolves a Property tree into a RequestConfiguration and calls client.execute(...). It already mutates the resolved configuration in place for cross-cutting concerns (tracer header injection) and already runs environment-queued hooks (taskDescriptorContextHooks, backing .description(_:enabled:onDescribe:)) at that exact point. A resume hint (offset + validator) would be read the same way, right before client.execute.
• Internals.CacheControl.getUpdatedHeadersForCache (Sources/RequestDL/Tasks/Sources/Raw Task/Cache Control/Internals.CacheControl.swift) already does the closest analogue to what this needs: reissue a request with conditional headers (If-None-Match/If-Modified-Since), inspect the status code that comes back (304 vs. not), and branch. The resumable case is the same shape with Range/If-Range and 206/200/416 instead of 304.

So the proposal is: an extension on RequestTask where Element == TaskResult<AsyncBytes> returning a ResumableTask wrapper, not a new task type users declare their request under.

Proposed contract (rough)

extension RequestTask where Element == TaskResult<AsyncBytes> {
    func resumable() -> ResumableTask<Self>
}

extension ResumableTask {

    enum Resumption: Sendable {
        /// 206 — server honored Range. `payload` is only the missing tail; append it after
        /// whatever the caller already persisted at `offset`.
        case resumed(TaskResult<AsyncBytes>)

        /// 200 — server ignored Range/If-Range (no range support, or the validator no longer
        /// matched). `payload` is the entire resource from byte 0; caller discards whatever it
        /// already wrote.
        case restarted(TaskResult<AsyncBytes>)

        /// 416 — `offset` doesn't exist on the resource anymore. No payload; `head` usually
        /// carries `Content-Range: bytes */<total>` to compare against `offset`.
        case unsatisfiable(head: ResponseHead)
    }

    func resume(from offset: Int64, validator: Validator?) async throws -> Resumption
}

struct Validator: Sendable {
    static func etag(_ value: String) -> Validator
    static func lastModified(_ value: String) -> Validator
}

offset/validator are supplied by the caller on every resume(from:validator:) call, not tracked internally — only the caller knows how many bytes it actually persisted (wrote to disk, etc.), which can lag behind how many bytes the stream delivered.

No explicit pause(): pausing is the caller stopping/cancelling its own for try await consumption of AsyncBytes. Whether that actually tears down the underlying connection cleanly on both transports (NIO and URLSession) rather than leaving it dangling until a timeout is unverified — see open questions.

Open questions

• Should Resumption hide the byte bookkeeping and hand back one byte-continuous stream (merging old + new bytes internally), instead of leaving concatenation entirely to the caller? Raised and pushed back on, not settled.
• Does cancelling the caller's consuming Task actually release the connection on both the NIO and URLSession backends, or does ResumableTask need its own explicit teardown path?
• Worth adding typed RangeHeader/IfRangeHeader (alongside AcceptEncodingHeader, AcceptLanguageHeader, etc.) as public headers independent of resumable(), for callers who want to build the request by hand?
• ResumableTask only covers same-process resume. Surviving an app relaunch is BackgroundDownloadTask's territory today — should stay that way, or should this converge with it eventually?
• Naming (ResumableTask vs. alternatives), and whether it belongs as a method on RequestTask at all vs. a dedicated initializer.

Out of scope (v1)

• Persisting offset/validator across app restarts — caller-owned state, ResumableTask doesn't serialize anything itself.
• Anything requiring OS-level background transfer semantics — already BackgroundDownloadTask.

Milestone: 4.5.0

[o-nnerb]

Closing the loop on this proposal: it is implemented on main, for 4.5.0. It ended up with a different shape than the draft, so here is what became of each part.

What shipped

• Pausing on purpose: RequestController and .controller(_:) suspend and resume a request while its connection stays open, in both directions, on every executor (Add internal suspend/resume plumbing for transfers #378, Add RequestController and resumingDownloads #379).
• Carrying on after a lost connection: .resumingDownloads(_:) reconnects with Range/If-Range and splices the rest, so the caller sees one continuous stream (Add RequestController and resumingDownloads #379).
• Carrying on after the app was closed: DownloadResumptionPoint (Codable, versioned format) and .continuingDownload(from:whenChanged:), which hands back only the rest of the resource, with DownloadResumptionError for anything that isn't exactly that (Add continuingDownload(from:) to continue a download after the app was closed #382, Add whenChanged to continuingDownload(from:) to start over when the resource isn't the same #390).
• Observing it: RequestMonitor and .monitor(_:) (states, byte counts measured on the network); progress(...) is deprecated, not removed (Add RequestMonitor and deprecate progress #381).
• Beyond the proposal: resumable uploads with .resumingUploads(.ietf) or .resumingUploads(.tus) (Add the groundwork for resumable uploads (body from any offset, IETF and tus dialects) #383, Add the driver that makes an upload resumable over any executor #384, Add resumingUploads(_:) to make an upload resumable #388).

The open questions

1. Hide the byte bookkeeping? Both. .resumingDownloads hands back one continuous stream. .continuingDownload(from:) hands back only the rest, because the caller is the one who knows how many bytes it persisted.
2. Does cancelling the consuming Task release the connection on NIO and URLSession? Yes, and it is tested on both, with no teardown path of its own.
3. Typed RangeHeader/IfRangeHeader? Not done. The modifiers above build those headers themselves, and CustomHeader covers anyone building a request by hand.
4. Converge with BackgroundDownloadTask? Not in 4.5.0. It stays the operating system's transfer (Darwin only: the system keeps the file, and keeps downloading with the app closed), and this is the in-app, any-platform complement. Possible follow-ups, if there is demand: exposing URLSession's resumeData for a background download that fails or is cancelled (and accepting it back), and suspending, resuming and cancelling those downloads by id. There is no safe conversion between a DownloadResumptionPoint and URLSession's resume data, whose format is undocumented.
5. Naming: not a ResumableTask. These are modifiers on RequestTask, in the same style as the others.

What changed from the draft's contract

The draft's Resumption enum (resumed / restarted / unsatisfiable) became errors plus an option. By default a response that isn't the rest of the same resource fails with a DownloadResumptionError, and whenChanged: .restart asks again for the whole resource instead, with a 200 in the head where the rest has a 206.

The default fails because the result is a stream the caller appends to what it keeps: starting over without saying so would append the whole resource after a partial one. For reference, curl -C - also refuses ("Cannot resume", exit 33), while URLSession's downloadTask(withResumeData:) starts over without an error; I checked the latter against a local server that changes the ETag and against one that ignores Range. URLSession owns its partial file, which is why it can.

Out of scope (v1)

The library still stores no bytes and no partial file. What it does now is hand back the pieces, a Codable point and the rest of the resource, and leave the file to the caller.