Run: request_retry sends a failed model request again after a wait

[deepfates]

Why

When a provider is down for minutes (a 502 with the connection refused upstream, a 429 that lasts), every request a run makes fails. The retries Imp already makes don't help, because they all come within a second or two of the failure: ReqLLM's own retry, and ReActV2's one more step or last request. A host whose turns have a deadline of several minutes would rather wait and try again inside the same run than fail at once.

What

Imp.Run.start/3 gets a new option, request_retry: [waits: [ms, ...], reserve: ms]. It is off unless given.

• When it retries: inside Imp.LM.request/2, if a request fails with an error that Imp.Errors.retryable?/1 accepts (408, 425, 429, 5xx, or a failed connection), it is sent again after the next wait.
• With a deadline: the last wait repeats, as long as the run's Imp.Deadline would still have reserve left after the wait (60 s by default). Otherwise the failure is returned as before.
• Without a deadline: the list is the bound, so a run can't wait forever.
• What the record shows: each attempt is its own :model_request and :model_response, with its own model_call_id. A failed response that is followed by another attempt carries retry_in_ms in its metadata. Anyone reading the run sees the failure, the wait and the next attempt.
• What the caller sees: only the last attempt's result.
• Streamed requests (Imp.stream/3 through Imp.LM.record/3) are not sent again, because their chunks may already have reached the caller.
• How the policy reaches the request: it lives on the run's control process. Imp.LM.request/2 asks for it only after a request has failed, so a request that succeeds costs nothing extra.
• What counts as retryable: this reuses Imp.Errors.retryable?/1 and adds no new classification.

After review

• Only the outermost request retries. A process-dict flag is set while a request that may retry is in flight. Without it, an LM that wraps an LM (an optimizer budget, a rollout LM, a host wrapper calling Imp.LM.generate/3 inside generate/3) would turn k retries into (k+1)² attempts.
• retry-after is honoured. The wait is now max(wait, retry-after), still under the deadline and reserve check. The header is read with the parser Imp.Clients.ReqLLMBatch already uses, exposed as retry_after_ms/1.
• waits must be a non-empty list. An empty list under a deadline raised an ArithmeticError; it is now refused when the run starts.
• Doc line on double billing. A timed-out attempt may have been finished and billed by the provider with no response to record, so a retry can be billed twice. This is more likely without a deadline.

Tests

test/run_request_retry_test.exs covers eight cases (the first five here; the three after review are a wrapper LM, refusing empty waits, and a 429 with retry-after: 1 waiting 1000 ms):

1. A run whose provider fails twice and then answers succeeds. There are three model calls, the two failures carry retry_in_ms 10 and 30, and the third response has no retry_in_ms.
2. Without the option, nothing is retried.
3. A 400 (not retryable) is not retried.
4. Without a deadline, the list is the bound.
5. With a deadline, the last wait repeats and stops while the reserve is still left.

mix check passes: 3718 tests, 0 failures. mix quality.check and mix dialyzer.check also pass. The CHANGELOG has an entry under Unreleased, and decisions.md has a new row.

How a host uses it

A host that runs agent turns under a deadline of several minutes can pass, for example, request_retry: [waits: [15_000, 45_000, 120_000, 240_000], reserve: 60_000]. A provider outage then costs a few requests per turn instead of the turn itself.