Description
The connectors codebase has three hand-written copies of the same retry skeleton: HttpRetryMiddleware::handle and check_connectivity_with_retry in core/connectors/sdk/src/retry.rs, and DorisSink::load_batch in core/connectors/sinks/doris_sink/src/lib.rs.
The Doris sink cannot reuse the middleware: Doris Stream Load signals failure as HTTP 200 with {"Status":"Fail"} in the body, and the middleware never inspects response bodies. Any connector whose backend reports errors in-band has the same problem, so each one ends up hand-rolling the loop.
The copies have already diverged on backoff convention: the SDK's two loops pass the already-incremented attempt to exponential_backoff, so their first retry waits 2 * base, while the Doris sink passes attempt - 1 so the first retry waits the configured base delay, matching the documented meaning of retry_delay.
Raised as a non-blocking follow-up in the review of #3574: #3574 (comment)
Affected area / component
Connectors
Proposed solution
Expose a generic retry_async(operation, policy) in iggy_connector_sdk::retry that owns attempt counting, the transient-error predicate, capped exponential backoff with jitter, and per-retry logging, then collapse the three call sites onto it.
The shared helper should keep the base-delay-first convention (base * 2^(k-1) for retry k), since that is what the retry_delay config field documents.
Alternatives considered
- Reuse
HttpRetryMiddleware in the Doris sink: not possible, the middleware classifies on status codes and never inspects response bodies, so an in-band {"Status":"Fail"} looks like success to it.
- Keep per-connector loops: the exponent divergence above shows the conventions drift as soon as the skeleton is copied.
Contribution
Good first issue
Description
The connectors codebase has three hand-written copies of the same retry skeleton:
HttpRetryMiddleware::handleandcheck_connectivity_with_retryincore/connectors/sdk/src/retry.rs, andDorisSink::load_batchincore/connectors/sinks/doris_sink/src/lib.rs.The Doris sink cannot reuse the middleware: Doris Stream Load signals failure as HTTP 200 with
{"Status":"Fail"}in the body, and the middleware never inspects response bodies. Any connector whose backend reports errors in-band has the same problem, so each one ends up hand-rolling the loop.The copies have already diverged on backoff convention: the SDK's two loops pass the already-incremented attempt to
exponential_backoff, so their first retry waits2 * base, while the Doris sink passesattempt - 1so the first retry waits the configured base delay, matching the documented meaning ofretry_delay.Raised as a non-blocking follow-up in the review of #3574: #3574 (comment)
Affected area / component
Connectors
Proposed solution
Expose a generic
retry_async(operation, policy)iniggy_connector_sdk::retrythat owns attempt counting, the transient-error predicate, capped exponential backoff with jitter, and per-retry logging, then collapse the three call sites onto it.The shared helper should keep the base-delay-first convention (
base * 2^(k-1)for retryk), since that is what theretry_delayconfig field documents.Alternatives considered
HttpRetryMiddlewarein the Doris sink: not possible, the middleware classifies on status codes and never inspects response bodies, so an in-band{"Status":"Fail"}looks like success to it.Contribution
Good first issue