fix(retry): avoid replaying unread body data - #4621
Merged
Merged
Conversation
cprayer
force-pushed
the
fix/replay-unpolled-body
branch
from
September 10, 2026 15:25
efad888 to
858dfc4
Compare
Contributor
|
hey @cprayer, could you rebase your branch? it LGTM |
anthoturc
approved these changes
Sep 18, 2026
Retrying a body that was dropped before its first poll leaves an empty replay buffer. The retry reads and records the first DATA frame, then replays that same frame again on its next poll. Consume the empty replay phase before reading the original body, while preserving the existing capped-buffer error path. The new regression test failed with duplicated bytes before the fix. All 24 retry tests, all-target clippy, and formatting checks pass. Refs linkerd/linkerd2#15649 Signed-off-by: Taemin Shin <cprayer13@gmail.com>
cprayer
force-pushed
the
fix/replay-unpolled-body
branch
from
September 18, 2026 16:31
858dfc4 to
dfe642f
Compare
Contributor
Author
|
@anthoturc Thanks! Rebased onto the latest main |
Contributor
|
also for other reviewers, i was able to throw a slightly simpler version of the repro that @cprayer provided and confirm that the expected retry behavior is working properly. kubectl -n replay-repro logs -f deploy/server -c server
2026/09/18 17:57:19 listening on :8080 (h2c)
2026/09/18 17:57:19 req 0: attempt 1, replying 503 without reading the body
2026/09/18 17:57:20 req 0: attempt 2, read 11 bytes "hello world" -- OK
2026/09/18 17:57:25 req 1: attempt 1, replying 503 without reading the body
2026/09/18 17:57:26 req 1: attempt 2, read 11 bytes "hello world" -- OK
2026/09/18 17:57:31 req 2: attempt 1, replying 503 without reading the body
2026/09/18 17:57:32 req 2: attempt 2, read 11 bytes "hello world" -- OK
2026/09/18 17:57:37 req 3: attempt 1, replying 503 without reading the body
2026/09/18 17:57:38 req 3: attempt 2, read 11 bytes "hello world" -- OK
2026/09/18 17:57:43 req 4: attempt 1, replying 503 without reading the body
2026/09/18 17:57:44 req 4: attempt 2, read 11 bytes "hello world" -- OK
2026/09/18 17:57:49 req 5: attempt 1, replying 503 without reading the body
2026/09/18 17:57:50 req 5: attempt 2, read 11 bytes "hello world" -- OKWith the fix applied, we can see that the server is only getting the original body as opposed to the duplicate (which would have lead to 22 bytes being read) |
raykroeker
approved these changes
Sep 18, 2026
raykroeker
left a comment
Member
There was a problem hiding this comment.
Thanks @cprayer and @anthoturc !
This was referenced Sep 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes linkerd/linkerd2#15649
Background
When a retry starts with an empty replay buffer,
replay_bodyremains set while the original body is polled. The first DATA chunk is forwarded and buffered, then replayed on the next poll within the same attempt.Changes
replay_bodywhen the replay buffer is empty and uncapped, before polling the original bodyValidation
a66af811passes all 25 retry crate testsAI tools assisted with the investigation, reproducer, and PR description.