fix(test-optimization): bound payload delivery lifecycle - #10044
fix(test-optimization): bound payload delivery lifecycle#10044juan-fernandez wants to merge 12 commits into
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c336b20ce6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
c336b20 to
e3736d0
Compare
Overall package sizeSelf size: 8.65 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.4.0 | 127.33 kB | 447.04 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
|
@codex review |
🎉 All green!🧪 All tests passed 🔄 Datadog retried 1 test - 1 passed on retry 🎯 Code Coverage (details) 🔗 Commit SHA: aba4765 | Docs | View more details | Give us feedback! |
![datadog-prod-us1-4[bot]](https://avatars.githubusercontent.com/u/365230?s=60&v=4){kind=small}
There was a problem hiding this comment.
The Agent fallback path still discards Test Optimization payloads during exporter backpressure. Readable request bodies also bypass the 64 MiB queue limit while the request layer buffers them.
🤖 Datadog Autotest · Commit c336b20 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
|
Codex Review: Didn't find any major issues. Another round soon, please! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
BenchmarksBenchmark execution time: 2026-09-01 15:59:09 Comparing candidate commit aba4765 in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 2296 metrics, 14 unstable metrics.
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #10044 +/- ##
==========================================
- Coverage 98.63% 98.48% -0.15%
==========================================
Files 996 997 +1
Lines 152198 152501 +303
Branches 12857 12830 -27
==========================================
+ Hits 150116 150191 +75
- Misses 2082 2310 +228
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9357c29e5f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2862369d20
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
A 1,000-log batch benchmark improved from 91-93 ms to 57-60 ms across two fresh runs (about 37% faster).
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ebf4b146be
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| retryTimer = setTimeout(attempt, Math.min(50, remaining), attemptIndex) | ||
| if (!commonRequest.writable) { | ||
| waitingForBackpressure = true | ||
| retryTimer = setTimeout(attempt, Math.min(BACKPRESSURE_RETRY_MS, remaining), attemptIndex) |
There was a problem hiding this comment.
When flushInterval === 0, BufferingExporter._export starts a request for every append, and a saturated common transport makes each retained request create and repeatedly reschedule its own 50 ms timer here. The byte cap does not bound request count or timer/closure overhead, so a burst of small traces or logs can accumulate tens of thousands of polling timers and cause a CPU/memory spike while the transport remains blocked; cap queued request count or use a shared backpressure waiter.
AGENTS.md reference: AGENTS.md:L194-L200
Useful? React with 👍 / 👎.
Cap screenshot retries even during finalization, let ready Test Optimization responses beat creation-time timeouts, keep owned agent-discovery retries alive, stabilize Playwright dynamic-name fixtures, and isolate nested integration runs from incomplete GitHub Actions metadata.
1 participant
What does this PR do?
Makes Test Optimization payload delivery more reliable when tests produce data faster than the intake can receive it.
error_typeand HTTPstatus_codewhen delivery fails.Only Test Optimization opts into retaining payloads while the transport is busy. Other products keep their existing behavior.
Motivation
Large test runs can finish many tests at once and create a short burst of payloads. Previously, a full transport buffer or one slow request could cause data loss during this burst.
This PR gives temporary failures time to recover without waiting forever or allowing memory use to grow forever. Payloads can still be dropped after the 64 MiB queue limit or the 60-second final-flush limit is reached.
Additional Notes
This work was extracted from #9987 and includes the changes previously split into #10045. It builds on the concurrency, HTTP retry, and final-flush timeout changes from #10038, #10039, and #10040.
Verified with focused request, exporter, writer, telemetry, and final-flush tests, plus repeated web-ui end-to-end runs.