feat(debugger): support agentless Dynamic Instrumentation - #10051
feat(debugger): support agentless Dynamic Instrumentation#10051BridgeAR wants to merge 11 commits into
Conversation
Agentless mode disables the Agent Remote Config path, which leaves Dynamic Instrumentation without a probe source. One Remote Config manager owns lifecycle and applied state for both transports. Libdatadog handles only direct backend polling. Stopped poll generations cannot re-arm or overlap a restarted poll after an in-flight request completes.
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## rochdev/apm-data-pipeline #10051 +/- ##
=============================================================
- Coverage 88.08% 87.99% -0.09%
=============================================================
Files 893 894 +1
Lines 135229 135484 +255
Branches 7775 7753 -22
=============================================================
+ Hits 119112 119220 +108
- Misses 16117 16264 +147 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d08593c9f1
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| // The native client reports at most one add and one update per path per poll, but it can | ||
| // report both when a config it stored while inactive becomes active with new contents. The | ||
| // second one would dispatch the config to its handler twice. | ||
| if (seenPaths.has(path)) continue |
There was a problem hiding this comment.
When the native client reports the documented add-then-update pair for a path whose contents changed while inactive, this check retains the first, stale add and discards the newer update. The handler consequently installs the old probe while setConfigState acknowledges the fetcher's current path state, so the backend can believe the newer version was applied and never resend it; coalesce duplicate paths to the final change instead of keeping the first.
Useful? React with 👍 / 👎.
| try { | ||
| this.#applyChanges(changes) | ||
| } catch (applyError) { | ||
| log.error('[RC] Could not apply remote config update', applyError) |
There was a problem hiding this comment.
If a batch handler throws, this catch only logs the error even though the fetcher has already committed the returned changes as seen. For example, JsRemoteConfigFetcher.#diff updates #files before invoking this callback, so an unchanged response on the next poll produces no changes and the failed config is never dispatched again; a transient tracing or AppSec batch failure can therefore leave the process permanently on stale configuration until the backend publishes another version.
Useful? React with 👍 / 👎.
![datadog-official[bot]](https://avatars.githubusercontent.com/in/1358509?s=60&v=4){kind=small}
There was a problem hiding this comment.
The dependency change does not update yarn.lock, so frozen installs fail. Remote Config can also keep old data or lose updates, and global agentless mode stops Feature Flagging exposure delivery.
🤖 Datadog Autotest · Commit d08593c · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
| }, | ||
| "optionalDependencies": { | ||
| "@datadog/libdatadog": "0.12.1", | ||
| "@datadog/libdatadog": "^0.19.0", |
There was a problem hiding this comment.
Repository installs that use the frozen lock file cannot complete.
Assertion details
- Input: Run a frozen Yarn install from this revision.
- Expected:
The lock file must resolve a libdatadog version that satisfies the new manifest range. - Actual:
The manifest requires libdatadog 0.19, but yarn.lock still contains only version 0.12.1. A frozen install fails.
Was this helpful? React 👍 or 👎
🤖 Datadog Autotest · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
| // The native client reports at most one add and one update per path per poll, but it can | ||
| // report both when a config it stored while inactive becomes active with new contents. The | ||
| // second one would dispatch the config to its handler twice. | ||
| if (seenPaths.has(path)) continue |
There was a problem hiding this comment.
A probe or configuration can remain at an old version until a later server change occurs.
Assertion details
- Input: The native client returns an add record and then a newer update record for the same path in one poll.
- Expected:
The code must apply the newest change for each path once. - Actual:
The loop keeps the first change for a path and ignores the later update. The fetcher has already stored the later version, so another poll does not correct the old applied data.
Was this helpful? React 👍 or 👎
🤖 Datadog Autotest · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
| * @returns {void} | ||
| */ | ||
| function enable (config) { | ||
| if (config.DD_AGENTLESS_ENABLED) return |
There was a problem hiding this comment.
Global agentless mode silently drops all Feature Flagging exposure events.
Assertion details
- Input: Enable DD_AGENTLESS_ENABLED and use the OpenFeature provider.
- Expected:
Feature Flagging must start the exposure writer and use its agentless delivery route. - Actual:
The new return prevents the exposure writer and its channel subscribers from starting, although the configuration selects the direct agentless delivery route.
Was this helpful? React 👍 or 👎
🤖 Datadog Autotest · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
| for (const [handler, products] of this.#batchHandlers) { | ||
| const transactionView = filterTransactionByProducts(transaction, products) | ||
| if (transactionView.toUnapply.length || transactionView.toApply.length || transactionView.toModify.length) { | ||
| handler(transactionView) |
There was a problem hiding this comment.
An APM or AppSec configuration can remain unapplied, or a removed configuration can remain active.
Assertion details
- Input: A Remote Config batch handler throws while it processes an add, update, or removal.
- Expected:
A handler failure must keep the update available for retry or complete dispatch with an error state. - Actual:
The fetcher commits the new file state before this handler runs. If the handler throws, dispatch stops, but later polls report no change and cannot retry the update.
Was this helpful? React 👍 or 👎
🤖 Datadog Autotest · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
Agentless polling duplicated the Agent parser, dispatch, and scheduler ownership, so acknowledgements and lifecycle fixes could drift between transports. Keep the existing RemoteConfig state machine and adapt libdatadog at its fetch boundary. Agentless mode no longer overrides the customer's Remote Config or Dynamic Instrumentation settings.
Agent and agentless updates used separate transaction and dispatch flows, so batch outcomes and acknowledgement behavior could drift. Translate Agent responses into the same change records libdatadog returns, then process both through one path.
The native fetch promise owns completion. The transaction outcome map owns whether a product handler runs. Duplicating both contracts let agent and agentless polling diverge.
DD_AGENTLESS_ENABLED does not require an API key. Adding the undefined value to the request headers makes Node reject the request before network I/O.
|
I see I was requested for a review but the PR is in draft mode, do you need a review now or wait? |
|
@leoromanovsky thanks, not right now :) |
2eb2803 to
8781d02
Compare
3 participants
Agentless mode disables the Agent Remote Config path, so Dynamic Instrumentation has no probe source without an Agent. This adds direct Remote Config polling, debugger logs, and diagnostics for that mode.
RemoteConfigremains the only scheduler, subscription registry, applied-config store, and acknowledgement owner. It sends each probe to the debugger worker. The worker acknowledgement updates the same applied-config record. The Agent transport stays callback-native. Libdatadog's Promise ends at the agentless adapter.Poll generations cannot re-arm after stop or overlap after restart. Agent response state commits only after complete validation. Stale acknowledgements cannot update replacement configs. Test Optimization does not enable the debugger automatically.