feat(stats)!: per-key cardinality limits#2211
Open
Eldolfin wants to merge 9 commits into
Open
Conversation
Contributor
📚 Documentation Check Results📦
|
Contributor
Clippy Allow Annotation ReportComparing clippy allow annotations between branches:
Summary by Rule
Annotation Counts by File
Annotation Stats by Crate
About This ReportThis report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality. |
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: d82ebfd | Docs | Datadog PR Page | Give us feedback! |
Contributor
🔒 Cargo Deny Results📦
|
Contributor
Author
|
@codex review |
Eldolfin
force-pushed
the
oscarld/stats-per-key-cardinality-limit
branch
from
a5a4786 to
2845fa8
Compare
save memory by storing value hashes only, u64 has enough values to make collisions rare and in case we still get a collision storing 1 extra key should be fine
Eldolfin
force-pushed
the
oscarld/stats-per-key-cardinality-limit
branch
from
2845fa8 to
fca0f65
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a5a4786925
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| client_side_stats: StatsComputationConfig { | ||
| status: ArcSwap::new(stats.into()), | ||
| stats_cardinality_limit: self.stats_cardinality_limit, | ||
| stats_cardinality_limits: self.stats_cardinality_limits, |
There was a problem hiding this comment.
For exporters configured with OTLP metrics, the stats concentrator is created earlier with None for the new cardinality limits, and check_agent_info returns early when otlp_stats_enabled, so the value stored here never reaches that path. In that configuration, callers using set_stats_cardinality_limit still get the defaults for OTLP trace metrics instead of their configured per-key limits; pass self.stats_cardinality_limits into the OTLP SpanConcentrator::new call as well.
Useful? React with 👍 / 👎.
…alityLimitConfig binding
Contributor
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d82ebfd172
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| catch_panic!( | ||
| if let Some(handle) = config { | ||
| handle.stats_cardinality_limit = Some(limit); | ||
| handle.stats_cardinality_limits = Some(limits); |
There was a problem hiding this comment.
For C/C++ callers this by-value config has no default constructor, so the natural migration from the old single-limit API is a designated initializer such as { .whole_key_limit = n } or a zero-initialized struct with only that field assigned. The omitted per-field limits then arrive here as 0, and StatsBucket::collapse_key_fields_cardinality treats those limits as already exceeded, replacing every resource/http endpoint/peer-tags value with tracer_blocked_value and stripping those dimensions from all stats. Provide an FFI default initializer or otherwise preserve the Rust defaults for unspecified fields before storing the limits.
Useful? React with 👍 / 👎.
Comment on lines
+610
to
+611
| if v.is_empty() { | ||
| k.to_string() |
There was a problem hiding this comment.
When a real peer tag has an empty string value, this branch now encodes it as just the key (for example peer.hostname) instead of the previous peer.hostname: form. Downstream consumers parse peer tags as key:value entries (the OTLP mapper in build_attributes drops entries without :), so empty-valued peer tag dimensions are lost or confused with the sentinel representation; only the sentinel key should use the bare form.
Useful? React with 👍 / 👎.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Implement per-key cardinality limits to stats computation.
Also:
[]->[tracer_blocked_value]encode_grouped_statsmeta with empty value:key:->key(breaking change ?)Motivation
CSS spec compliance.
Additional Notes
Todo:
How to test the change?