[petruanica]

Important

This is a breaking change and must ship in a major chart version bump (6.x to 7.0.0). It removes 25 node-label dimensions from every Container Insights metric emitted from a Karpenter-managed node. Any dashboard, alarm, or query that selects or groups on one of them stops matching the moment the new chart version is installed. There is no deprecation window and no values-level opt-out in this change: the attributes are dropped at the processor, so they disappear in a single release. The paired agent change carries the same constraint, and neither should be picked into a minor or patch release.

Note

Release coordination (working assumption, not yet decided). This PR is written assuming it ships together with the two sibling Container Insights changes in a single major chart release, v7.0.0. That bundling has not been agreed yet — it is recorded here so reviewers know what the description assumes, and it needs confirming before any of the three merge. If they end up shipping separately, each breaking change still needs its own major bump.

Issue #, if available:

n/a

Description of changes:

Adds 25 Karpenter node-label keys to awsattributelimit/cw_k8s_ci_v0.unconditional_removal_keys in both collector configs — templates/linux/_otel-container-insights-config.tpl (node agent) and templates/linux/_otel-container-insights-cluster-scraper-config.tpl (cluster scraper). 22 are karpenter.k8s.aws/instance-* and 3 are karpenter.sh/ lifecycle labels. Additive only, 25 lines per file, no other behaviour change.

Karpenter applies these labels to every node it manages, and the Kubernetes attribute enrichment copies them onto the resource block, so their key text is attached to every datapoint of every series from a Karpenter-managed node. Most are derived from instance type details, so they restate host.type, which these pipelines already publish. The denylist already removes comparable platform-injected families (feature.node.kubernetes.io/, beta.kubernetes.io/, alpha.eksctl.io/, sourceLaunchTemplateId); the Karpenter equivalents were never added.

What is removed and what is kept

The list is now complete against upstream's WellKnownLabels. Karpenter defines 28 labels in the karpenter.k8s.aws/ domain and 5 in karpenter.sh/; this denies 22 and 3 of them respectively, and retains the remaining 8 deliberately.

Retained, because they are not instance properties and are legitimate grouping dimensions:

• karpenter.sh/nodepool, karpenter.k8s.aws/ec2nodeclass, karpenter.sh/capacity-type — provisioning identity, with capacity-type the only signal distinguishing spot from on-demand capacity.
• karpenter.k8s.aws/capacity-reservation-id, -type, -interruptible — which capacity reservation a node consumed, whether it is an ordinary On-Demand Capacity Reservation or a Capacity Block, and whether EC2 may reclaim it. Not derivable from the instance type.
• karpenter.k8s.aws/placement-group-id, -partition — which EC2 placement group the instance launched into, and for partition placement groups which partition. A partition is an independent failure domain, so this is a physical fault boundary rather than an instance property.

That is also why the change lists exact keys rather than adding karpenter.k8s.aws/ and karpenter.sh/ to unconditional_removal_prefixes, which would strip all 8 retained keys as well.

GPU and accelerator labels

The four instance-gpu-* and three instance-accelerator-* keys were added after review feedback. Karpenter populates them conditionally — computeRequirements sets the GPU labels only when the instance type reports GPU info, and the accelerator labels only for Inferentia or Neuron devices — so they are absent from clusters with no accelerated nodes and were missed when the list was first assembled. They are the same class as instance-cpu and instance-memory: properties of the instance type that restate host.type.

GPU and Neuron dashboards are unaffected, because device identity travels on the DCGM_FI_DEV_* and neuroncore_* metrics' own datapoint attributes, which this change does not touch. Note that the Neuron branch also populates the accelerator labels, so this covers Trainium and Inferentia2 as well as Inferentia1.

Breaking change and release sequencing

Customers see the removed dimensions on two surfaces: the k8s_node_label_karpenter_* metric label, and the @resource. prefix in PromQL. Both stop resolving for the 25 denied keys. Concretely, a query grouping by karpenter.k8s.aws/instance-family to compare utilisation across instance families returns a single unlabelled series afterwards, and an alarm scoped to a specific instance-size stops matching any series at all.

Because the keys are listed in the template with no conditional and no values.yaml toggle, a customer who relies on one of them has no way to retain it in the version that removes it. That is the reason this needs a major chart version bump rather than a minor, and it should be called out in the release notes alongside the paired agent change.

Testing:

helm lint clean with a region set; helm template with otelContainerInsights.enabled=true shows all 25 keys in both collector configs and none of the 8 retained keys in either denylist. The chart's own suites pass: tests/flag_matrix.sh 8/8, tests/pod_level_values.sh 22/22, tests/yaml_document_injection.sh 17/17.

New integration coverage in aws/amazon-cloudwatch-agent-test (aws/amazon-cloudwatch-agent-test#775): TestKarpenterInstanceLabelsRemoved asserts none of the 25 denied keys reach resource scope, and TestKarpenterGroupingLabelsRetained asserts the three keys Karpenter sets on every managed node still arrive. The pair makes the exact-keys decision testable — a regression that switched to prefixes would pass the first and fail the second.

Both were validated on a live EKS cluster, since the claim is about what arrives in CloudWatch rather than what the config renders to. The changed configuration was applied to the running agent and cluster scraper, the pods were restarted, and metrics were given 30 minutes to settle; test/otel/standard -computeType=EKS then asserts on datapoints read back from CloudWatch. The suite was also run before the change, because an "attribute is absent" assertion passes trivially against empty data.

Assertion	Before	After
TestKarpenterInstanceLabelsRemoved	18 fail	18 pass
TestKarpenterGroupingLabelsRetained (guard)	18 pass	18 pass

No failures, no skips. The guard passing both before and after shows only the intended keys were removed, and establishes that the result set genuinely contained Karpenter labels.

The same list is applied to the agent-side pipeline configs in aws/amazon-cloudwatch-agent (companion PR) so both config paths stay consistent.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

PR Checklist

• Commits are squashed into a logical, reviewable set (one commit for a single change)
• Commits and PR description comply with Amazon internal guidelines
• Chart renders and lints locally
• All GitHub Actions checks on the PR are passing
• Integration test evidence: validated end-to-end on a live EKS cluster; baseline and post-change assertion counts are in the Testing section above. There is no public CI link because the integration suite needs a provisioned cluster and credentials that fork PRs cannot reach.
• New or updated integration test coverage: test(otel): cover Karpenter node-label and pod-label counterpart removal aws/amazon-cloudwatch-agent-test#775
• New functionality has unit tests; bug fixes have a reproducing test
• N/A — no config translation in this repo, so no golden files
• Breaking or customer-visible changes are called out in the PR description