Skip to content

EKS workloads: show CPU usage as a percentage of the pods' CPU request - #234

Merged
ravion-ops[bot] merged 4 commits into
mainfrom
devin/1792861259-eks-cpu-percent-of-request
Oct 25, 2026
Merged

ravion-ops[bot] merged 4 commits into
mainfrom
devin/1792861259-eks-cpu-percent-of-request

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor
  • In merge queue / Merged
  • Prioritize

Summary

The EKS web, worker and cron CPU charts (the AMP and in-cluster Prometheus links of partials/ui/eks-workload-pod-metrics.yml) used to plot summed CPU cores, so each card scaled to its own peak (0.92, 0.01...). They now plot utilization as a percentage of the running pods' CPU request. That's the same basis as the autoscaler's CPU target, and the same idea as ECS CPUUtilization. The UI pins Percent charts to 0–100%.

RUNNING = (kube_pod_status_phase{ns, pod=~…, phase="Running"} == 1)
100 * sum(rate(container_cpu_usage_seconds_total{ns, pod=~…, container!="", container!="POD"}[$__interval]) * on(namespace, pod) group_left() RUNNING)
    / (sum(kube_pod_container_resource_requests{ns, pod=~…, resource="cpu"} * on(namespace, pod) group_left() RUNNING) > 0)
unit: Percent
  • container!="" drops cAdvisor's pod-level cgroup series. The old query summed those together with the per-container series, which likely double-counted.
  • Usage and requests both count running pods only. Pending pods, finished cron pods and pods that were just removed don't skew the ratio.
  • > 0: a workload with a zero CPU request shows no CPU data instead of Inf.
  • Both metrics are already on the add-ons kube-state-metrics allow-list (compute/eks/addons/locals.tf).
  • The CloudWatch link is unchanged and still reports cores, because Container Insights has no per-request equivalent here. The READMEs say so.
  • Bumps: rvn-eks-web 1.12.2 and rvn-eks-worker 0.14.2. rvn-eks-cron is published: false, so only its README text changed.

Not yet checked against live Prometheus data.

Link to Devin session: https://app.devin.ai/sessions/1acd1d2117304f93b7a3526d86b124d0
Open in Devin Desktop: https://app.devin.ai/desktop/session/1acd1d2117304f93b7a3526d86b124d0?variant=devin

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; no new actionable issue remains.

Summary

The PR changes Prometheus CPU charts from cores to a percentage of running pods’ CPU requests. CloudWatch continues to show cores.

  • EKS Prometheus CPU charts compare usage with running pods' CPU requests.

Reviews (2) · Last reviewed commit: "Count only running pods in CPU usage and..." · Reviewed by Greptile

devin-ai-integration Bot and others added 2 commits October 24, 2026 17:13
Co-Authored-By: brandon <brandon@flightcontrol.dev>
Co-Authored-By: brandon <brandon@flightcontrol.dev>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Ravion Module Publish Plan

Dry run only. No Ravion API mutations were made.

Module Current Version New Version Description
rvn-eks-web 1.12.1 1.12.2 Show CPU usage as a percentage of the pods' CPU request.
rvn-eks-worker 0.14.1 0.14.2 Show CPU usage as a percentage of the pods' CPU request.

Diffs

rvn-eks-web 1.12.1 -> 1.12.2

--- remote
+++ compiled
     source:
       base_path: charts/rvn-eks-web
       branch: main
-      ref: rvn-eks-web@1.12.1
+      ref: rvn-eks-web@1.12.2
       repo_url: https://github.com/ravionhq/modules
       type: git
     type: helm
@@
 
   Terraform owns the `aws_lb_target_group` and `aws_lb_listener_rule` against a shared load balancer, and the AWS Load Balancer Controller registers pod IPs into that target group. The chart does not create an Ingress, so no per-service load balancer is provisioned. This is the same routing shape as ECS Web Server, so a service moving from ECS keeps its routing model.
 
-  Chart source: [ravionhq/modules/charts/rvn-eks-web](https://github.com/ravionhq/modules/tree/rvn-eks-web@1.12.1/charts/rvn-eks-web)
+  Chart source: [ravionhq/modules/charts/rvn-eks-web](https://github.com/ravionhq/modules/tree/rvn-eks-web@1.12.2/charts/rvn-eks-web)
 
-  Terraform source: [ravionhq/modules/compute/eks_service](https://github.com/ravionhq/modules/tree/rvn-eks-web@1.12.1/compute/eks_service)
+  Terraform source: [ravionhq/modules/compute/eks_service](https://github.com/ravionhq/modules/tree/rvn-eks-web@1.12.2/compute/eks_service)
 
   ## Use cases
 
@@
 
   Charts are the same five golden signals whichever store answers: CPU, memory working set, container restarts, network in, and network out, plus replicas available and desired. Two things differ on the CloudWatch link of the chain, and are worth knowing before you read a number off it:
 
-  - **Container Insights reports per pod, Prometheus reports across pods.** A CloudWatch chart is an average over the workload's pods; the Prometheus charts sum them. Same shape, different magnitude.
+  - **CPU uses different units by provider.** Prometheus shows usage as a percentage of the running pods' CPU requests, matching the autoscaler's CPU target; CloudWatch reports cores. Other Prometheus charts sum across pods; CloudWatch averages across them.
   - **It needs enhanced observability**, which the add-ons module turns on with CloudWatch by default. Without it, CPU and memory come back empty.
 
   Replicas available falls back to Container Insights' running-pod count for this module's Kubernetes Service. Replicas desired has no CloudWatch equivalent, so with CloudWatch as the only metrics provider that one chart is empty.
@@
         execution_environment_overrides:
           security_group: << module.input.ravion_runner_security_group_id >>
         plan_permissions: '<< module.input.terraform_plan_default_policies_enabled == false ? {"replace": module.input.terraform_plan_iam_policy_arns} : (module.input.terraform_plan_iam_policy_arns ? {"attach": module.input.terraform_plan_iam_policy_arns} : nil) >>'
-        ref: rvn-eks-web@1.12.1
+        ref: rvn-eks-web@1.12.2
         repo: https://github.com/ravionhq/modules
         stack_id: <<stack.id>>
         terraform_variables:
@@
       name: CPU usage
       source:
         aws_account_id: << module.input.aws_account_id >>
-        query: 'sum(rate(container_cpu_usage_seconds_total{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+"}[$__interval]))'
+        query: '100 * sum(rate(container_cpu_usage_seconds_total{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",container!="",container!="POD"}[$__interval]) * on(namespace, pod) group_left() (kube_pod_status_phase{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",phase="Running"} == 1)) / (sum(kube_pod_container_resource_requests{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",resource="cpu"} * on(namespace, pod) group_left() (kube_pod_status_phase{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",phase="Running"} == 1)) > 0)'
         region: << module.input.amp_workspace_region >>
         type: amp
-        unit: cores
+        unit: Percent
         workspace_id: << module.input.amp_workspace_id >>
       type: line
     - enabled: << module.input.metrics_rendering_providers != nil && ("prometheus" in module.input.metrics_rendering_providers || "mimir" in module.input.metrics_rendering_providers) >>
@@
       source:
         cluster_arn: << module.input.cluster_arn >>
         endpoint: << module.input.prometheus_endpoint >>
-        query: 'sum(rate(container_cpu_usage_seconds_total{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+"}[$__interval]))'
+        query: '100 * sum(rate(container_cpu_usage_seconds_total{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",container!="",container!="POD"}[$__interval]) * on(namespace, pod) group_left() (kube_pod_status_phase{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",phase="Running"} == 1)) / (sum(kube_pod_container_resource_requests{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",resource="cpu"} * on(namespace, pod) group_left() (kube_pod_status_phase{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",phase="Running"} == 1)) > 0)'
         type: eks_prometheus
-        unit: cores
+        unit: Percent
       type: line
     - enabled: << module.input.metrics_rendering_providers != nil && "cloudwatch" in module.input.metrics_rendering_providers >>
       id: cpu

rvn-eks-worker 0.14.1 -> 0.14.2

--- remote
+++ compiled
     source:
       base_path: charts/rvn-eks-worker
       branch: main
-      ref: rvn-eks-worker@0.14.1
+      ref: rvn-eks-worker@0.14.2
       repo_url: https://github.com/ravionhq/modules
       type: git
     type: helm
@@
 
   The Terraform stack creates the ECR repository that Dockerfile and Railpack builds push to and, when Fargate is selected, a workload-specific EKS Fargate profile and pod execution role. On the Pull from image registry path with automatic or EC2 compute, the stack creates no AWS resources. Everything else the module needs - cluster endpoint, certificate authority data, private subnet IDs, and the Ravion Runner role - comes from the selected EKS Cluster module.
 
-  Chart source: [ravionhq/modules/charts/rvn-eks-worker](https://github.com/ravionhq/modules/tree/rvn-eks-worker@0.14.1/charts/rvn-eks-worker)
+  Chart source: [ravionhq/modules/charts/rvn-eks-worker](https://github.com/ravionhq/modules/tree/rvn-eks-worker@0.14.2/charts/rvn-eks-worker)
 
-  Terraform source: [ravionhq/modules/compute/eks_service](https://github.com/ravionhq/modules/tree/rvn-eks-worker@0.14.1/compute/eks_service)
+  Terraform source: [ravionhq/modules/compute/eks_service](https://github.com/ravionhq/modules/tree/rvn-eks-worker@0.14.2/compute/eks_service)
 
   ## Use cases
 
@@
 
   Charts are the same five golden signals whichever store answers: CPU, memory working set, container restarts, network in, and network out. Two things differ on the CloudWatch link of the chain, and are worth knowing before you read a number off it:
 
-  - **Container Insights reports per pod, Prometheus reports across pods.** A CloudWatch chart is an average over the worker's pods; the Prometheus charts sum them. Same shape, different magnitude.
+  - **CPU uses different units by provider.** Prometheus shows usage as a percentage of the running pods' CPU requests, matching the autoscaler's CPU target; CloudWatch reports cores. Other Prometheus charts sum across pods; CloudWatch averages across them.
   - **It needs enhanced observability**, which the add-ons module turns on with CloudWatch by default. Without it, CPU and memory come back empty.
 
   Replicas available and desired have no CloudWatch equivalent. Container Insights' nearest metric counts pods behind a Kubernetes Service, and a worker renders no Service, so with CloudWatch as the only metrics provider those two charts are empty while the pod charts still work.
@@
         execution_environment_overrides:
           security_group: << module.input.ravion_runner_security_group_id >>
         plan_permissions: '<< module.input.terraform_plan_default_policies_enabled == false ? {"replace": module.input.terraform_plan_iam_policy_arns} : (module.input.terraform_plan_iam_policy_arns ? {"attach": module.input.terraform_plan_iam_policy_arns} : nil) >>'
-        ref: rvn-eks-worker@0.14.1
+        ref: rvn-eks-worker@0.14.2
         repo: https://github.com/ravionhq/modules
         stack_id: <<stack.id>>
         terraform_variables:
@@
       name: CPU usage
       source:
         aws_account_id: << module.input.aws_account_id >>
-        query: 'sum(rate(container_cpu_usage_seconds_total{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+"}[$__interval]))'
+        query: '100 * sum(rate(container_cpu_usage_seconds_total{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",container!="",container!="POD"}[$__interval]) * on(namespace, pod) group_left() (kube_pod_status_phase{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",phase="Running"} == 1)) / (sum(kube_pod_container_resource_requests{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",resource="cpu"} * on(namespace, pod) group_left() (kube_pod_status_phase{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",phase="Running"} == 1)) > 0)'
         region: << module.input.amp_workspace_region >>
         type: amp
-        unit: cores
+        unit: Percent
         workspace_id: << module.input.amp_workspace_id >>
       type: line
     - enabled: << module.input.metrics_rendering_providers != nil && ("prometheus" in module.input.metrics_rendering_providers || "mimir" in module.input.metrics_rendering_providers) >>
@@
       source:
         cluster_arn: << module.input.cluster_arn >>
         endpoint: << module.input.prometheus_endpoint >>
-        query: 'sum(rate(container_cpu_usage_seconds_total{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+"}[$__interval]))'
+        query: '100 * sum(rate(container_cpu_usage_seconds_total{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",container!="",container!="POD"}[$__interval]) * on(namespace, pod) group_left() (kube_pod_status_phase{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",phase="Running"} == 1)) / (sum(kube_pod_container_resource_requests{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",resource="cpu"} * on(namespace, pod) group_left() (kube_pod_status_phase{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",phase="Running"} == 1)) > 0)'
         type: eks_prometheus
-        unit: cores
+        unit: Percent
       type: line
     - enabled: << module.input.metrics_rendering_providers != nil && "cloudwatch" in module.input.metrics_rendering_providers >>
       id: cpu

Comment thread partials/ui/eks-workload-pod-metrics.yml
Comment thread partials/ui/eks-workload-pod-metrics.yml Outdated
Comment thread partials/ui/eks-workload-pod-metrics.yml Outdated
Co-Authored-By: brandon <brandon@flightcontrol.dev>
@github-actions
github-actions Bot requested a review from flybayer October 24, 2026 17:22

Copy link
Copy Markdown
Member

@greptileai

@flybayer
flybayer requested a review from mabadir October 24, 2026 17:23
@ravion-ops
ravion-ops Bot merged commit 1209082 into main Oct 25, 2026
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants