Repository navigation
EKS workloads: show CPU usage as a percentage of the pods' CPU request - #234
Merged
ravion-ops[bot] merged 4 commits intoOct 25, 2026
Merged
Conversation
Co-Authored-By: brandon <brandon@flightcontrol.dev>
Co-Authored-By: brandon <brandon@flightcontrol.dev>
Contributor
Author
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
Ravion Module Publish PlanDry run only. No Ravion API mutations were made.
Diffsrvn-eks-web 1.12.1 -> 1.12.2--- remote
+++ compiled
source:
base_path: charts/rvn-eks-web
branch: main
- ref: rvn-eks-web@1.12.1
+ ref: rvn-eks-web@1.12.2
repo_url: https://github.com/ravionhq/modules
type: git
type: helm
@@
Terraform owns the `aws_lb_target_group` and `aws_lb_listener_rule` against a shared load balancer, and the AWS Load Balancer Controller registers pod IPs into that target group. The chart does not create an Ingress, so no per-service load balancer is provisioned. This is the same routing shape as ECS Web Server, so a service moving from ECS keeps its routing model.
- Chart source: [ravionhq/modules/charts/rvn-eks-web](https://github.com/ravionhq/modules/tree/rvn-eks-web@1.12.1/charts/rvn-eks-web)
+ Chart source: [ravionhq/modules/charts/rvn-eks-web](https://github.com/ravionhq/modules/tree/rvn-eks-web@1.12.2/charts/rvn-eks-web)
- Terraform source: [ravionhq/modules/compute/eks_service](https://github.com/ravionhq/modules/tree/rvn-eks-web@1.12.1/compute/eks_service)
+ Terraform source: [ravionhq/modules/compute/eks_service](https://github.com/ravionhq/modules/tree/rvn-eks-web@1.12.2/compute/eks_service)
## Use cases
@@
Charts are the same five golden signals whichever store answers: CPU, memory working set, container restarts, network in, and network out, plus replicas available and desired. Two things differ on the CloudWatch link of the chain, and are worth knowing before you read a number off it:
- - **Container Insights reports per pod, Prometheus reports across pods.** A CloudWatch chart is an average over the workload's pods; the Prometheus charts sum them. Same shape, different magnitude.
+ - **CPU uses different units by provider.** Prometheus shows usage as a percentage of the running pods' CPU requests, matching the autoscaler's CPU target; CloudWatch reports cores. Other Prometheus charts sum across pods; CloudWatch averages across them.
- **It needs enhanced observability**, which the add-ons module turns on with CloudWatch by default. Without it, CPU and memory come back empty.
Replicas available falls back to Container Insights' running-pod count for this module's Kubernetes Service. Replicas desired has no CloudWatch equivalent, so with CloudWatch as the only metrics provider that one chart is empty.
@@
execution_environment_overrides:
security_group: << module.input.ravion_runner_security_group_id >>
plan_permissions: '<< module.input.terraform_plan_default_policies_enabled == false ? {"replace": module.input.terraform_plan_iam_policy_arns} : (module.input.terraform_plan_iam_policy_arns ? {"attach": module.input.terraform_plan_iam_policy_arns} : nil) >>'
- ref: rvn-eks-web@1.12.1
+ ref: rvn-eks-web@1.12.2
repo: https://github.com/ravionhq/modules
stack_id: <<stack.id>>
terraform_variables:
@@
name: CPU usage
source:
aws_account_id: << module.input.aws_account_id >>
- query: 'sum(rate(container_cpu_usage_seconds_total{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+"}[$__interval]))'
+ query: '100 * sum(rate(container_cpu_usage_seconds_total{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",container!="",container!="POD"}[$__interval]) * on(namespace, pod) group_left() (kube_pod_status_phase{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",phase="Running"} == 1)) / (sum(kube_pod_container_resource_requests{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",resource="cpu"} * on(namespace, pod) group_left() (kube_pod_status_phase{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",phase="Running"} == 1)) > 0)'
region: << module.input.amp_workspace_region >>
type: amp
- unit: cores
+ unit: Percent
workspace_id: << module.input.amp_workspace_id >>
type: line
- enabled: << module.input.metrics_rendering_providers != nil && ("prometheus" in module.input.metrics_rendering_providers || "mimir" in module.input.metrics_rendering_providers) >>
@@
source:
cluster_arn: << module.input.cluster_arn >>
endpoint: << module.input.prometheus_endpoint >>
- query: 'sum(rate(container_cpu_usage_seconds_total{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+"}[$__interval]))'
+ query: '100 * sum(rate(container_cpu_usage_seconds_total{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",container!="",container!="POD"}[$__interval]) * on(namespace, pod) group_left() (kube_pod_status_phase{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",phase="Running"} == 1)) / (sum(kube_pod_container_resource_requests{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",resource="cpu"} * on(namespace, pod) group_left() (kube_pod_status_phase{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",phase="Running"} == 1)) > 0)'
type: eks_prometheus
- unit: cores
+ unit: Percent
type: line
- enabled: << module.input.metrics_rendering_providers != nil && "cloudwatch" in module.input.metrics_rendering_providers >>
id: cpurvn-eks-worker 0.14.1 -> 0.14.2--- remote
+++ compiled
source:
base_path: charts/rvn-eks-worker
branch: main
- ref: rvn-eks-worker@0.14.1
+ ref: rvn-eks-worker@0.14.2
repo_url: https://github.com/ravionhq/modules
type: git
type: helm
@@
The Terraform stack creates the ECR repository that Dockerfile and Railpack builds push to and, when Fargate is selected, a workload-specific EKS Fargate profile and pod execution role. On the Pull from image registry path with automatic or EC2 compute, the stack creates no AWS resources. Everything else the module needs - cluster endpoint, certificate authority data, private subnet IDs, and the Ravion Runner role - comes from the selected EKS Cluster module.
- Chart source: [ravionhq/modules/charts/rvn-eks-worker](https://github.com/ravionhq/modules/tree/rvn-eks-worker@0.14.1/charts/rvn-eks-worker)
+ Chart source: [ravionhq/modules/charts/rvn-eks-worker](https://github.com/ravionhq/modules/tree/rvn-eks-worker@0.14.2/charts/rvn-eks-worker)
- Terraform source: [ravionhq/modules/compute/eks_service](https://github.com/ravionhq/modules/tree/rvn-eks-worker@0.14.1/compute/eks_service)
+ Terraform source: [ravionhq/modules/compute/eks_service](https://github.com/ravionhq/modules/tree/rvn-eks-worker@0.14.2/compute/eks_service)
## Use cases
@@
Charts are the same five golden signals whichever store answers: CPU, memory working set, container restarts, network in, and network out. Two things differ on the CloudWatch link of the chain, and are worth knowing before you read a number off it:
- - **Container Insights reports per pod, Prometheus reports across pods.** A CloudWatch chart is an average over the worker's pods; the Prometheus charts sum them. Same shape, different magnitude.
+ - **CPU uses different units by provider.** Prometheus shows usage as a percentage of the running pods' CPU requests, matching the autoscaler's CPU target; CloudWatch reports cores. Other Prometheus charts sum across pods; CloudWatch averages across them.
- **It needs enhanced observability**, which the add-ons module turns on with CloudWatch by default. Without it, CPU and memory come back empty.
Replicas available and desired have no CloudWatch equivalent. Container Insights' nearest metric counts pods behind a Kubernetes Service, and a worker renders no Service, so with CloudWatch as the only metrics provider those two charts are empty while the pod charts still work.
@@
execution_environment_overrides:
security_group: << module.input.ravion_runner_security_group_id >>
plan_permissions: '<< module.input.terraform_plan_default_policies_enabled == false ? {"replace": module.input.terraform_plan_iam_policy_arns} : (module.input.terraform_plan_iam_policy_arns ? {"attach": module.input.terraform_plan_iam_policy_arns} : nil) >>'
- ref: rvn-eks-worker@0.14.1
+ ref: rvn-eks-worker@0.14.2
repo: https://github.com/ravionhq/modules
stack_id: <<stack.id>>
terraform_variables:
@@
name: CPU usage
source:
aws_account_id: << module.input.aws_account_id >>
- query: 'sum(rate(container_cpu_usage_seconds_total{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+"}[$__interval]))'
+ query: '100 * sum(rate(container_cpu_usage_seconds_total{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",container!="",container!="POD"}[$__interval]) * on(namespace, pod) group_left() (kube_pod_status_phase{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",phase="Running"} == 1)) / (sum(kube_pod_container_resource_requests{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",resource="cpu"} * on(namespace, pod) group_left() (kube_pod_status_phase{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",phase="Running"} == 1)) > 0)'
region: << module.input.amp_workspace_region >>
type: amp
- unit: cores
+ unit: Percent
workspace_id: << module.input.amp_workspace_id >>
type: line
- enabled: << module.input.metrics_rendering_providers != nil && ("prometheus" in module.input.metrics_rendering_providers || "mimir" in module.input.metrics_rendering_providers) >>
@@
source:
cluster_arn: << module.input.cluster_arn >>
endpoint: << module.input.prometheus_endpoint >>
- query: 'sum(rate(container_cpu_usage_seconds_total{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+"}[$__interval]))'
+ query: '100 * sum(rate(container_cpu_usage_seconds_total{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",container!="",container!="POD"}[$__interval]) * on(namespace, pod) group_left() (kube_pod_status_phase{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",phase="Running"} == 1)) / (sum(kube_pod_container_resource_requests{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",resource="cpu"} * on(namespace, pod) group_left() (kube_pod_status_phase{namespace="<< module.input.namespace >>",pod=~"<< module.input.name >><< module.input.spot_burst_enabled == true && module.input.compute_target == "on_demand" ? "(-spot)?" : "" >>-[a-z0-9]+-[a-z0-9]+",phase="Running"} == 1)) > 0)'
type: eks_prometheus
- unit: cores
+ unit: Percent
type: line
- enabled: << module.input.metrics_rendering_providers != nil && "cloudwatch" in module.input.metrics_rendering_providers >>
id: cpu |
Co-Authored-By: brandon <brandon@flightcontrol.dev>
Member
mabadir
approved these changes
Oct 24, 2026
flybayer
approved these changes
Oct 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The EKS web, worker and cron CPU charts (the AMP and in-cluster Prometheus links of
partials/ui/eks-workload-pod-metrics.yml) used to plot summed CPU cores, so each card scaled to its own peak (0.92, 0.01...). They now plot utilization as a percentage of the running pods' CPU request. That's the same basis as the autoscaler's CPU target, and the same idea as ECSCPUUtilization. The UI pinsPercentcharts to 0–100%.container!=""drops cAdvisor's pod-level cgroup series. The old query summed those together with the per-container series, which likely double-counted.> 0: a workload with a zero CPU request shows no CPU data instead of Inf.compute/eks/addons/locals.tf).rvn-eks-web1.12.2 andrvn-eks-worker0.14.2.rvn-eks-cronispublished: false, so only its README text changed.Not yet checked against live Prometheus data.
Link to Devin session: https://app.devin.ai/sessions/1acd1d2117304f93b7a3526d86b124d0
Open in Devin Desktop: https://app.devin.ai/desktop/session/1acd1d2117304f93b7a3526d86b124d0?variant=devin
The PR appears safe to merge; no new actionable issue remains.
Summary
The PR changes Prometheus CPU charts from cores to a percentage of running pods’ CPU requests. CloudWatch continues to show cores.
Reviews (2) · Last reviewed commit: "Count only running pods in CPU usage and..." · Reviewed by Greptile