Skip to content

test 5.0 cri-o revert#80786

Open
mcornea wants to merge 1 commit into
openshift:mainfrom
mcornea:test_5.0_crio
Open

test 5.0 cri-o revert#80786
mcornea wants to merge 1 commit into
openshift:mainfrom
mcornea:test_5.0_crio

Conversation

@mcornea

@mcornea mcornea commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Summary by CodeRabbit

This PR modifies the OpenShift QE infrastructure provisioning step to patch the cri-o container runtime across the cluster. Specifically, it updates the openshift-qe/workers-infra-workload CI step used during OpenShift 5.0 test deployments.

What Changed:
The script now applies three MachineConfig resources (for worker, master, and infra node roles) that update the RHCOS/OS image to a specific build containing cri-o version 1.35.2-5.rhaos4.22. These machine configs are applied via oc apply -f- after the existing node and machineset provisioning logic.

Additionally, the script adds a cluster stabilization wait step (oc adm wait-for-stable-cluster --minimum-stable-period 2m) at the end to ensure the cluster has settled after applying the OS image updates.

Impact:
This affects the CI infrastructure used for OpenShift QE testing, specifically the infrastructure workload provisioning pipeline. The change ensures that test clusters being deployed for quality engineering validation run a specific version of cri-o, which is important for testing cri-o-related changes or verifying compatibility in the 5.0 release cycle.

@coderabbitai

coderabbitai Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Walkthrough

The script gains two additions: a heredoc that applies a List of three MachineConfig resources (worker, master, infra) each setting a pinned osImageURL to update cri-o to 1.35.2-5.rhaos4.22, and a post-provisioning call to oc adm wait-for-stable-cluster --minimum-stable-period 2m.

Changes

MachineConfig RHCOS patch and cluster stability wait

Layer / File(s) Summary
MachineConfig osImageURL patch (worker/master/infra) and stability wait
ci-operator/step-registry/openshift-qe/workers-infra-workload/openshift-qe-workers-infra-workload-commands.sh
Inserts an oc apply -f- heredoc that creates three MachineConfig objects (worker, master, infra) with a pinned osImageURL to apply the cri-o 1.35.2-5.rhaos4.22 patch, then adds oc adm wait-for-stable-cluster --minimum-stable-period 2m after the node/machineset provisioning block.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Suggested reviewers

  • wgahnagl
  • hasan4791
🚥 Pre-merge checks | ✅ 15
✅ Passed checks (15 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'test 5.0 cri-o revert' is specific and directly related to the PR's objective of testing cri-o version updates, though it doesn't fully convey the script's expanded scope including MachineConfig application and cluster stability waiting logic.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed No Ginkgo test files are modified in this PR. The only changed file is a bash shell script, not a Go test file with It(), Describe(), or other Ginkgo test patterns.
Test Structure And Quality ✅ Passed Custom check for Ginkgo test quality is not applicable; PR modifies only shell scripts in CI operator step registry, not Go/Ginkgo test code.
Microshift Test Compatibility ✅ Passed No new Ginkgo e2e tests are added in this PR. The changes are infrastructure/setup code (bash shell script for MachineConfig and cluster stability), not test code subject to MicroShift compatibilit...
Single Node Openshift (Sno) Test Compatibility ✅ Passed This PR does not add any Ginkgo e2e tests. It only modifies a bash shell script for CI infrastructure. The check applies only to new Ginkgo e2e tests, making it not applicable here.
Topology-Aware Scheduling Compatibility ✅ Passed PR modifies CI/CD test script with MachineConfig/MachineConfigPool resources for OS patching, not deployment manifests, operators, or controllers with pod scheduling constraints.
Ote Binary Stdout Contract ✅ Passed This PR modifies a shell script in the CI operator step registry, not a Go OTE binary. The custom check for "OTE Binary Stdout Contract" applies only to Go test binaries that communicate JSON via s...
Ipv6 And Disconnected Network Test Compatibility ✅ Passed PR does not add any Ginkgo e2e tests. The custom check targets IPv6/disconnected compatibility in e2e tests; this PR modifies a bash shell script only.
No-Weak-Crypto ✅ Passed No weak cryptography detected. Script uses SHA256 hashes only for container image digests, which are appropriate security measures with no weak algorithms, custom implementations, or insecure compa...
Container-Privileges ✅ Passed No privileged container settings found. Changes only add MachineConfig resources with osImageURL specifications, lacking any container specs or security context flags.
No-Sensitive-Data-In-Logs ✅ Passed No sensitive data (passwords, tokens, API keys, PII, session IDs, or internal hostnames) detected. The developer name in log message and Docker image digest are not sensitive data.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci

openshift-ci Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mcornea

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 19, 2026
@openshift-ci openshift-ci Bot requested review from capolrik and chentex June 19, 2026 12:05
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

[REHEARSALNOTIFIER]
@mcornea: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name Repo Type Reason
pull-ci-openshift-kueue-operator-main-perfscale-pr-kueue-operator-9nodes openshift/kueue-operator presubmit Registry content changed
pull-ci-openshift-kueue-operator-release-1.4-perfscale-pr-kueue-operator-9nodes openshift/kueue-operator presubmit Registry content changed
pull-ci-openshift-kueue-operator-release-1.3-perfscale-pr-kueue-operator-9nodes openshift/kueue-operator presubmit Registry content changed
pull-ci-openshift-eng-ocp-qe-perfscale-ci-main-4.20-nightly-control-plane-nutanix-24nodes openshift-eng/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-4.21-nightly-x86-udn-density-l3-24nodes openshift-eng/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-5.0-nightly-x86-loaded-upgrade-from-4.22-loaded-upgrade-180nodes openshift-eng/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-5.0-nightly-x86-loaded-upgrade-from-4.22-loaded-upgrade-3nodes-udn openshift-eng/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-5.0-nightly-x86-loaded-upgrade-from-4.22-loaded-upgrade-24nodes-72udn openshift-eng/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-5.0-nightly-x86-loaded-upgrade-from-4.22-loaded-upgrade-24nodes-layer2-72udn openshift-eng/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-4.22-nightly-x86-loaded-upgrade-from-4.21-loaded-upgrade-180nodes openshift-eng/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-4.22-nightly-x86-loaded-upgrade-from-4.21-loaded-upgrade-3nodes-udn openshift-eng/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-4.22-nightly-x86-loaded-upgrade-from-4.21-loaded-upgrade-24nodes-72udn openshift-eng/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-4.22-nightly-x86-loaded-upgrade-from-4.21-loaded-upgrade-24nodes-layer2-72udn openshift-eng/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-eng-ocp-qe-perfscale-ci-main-4.21-nightly-node-density-heavy-aws-3nodes-1500ppn openshift-eng/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-4.23-nightly-x86-loaded-upgrade-from-4.22-loaded-upgrade-180nodes openshift-eng/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-4.23-nightly-x86-loaded-upgrade-from-4.22-loaded-upgrade-3nodes-udn openshift-eng/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-4.23-nightly-x86-loaded-upgrade-from-4.22-loaded-upgrade-24nodes-72udn openshift-eng/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-4.23-nightly-x86-loaded-upgrade-from-4.22-loaded-upgrade-24nodes-layer2-72udn openshift-eng/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-eng-ocp-qe-perfscale-ci-main-4.22-nightly-node-density-heavy-aws-3nodes-1500ppn openshift-eng/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-4.23-nightly-x86-udn-density-l3-24nodes openshift-eng/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-4.21-nightly-x86-netpol-24nodes openshift-eng/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-4.21-nightly-x86-payload-control-plane-6nodes openshift-eng/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-4.21-nightly-x86-control-plane-120nodes openshift-eng/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-4.21-nightly-x86-control-plane-24nodes openshift-eng/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-4.21-nightly-x86-conc-builds-3nodes openshift-eng/ocp-qe-perfscale-ci presubmit Registry content changed

A total of 715 jobs have been affected by this change. The above listing is non-exhaustive and limited to 25 jobs.

A full list of affected jobs can be found here
Prior to this PR being merged, you will need to either run and acknowledge or opt to skip these rehearsals.

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@ci-operator/step-registry/openshift-qe/workers-infra-workload/openshift-qe-workers-infra-workload-commands.sh`:
- Around line 553-585: Remove the echo statement containing the developer name
"mcornea" on line 553, as it indicates this is temporary test code in shared
infrastructure. Either remove the entire oc apply block with the MachineConfig
resources for os-layer-custom-worker, os-layer-custom-master, and
os-layer-custom-infra if this is a temporary test, or if this change is intended
to be permanent, remove the developer attribution from the echo statement and
add documentation explaining what the hardcoded osImageURL digest is pinning to
and the rationale for using it.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 57fa0cc8-076e-46c1-ae9a-46f8c3766fba

📥 Commits

Reviewing files that changed from the base of the PR and between 878b7ce and d7c3c21.

📒 Files selected for processing (1)
  • ci-operator/step-registry/openshift-qe/workers-infra-workload/openshift-qe-workers-infra-workload-commands.sh

Comment on lines +553 to +585
echo "mcornea patching rhcos version with cri-o 1.35.2-5.rhaos4.22 and wait"
# Apply the changes
oc apply -f- <<EOF
apiVersion: v1
items:
- apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: worker
name: os-layer-custom-worker
spec:
osImageURL: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:81e97c192d3fed112c182dba8c4bbbbb6b1c15dfb3cf9ee4f3585267ba53ef16
- apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: master
name: os-layer-custom-master
spec:
osImageURL: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:81e97c192d3fed112c182dba8c4bbbbb6b1c15dfb3cf9ee4f3585267ba53ef16
- apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/role: infra
name: os-layer-custom-infra
spec:
osImageURL: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:81e97c192d3fed112c182dba8c4bbbbb6b1c15dfb3cf9ee4f3585267ba53ef16
kind: List
metadata:
resourceVersion: ""
EOF

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚖️ Poor tradeoff

Test/debug code appears to be intended for merge into shared infrastructure.

The echo on line 553 containing "mcornea patching..." indicates this is developer-specific test code. Based on the PR title "test 5.0 cri-o revert", this appears to be a temporary testing change rather than a permanent addition to the step registry.

Key concerns:

  1. Developer name in echo statement suggests this is not production-ready
  2. Hardcoded osImageURL digest with no documentation explaining what cri-o version or RHCOS content it pins to
  3. This modifies shared QE infrastructure that other jobs depend on

If this is meant to be a temporary test, consider using a separate test workflow or configuration variant instead of modifying the shared step. If this change is intended to be permanent, please remove the developer attribution and add documentation about the pinned image.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/step-registry/openshift-qe/workers-infra-workload/openshift-qe-workers-infra-workload-commands.sh`
around lines 553 - 585, Remove the echo statement containing the developer name
"mcornea" on line 553, as it indicates this is temporary test code in shared
infrastructure. Either remove the entire oc apply block with the MachineConfig
resources for os-layer-custom-worker, os-layer-custom-master, and
os-layer-custom-infra if this is a temporary test, or if this change is intended
to be permanent, remove the developer attribution from the echo statement and
add documentation explaining what the hardcoded osImageURL digest is pinning to
and the rationale for using it.

@mcornea

mcornea commented Jun 19, 2026

Copy link
Copy Markdown
Contributor Author

/pj-rehearse periodic-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-5.0-nightly-x86-payload-control-plane-6nodes

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@mcornea: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@mcornea

mcornea commented Jun 19, 2026

Copy link
Copy Markdown
Contributor Author

/pj-rehearse periodic-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-5.0-nightly-x86-payload-control-plane-6nodes

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@mcornea: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-ci

openshift-ci Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

@mcornea: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/rehearse/periodic-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-5.0-nightly-x86-payload-control-plane-6nodes d7c3c21 link unknown /pj-rehearse periodic-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-5.0-nightly-x86-payload-control-plane-6nodes

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant