UPSTREAM: <carry>: Fix flags passed to test-experimental-e2e by tmshort · Pull Request #723 · openshift/operator-framework-operator-controller

tmshort · 2026-05-11T18:14:31Z

Summary by CodeRabbit

Tests
- Experimental end-to-end test target is now disabled in automated runs and immediately succeeds, skipping downstream experimental checks.
- Main end-to-end test target now uses a longer 50-minute timeout and passes that timeout to the test runner to prevent hangs.
- The exported experimental E2E flags have been removed.

coderabbitai · 2026-05-11T18:14:45Z

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

@coderabbitai resume to resume automatic reviews.
@coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

▶️ Resume reviews
🔍 Trigger review

Walkthrough

Makefile edits in openshift/Makefile: test-experimental-e2e was replaced with a no-op (/bin/true) and the exported DOWNSTREAM_EXPERIMENTAL_E2E_FLAGS was removed; DOWNSTREAM_E2E_TIMEOUT := 50m was added and test-e2e's go test invocation now includes -timeout=$(DOWNSTREAM_E2E_TIMEOUT) and is reformatted across lines.

Changes

Openshift Makefile — E2E targets

Layer / File(s)	Summary
Timeout variable and comment `openshift/Makefile`	Add `export DOWNSTREAM_E2E_TIMEOUT := 50m` and a comment noting the default timeout bump.
test-e2e: test invocation formatting & timeout `openshift/Makefile`	Refactor `test-e2e` recipe's `go test` invocation across multiple lines and add `-timeout=$(DOWNSTREAM_E2E_TIMEOUT)` while keeping `--godog.tags="$(DOWNSTREAM_GODOG_FLAGS)"` and `--k8s.cli=oc`.
test-experimental-e2e: no-op replacement & flag removal `openshift/Makefile`	Replace previous build/test steps for `test-experimental-e2e` with `/bin/true` and remove the exported `DOWNSTREAM_EXPERIMENTAL_E2E_FLAGS` variable.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 12

✅ Passed checks (12 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title directly addresses the main change: fixing flags for test-experimental-e2e, which aligns with the primary modification of adjusting the target recipe and removing/adding related variables.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	PR only modifies openshift/Makefile (build config). No Ginkgo test files or test names are changed, so the check is not applicable.
Test Structure And Quality	✅ Passed	PR modifies only openshift/Makefile. No Ginkgo test code files in test/ were changed. Custom check for test code quality is not applicable to this PR.
Microshift Test Compatibility	✅ Passed	This PR modifies openshift/Makefile only—adding a timeout flag to test-e2e and disabling test-experimental-e2e. No new Ginkgo tests are added. The check is not applicable.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	PR only modifies openshift/Makefile (test execution targets), not test code. No new Ginkgo e2e tests are added, so custom check for SNO test compatibility is not applicable.
Topology-Aware Scheduling Compatibility	✅ Passed	PR modifies only openshift/Makefile test infrastructure. No deployment manifests, operator code, controllers, or scheduling constraints are introduced.
Ote Binary Stdout Contract	✅ Passed	PR only modifies openshift/Makefile for test orchestration. The test/e2e code added is a Godog BDD test suite, not an OTE binary. OTE stdout contract check does not apply.
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	This PR does not add new Ginkgo e2e tests. It only modifies openshift/Makefile to disable test-experimental-e2e and add timeout to test-e2e. Check applies only when new tests are added.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

Generate code and open pull requests
Plan features and break down work
Investigate incidents and troubleshoot customer tickets together
Automate recurring tasks and respond to alerts with triggers
Summarize progress and report instantly

Built for teams:

Shared memory across your entire org—no repeating context
Per-thread sandboxes to safely plan and execute work
Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

openshift-ci · 2026-05-11T18:15:05Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tmshort

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~DOWNSTREAM_OWNERS~~ [tmshort]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

tmshort · 2026-05-11T18:15:15Z

/test openshift-e2e-aws-techpreview

tmshort · 2026-05-11T18:28:08Z

/test openshift-e2e-aws-techpreview

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@openshift/Makefile`:
- Around line 47-48: Replace the short-circuit in the Makefile target
test-experimental-e2e (remove the "/bin/true" line) and restore the real
command(s) that run the experimental e2e suite so the target actually executes
tests; ensure the restored command invokes the same test runner used elsewhere
(e.g., the existing e2e script or binary) and passes the corrected flags/ENV
used for experimental runs so CI executes the suite instead of always
succeeding.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: c1ad50ea-2428-4502-86f2-c79d425a54c2

📥 Commits

Reviewing files that changed from the base of the PR and between 0c7d0b1 and 7aa1838.

📒 Files selected for processing (1)

openshift/Makefile

coderabbitai · 2026-05-11T18:29:17Z

 test-experimental-e2e: ## Run the experimental e2e tests.
-	$(DIR)/operator-controller/build-test-registry.sh $(E2E_REGISTRY_NAMESPACE) $(E2E_REGISTRY_NAME)
-	cd $(DIR)/../; \
-	go test $(DOWNSTREAM_EXPERIMENTAL_E2E_FLAGS) ./test/experimental-e2e/...;
+	/bin/true # keep - because it's triggered, but always succeed


⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Restore real test execution in test-experimental-e2e

Line 48 makes this target always pass, which bypasses experimental e2e coverage and can mask regressions in CI. If the goal is to fix flags, this should still run the suite with corrected flags instead of short-circuiting.

Suggested fix

.PHONY: test-experimental-e2e test-experimental-e2e: ## Run the experimental e2e tests. - /bin/true # keep - because it's triggered, but always succeed + $(DIR)/operator-controller/build-test-registry.sh $(E2E_REGISTRY_NAMESPACE) $(E2E_REGISTRY_NAME) + cd $(DIR)/../; \ + go test $(DOWNSTREAM_E2E_FLAGS) ./test/experimental-e2e/... --k8s.cli=oc

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openshift/Makefile` around lines 47 - 48, Replace the short-circuit in the Makefile target test-experimental-e2e (remove the "/bin/true" line) and restore the real command(s) that run the experimental e2e suite so the target actually executes tests; ensure the restored command invokes the same test runner used elsewhere (e.g., the existing e2e script or binary) and passes the corrected flags/ENV used for experimental runs so CI executes the suite instead of always succeeding.

This is intentional. The experimental e2e test now looks at the deployments featuregates to determine which features to test. So, having a separate test for experimental is redundant. It will eventually be removed, but right now it's still invoked, and if it were removed, it would cause the CI to fail.

@tmshort, thanks for the context — that makes sense. Since the experimental e2e functionality has been folded into the regular e2e suite (driven by deployment feature gates), keeping this target as a no-op prevents CI failures while the trigger is still wired up. My concern no longer applies.

✏️ Learnings added

Learnt from: tmshort Repo: openshift/operator-framework-operator-controller PR: 723 File: openshift/Makefile:47-48 Timestamp: 2026-05-11T18:39:21.641Z Learning: In `openshift/Makefile`, the `test-experimental-e2e` target is intentionally set to `/bin/true` (always succeed). The experimental e2e tests now look at deployment feature gates to determine which features to test, making a separate experimental test target redundant. The target is kept as a no-op to avoid CI failures while the CI job still invokes it; it will be removed once CI stops triggering it.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

tmshort · 2026-05-11T19:57:15Z

/test openshift-e2e-aws-techpreview

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@openshift/Makefile`:
- Around line 46-49: The Go test command in the Makefile uses the misspelled
flag "-tieout" which should be "-timeout"; update the invocation (the go test
line that includes $(DOWNSTREAM_E2E_FLAGS) and $(DOWNSTREAM_E2E_TIMEOUT)) to use
"-timeout=$(DOWNSTREAM_E2E_TIMEOUT)" so the Go test timeout is applied and the
make test-e2e target no longer fails due to an unknown flag.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 83c75b64-475b-430d-885d-47b1d524fd27

📥 Commits

Reviewing files that changed from the base of the PR and between 7aa1838 and d36da45.

📒 Files selected for processing (1)

openshift/Makefile

tmshort · 2026-05-12T14:01:36Z

/test openshift-e2e-aws-techpreview

tmshort · 2026-05-12T15:46:32Z

/test openshift-e2e-aws-techpreview

tmshort · 2026-05-12T18:55:23Z

/test openshift-e2e-aws-techpreview

Signed-off-by: Todd Short <tshort@redhat.com>

tmshort · 2026-05-13T12:59:45Z

/test openshift-e2e-aws-techpreview

@serial

Takes advantage of changes made to isolate test runs to execute as many tests in parallel as possible. For tests that must be run serially, the @serial tag has been added to the beginning of relevant feature file(s). Signed-off-by: Daniel Franz <dfranz@redhat.com> Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>

openshift-ci · 2026-05-13T13:31:07Z

@tmshort: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/openshift-e2e-aws	`f17a6e4`	link	true	`/test openshift-e2e-aws`
ci/prow/verify-commits	`f17a6e4`	link	true	`/test verify-commits`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

tmshort · 2026-05-13T13:48:37Z

/test openshift-e2e-aws-techpreview

openshift-ci Bot requested review from fgiudici and perdasilva May 11, 2026 18:15

openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 11, 2026

tmshort force-pushed the fix-experimental-e2e branch from 0c7d0b1 to 7aa1838 Compare May 11, 2026 18:26

coderabbitai Bot reviewed May 11, 2026

View reviewed changes

tmshort force-pushed the fix-experimental-e2e branch from 7aa1838 to d36da45 Compare May 11, 2026 19:52

coderabbitai Bot reviewed May 11, 2026

View reviewed changes

Comment thread openshift/Makefile

tmshort force-pushed the fix-experimental-e2e branch from d36da45 to c79c8d7 Compare May 12, 2026 13:33

tmshort force-pushed the fix-experimental-e2e branch from c79c8d7 to e073781 Compare May 12, 2026 15:46

tmshort force-pushed the fix-experimental-e2e branch from e073781 to 2687456 Compare May 12, 2026 18:51

tmshort added 2 commits May 12, 2026 19:36

UPSTREAM: <carry>: Fix flags passed to test-experimental-e2e

f20ef8c

Signed-off-by: Todd Short <tshort@redhat.com>

fixup! UPSTREAM: <carry>: Fix flags passed to test-experimental-e2e

f17a6e4

tmshort force-pushed the fix-experimental-e2e branch from 2687456 to f17a6e4 Compare May 12, 2026 23:52

Conversation

tmshort commented May 11, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented May 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviews paused

Walkthrough

Changes

Estimated code review effort

Uh oh!

openshift-ci Bot commented May 11, 2026

Uh oh!

tmshort commented May 11, 2026

Uh oh!

tmshort commented May 11, 2026

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

coderabbitai Bot May 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tmshort May 11, 2026

Choose a reason for hiding this comment

Uh oh!

coderabbitai Bot May 11, 2026

Choose a reason for hiding this comment

Uh oh!

tmshort commented May 11, 2026

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tmshort commented May 12, 2026

Uh oh!

tmshort commented May 12, 2026

Uh oh!

tmshort commented May 12, 2026

Uh oh!

tmshort commented May 13, 2026

Uh oh!

openshift-ci Bot commented May 13, 2026

Uh oh!

tmshort commented May 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

tmshort commented May 11, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented May 11, 2026 •

edited

Loading

coderabbitai Bot May 11, 2026 •

edited

Loading