Update plc eval test case for version-aware verdicts - #2
Merged
jhadvig merged 1 commit intoJul 24, 2026
Conversation
Align plc_proposal_olm_batch_check with the new olm-check output from openshift#43: test the version-aware found/not-found split instead of the old all-versions dump. Changes: - Switch cluster-logging to v6.3.1 (tracked but not OCP 4.21 compatible) - Replace operators_with/without_lifecycle_data with three finer counts: api_tracked (4), version_tracked (3), ocp_compatible (2) - Update query to ask for per-version and compatibility breakdown
3 tasks
Collaborator
Author
|
tests here openshift#43 (comment) and here openshift#43 (comment) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
plc_proposal_olm_batch_checkeval test case to match thenew per-operator verdict output from OTA-2070: Return per-operator verdicts from olm-check openshift/agentic-skills#43
granular counts:
operators_api_tracked_count,operators_version_tracked_count,operators_ocp_compatible_count"tracked but not OCP-compatible" case
Context
PR openshift#43 refactored
olm-checkto return explicitfound: true/falseverdicts per operator+version instead of dumping all version entries.
This eval change exercises that new output structure so evals validate
the correct 4/3/2 split in the test
plc_proposal_olm_batch_check:operators_api_tracked_countoperators_version_tracked_countoperators_ocp_compatible_countTest plan