Skip to content

Bug OCPBUGS-104497: Fix [sig-ci] prow job name OS version test for 4.23 rhcos9 - #31481

Open
redhat-chai-bot wants to merge 1 commit into
openshift:mainfrom
redhat-chai-bot:fix-4.23-rhcos9-os-version-test
Open

Bug OCPBUGS-104497: Fix [sig-ci] prow job name OS version test for 4.23 rhcos9#31481
redhat-chai-bot wants to merge 1 commit into
openshift:mainfrom
redhat-chai-bot:fix-4.23-rhcos9-os-version-test

Conversation

@redhat-chai-bot

@redhat-chai-bot redhat-chai-bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Summary

4.23 nightly jobs now run with RHCOS 9 (rhel-9 stream), but the [sig-ci] [Early] prow job name should match os version test in validateStandaloneNodeOS defaults to expecting rhel-10 when the job name has no explicit rhcos9/rhcos10 marker. This causes a permafail across all 4.23 e2e jobs starting with payload 4.23.0-0.nightly-2026-08-01-032511.

Root Cause

The if/else chain in validateStandaloneNodeOS that determines the expected targetStream handles explicit markers (rhcos9, rhcos10), -runc jobs (#31479), and upgrade jobs. For all other jobs it falls through to the rhel-10 default — which is correct for 5.0+ but wrong for 4.23 (which is still a 4.x release running RHEL 9).

The error:

MCP master uses rhel-9 as stream but was expecting rhel-10

Fix

Added an else clause that checks the cluster's desired version: if Major < 5, default targetStream to rhel-9. This correctly handles 4.23 (and any future 4.x releases) without affecting 5.0+ jobs.

Testing

  • go vet ./test/extended/ci/... — passes
  • go test ./test/extended/ci/... — passes (no unit test files for this Ginkgo e2e package, but compiles cleanly)

References


AI-generated. Review for accuracy.

@petr-muller requested in Slack thread

Summary by CodeRabbit

  • Bug Fixes
    • Corrected operating system selection for standalone node jobs based on the cluster’s installation version.
    • Older clusters now use RHEL 9, while newer clusters continue using RHEL 10.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

@openshift-ci openshift-ci Bot added the ready-for-human-review Indicates a PR has been reviewed by automated tools and is ready for human review label Aug 4, 2026
@openshift-ci-robot openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Aug 4, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@redhat-chai-bot: This pull request references Jira Issue OCPBUGS-104497, which is invalid:

  • expected the bug to target the "5.0.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Summary

4.23 nightly jobs now run with RHCOS 9 (rhel-9 stream), but the [sig-ci] [Early] prow job name should match os version test in validateStandaloneNodeOS defaults to expecting rhel-10 when the job name has no explicit rhcos9/rhcos10 marker. This causes a permafail across all 4.23 e2e jobs starting with payload 4.23.0-0.nightly-2026-08-01-032511.

Root Cause

The if/else chain in validateStandaloneNodeOS that determines the expected targetStream handles explicit markers (rhcos9, rhcos10), -runc jobs (#31479), and upgrade jobs. For all other jobs it falls through to the rhel-10 default — which is correct for 5.0+ but wrong for 4.23 (which is still a 4.x release running RHEL 9).

The error:

MCP master uses rhel-9 as stream but was expecting rhel-10

Fix

Added an else clause that checks the cluster's desired version: if Major < 5, default targetStream to rhel-9. This correctly handles 4.23 (and any future 4.x releases) without affecting 5.0+ jobs.

Testing

  • go vet ./test/extended/ci/... — passes
  • go test ./test/extended/ci/... — passes (no unit test files for this Ginkgo e2e package, but compiles cleanly)

References


AI-generated. Review for accuracy.

@petr-muller requested in Slack thread

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci
openshift-ci Bot requested review from deads2k and sjenning August 4, 2026 11:53
@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: fa0d1465-1241-481e-b62c-086f6e0f8ec2

📥 Commits

Reviewing files that changed from the base of the PR and between 4ccfabd and b67e0bd.

📒 Files selected for processing (1)
  • test/extended/ci/job_names.go

Walkthrough

validateStandaloneNodeOS now maps -runc jobs to rhel-9 and uses the cluster install major version for jobs without explicit OS markers. Install versions below 5 use rhel-9; other jobs keep rhel-10.

Changes

Standalone node OS selection

Layer / File(s) Summary
RHEL stream selection
test/extended/ci/job_names.go
-runc jobs now use rhel-9. Jobs without explicit OS markers use getInstallVersion; install major versions below 5 use rhel-9, and other versions use rhel-10.

Estimated code review effort: 2 (Simple) | ~5 minutes

Suggested reviewers: deads2k, sjenning, pablintino

🚥 Pre-merge checks | ✅ 15
✅ Passed checks (15 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the fix to the OS version test for 4.23 RHCOS 9 jobs, which matches the main code change.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed The PR changes only OS-stream logic; all Ginkgo titles in the file are static string literals with no dynamic values.
Test Structure And Quality ✅ Passed The PR modifies a helper function within existing test code. All existing Ginkgo tests meet quality requirements: single responsibility, appropriate setup patterns, proper assertion messages, and c...
Microshift Test Compatibility ✅ Passed The PR changes only existing validateStandaloneNodeOS logic; the diff adds no new Ginkgo It, Describe, Context, or When tests, so this check is not applicable.
Single Node Openshift (Sno) Test Compatibility ✅ Passed The patch changes existing target-stream logic only; it adds no Ginkgo test or multi-node assumption, so the SNO compatibility check is not applicable.
Topology-Aware Scheduling Compatibility ✅ Passed The patch changes only test/extended/ci/job_names.go and only adjusts OS stream selection; it adds no manifests, controllers, replicas, affinity, tolerations, selectors, or other scheduling constra...
Ote Binary Stdout Contract ✅ Passed The PR only changes targetStream logic in job_names.go; it adds no main/init/suite output, fmt.Print, klog, or stdout writes. NewCLI setup registers Ginkgo hooks without output.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed PR modifies existing validation logic, not adding new Ginkgo e2e tests (It(), Describe(), etc.); check is not applicable.
No-Weak-Crypto ✅ Passed The patch only adds version-based rhel-9 selection; added-line scans and import comparison show no weak crypto, custom crypto, or secret/token comparison.
Container-Privileges ✅ Passed The commit changes only Go test logic in test/extended/ci/job_names.go; its patch contains no privileged, host namespace, SYS_ADMIN, or privilege-escalation settings.
No-Sensitive-Data-In-Logs ✅ Passed PR adds no new logging statements. Existing error messages log only non-sensitive system data (MCP names, stream versions, cluster versions), not passwords, tokens, keys, PII, or customer data.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

@redhat-chai-bot

Copy link
Copy Markdown
Contributor Author

/retest-required

GCP job failures are infra-related (node provisioning timeout: "6 ready nodes expected, found 5"), not caused by this change. All AWS/vSphere/metal jobs passed.


AI-generated. Review for accuracy.

@redhat-chai-bot

Copy link
Copy Markdown
Contributor Author

/test e2e-gcp-csi
/test e2e-gcp-ovn


AI-generated. Review for accuracy.

@neisw

neisw commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

/payload-job periodic-ci-openshift-release-main-ci-4.23-e2e-aws-ovn-techpreview

@openshift-ci

openshift-ci Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

@neisw: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-main-ci-4.23-e2e-aws-ovn-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/a13fb8c0-904f-11f1-864b-9ddbddabc300-0

Comment thread test/extended/ci/job_names.go Outdated
Comment on lines +285 to +286
// For non-upgrade, non-explicit-rhcos jobs: 4.x clusters default to rhel-9
desiredVersion, err := utilversion.ParseGeneric(clusterVersion.Status.Desired.Version)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@redhat-chai-bot Is there a good reason for having separate branches for upgrade and "else if current version < 5"?

It seems to me that we can have a single "if was installed as <5 then expect rhel-9" (unless explicit, per rhcos10 marker, handled by earlier branch), and maybe just add a comment/log about the upgrade special case (OCP5 clusters will have RHEL9 if they were installed as OCP4 and upgraded)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. Yes, there is a reason they're separate: for upgrade tests (4.x → 5.x), the cluster's desiredVersion after upgrade is 5.x, so the Major() < 5 check in the else branch would be false — it would incorrectly default to rhel-10. But the MCP stream is still rhel-9 because that's what was originally installed. The upgrade branch catches this via the job name.

In other words:

  • Upgrade job (4.x→5.x): desiredVersion = 5.x, MCP stream = rhel-9 → needs job-name-based detection
  • Non-upgrade 4.x job: desiredVersion = 4.x, MCP stream = rhel-9 → caught by version check

A single "if was installed as <5" branch could work if we checked the ClusterVersion's .status.history for the original installation version instead of desired.version, but that's a bigger change. Happy to refactor that way if you'd prefer, or we could add a comment here explaining the distinction and consider the consolidation as a follow-up.


AI-generated. Review for accuracy.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@redhat-chai-bot if we only check the installation version through the helper then we do not need to care about desired version at all

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right — checking the installation version through the ClusterVersion history would let us consolidate the upgrade and else branches into a single check. I'll refactor to use .status.history to find the initial installation version. If its Major < 5, we expect rhel-9 regardless of whether it's an upgrade or not.

Working on this now.


AI-generated. Review for accuracy.

@redhat-chai-bot

Copy link
Copy Markdown
Contributor Author

/test e2e-gcp-csi


AI-generated. Review for accuracy.

@petr-muller

Copy link
Copy Markdown
Member

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Aug 5, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@petr-muller: This pull request references Jira Issue OCPBUGS-104497, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)
Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

…23 rhcos9

4.23 nightly jobs now run with RHCOS 9 (rhel-9 stream). The
validateStandaloneNodeOS check defaulted the expected stream to rhel-10
for jobs without an explicit rhcos9/rhcos10 marker, causing all 4.23 e2e
jobs to fail with 'MCP master uses rhel-9 as stream but was expecting
rhel-10'. Add an else clause so non-upgrade, non-explicit-rhcos jobs on
4.x clusters (Major < 5) default to rhel-9.
@redhat-chai-bot
redhat-chai-bot force-pushed the fix-4.23-rhcos9-os-version-test branch from 4b6c842 to b67e0bd Compare August 5, 2026 12:15
@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@redhat-chai-bot

Copy link
Copy Markdown
Contributor Author

/payload-job periodic-ci-openshift-release-main-ci-5.0-e2e-aws-ovn periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-aws-ovn-upgrade periodic-ci-openshift-release-main-ci-4.23-upgrade-from-stable-4.22-e2e-aws-ovn-upgrade periodic-ci-openshift-release-main-nightly-4.23-e2e-aws-ovn-serial-1of2


AI-generated. Review for accuracy.

@openshift-ci

openshift-ci Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

@redhat-chai-bot: trigger 4 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-main-ci-5.0-e2e-aws-ovn
  • periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-aws-ovn-upgrade
  • periodic-ci-openshift-release-main-ci-4.23-upgrade-from-stable-4.22-e2e-aws-ovn-upgrade
  • periodic-ci-openshift-release-main-nightly-4.23-e2e-aws-ovn-serial-1of2

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/72f92be0-90c7-11f1-8e18-dc1dc88e9af8-0

@openshift-ci-robot

Copy link
Copy Markdown

@redhat-chai-bot: This pull request references Jira Issue OCPBUGS-104497, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
Details

In response to this:

Summary

4.23 nightly jobs now run with RHCOS 9 (rhel-9 stream), but the [sig-ci] [Early] prow job name should match os version test in validateStandaloneNodeOS defaults to expecting rhel-10 when the job name has no explicit rhcos9/rhcos10 marker. This causes a permafail across all 4.23 e2e jobs starting with payload 4.23.0-0.nightly-2026-08-01-032511.

Root Cause

The if/else chain in validateStandaloneNodeOS that determines the expected targetStream handles explicit markers (rhcos9, rhcos10), -runc jobs (#31479), and upgrade jobs. For all other jobs it falls through to the rhel-10 default — which is correct for 5.0+ but wrong for 4.23 (which is still a 4.x release running RHEL 9).

The error:

MCP master uses rhel-9 as stream but was expecting rhel-10

Fix

Added an else clause that checks the cluster's desired version: if Major < 5, default targetStream to rhel-9. This correctly handles 4.23 (and any future 4.x releases) without affecting 5.0+ jobs.

Testing

  • go vet ./test/extended/ci/... — passes
  • go test ./test/extended/ci/... — passes (no unit test files for this Ginkgo e2e package, but compiles cleanly)

References


AI-generated. Review for accuracy.

@petr-muller requested in Slack thread

Summary by CodeRabbit

  • Bug Fixes
  • Corrected operating system selection for standalone node jobs based on the cluster’s installation version.
  • Older clusters now use RHEL 9, while newer clusters continue using RHEL 10.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Aug 5, 2026
@openshift-ci

openshift-ci Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: petr-muller, redhat-chai-bot

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 5, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. ready-for-human-review Indicates a PR has been reviewed by automated tools and is ready for human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants