OCPBUGS-115111: skip Watchdog invariant for External topology

[vsolanki12]

Summary

The [bz-monitoring][invariant] alert/Watchdog must have no gaps or changes invariant fails with "Watchdog alert not found" on HyperShift External topology clusters. In External topology, the monitoring stack runs on the management cluster, so Watchdog alert intervals are not captured from the guest cluster's perspective.

Skip the Watchdog invariant check when jobType.Topology == "external" since the alert absence is expected, not a Prometheus outage.

Evidence

Conformance test run against HyperShift External topology cluster (4.18.12, aws, External, OVN):

FAIL: [Monitor:legacy-test-framework-invariants-alerts][bz-monitoring][invariant] alert/Watchdog must have no gaps or changes
  Watchdog alert not found

Note: KubePodNotReady alerts all PASSED — the previous implementation fixing KubePodNotReady grace periods was addressing the wrong failure.

Test plan

• New unit tests pass (go test -run TestWatchdogInvariantCheck ./pkg/monitortestlibrary/allowedalerts/)
• Tests cover: HA with/without Watchdog, External with/without Watchdog
• Conformance test on External topology no longer fails Watchdog invariant

/jira OCPBUGS-115111

Summary by CodeRabbit

• Bug Fixes

  • External-topology jobs now pass monitoring checks when Watchdog intervals are missing, since monitoring runs on the management cluster.
  • Existing failure behavior for missing Watchdog intervals remains unchanged for other topologies.

• Tests

  • Added coverage for Watchdog alert validation across supported topologies, including expected pass and failure scenarios.

[openshift-merge-bot]

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[openshift-ci-robot]

@vsolanki12: This pull request references Jira Issue OCPBUGS-115111, which is valid.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (5.1.0) matches configured target version for branch (5.1.0)
• bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

What does this PR do?

Tolerates bounded, expected KubePodNotReady alerts during HyperShift guest-cluster startup.

The exception applies only to external control-plane topology and the affected
openshift-dns, openshift-insights, and openshift-ingress-canary
namespaces.

Why is this change needed?

HyperShift guest clusters can temporarily report these alerts while initial
platform pods converge. The monitor previously evaluated the startup intervals
with normal thresholds, causing false failures.

Intervals that continue beyond the five-minute startup grace period remain
subject to the existing checks.

How was this tested?

• go test ./pkg/monitortests/testframework/legacytestframeworkmonitortests
• go test ./pkg/monitortests/testframework/... ./pkg/cmd/openshift-tests/dev
• go test ./pkg/monitortests/...

Jira

https://issues.redhat.com/browse/OCPBUGS-115111

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: vsolanki12
Once this PR has been reviewed and has the lgtm label, please assign miyadav for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Advanced

Run ID: a13ad15a-5a94-4fd6-808c-adeec057215d

📥 Commits

Reviewing files that changed from the base of the PR and between 3c85f76 and d8a0e42.

📒 Files selected for processing (2)

• pkg/monitortestlibrary/allowedalerts/watchdog.go
• pkg/monitortestlibrary/allowedalerts/watchdog_test.go

Included review availability: Your plan provides up to 8 included reviews per hour; 3 remain after this review.

---

Walkthrough

InvariantCheck now passes when external-topology jobs have no Watchdog intervals. Other topologies retain the existing failure. New tests cover four alert and topology combinations.

Changes

Watchdog invariant handling

Layer / File(s)	Summary
Topology-aware Watchdog check pkg/monitortestlibrary/allowedalerts/watchdog.go, pkg/monitortestlibrary/allowedalerts/watchdog_test.go	External topology passes without a Watchdog alert. HA topology still fails without the alert. Tests cover both topologies with and without the alert.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to d8a0e

The Watchdog invariant now permits missing intervals for external-topology jobs while preserving the HA requirement, with tests covering both behaviors. The change is mergeable.

🚥 Pre-merge checks | ✅ 14 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 6 functions across 5 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (14 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: skipping the Watchdog invariant check for External topology.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	The added test uses a static TestWatchdogInvariantCheck name and four static t.Run names. The names contain only fixed topology and Watchdog-state descriptions. They contain no pod names, timestam…
Test Structure And Quality	✅ Passed	PASS. The changed test is a standard Go testing table-driven test with testify, not Ginkgo code. Each subtest checks one Watchdog invariant case. It creates no cluster resources and performs no wa…
Microshift Test Compatibility	✅ Passed	PASS: The pull request adds a standard Go testing unit test, TestWatchdogInvariantCheck, and does not add any Ginkgo It, Describe, Context, or When e2e test. The MicroShift compatibility c…
Single Node Openshift (Sno) Test Compatibility	✅ Passed	The PR adds a standard Go testing unit test, TestWatchdogInvariantCheck, with t.Run and Testify assertions. It does not add a Ginkgo e2e test or any multi-node/HA assumption. The custom SNO chec…
Topology-Aware Scheduling Compatibility	✅ Passed	PASS: The pull request changes only the Watchdog invariant logic and its unit test in pkg/monitortestlibrary/allowedalerts/. It does not add or modify deployment manifests, operators, controllers, r…
Ote Binary Stdout Contract	✅ Passed	The pull request introduces no process-level stdout writes. The production change only returns a JUnit test case for External topology. The added test contains no main, init, suite hook, logging, or s…
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	PASS: The pull request adds a standard Go testing unit test, not a Ginkgo e2e test. The test only constructs local monitorapi intervals and calls InvariantCheck; it contains no IPv4 assumptions,…
No-Weak-Crypto	✅ Passed	The pull request changes only the Watchdog invariant and its unit test. The changed code adds an external topology branch that returns a passing JUnit result and adds test data; it introduces no MD5…
Container-Privileges	✅ Passed	PASS. The PR changes only two Go source/test files. It adds no container or Kubernetes manifest settings. Scans of the changed files and added lines found no privileged:true, hostPID, hostNetwork, hos…
No-Sensitive-Data-In-Logs	✅ Passed	The pull request adds only topology branching, comments, and unit-test code. It does not add logging or output of passwords, tokens, API keys, PII, session IDs, hostnames, or customer data. Existing W…

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create a new PR

---

_{Comment @coderabbitai help to get the list of available commands.}

[openshift-merge-bot]

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6

[vsolanki12]

@coderabbitai resume

[coderabbitai]

⚠️ Action not completed

Already reviewed the last commit. Use @coderabbitai full review to rerun a review of the entire changeset.

[vsolanki12]

@coderabbitai resume

[coderabbitai]

⚠️ Action not completed

Already reviewed the last commit. Use @coderabbitai full review to rerun a review of the entire changeset.

[vsolanki12]

@coderabbitai full review

[coderabbitai]

✅ Action performed

Full review finished.

[vsolanki12]

/test verify

[openshift-ci]

@vsolanki12: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/verify	d8a0e42	link	true	/test verify

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.