Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions web/cypress.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ export default defineConfig({
typeDelay: 200,
},
fixturesFolder: 'cypress/fixtures',
pageLoadTimeout: 60000,
defaultCommandTimeout: 80000, //due to performance loading issue on console
readyTimeoutMilliseconds: 120000,
installTimeoutMilliseconds: 600000,
Expand Down
28 changes: 7 additions & 21 deletions web/cypress/e2e/incidents/01.incidents.cy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,11 @@
The test verifies the basic functionality of the Incidents page and serves
as a verification that the Incidents View is working as expected.

Currently, it depends on an alert being present in the cluster.
In the future, mocking requests / injecting alerts should be considered.
Natural creation of the alert is done in the 00.coo_incidents_e2e.cy.ts test,
but takes significant time.
All tests use mocked data. Tests 1-3 use a default fixture (incident content
is irrelevant for toolbar/filter verification). Tests 4-5 switch mocks
mid-test for empty state and traversal scenarios.
*/

import { commonPages } from '../../views/common';
import { incidentsPage } from '../../views/incidents-page';

const MCP = {
Expand All @@ -26,24 +24,13 @@ const MP = {
operatorName: 'Cluster Monitoring Operator',
};

const ALERTNAME = 'Watchdog';
const NAMESPACE = 'openshift-monitoring';
const SEVERITY = 'Critical';
const ALERT_DESC = 'This is an alert meant to ensure that the entire alerting pipeline is functional. This alert is always firing, therefore it should always be firing in Alertmanager and always fire against a receiver. There are integrations with various notification mechanisms that send a notification when this alert is not firing. For example the "DeadMansSnitch" integration in PagerDuty.'
const ALERT_SUMMARY = 'An alert that should always be firing to certify that Alertmanager is working properly.'
describe('BVT: Incidents - UI', { tags: ['@smoke', '@incidents'] }, () => {
before(() => {
cy.beforeBlockCOO(MCP, MP, { dashboards: false, troubleshootingPanel: false });
cy.mockIncidentFixture('incident-scenarios/1-single-incident-firing-critical-and-warning-alerts.yaml');
});


beforeEach(() => {
cy.log('Navigate to Observe → Incidents');
incidentsPage.goTo();
// Temporary workaround for testing against locally built instances.
cy.transformMetrics();
});

it('1. Admin perspective - Incidents page - Toolbar and charts toggle functionality', () => {
cy.log('1.1 Verify toolbar and toggle charts button');
incidentsPage.elements.toolbar().should('be.visible');
Expand Down Expand Up @@ -80,24 +67,23 @@ describe('BVT: Incidents - UI', { tags: ['@smoke', '@incidents'] }, () => {
cy.log('4.1 Verify chart titles are visible');
incidentsPage.elements.incidentsChartTitle().should('be.visible');
incidentsPage.elements.alertsChartTitle().should('be.visible');

cy.log('4.2 Verify alerts chart shows empty state');
incidentsPage.elements.alertsChartEmptyState().should('exist');
});

it('5. Admin perspective - Incidents page - Traverse Incident Table', () => {
incidentsPage.goTo();

cy.log('5.1 Traverse incident table');
incidentsPage.clearAllFilters();
cy.mockIncidents([]);
incidentsPage.findIncidentWithAlert('TargetAlert').should('be.false');
Comment on lines +75 to 79
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Set the mock before the first page load.

incidentsPage.goTo() runs before cy.mockIncidents([]), so the initial Incidents requests can fire against stale/no intercepts and then the helper navigates again. Let the mock helper own the navigation.

Suggested fix
   it('5. Admin perspective - Incidents page - Traverse Incident Table', () => {
-    incidentsPage.goTo();
-
     cy.log('5.1 Traverse incident table');
     cy.mockIncidents([]);
     incidentsPage.findIncidentWithAlert('TargetAlert').should('be.false');
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
incidentsPage.goTo();
cy.log('5.1 Traverse incident table');
incidentsPage.clearAllFilters();
cy.mockIncidents([]);
incidentsPage.findIncidentWithAlert('TargetAlert').should('be.false');
it('5. Admin perspective - Incidents page - Traverse Incident Table', () => {
cy.log('5.1 Traverse incident table');
cy.mockIncidents([]);
incidentsPage.findIncidentWithAlert('TargetAlert').should('be.false');
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@web/cypress/e2e/incidents/01.incidents.cy.ts` around lines 75 - 79, Move the
mock setup before navigating so the network intercepts are in place: call
cy.mockIncidents([]) prior to incidentsPage.goTo(), or remove the explicit
incidentsPage.goTo() and let cy.mockIncidents([]) perform the navigation itself;
ensure the sequence around
incidentsPage.findIncidentWithAlert('TargetAlert').should('be.false') happens
only after the mock is applied (referencing cy.mockIncidents,
incidentsPage.goTo, and incidentsPage.findIncidentWithAlert).


cy.log('5.2 Verify traversing incident table works when the alert is not present');
cy.mockIncidentFixture('incident-scenarios/1-single-incident-firing-critical-and-warning-alerts.yaml');
incidentsPage.findIncidentWithAlert('TargetAlert').should('be.false');

incidentsPage.clearAllFilters
cy.log('5.3 Verify traversing incident table works when the alert is present');
cy.mockIncidentFixture('incident-scenarios/6-multi-incident-target-alert-scenario.yaml');
incidentsPage.findIncidentWithAlert('TargetAlert').should('be.true');
incidentsPage.findIncidentWithAlert('TargetAlert').should('be.true');
});
});
2 changes: 0 additions & 2 deletions web/cypress/e2e/incidents/regression/01.reg_filtering.cy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,6 @@ describe('Regression: Incidents Filtering', { tags: ['@incidents'] }, () => {
});

beforeEach(() => {
cy.log('Navigate to Observe → Incidents');
incidentsPage.goTo();
cy.log('Setting up comprehensive filtering test scenarios');
cy.mockIncidentFixture('incident-scenarios/7-comprehensive-filtering-test-scenarios.yaml');
});
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,6 @@ describe('Regression: Charts UI - Comprehensive', { tags: ['@incidents'] }, () =
});

beforeEach(() => {
incidentsPage.goTo();
cy.mockIncidentFixture('incident-scenarios/12-charts-ui-comprehensive.yaml');
});

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
/*
Regression test for mixed severity interval boundary times (Section 2.3.3).

Test 1 (OU-1205): Verifies tooltip End times at severity interval boundaries
are rounded to 5-minute precision. Without rounding, consecutive interval end
times can land on non-5-minute values (e.g., "10:58" instead of "11:00").

Test 2 (OU-1221, XFAIL): Verifies Start times shown in incident tooltips match
alert tooltips and alerts table, and that consecutive segment boundaries align
with no 5-minute gap between End of one segment and Start of the next.
*/

import { incidentsPage } from '../../../views/incidents-page';

const MCP = {
namespace: Cypress.env('COO_NAMESPACE'),
packageName: 'cluster-observability-operator',
operatorName: 'Cluster Observability Operator',
config: {
kind: 'UIPlugin',
name: 'monitoring',
},
};

const MP = {
namespace: 'openshift-monitoring',
operatorName: 'Cluster Monitoring Operator',
};

describe('Regression: Mixed Severity Interval Boundary Times', { tags: ['@incidents', '@xfail'] }, () => {

before(() => {
cy.beforeBlockCOO(MCP, MP, { dashboards: false, troubleshootingPanel: false });
});

beforeEach(() => {
cy.mockIncidentFixture('incident-scenarios/21-multi-severity-boundary-times.yaml');
});

const extractTime = (tooltipText: string, field: 'Start' | 'End'): string => {
const afterField = tooltipText.split(field)[1] || '';
const match = afterField.match(/(\d{1,2}:\d{2}(\s*[AP]M)?)/);
return match ? match[1].trim() : '';
};

it('1. Tooltip End times at severity boundaries show 5-minute rounded values', () => {
const verifyEndTimeRounded = (label: string) => {
incidentsPage.elements.tooltip()
.invoke('text')
.then((text) => {
cy.log(`${label} tooltip: "${text}"`);

if (text.match(/End.*---/)) {
cy.log(`${label}: Firing, End shows --- (skipped)`);
return;
}

const endPart = text.split('End')[1];
expect(endPart, `${label}: should contain End time`).to.exist;

const timeMatch = endPart.match(/(\d{1,2}):(\d{2})/);
expect(timeMatch, `${label}: End time should be parseable`).to.not.be.null;

const minutes = parseInt(timeMatch[2], 10);
const remainder = minutes % 5;
expect(remainder, `${label}: End minutes (${minutes}) should be divisible by 5, remainder=${remainder}`).to.equal(0);
cy.log(`${label}: End ${timeMatch[1]}:${timeMatch[2]} - minutes divisible by 5`);
});
};

cy.log('1.1 Verify multi-severity incident loaded');
incidentsPage.clearAllFilters();
incidentsPage.setDays('1 day');
incidentsPage.elements.incidentsChartContainer().should('be.visible');
incidentsPage.elements.incidentsChartBarsGroups().should('have.length', 1);

cy.log('1.2 Verify bar has multiple severity segments');
incidentsPage.getIncidentBarVisibleSegments(0).then((segments) => {
expect(segments.length, 'Multi-severity bar should have at least 2 visible segments')
.to.be.greaterThan(1);
cy.log(`Found ${segments.length} visible severity segments`);
});

cy.log('1.3 Check first segment end time (Info -> Warning boundary)');
incidentsPage.hoverOverIncidentBarSegment(0, 0);
verifyEndTimeRounded('First segment');

cy.log('1.4 Check second segment end time (Warning -> Critical boundary)');
incidentsPage.hoverOverIncidentBarSegment(0, 1);
verifyEndTimeRounded('Second segment');

cy.log('1.5 Check third segment end time (Critical end)');
incidentsPage.hoverOverIncidentBarSegment(0, 2);
verifyEndTimeRounded('Third segment');

cy.log('Verified: All tooltip End times at severity boundaries are at 5-minute precision (OU-1205)');
});

it('2. Start times match between incident tooltip, alert tooltip, and table; consecutive boundaries align',
() => {
cy.log('2.1 Setup: verify multi-severity incident loaded');
incidentsPage.clearAllFilters();
incidentsPage.setDays('1 day');
incidentsPage.elements.incidentsChartContainer().should('be.visible');
incidentsPage.elements.incidentsChartBarsGroups().should('have.length', 1);
cy.pause();

cy.log('2.2 Consecutive interval boundaries: End of segment 1 should equal Start of segment 2');
incidentsPage.hoverOverIncidentBarSegment(0, 0);
incidentsPage.elements.tooltip().invoke('text').then((firstText) => {
const firstEnd = extractTime(firstText, 'End');
cy.log(`First segment End: ${firstEnd}`);
expect(firstEnd, 'First segment End should be parseable').to.not.be.empty;

incidentsPage.hoverOverIncidentBarSegment(0, 1);
incidentsPage.elements.tooltip().invoke('text').then((secondText) => {
const secondStart = extractTime(secondText, 'Start');
cy.log(`Second segment Start: ${secondStart}`);
expect(secondStart, 'Second segment Start should be parseable').to.not.be.empty;
expect(secondStart,
`No 5-min gap: second Start (${secondStart}) should equal first End (${firstEnd})`
).to.equal(firstEnd);
});
});
cy.pause();

cy.log('2.3 Incident tooltip Start vs alert tooltip Start vs alerts table Start');
incidentsPage.hoverOverIncidentBarSegment(0, 0);
incidentsPage.elements.tooltip().invoke('text').then((incidentText) => {
const incidentStart = extractTime(incidentText, 'Start');
cy.log(`Incident tooltip Start: ${incidentStart}`);
expect(incidentStart, 'Incident Start should be parseable').to.not.be.empty;

cy.log('2.4 Select incident and get alert tooltip Start');
incidentsPage.selectIncidentById('monitoring-multi-severity-boundary-001');
incidentsPage.elements.alertsChartCard().should('be.visible');

incidentsPage.hoverOverAlertBar(0);
incidentsPage.elements.alertsChartTooltip().invoke('text').then((alertText) => {
const alertStart = extractTime(alertText, 'Start');
cy.log(`Alert tooltip Start: ${alertStart}`);
expect(incidentStart,
`Incident Start (${incidentStart}) should match alert Start (${alertStart})`
).to.equal(alertStart);
});

cy.log('2.5 Compare incident tooltip Start with alerts table Start');
incidentsPage.getSelectedIncidentAlerts().then((alerts) => {
expect(alerts.length, 'Should have at least 1 alert row').to.be.greaterThan(0);
alerts[0].getStartCell().invoke('text').then((cellText) => {
const tableMatch = cellText.trim().match(/(\d{1,2}:\d{2}(\s*[AP]M)?)/);
expect(tableMatch, 'Table Start time should be parseable').to.not.be.null;
const tableStart = tableMatch[1].trim();
cy.log(`Alerts table Start: ${tableStart}`);
expect(incidentStart,
`Incident Start (${incidentStart}) should match table Start (${tableStart})`
).to.equal(tableStart);
});
});
});
cy.pause();

cy.log('Expected failure: Incident tooltip Start times are 5 minutes off (OU-1221)');
});
});
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,13 @@ const MP = {
describe('Regression: Time-Based Alert Resolution (E2E with Firing Alerts)', { tags: ['@incidents', '@slow', '@e2e-real'] }, () => {
let currentAlertName: string;


before(() => {
cy.beforeBlockCOO(MCP, MP, { dashboards: false, troubleshootingPanel: false });

cy.log('Create or reuse firing alert for testing');
cy.createKubePodCrashLoopingAlert('TimeBasedResolution2').then((alertName) => {
cy.log('Create firing alert for testing');
cy.cleanupIncidentPrometheusRules();
cy.createKubePodCrashLoopingAlert().then((alertName) => {
Comment on lines +43 to +45
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

# First, locate and read the test file at the specified lines
fd "03-04.reg_e2e_firing_alerts.cy.ts" --type f

Repository: openshift/monitoring-plugin

Length of output: 141


🏁 Script executed:

# Read the test file around lines 43-45 with context
cat -n web/cypress/e2e/incidents/regression/03-04.reg_e2e_firing_alerts.cy.ts | sed -n '30,80p'

Repository: openshift/monitoring-plugin

Length of output: 2164


🏁 Script executed:

# Check the cleanup command implementation
cat -n web/cypress/support/commands/incident-commands.ts | sed -n '80,110p'

Repository: openshift/monitoring-plugin

Length of output: 1117


🏁 Script executed:

# Check if there are fixed waits at the mentioned lines
cat -n web/cypress/e2e/incidents/regression/03-04.reg_e2e_firing_alerts.cy.ts | sed -n '50,60p'
cat -n web/cypress/e2e/incidents/regression/03-04.reg_e2e_firing_alerts.cy.ts | sed -n '160,175p'
cat -n web/cypress/e2e/incidents/regression/03-04.reg_e2e_firing_alerts.cy.ts | sed -n '260,275p'

Repository: openshift/monitoring-plugin

Length of output: 1855


🏁 Script executed:

# Verify the structure of cy.exec to confirm it doesn't wait for K8s resource termination
cat -n web/cypress/support/commands/incident-commands.ts | sed -n '85,105p'

Repository: openshift/monitoring-plugin

Length of output: 979


🏁 Script executed:

# Check if there's a wait wrapper or polling after cleanup calls
rg "cleanupIncidentPrometheusRules" -A 5 -B 2 web/cypress/

Repository: openshift/monitoring-plugin

Length of output: 3027


Add explicit wait for resource deletion before recreating alert.

cy.cleanupIncidentPrometheusRules() only initiates the deletion via cy.exec(); it does not wait for the PrometheusRule and fixtures to actually disappear from the cluster. When cy.createKubePodCrashLoopingAlert() runs immediately after, old resources may still be terminating, creating a race condition that causes nondeterministic test behavior.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@web/cypress/e2e/incidents/regression/03-04.reg_e2e_firing_alerts.cy.ts`
around lines 43 - 45, cy.cleanupIncidentPrometheusRules() only triggers deletion
but doesn't wait for resources to be gone, causing a race when
cy.createKubePodCrashLoopingAlert() runs next; update the test to wait until the
PrometheusRule and related fixtures are fully deleted before calling
cy.createKubePodCrashLoopingAlert() by adding an explicit polling/wait step
(e.g., loop with cy.exec('kubectl get prometheusrule ...') or use an existing
helper like waitForPrometheusRuleDeletion) that verifies absence of the specific
alert name returned by cy.cleanupIncidentPrometheusRules() or by querying the
cluster until resources are removed.

currentAlertName = alertName;
cy.log(`Test will monitor alert: ${currentAlertName}`);
});
Expand All @@ -50,6 +52,7 @@ describe('Regression: Time-Based Alert Resolution (E2E with Firing Alerts)', { t
it('1. Section 3.3 - Alert not incorrectly marked as resolved after time passes', () => {
cy.log('1.1 Navigate to Incidents page and clear filters');
incidentsPage.goTo();
cy.wait(10000);
incidentsPage.clearAllFilters();

const intervalMs = 60_000;
Expand Down Expand Up @@ -161,6 +164,7 @@ describe('Regression: Time-Based Alert Resolution (E2E with Firing Alerts)', { t
it('2. Section 4.7 - Prometheus query end time updates to current time on filter refresh', () => {
cy.log('2.1 Navigate to Incidents page and clear filters');
incidentsPage.goTo();
cy.wait(10000);
incidentsPage.clearAllFilters();

cy.log('2.2 Capture initial page load time');
Expand Down Expand Up @@ -262,6 +266,7 @@ describe('Regression: Time-Based Alert Resolution (E2E with Firing Alerts)', { t
it('3. Verify alert lifecycle - alert continues firing throughout test', () => {
cy.log('3.1 Navigate to Incidents page');
incidentsPage.goTo();
cy.wait(10000);
incidentsPage.clearAllFilters();

cy.log('3.2 Search for and select incident with custom alert');
Expand Down
52 changes: 45 additions & 7 deletions web/cypress/e2e/incidents/regression/03.reg_api_calls.cy.ts
Original file line number Diff line number Diff line change
@@ -1,15 +1,20 @@
/*
Regression test for Silences Not Applied Correctly (Section 3.2)
Regression tests for API Calls and Data Loading (Section 3)

BUG: Silences were being matched by name only, not by name + namespace + severity.
This test verifies that silence matching uses: alertname + namespace + severity.
Tests:
1. Silences Not Applied Correctly (Section 3.2)
BUG: Silences were being matched by name only, not by name + namespace + severity.
This test verifies that silence matching uses: alertname + namespace + severity.

While targeting the bug, it verifies the basic Silences Implementation.
2. Permission Denied Handling (Section 3.5)
Tests graceful handling of 403 Forbidden responses from rules/silences endpoints.
Incidents page should still function when user lacks permissions to view rules/silences.

Verifies: OU-1020, OU-706
Verifies: OU-1020, OU-706, OU-1213
*/

import { incidentsPage } from '../../../views/incidents-page';
import { nav } from '../../../views/nav';

const MCP = {
namespace: Cypress.env('COO_NAMESPACE'),
Expand All @@ -33,8 +38,6 @@ describe('Regression: Silences Not Applied Correctly', { tags: ['@incidents'] },
});

beforeEach(() => {
cy.log('Navigate to Observe → Incidents');
incidentsPage.goTo();
cy.log('Setting up silenced alerts mixed scenario');
cy.mockIncidentFixture('incident-scenarios/9-silenced-alerts-mixed-scenario.yaml');
});
Expand Down Expand Up @@ -125,4 +128,39 @@ describe('Regression: Silences Not Applied Correctly', { tags: ['@incidents'] },
});
});

describe('Regression: Permission Denied Handling', { tags: ['@incidents'] }, () => {

before(() => {
cy.beforeBlockCOO(MCP, MP, { dashboards: false, troubleshootingPanel: false });
});

beforeEach(() => {
cy.log('Mock all API endpoints as 403 Forbidden');
cy.mockPermissionDenied();
cy.log('Navigate to Observe -> Incidents');
// Using custom navigation commands to avoid waiting for the page to load which never happens in this test
nav.sidenav.clickNavLink(['Observe', 'Alerting']);
nav.tabs.switchTab('Incidents');

});

it('Page displays access denied state when all API endpoints return 403 Forbidden', () => {
cy.log('1.1 Verify 403 requests were intercepted');
const waitTimeout = { timeout: 120000 };
cy.wait('@rulesPermissionDenied', waitTimeout)
.its('response').should('exist')
.its('statusCode').should('eq', 403);
cy.wait('@silencesPermissionDenied', waitTimeout)
.its('response').should('exist')
.its('statusCode').should('eq', 403);
cy.wait('@prometheusQueryRangePermissionDenied', waitTimeout)
.its('response').should('exist')
.its('statusCode').should('eq', 403);

cy.log('1.2 Verify access denied empty state is displayed');
cy.byTestID('access-denied').should('be.visible');
cy.byTestID('access-denied').should('contain.text', 'You don\'t have access to this section due to cluster policy');

cy.log('Verified: Page displays restricted access state for permission denied');
});
});
Loading