Batch etcdlog intervals to prevent hundreds of thousands slamming browser charts

[dgoodwin]

Runs like https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-openshift-release-master-nightly-4.22-e2e-rosa-sts-ovn/2018612074661285888 have 200k etcdlog intervals. Turns out etcd can log the messages we watch for a LOT. These intervals are in memory on any prow job page load, and make interval charts brutally slow to load if they do at all.

This change batches them on minute boundaries, we'll see the message, the locator, and a count within that minute, but it cuts hundreds of thousands of intervals down to less than 400 in this case.

Assisted-by: Claude

[openshift-ci-robot]

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dgoodwin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [dgoodwin]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci-robot]

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

[openshift-trt]

Risk analysis has seen new tests most likely introduced by this PR.
Please ensure that new tests meet guidelines for naming and stability.

New tests seen in this PR at sha: 5accfa4

• "[Monitor:audit-log-analyzer][Jira:"Storage"] operator service account gcp-pd-csi-driver-operator should not create excessive watch requests" [Total: 3, Pass: 3, Fail: 0, Flake: 0]
• "[Monitor:audit-log-analyzer][Jira:"Test Framework"] operator service account aws-ebs-csi-driver-operator should not create excessive watch requests" [Total: 4, Pass: 4, Fail: 0, Flake: 0]
• "[Monitor:audit-log-analyzer][Jira:"Test Framework"] operator service account vmware-vsphere-csi-driver-operator should not create excessive watch requests" [Total: 2, Pass: 2, Fail: 0, Flake: 0]
• "[Monitor:audit-log-analyzer][Jira:"Test Framework"] operator service account vsphere-problem-detector-operator should not create excessive watch requests" [Total: 2, Pass: 2, Fail: 0, Flake: 0]

[openshift-ci]

@dgoodwin: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-vsphere-ovn	5accfa4	link	true	/test e2e-vsphere-ovn
ci/prow/e2e-aws-ovn-microshift	5accfa4	link	true	/test e2e-aws-ovn-microshift
ci/prow/e2e-vsphere-ovn-upi	5accfa4	link	true	/test e2e-vsphere-ovn-upi
ci/prow/e2e-gcp-ovn	5accfa4	link	true	/test e2e-gcp-ovn
ci/prow/e2e-aws-ovn-fips	5accfa4	link	true	/test e2e-aws-ovn-fips
ci/prow/e2e-gcp-csi	5accfa4	link	true	/test e2e-gcp-csi
ci/prow/e2e-aws-ovn-serial-2of2	5accfa4	link	true	/test e2e-aws-ovn-serial-2of2
ci/prow/e2e-metal-ipi-ovn-ipv6	5accfa4	link	true	/test e2e-metal-ipi-ovn-ipv6
ci/prow/verify	505a0e3	link	true	/test verify

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[dgoodwin]

@CodeRabbit configuration

[dgoodwin]

@CodeRabbit configuration