mixin: add AlertmanagerClusterFailedPeers alert

[trouaux]

Pull Request Checklist

Please check all the applicable boxes.

• Please list all open issue(s) discussed with maintainers related to this change
  • Fixes #
• Is this a new Receiver integration?
  • I have already tried to use the Webhook Receiver Integration and 3rd party integrations before adding this new Receiver Integration
• Is this a bugfix?
  • I have added tests that can reproduce the bug which pass with this bugfix applied
• Is this a new feature?
  • I have added tests that test the new feature's functionality
• Does this change affect performance?
  • I have provided benchmarks comparison that shows performance is improved or is not degraded
    • You can use benchstat to compare benchmarks
  • I have added new benchmarks if required or requested by maintainers
• Is this a breaking change?
  • My changes do not break the existing cluster messages
  • My changes do not break the existing api
• I have added/updated the required documentation
• I have signed-off my commits
• I will follow best practices for contributing to this project

Which user-facing changes does this PR introduce?

- add AlertmanagerClusterFailedPeers alert

current alerts only trigger unhealthy HA cluster state when both pods run in the same cluster, adding an alert to trigger on unhealthy HA state of an alertmanager instance

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 222b104d-7f66-4e1b-a46e-01ba619c3d9c

📥 Commits

Reviewing files that changed from the base of the PR and between 2eedf4f and 651bd52.

📒 Files selected for processing (1)

• doc/alertmanager-mixin/alerts.libsonnet

---

📝 Walkthrough

Walkthrough

This PR adds a new Prometheus alert rule AlertmanagerClusterFailedPeers to the Alertmanager monitoring mixin. The rule detects cluster peer failures by monitoring the alertmanager_cluster_failed_peers metric, triggers when the count exceeds zero within a 5-minute window, and generates a warning-severity alert after a 15-minute hold period with instance and cluster context in annotations.

Changes

Cluster Peer Failure Monitoring Alert

Layer / File(s)	Summary
AlertmanagerClusterFailedPeers alert rule doc/alertmanager-mixin/alerts.libsonnet	New alert rule monitors alertmanager_cluster_failed_peers metric, triggers when failed peers count exceeds zero over 5m window, evaluates for 15m, sets warning severity, and includes annotations with instance name and cluster identifier.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description is largely incomplete, missing critical required sections like linked issues and release notes detail, though some checklist items are addressed.	Add a 'Fixes #' link, provide more detailed release notes following the [FEATURE]/[BUGFIX] format, and ensure all applicable checklist items are properly documented.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes the main change: adding a new AlertmanagerClusterFailedPeers alert to the mixin.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}