IncidentRelay

Self-hosted on-call routing, alert delivery, and escalation for teams that want control over their incident workflow.

IncidentRelay helps SRE, DevOps, platform, infrastructure, and operations teams route alerts to the right people through the right channels — without depending on a hosted incident-management platform.

It gives you the core building blocks of an on-call system:

• access groups and RBAC-style group roles;
• teams and on-call rotations;
• alert intake routes with per-route tokens;
• Alertmanager, Zabbix, and generic webhook integrations;
• Mattermost, Slack, Telegram, Discord, Microsoft Teams, email, webhook, and voice-call notifications;
• acknowledge and resolve workflows;
• reminders and escalation to the next on-call user;
• rotation overrides;
• calendar view for on-call schedules;
• personal API tokens;
• Swagger/OpenAPI documentation.

IncidentRelay is designed for self-hosted environments where teams need predictable behavior, clear ownership, easy integrations, and full control over alert routing.

---

Why IncidentRelay?

Many teams need on-call routing, but do not always need a large SaaS incident platform.

IncidentRelay focuses on the practical workflow:

Monitoring system -> Route -> Team -> Rotation -> Notification channels -> ACK / Resolve

A route owns its own intake token, so external systems send alerts to an exact alert path:

ROUTE_INTAKE_TOKEN -> Route -> Team -> Rotation -> Channels

Channels only describe where notifications are delivered.
Routes decide which team receives the alert and which channels are used.

This keeps alert delivery easier to reason about, easier to audit, and safer for self-hosted deployments.

---

Highlights

Self-hosted by design

Run IncidentRelay in your own environment, with your own database, your own network rules, and your own operational policies.

Route-based alert intake

Alert intake tokens belong to routes, not channels. This makes it clear which incoming integration is allowed to submit alerts to which team and rotation.

Multi-team and multi-group support

Use groups as access boundaries. Teams, rotations, routes, channels, alerts, and silences are scoped through groups and memberships.

Escalation and reminders

Unacknowledged alerts can trigger repeated reminders and then escalate to the next on-call user according to the team configuration.

Mattermost with real actions

Mattermost Bot API mode supports interactive Acknowledge and Resolve buttons, message updates, and severity-based attachment colors.

Pluggable voice calls

IncidentRelay can be extended with custom voice providers for self-hosted installations. Providers can implement text-to-speech calls, call status callbacks, DTMF button callbacks, ACK / Resolve actions from phone keypad, and optional call status polling.

API-first

IncidentRelay includes Swagger/OpenAPI documentation and personal API tokens with scopes for alerts, resources, and profile access.

---

Supported integrations

Incoming alert sources

Source	Endpoint
Alertmanager	POST /api/integrations/alertmanager
Zabbix	POST /api/integrations/zabbix
Generic webhook	POST /api/integrations/webhook

Notification channels

Channel	Notes
Mattermost	Incoming webhook mode or Bot API mode with buttons and updates
Slack	Webhook notifications
Telegram	Bot notifications
Discord	Webhook notifications
Microsoft Teams	Webhook notifications
Email	Email recipients
Webhook	Generic outbound webhook
Voice call	Pluggable provider API for self-hosted voice integrations

---

Quick start

Systemd installation

IncidentRelay can be installed as two systemd services:

incidentrelay-web.service        # HTTP API, UI, webhooks
incidentrelay-scheduler.service  # reminders, escalations, periodic jobs

Quick start:

sudo git clone https://github.com/roxy-wi/IncidentRelay.git /var/www/incedentrelay
cd /var/www/incedentrelay

sudo python3 -m venv /var/www/incedentrelay/venv
sudo /var/www/incedentrelay/venv/bin/pip install --upgrade pip
sudo /var/www/incedentrelay/venv/bin/pip install -r requirements.txt
sudo /var/www/incedentrelay/venv/bin/pip install gunicorn

sudo mkdir -p /etc/incedentrelay /var/lib/incidentrelay /var/log/incidentrelay
sudo cp etc/incedentrelay/incedentrelay.conf /etc/incedentrelay/incedentrelay.conf

sudo chown -R www-data:www-data /var/www/incedentrelay
sudo chown -R www-data:www-data /var/lib/incidentrelay
sudo chown -R www-data:www-data /var/log/incidentrelay

sudo cp systemd/incidentrelay-web.service /etc/systemd/system/
sudo cp systemd/incidentrelay-scheduler.service /etc/systemd/system/
sudo systemctl daemon-reload

sudo -u www-data \
  INCEDENTRELAY_CONFIG_FILE=/etc/incedentrelay/incedentrelay.conf \
  /var/www/incedentrelay/venv/bin/python app/migrate.py migrate

sudo systemctl enable --now incidentrelay-web
sudo systemctl enable --now incidentrelay-scheduler

Create the first admin user:

sudo -u www-data \
  INCEDENTRELAY_CONFIG_FILE=/etc/incedentrelay/incedentrelay.conf \
  /var/www/incedentrelay/venv/bin/python manage.py create-admin \
    --username admin \
    --password 'change-me-123' \
    --email admin@example.com

Read more:

Systemd Installation

Docker installation

The recommended self-hosted installation method is Docker Compose.

By default, IncidentRelay starts with SQLite and two containers:

incidentrelay-web        # HTTP API, UI, webhooks
incidentrelay-scheduler  # reminders, escalations, periodic jobs

Start:

docker compose up -d --build

SQLite data is stored in the incidentrelay-data Docker volume.

The scheduler runs in a separate container, so reminder and escalation jobs are not duplicated by web workers.

For PostgreSQL:

docker compose \
  -f docker-compose.yml \
  -f docker-compose.postgres.yml \
  up -d

For production behind Nginx/HAProxy, bind the container port to localhost:

ports:
  - "127.0.0.1:8080:8080"

and set:

[server]
public_base_url = https://incidentrelay.example.com

Read more:

• Docker installation
• Scheduler

---

Basic setup flow

After the first login:

1. Create a group
2. Create users
3. Add users to the group
4. Create a team
5. Add users to the team
6. Create a rotation
7. Add rotation members
8. Create notification channels
9. Create a route
10. Copy the route intake token
11. Configure Alertmanager, Zabbix, or webhook sender
12. Send a test alert
13. Acknowledge or resolve the alert

Detailed guide:

First login and initial setup

---

Example Alertmanager request

curl -X POST http://127.0.0.1:8080/api/integrations/alertmanager \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer ALERTMANAGER_ROUTE_TOKEN' \
  -d '{
    "status": "firing",
    "alerts": [
      {
        "status": "firing",
        "labels": {
          "alertname": "DiskFull",
          "severity": "critical",
          "team": "infra",
          "instance": "host1"
        },
        "annotations": {
          "summary": "Disk is full",
          "description": "/var is 95% full"
        },
        "fingerprint": "disk-full-host1-var"
      }
    ]
  }'

More examples:

• Alertmanager integration
• Zabbix integration
• Generic webhook integration

---

Mattermost buttons and message updates

Mattermost has two modes.

Incoming webhook mode sends plain messages only.

Bot API mode is recommended when you want:

• Acknowledge button;
• Resolve button;
• message updates after ACK / Resolve;
• severity-based colors.

More details:

Mattermost integration

---

Custom voice providers

IncidentRelay supports custom voice providers for self-hosted installations.

A provider is a Python module that can be placed into:

/usr/local/lib/incidentrelay/voice_providers

Custom providers can implement:

• text-to-speech call creation;
• provider call ID tracking;
• call status callbacks;
• DTMF button callbacks;
• ACK / Resolve actions from phone keypad;
• optional call status polling.

Start here:

• Custom Voice Providers
• Provider API
• Configuration
• Callbacks and DTMF
• Security
• Troubleshooting
• Example providers

---

Documentation

Topic	Link
Getting started	docs/getting-started/
Installation	docs/getting-started/installation.md
Configuration	docs/getting-started/configuration.md
First login	docs/getting-started/first-login.md
Groups and RBAC	docs/concepts/groups-and-rbac.md
Teams, rotations, routes	docs/concepts/teams-rotations-routes.md
Route intake tokens	docs/concepts/route-intake-tokens.md
Channels	docs/concepts/channels.md
Alertmanager	docs/integrations/alertmanager.md
Zabbix	docs/integrations/zabbix.md
Generic webhook	docs/integrations/generic-webhook.md
Mattermost	docs/integrations/mattermost.md
Alerts	docs/usage/alerts.md
Calendar	docs/usage/calendar.md
Silences	docs/usage/silences.md
Rotation overrides	docs/usage/rotation-overrides.md
Profile and API tokens	docs/usage/profile-and-tokens.md
Logging	docs/administration/logging.md
Troubleshooting	docs/administration/troubleshooting.md
Custom voice providers	docs/voice-providers/index.md
Swagger/OpenAPI notes	docs/api/voice-call-openapi.md

Swagger UI is available at:

/docs

OpenAPI JSON is available at:

/api/openapi.json

---

Documentation website

The documentation is ready to be published with MkDocs Material.

Install MkDocs Material:

pip install mkdocs-material

Run local preview:

mkdocs serve

Open:

http://127.0.0.1:8000

---

Demo data

Create demo data:

python manage.py demo-data

The command creates demo groups, users, teams, rotations, channels, routes, and route intake tokens.

Static demo-data check:

python app/check_demo_data.py

More details:

Demo data

---

Schema check

After running migrations, verify that all Peewee model tables and columns exist in the configured database:

python app/check_schema.py

Expected output:

Schema check OK: all model tables and columns exist.

More details:

Schema check

---

Troubleshooting

If an alert is not visible or not delivered:

1. Check that the correct route intake token was used.
2. Check that the endpoint matches the route source.
3. Check that route matchers match alert labels.
4. Check that the group is active.
5. Check that the team is active.
6. Check that the UI active group is correct.
7. Select "All my groups" and reload the Alerts page.
8. Check routing_error in the integration response.
9. Check JSON logs by error_id if the server returned one.

More details:

Troubleshooting

---

License

Add your project license here.