docs(integration-toolkit): document monitoring and alerting, fix stale facts

docs/integrations/integration-toolkit/configuration.md

@@ -22,7 +22,7 @@ Integrations support two types:
 ### Creating an Integration
 
 ```bash
-curl -X POST 'https://erp-integration.sls.epilot.io/v2/integrations' \
+curl -X POST 'https://integration-toolkit.sls.epilot.io/v2/integrations' \
   -H 'Authorization: Bearer <token>' \
   -H 'Content-Type: application/json' \
   -d '{
@@ -34,7 +34,7 @@ curl -X POST 'https://erp-integration.sls.epilot.io/v2/integrations' \
 **Creating a Connector Integration:**
 
 ```bash
-curl -X POST 'https://erp-integration.sls.epilot.io/v2/integrations' \
+curl -X POST 'https://integration-toolkit.sls.epilot.io/v2/integrations' \
   -H 'Authorization: Bearer <token>' \
   -H 'Content-Type: application/json' \
   -d '{
@@ -68,14 +68,14 @@ curl -X POST 'https://erp-integration.sls.epilot.io/v2/integrations' \
 ### Listing Integrations
 
 ```bash
-curl -X GET 'https://erp-integration.sls.epilot.io/v2/integrations' \
+curl -X GET 'https://integration-toolkit.sls.epilot.io/v2/integrations' \
   -H 'Authorization: Bearer <token>'
 ```
 
 ### Updating an Integration
 
 ```bash
-curl -X PUT 'https://erp-integration.sls.epilot.io/v2/integrations/{integrationId}' \
+curl -X PUT 'https://integration-toolkit.sls.epilot.io/v2/integrations/{integrationId}' \
   -H 'Authorization: Bearer <token>' \
   -H 'Content-Type: application/json' \
   -d '{
@@ -91,7 +91,7 @@ Deleting an integration also removes all associated use cases.
 :::
 
 ```bash
-curl -X DELETE 'https://erp-integration.sls.epilot.io/v2/integrations/{integrationId}' \
+curl -X DELETE 'https://integration-toolkit.sls.epilot.io/v2/integrations/{integrationId}' \
   -H 'Authorization: Bearer <token>'
 ```
 
@@ -100,7 +100,7 @@ curl -X DELETE 'https://erp-integration.sls.epilot.io/v2/integrations/{integrati
 ### Creating a Use Case
 
 ```bash
-curl -X POST 'https://erp-integration.sls.epilot.io/v1/integrations/{integrationId}/use-cases' \
+curl -X POST 'https://integration-toolkit.sls.epilot.io/v1/integrations/{integrationId}/use-cases' \
   -H 'Authorization: Bearer <token>' \
   -H 'Content-Type: application/json' \
   -d '{
@@ -132,7 +132,7 @@ curl -X POST 'https://erp-integration.sls.epilot.io/v1/integrations/{integration
 ### Enabling/Disabling a Use Case
 
 ```bash
-curl -X PUT 'https://erp-integration.sls.epilot.io/v1/integrations/{integrationId}/use-cases/{useCaseId}' \
+curl -X PUT 'https://integration-toolkit.sls.epilot.io/v1/integrations/{integrationId}/use-cases/{useCaseId}' \
   -H 'Authorization: Bearer <token>' \
   -H 'Content-Type: application/json' \
   -d '{
@@ -151,7 +151,7 @@ incoming events. Plan cutovers accordingly.
 View the change history for a use case:
 
 ```bash
-curl -X GET 'https://erp-integration.sls.epilot.io/v1/integrations/{integrationId}/use-cases/{useCaseId}/history' \
+curl -X GET 'https://integration-toolkit.sls.epilot.io/v1/integrations/{integrationId}/use-cases/{useCaseId}/history' \
   -H 'Authorization: Bearer <token>'
 ```
 
@@ -160,7 +160,7 @@ curl -X GET 'https://erp-integration.sls.epilot.io/v1/integrations/{integrationI
 Outbound use cases deliver standardized epilot events (event-catalog events) to your external system. The configuration consists of an `event_catalog_event` and one or more `mappings`:
 
 ```bash
-curl -X POST 'https://erp-integration.sls.epilot.io/v1/integrations/{integrationId}/use-cases' \
+curl -X POST 'https://integration-toolkit.sls.epilot.io/v1/integrations/{integrationId}/use-cases' \
   -H 'Authorization: Bearer <token>' \
   -H 'Content-Type: application/json' \
   -d '{
@@ -350,7 +350,7 @@ Managed call use cases define synchronous API operations against external partne
 ### Creating a Managed Call Use Case
 
 ```bash
-curl -X POST 'https://erp-integration.sls.epilot.io/v1/integrations/{integrationId}/use-cases' \
+curl -X POST 'https://integration-toolkit.sls.epilot.io/v1/integrations/{integrationId}/use-cases' \
   -H 'Authorization: Bearer <token>' \
   -H 'Content-Type: application/json' \
   -d '{
@@ -377,7 +377,7 @@ curl -X POST 'https://erp-integration.sls.epilot.io/v1/integrations/{integration
 Managed calls are executed via the `/v1/managed-call/{slug}/execute` endpoint, where the slug acts as the RPC method name:
 
 ```bash
-curl -X POST 'https://erp-integration.sls.epilot.io/v1/managed-call/get-customer/execute' \
+curl -X POST 'https://integration-toolkit.sls.epilot.io/v1/managed-call/get-customer/execute' \
   -H 'Authorization: Bearer <token>' \
   -H 'Content-Type: application/json' \
   -d '{
@@ -433,7 +433,7 @@ Secure proxy use cases route HTTP requests through epilot's dedicated proxy infr
 ### Creating a Secure Proxy Use Case
 
 ```bash
-curl -X POST 'https://erp-integration.sls.epilot.io/v1/integrations/{integrationId}/use-cases' \
+curl -X POST 'https://integration-toolkit.sls.epilot.io/v1/integrations/{integrationId}/use-cases' \
   -H 'Authorization: Bearer <token>' \
   -H 'Content-Type: application/json' \
   -d '{
@@ -458,7 +458,7 @@ curl -X POST 'https://erp-integration.sls.epilot.io/v1/integrations/{integration
 ### Sending a Proxy Request
 
 ```bash
-curl -X POST 'https://erp-integration.sls.epilot.io/v1/secure-proxy' \
+curl -X POST 'https://integration-toolkit.sls.epilot.io/v1/secure-proxy' \
   -H 'Authorization: Bearer <token>' \
   -H 'Content-Type: application/json' \
   -d '{
@@ -602,48 +602,22 @@ Include a correlation ID to track related events:
 - Filter monitoring data by batch
 - Trace event processing through the pipeline
 
-## Monitoring Configuration
+## Monitoring
 
-### Query Events
+Every inbound and outbound event the toolkit processes is recorded as a monitoring
+event, queryable per integration and visible in the Integration Hub's **Monitoring**
+tab. A one-off example — the last day's error events for one integration:
 
 ```bash
-curl -X POST 'https://erp-integration.sls.epilot.io/v1/monitoring/events' \
+curl -X POST 'https://integration-toolkit.sls.epilot.io/v2/integrations/{integrationId}/monitoring/events' \
   -H 'Authorization: Bearer <token>' \
   -H 'Content-Type: application/json' \
-  -d '{
-    "integration_id": "<integration-id>",
-    "from": "2024-01-15T00:00:00Z",
-    "to": "2024-01-15T23:59:59Z",
-    "status": ["error"],
-    "limit": 100
-  }'
+  -d '{ "level": "error", "from_date": "2026-01-15T00:00:00Z", "limit": 50 }'
 ```
 
-### Query Statistics
+Querying, stats and time series, event traces, replay, the code reference, and
+alerting are documented in the [Monitoring section](./monitoring/overview.md).
 
-```bash
-curl -X POST 'https://erp-integration.sls.epilot.io/v1/monitoring/stats' \
-  -H 'Authorization: Bearer <token>' \
-  -H 'Content-Type: application/json' \
-  -d '{
-    "integration_id": "<integration-id>",
-    "from": "2024-01-01T00:00:00Z",
-    "to": "2024-01-31T23:59:59Z"
-  }'
-```
-
-### Event Replay
-
-Reprocess failed or specific events:
-
-```bash
-curl -X POST 'https://erp-integration.sls.epilot.io/v1/integrations/{integrationId}/events/replay' \
-  -H 'Authorization: Bearer <token>' \
-  -H 'Content-Type: application/json' \
-  -d '{
-    "event_ids": ["evt-001", "evt-002", "evt-003"]
-  }'
-```
 
 ## Rate Limits
 
@@ -668,9 +642,28 @@ Each organization's data is fully isolated. Integration IDs are scoped to organi
 
 ### Permissions
 
-| Action | Permission |
-|--------|------------|
-| Read integrations | `erp:read` |
-| Create/Update integrations | `erp:write` |
-| Send events | `erp:events` |
-| View monitoring | `erp:monitoring` |
+Three actions govern the Integration Toolkit. They are checked against the
+organization the token belongs to, and — where the endpoint names one — against the
+specific integration.
+
+| Permission | Grants |
+|---|---|
+| `integration:view` | Read integrations and use cases; read monitoring events, stats, time series, traces, access logs, notification history and status |
+| `integration:manage` | Create, update and delete integrations and use cases; edit notification configuration; push external monitoring events; send test notifications; replay events |
+| `integration:consume` | Poll and acknowledge the outbound queue. Scoped to [Pollable Outbound](./pollable-outbound.md) only — it grants nothing else |
+
+The `view` / `manage` split is the one that catches people out when minting a scoped
+token for a middleware:
+
+| Endpoint | Needs |
+|---|---|
+| `POST /v2/integrations/{id}/monitoring/events`, `…/stats`, `…/time-series` | `integration:view` |
+| `GET /v2/integrations/{id}/monitoring/traces/{correlationId}` | `integration:view` |
+| `GET /v2/integrations/{id}/notifications/history`, `…/status` | `integration:view` |
+| `POST /v2/integrations/{id}/monitoring/external-events` | `integration:manage` |
+| `POST /v2/integrations/{id}/notifications/test` | `integration:manage` |
+| `POST /v1/integrations/{id}/events/replay` | `integration:manage` |
+| `POST /v1/integrations/{id}/outbound/messages/poll`, `…/ack` | `integration:consume` |
+
+Inbound event submission authenticates as the integration's own API token rather
+than through these actions — see [Inbound Getting Started](./inbound/getting-started.md).

docs/integrations/integration-toolkit/inbound/examples.md

@@ -93,7 +93,7 @@ Synchronize customer data from an ERP system into epilot contacts.
 ### API Request
 
 ```bash
-curl -X POST 'https://erp-integration.sls.epilot.io/v3/erp/updates/events' \
+curl -X POST 'https://integration-toolkit.sls.epilot.io/v3/erp/updates/events' \
   -H 'Authorization: Bearer <token>' \
   -H 'Content-Type: application/json' \
   -d '{
@@ -480,7 +480,7 @@ Handle different entity types based on payload conditions.
 Before deploying your mapping, test it using the simulation endpoint:
 
 ```bash
-curl -X POST 'https://erp-integration.sls.epilot.io/v2/erp/updates/mapping_simulation' \
+curl -X POST 'https://integration-toolkit.sls.epilot.io/v2/erp/updates/mapping_simulation' \
   -H 'Authorization: Bearer <token>' \
   -H 'Content-Type: application/json' \
   -d '{

docs/integrations/integration-toolkit/inbound/getting-started.md

@@ -19,7 +19,7 @@ This guide walks you through setting up an inbound integration to synchronize da
 Create a new integration to represent your ERP connection:
 
 ```bash title="Create an integration"
-curl -X POST 'https://erp-integration.sls.epilot.io/v2/integrations' \
+curl -X POST 'https://integration-toolkit.sls.epilot.io/v2/integrations' \
   -H 'Authorization: Bearer <your-token>' \
   -H 'Content-Type: application/json' \
   -d '{
@@ -49,7 +49,7 @@ Save the `id` — you'll need it for subsequent API calls.
 A use case defines how specific data types are mapped and synchronized. Create an inbound use case for customer data:
 
 ```bash title="Create a use case"
-curl -X POST 'https://erp-integration.sls.epilot.io/v1/integrations/{integrationId}/use-cases' \
+curl -X POST 'https://integration-toolkit.sls.epilot.io/v1/integrations/{integrationId}/use-cases' \
   -H 'Authorization: Bearer <your-token>' \
   -H 'Content-Type: application/json' \
   -d '{
@@ -78,7 +78,7 @@ curl -X POST 'https://erp-integration.sls.epilot.io/v1/integrations/{integration
 Push data from your ERP system using the events endpoint:
 
 ```bash title="Send an inbound event"
-curl -X POST 'https://erp-integration.sls.epilot.io/v3/erp/updates/events' \
+curl -X POST 'https://integration-toolkit.sls.epilot.io/v3/erp/updates/events' \
   -H 'Authorization: Bearer <your-token>' \
   -H 'Content-Type: application/json' \
   -d '{

docs/integrations/integration-toolkit/monitoring/_category_.json

@@ -0,0 +1,4 @@
+{
+  "label": "Monitoring & Alerting",
+  "position": 9
+}

docs/integrations/integration-toolkit/monitoring/acks.md

@@ -0,0 +1,135 @@
+---
+sidebar_position: 5
+title: ACK Tracking
+description: How ERPs acknowledge events they processed, the ACK lifecycle and timeout, and how to turn tracking off per use case
+slug: /integrations/integration-toolkit/monitoring/acks
+---
+
+# ACK Tracking
+
+epilot knows it *delivered* an outbound event — the webhook returned 2xx, or the
+message was polled. It does not know the ERP actually **processed** it. An
+acknowledgement closes that gap: your middleware confirms the work is done, and the
+event's status in the Integration Hub becomes end-to-end rather than delivery-only.
+
+## The flow
+
+```mermaid
+sequenceDiagram
+    participant EP as epilot
+    participant MW as Your middleware
+
+    EP->>MW: outbound event (payload carries _ack_id)
+    Note over EP: ACK_PENDING recorded
+    MW->>MW: process the event
+    MW->>EP: POST /v1/erp/tracking/acknowledgement { ack_id }
+    Note over EP: ACK_CONFIRMED — tracking record cleared
+```
+
+If the acknowledgement never arrives, the record goes stale and epilot records
+`ACK_TIMEOUT` instead.
+
+## Where `ack_id` comes from
+
+It arrives **in the event payload**, as the `_ack_id` field, alongside the other
+common metadata every core event carries (`_event_version`, `_event_source`, …). It is
+not an HTTP header, and it is not something you construct.
+
+```jsonc
+{
+  "_event_source": "epilot",
+  "_ack_id": "ack_01HZY…",     // ← acknowledge with this
+  "contract": { "...": "..." }
+}
+```
+
+Each delivered event gets its own `_ack_id`. Acknowledge each one individually — an
+id is consumed once, and acknowledging clears its tracking record.
+
+:::note
+A JSONata outbound mapping controls what your ERP receives. If your mapping builds a
+brand-new object rather than extending the event, carry `_ack_id` through explicitly,
+or your middleware will never see the id it needs to acknowledge with.
+:::
+
+## Sending the acknowledgement
+
+```bash title="Send ACK"
+curl -X POST 'https://integration-toolkit.sls.epilot.io/v1/erp/tracking/acknowledgement' \
+  -H 'Content-Type: application/json' \
+  -d '{ "ack_id": "ack_01HZY…" }'
+```
+
+`ack_id` is the **only** field. There is no status to report: sending the
+acknowledgement *is* the signal that processing succeeded. A failure is simply an ACK
+that never arrives, which the timeout below turns into a visible warning.
+
+| Response | Meaning |
+|---|---|
+| `200` | Acknowledged; the tracking record is cleared |
+| `400` | `ack_id` missing from the body |
+| `404` | No tracking record — already acknowledged, already timed out, or an unknown id |
+
+A `404` after a successful `200` is normal if you retry: the record is gone because
+the first call consumed it. Treat it as success.
+
+## The lifecycle and its codes
+
+Three [monitoring codes](./codes.md) tell the whole story, and all three are
+filterable in the Monitoring tab:
+
+| Code | Level | When |
+|---|---|---|
+| `ACK_PENDING` | info | The event was delivered and epilot is waiting for the acknowledgement |
+| `ACK_CONFIRMED` | info | Your acknowledgement arrived |
+| `ACK_TIMEOUT` | warning | No acknowledgement within the timeout window |
+
+`ACK_PENDING` and `ACK_CONFIRMED` are **info**-level: they are lifecycle markers, not
+outcomes, so they are counted in total events but deliberately excluded from the
+success rate. `ACK_TIMEOUT` is a **warning** — the delivery itself worked, so it is
+not an error on epilot's side, but something on yours needs attention.
+
+### The timeout window
+
+A checker runs every **10 minutes** and times out any record older than **15
+minutes**. In practice an unacknowledged event surfaces as `ACK_TIMEOUT` within about
+25 minutes of delivery — so do not treat a missing ACK as final for roughly half an
+hour.
+
+`ACK_TIMEOUT` is also promoted to its own figure in the stats response,
+`ack_timeout_count`, so you can chart "how often is the ERP failing to confirm" without
+filtering the event stream by code:
+
+```bash
+curl -X POST 'https://integration-toolkit.sls.epilot.io/v2/integrations/{integrationId}/monitoring/stats' \
+  -H 'Authorization: Bearer <token>' \
+  -H 'Content-Type: application/json' \
+  -d '{ "from_date": "2026-01-01T00:00:00Z", "use_case_type": "outbound" }'
+```
+
+## Turning tracking off
+
+ACK tracking is per outbound use case, via `ack_tracking`:
+
+| Value | Behaviour |
+|---|---|
+| `on` *(default)* | A tracking record is written, `ACK_PENDING` is recorded, and an unacknowledged event eventually raises `ACK_TIMEOUT` |
+| `off` | No tracking record, no `ACK_PENDING`, and no `ACK_TIMEOUT` |
+
+Set it to `off` for consumers that will never acknowledge, and for deliveries that
+already keep their own durable per-item record — [Pollable
+Outbound](../pollable-outbound.md) queues, for instance, track consumption through the
+queue's own `MSG_ACKED` lifecycle, so ACK tracking on top of it produces timeouts that
+mean nothing.
+
+:::warning
+Leaving `ack_tracking: on` for a consumer that never acknowledges generates a steady
+stream of `ACK_TIMEOUT` warnings. That is noise on its own, and it will trip a
+`warning_threshold` [alert rule](./alerting.md) if you enable one.
+:::
+
+## Related
+
+- [Monitoring Codes](./codes.md) — the full code reference
+- [Investigating events](./investigating.md) — traces and replay
+- [Pollable Outbound](../pollable-outbound.md) — the queue's own delivery lifecycle