Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions content/docs/core/herald/api-keys.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
---
title: API keys
description: Authenticate to Herald and scope access
---

Herald authenticates every request with a bearer API key that resolves to a
single tenant. Pass it as `authorization: Bearer <key>` on REST and GraphQL
requests alike.

## Permission levels

| Level | Can send | Can manage tenant (keys, domains, templates, suppressions, settings) |
| --------- | -------- | -------------------------------------------------------------------- |
| `full` | yes | yes |
| `sending` | yes | no |

Use a `sending` key for the application code path that only emits mail, and keep
`full` keys for administrative tooling.

## Creating a key

From the dashboard (**API Keys → New key**) or the `createApiKey` mutation:

```graphql
mutation {
createApiKey(
input: { name: "production-sender", permission: "sending", expiresAt: "2027-01-01T00:00:00Z" }
) {
apiKeyId
secret
}
}
```

The `secret` is returned **once** and stored only as a peppered hash; Herald
cannot show it again. The leading prefix is retained so you can recognise a key
in the dashboard without revealing it.

`expiresAt` is optional; after it passes the key stops authenticating.

## Rotating and revoking

Create the replacement key, deploy it, then revoke the old one with
`revokeApiKey(apiKeyId: ...)`. Revocation is immediate. A revoked or expired key
returns `401`.

## Keep keys secret

Treat keys like passwords: never commit them, never log them, and prefer the
shortest-lived key that works. Herald never logs the key, the recipient, or the
message body.
80 changes: 80 additions & 0 deletions content/docs/core/herald/architecture.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
---
title: Architecture
description: How Herald is built
---

Herald is two pieces: a stateless send API and a self-hosted MTA. The API owns
tenancy, validation, suppression, and delivery state; the MTA owns signing and
the SMTP path.

## Components

```
caller (service or Vortex workflow)
│ POST /messages (Bearer <tenant API key>)
┌──────────────────┐ inject (HTTP, Basic auth)
│ herald-api │ ───────────────────────────────┐
│ (Bun + Elysia) │ ▼
└────────┬─────────┘ ┌──────────────────┐
│ │ KumoMTA │
┌────────▼─────────┐ webhook (log_hooks) │ (mail.omni.dev) │
│ PostgreSQL │ ◀──────────────────────┤ DKIM + SMTP :25 │
│ tenants, keys, │ └────────┬─────────┘
│ messages, events,│ │ deliver
│ suppression │ ▼
└──────────────────┘ recipient mail servers
```

### herald-api

A Bun + Elysia service. Responsibilities:

- **Auth & tenancy**: an `Authorization: Bearer <key>` is hashed and matched to a tenant; routes decide whether a tenant is required. There is no public GraphQL surface in this phase.
- **Send (`POST /messages`)**: validate the body at the boundary, reject suppressed recipients, short-circuit duplicate idempotency keys, persist a `queued` message, then inject into the MTA. On an engine failure the message is marked `failed` and the caller gets a `502` so it can fail over.
- **Webhook (`POST /webhooks/kumomta`)**: ingest delivery lifecycle records, advance `message.status`, and auto-suppress hard bounces and complaints. Always acks `200` so the MTA does not retry forever.
- **Events**: emit `herald.message.*` CloudEvents to Vortex (source `omni.herald`).

The mail engine is injected behind a `MailEngine` interface, so KumoMTA can be
swapped without touching the routes.

### KumoMTA host

A dedicated, isolated Hetzner box (`mail.omni.dev`) with a dedicated mail IP and
PTR, never sharing application egress. It runs:

- an **HTTP injection** endpoint behind Basic auth (Caddy terminates TLS, proxies to KumoMTA),
- **DKIM signing** for the sending domain (selector `herald`),
- an **SMTP listener** on `:25` to receive bounces and DSNs,
- a **`log_hooks`** module that POSTs delivery/bounce/complaint records to `herald-api`.

## Correlation

KumoMTA's injection API does not return a per-message id, so Herald threads its
own message id through an `X-Herald-Message-Id` header on the injected content.
The MTA's `log_hooks` is configured to echo that header on every delivery record,
and the webhook correlates events back to the originating message by it (falling
back to the stored external id). This is what lets an asynchronous bounce or
complaint, arriving seconds or minutes later, find its message and auto-suppress
the recipient.

## Data model (essentials)

| Table | Holds |
|-------|-------|
| `tenant` | one row per sending tenant (org binding, name) |
| `api_key` | hashed API keys, `lastUsedAt`, tenant FK |
| `sending_domain` | per-tenant verified domains + DKIM selector |
| `message` | every send: addresses, subject, status, external id, idempotency key |
| `message_event` | immutable raw delivery/bounce/complaint records |
| `suppression` | per-tenant suppressed addresses with a reason |

## Send path, end to end

1. Caller `POST /messages` with a tenant API key.
2. Resolve tenant → validate body → suppression check → insert `queued` message.
3. Inject into KumoMTA with `X-Herald-Message-Id`; store the returned id; mark sent.
4. KumoMTA DKIM-signs and delivers over SMTP `:25`.
5. KumoMTA `log_hooks` POSTs the outcome to `/webhooks/kumomta`.
6. Herald correlates by the echoed header, advances status, auto-suppresses hard
bounces/complaints, and emits a `herald.message.*` CloudEvent.
53 changes: 53 additions & 0 deletions content/docs/core/herald/domains.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
---
title: Sending domains
description: Add and verify a domain to send from
---

Herald sends only from domains you have verified. Verification proves control of
the domain and publishes a DKIM key so the MTA can sign your mail, which is what
makes it deliverable.

## Add a domain

From the dashboard (**Domains → Add domain**) or `createSendingDomain`:

```graphql
mutation {
createSendingDomain(input: { domain: "send.example.com" }) {
domainId
dkimSelector
dkimPublicKey
verified
}
}
```

Herald generates a DKIM key pair and returns the public half. Use a dedicated
sending subdomain (for example `send.example.com`) rather than your root domain,
so sending reputation stays isolated from the rest of your mail.

## Publish DNS and verify

Add the DKIM public key as a `TXT` record at
`<selector>._domainkey.<domain>` (the dashboard shows the exact record), then
verify:

```graphql
mutation {
verifySendingDomain(id: "<domainId>") {
verified
}
}
```

Verification resolves the record and checks it matches the key Herald generated.
A missing record reads as "not yet verified" rather than an error, so you can
re-run it after DNS propagates.

For the best inbox placement, also publish `SPF` and `DMARC` records for the
sending domain. Herald surfaces deliverability recommendations in the dashboard.

## The send gate

Until a domain is verified, any send whose `from` address uses it is rejected.
Once verified, messages from that domain are signed and sent.
80 changes: 80 additions & 0 deletions content/docs/core/herald/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
---
title: Herald
description: Transactional email platform for the Omni stack
---

import { ProductOverview } from "@/components/docs";
import app from "@/lib/config/app.config";
import { PiEnvelopeSimple, PiPaperPlaneTilt, PiShieldCheck } from "react-icons/pi";

<ProductOverview
tags={[
{
label: "Email",
icon: <PiEnvelopeSimple />,
className: "bg-blue-100 text-blue-800 dark:bg-blue-900/20 dark:text-blue-300",
},
{
label: "Transactional",
icon: <PiPaperPlaneTilt />,
className: "bg-purple-100 text-purple-800 dark:bg-purple-900/20 dark:text-purple-300",
},
{
label: "Deliverability",
icon: <PiShieldCheck />,
className: "bg-green-100 text-green-800 dark:bg-green-900/20 dark:text-green-300",
},
]}
links={[
{
href: `${app.socials.github}/herald-api`,
label: "Visit Repository",
type: "repository",
},
]}
alerts={[
{
title: "Big Idea",
description:
"Own the transactional email path end to end: a multi-tenant send API in front of a self-hosted MTA with a dedicated sending reputation, suppression, and delivery feedback.",
},
]}
/>

**Herald** is Omni's transactional email platform. It pairs a small multi-tenant
send API with a self-hosted [KumoMTA](https://kumomta.com) mail host on a
dedicated IP, so product emails (order receipts, notifications, verification)
flow over infrastructure Omni controls rather than a third-party ESP, while
keeping a third-party provider available as a fallback.

## Key Features

- **Multi-tenant send API**: API-key auth resolves a tenant; each tenant has its own suppression list and verified sending domains.
- **Self-hosted MTA**: KumoMTA on a dedicated Hetzner host with a dedicated mail IP, PTR, and DKIM signing, isolated from application egress to protect sending reputation.
- **Suppression built in**: hard bounces and spam complaints auto-suppress the recipient; suppressed addresses are rejected before any send.
- **Delivery feedback**: KumoMTA posts delivery/bounce/complaint events back to Herald, which advances message status and emits CloudEvents.
- **Idempotent sends**: an optional idempotency key dedupes retries so a redelivered request never sends twice.
- **Swappable engine**: the mail engine sits behind an interface, so the underlying MTA can change without touching callers.

## How it fits

Herald is consumed two ways:

1. **Directly** by services that send their own mail (`POST /messages` with a tenant API key).
2. **Through [Vortex](/docs/core/vortex)** workflows, which render an email and hand it to Herald, falling back to a third-party provider on failure.

Inter-service delivery events are published as [CloudEvents](https://cloudevents.io)
(`herald.message.delivered`, `herald.message.bounced`, `herald.message.complained`)
under the source `omni.herald`.

## Sending domain

Herald signs and sends from a dedicated sending subdomain (`send.omni.dev`) with
SPF, a `herald` DKIM selector, and DMARC configured, while the MTA host itself
(`mail.omni.dev`) carries the PTR and receives bounce/feedback mail. Keeping the
sending domain separate from the primary domain isolates reputation.

## Status

Herald is an internal platform service. See [Architecture](/docs/core/herald/architecture)
for how the API and MTA fit together.
14 changes: 14 additions & 0 deletions content/docs/core/herald/meta.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"title": "Herald",
"pages": [
"index",
"quickstart",
"api-keys",
"domains",
"templates",
"tracking",
"webhooks",
"sdk",
"architecture"
]
}
84 changes: 84 additions & 0 deletions content/docs/core/herald/quickstart.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
---
title: Quickstart
description: Send your first email with Herald
---

This guide takes you from nothing to a delivered email: create an API key,
verify a sending domain, and send a message over REST or GraphQL.

## 1. Create an API key

Every request authenticates as a tenant with a bearer API key. Create one from
the dashboard (**API Keys → New key**) or with the `createApiKey` mutation. The
secret is shown **once**, so store it immediately.

Keys carry a permission level:

- `full` manages the tenant (keys, domains, templates, suppressions) and sends.
- `sending` may only send messages.

See [API keys](/docs/core/herald/api-keys) for expiry and rotation.

## 2. Verify a sending domain

Herald only sends from a domain you have verified, which proves you control it
and lets Herald publish a DKIM key for signing. Add a domain (**Domains → Add
domain**), then publish the DKIM `TXT` record Herald shows you and click
**Verify**. See [Sending domains](/docs/core/herald/domains).

## 3. Send a message

### REST

```bash
curl https://api.herald.omni.dev/messages \
-H "authorization: Bearer $HERALD_API_KEY" \
-H "content-type: application/json" \
-d '{
"to": "recipient@example.com",
"from": "hello@send.example.com",
"subject": "Welcome aboard",
"html": "<p>Thanks for signing up.</p>",
"idempotencyKey": "signup-12345"
}'
```

`from` must belong to a verified sending domain or the send is rejected. An
optional `idempotencyKey` dedupes retries so a redelivered request never sends
twice. Other fields: `text`, `cc`, `bcc`, `replyTo`, `attachments`, `sendAt`
(future ISO-8601 to schedule), `variables`, `trackOpens`, `trackClicks`.

### GraphQL

```graphql
mutation Send {
sendMessage(
input: {
to: "recipient@example.com"
from: "hello@send.example.com"
subject: "Welcome aboard"
html: "<p>Thanks for signing up.</p>"
}
) {
messageId
status
}
}
```

The GraphQL endpoint is `https://api.herald.omni.dev/graphql`, authenticated with
the same bearer key.

## 4. Track delivery

Herald records every message and its delivery events (`delivered`, `bounced`,
`complained`, and, when enabled, `opened`/`clicked`). View them in the dashboard
**Messages** log, query the `messages` connection, or subscribe to
[webhooks](/docs/core/herald/webhooks). Hard bounces and complaints add the
recipient to your suppression list automatically.

## Next steps

- [Templates and variables](/docs/core/herald/templates) for reusable, personalized content
- [Open and click tracking](/docs/core/herald/tracking)
- [The TypeScript SDK and MCP server](/docs/core/herald/sdk)
Loading
Loading