From c966bfde592e2d9120b7c5b077d423c46f389b59 Mon Sep 17 00:00:00 2001 From: Brian Cooper Date: Tue, 23 Jun 2026 13:10:58 -0700 Subject: [PATCH] docs(herald): product documentation Adds the Herald docs section: quickstart, API keys, sending domains, templates and variables, open/click tracking, webhooks, and the SDK/MCP server, alongside the existing overview and architecture pages. Navigation ordered via meta.json. Grounded in the shipped API (REST + GraphQL). --- content/docs/core/herald/api-keys.mdx | 51 ++++++++++++++ content/docs/core/herald/architecture.mdx | 80 +++++++++++++++++++++ content/docs/core/herald/domains.mdx | 53 ++++++++++++++ content/docs/core/herald/index.mdx | 80 +++++++++++++++++++++ content/docs/core/herald/meta.json | 14 ++++ content/docs/core/herald/quickstart.mdx | 84 +++++++++++++++++++++++ content/docs/core/herald/sdk.mdx | 52 ++++++++++++++ content/docs/core/herald/templates.mdx | 63 +++++++++++++++++ content/docs/core/herald/tracking.mdx | 54 +++++++++++++++ content/docs/core/herald/webhooks.mdx | 52 ++++++++++++++ 10 files changed, 583 insertions(+) create mode 100644 content/docs/core/herald/api-keys.mdx create mode 100644 content/docs/core/herald/architecture.mdx create mode 100644 content/docs/core/herald/domains.mdx create mode 100644 content/docs/core/herald/index.mdx create mode 100644 content/docs/core/herald/meta.json create mode 100644 content/docs/core/herald/quickstart.mdx create mode 100644 content/docs/core/herald/sdk.mdx create mode 100644 content/docs/core/herald/templates.mdx create mode 100644 content/docs/core/herald/tracking.mdx create mode 100644 content/docs/core/herald/webhooks.mdx diff --git a/content/docs/core/herald/api-keys.mdx b/content/docs/core/herald/api-keys.mdx new file mode 100644 index 0000000..e42bd9a --- /dev/null +++ b/content/docs/core/herald/api-keys.mdx @@ -0,0 +1,51 @@ +--- +title: API keys +description: Authenticate to Herald and scope access +--- + +Herald authenticates every request with a bearer API key that resolves to a +single tenant. Pass it as `authorization: Bearer ` on REST and GraphQL +requests alike. + +## Permission levels + +| Level | Can send | Can manage tenant (keys, domains, templates, suppressions, settings) | +| --------- | -------- | -------------------------------------------------------------------- | +| `full` | yes | yes | +| `sending` | yes | no | + +Use a `sending` key for the application code path that only emits mail, and keep +`full` keys for administrative tooling. + +## Creating a key + +From the dashboard (**API Keys → New key**) or the `createApiKey` mutation: + +```graphql +mutation { + createApiKey( + input: { name: "production-sender", permission: "sending", expiresAt: "2027-01-01T00:00:00Z" } + ) { + apiKeyId + secret + } +} +``` + +The `secret` is returned **once** and stored only as a peppered hash; Herald +cannot show it again. The leading prefix is retained so you can recognise a key +in the dashboard without revealing it. + +`expiresAt` is optional; after it passes the key stops authenticating. + +## Rotating and revoking + +Create the replacement key, deploy it, then revoke the old one with +`revokeApiKey(apiKeyId: ...)`. Revocation is immediate. A revoked or expired key +returns `401`. + +## Keep keys secret + +Treat keys like passwords: never commit them, never log them, and prefer the +shortest-lived key that works. Herald never logs the key, the recipient, or the +message body. diff --git a/content/docs/core/herald/architecture.mdx b/content/docs/core/herald/architecture.mdx new file mode 100644 index 0000000..91f24ad --- /dev/null +++ b/content/docs/core/herald/architecture.mdx @@ -0,0 +1,80 @@ +--- +title: Architecture +description: How Herald is built +--- + +Herald is two pieces: a stateless send API and a self-hosted MTA. The API owns +tenancy, validation, suppression, and delivery state; the MTA owns signing and +the SMTP path. + +## Components + +``` + caller (service or Vortex workflow) + │ POST /messages (Bearer ) + ▼ + ┌──────────────────┐ inject (HTTP, Basic auth) + │ herald-api │ ───────────────────────────────┐ + │ (Bun + Elysia) │ ▼ + └────────┬─────────┘ ┌──────────────────┐ + │ │ KumoMTA │ + ┌────────▼─────────┐ webhook (log_hooks) │ (mail.omni.dev) │ + │ PostgreSQL │ ◀──────────────────────┤ DKIM + SMTP :25 │ + │ tenants, keys, │ └────────┬─────────┘ + │ messages, events,│ │ deliver + │ suppression │ ▼ + └──────────────────┘ recipient mail servers +``` + +### herald-api + +A Bun + Elysia service. Responsibilities: + +- **Auth & tenancy**: an `Authorization: Bearer ` is hashed and matched to a tenant; routes decide whether a tenant is required. There is no public GraphQL surface in this phase. +- **Send (`POST /messages`)**: validate the body at the boundary, reject suppressed recipients, short-circuit duplicate idempotency keys, persist a `queued` message, then inject into the MTA. On an engine failure the message is marked `failed` and the caller gets a `502` so it can fail over. +- **Webhook (`POST /webhooks/kumomta`)**: ingest delivery lifecycle records, advance `message.status`, and auto-suppress hard bounces and complaints. Always acks `200` so the MTA does not retry forever. +- **Events**: emit `herald.message.*` CloudEvents to Vortex (source `omni.herald`). + +The mail engine is injected behind a `MailEngine` interface, so KumoMTA can be +swapped without touching the routes. + +### KumoMTA host + +A dedicated, isolated Hetzner box (`mail.omni.dev`) with a dedicated mail IP and +PTR, never sharing application egress. It runs: + +- an **HTTP injection** endpoint behind Basic auth (Caddy terminates TLS, proxies to KumoMTA), +- **DKIM signing** for the sending domain (selector `herald`), +- an **SMTP listener** on `:25` to receive bounces and DSNs, +- a **`log_hooks`** module that POSTs delivery/bounce/complaint records to `herald-api`. + +## Correlation + +KumoMTA's injection API does not return a per-message id, so Herald threads its +own message id through an `X-Herald-Message-Id` header on the injected content. +The MTA's `log_hooks` is configured to echo that header on every delivery record, +and the webhook correlates events back to the originating message by it (falling +back to the stored external id). This is what lets an asynchronous bounce or +complaint, arriving seconds or minutes later, find its message and auto-suppress +the recipient. + +## Data model (essentials) + +| Table | Holds | +|-------|-------| +| `tenant` | one row per sending tenant (org binding, name) | +| `api_key` | hashed API keys, `lastUsedAt`, tenant FK | +| `sending_domain` | per-tenant verified domains + DKIM selector | +| `message` | every send: addresses, subject, status, external id, idempotency key | +| `message_event` | immutable raw delivery/bounce/complaint records | +| `suppression` | per-tenant suppressed addresses with a reason | + +## Send path, end to end + +1. Caller `POST /messages` with a tenant API key. +2. Resolve tenant → validate body → suppression check → insert `queued` message. +3. Inject into KumoMTA with `X-Herald-Message-Id`; store the returned id; mark sent. +4. KumoMTA DKIM-signs and delivers over SMTP `:25`. +5. KumoMTA `log_hooks` POSTs the outcome to `/webhooks/kumomta`. +6. Herald correlates by the echoed header, advances status, auto-suppresses hard + bounces/complaints, and emits a `herald.message.*` CloudEvent. diff --git a/content/docs/core/herald/domains.mdx b/content/docs/core/herald/domains.mdx new file mode 100644 index 0000000..4e5d9a1 --- /dev/null +++ b/content/docs/core/herald/domains.mdx @@ -0,0 +1,53 @@ +--- +title: Sending domains +description: Add and verify a domain to send from +--- + +Herald sends only from domains you have verified. Verification proves control of +the domain and publishes a DKIM key so the MTA can sign your mail, which is what +makes it deliverable. + +## Add a domain + +From the dashboard (**Domains → Add domain**) or `createSendingDomain`: + +```graphql +mutation { + createSendingDomain(input: { domain: "send.example.com" }) { + domainId + dkimSelector + dkimPublicKey + verified + } +} +``` + +Herald generates a DKIM key pair and returns the public half. Use a dedicated +sending subdomain (for example `send.example.com`) rather than your root domain, +so sending reputation stays isolated from the rest of your mail. + +## Publish DNS and verify + +Add the DKIM public key as a `TXT` record at +`._domainkey.` (the dashboard shows the exact record), then +verify: + +```graphql +mutation { + verifySendingDomain(id: "") { + verified + } +} +``` + +Verification resolves the record and checks it matches the key Herald generated. +A missing record reads as "not yet verified" rather than an error, so you can +re-run it after DNS propagates. + +For the best inbox placement, also publish `SPF` and `DMARC` records for the +sending domain. Herald surfaces deliverability recommendations in the dashboard. + +## The send gate + +Until a domain is verified, any send whose `from` address uses it is rejected. +Once verified, messages from that domain are signed and sent. diff --git a/content/docs/core/herald/index.mdx b/content/docs/core/herald/index.mdx new file mode 100644 index 0000000..1d2a249 --- /dev/null +++ b/content/docs/core/herald/index.mdx @@ -0,0 +1,80 @@ +--- +title: Herald +description: Transactional email platform for the Omni stack +--- + +import { ProductOverview } from "@/components/docs"; +import app from "@/lib/config/app.config"; +import { PiEnvelopeSimple, PiPaperPlaneTilt, PiShieldCheck } from "react-icons/pi"; + +, + className: "bg-blue-100 text-blue-800 dark:bg-blue-900/20 dark:text-blue-300", + }, + { + label: "Transactional", + icon: , + className: "bg-purple-100 text-purple-800 dark:bg-purple-900/20 dark:text-purple-300", + }, + { + label: "Deliverability", + icon: , + className: "bg-green-100 text-green-800 dark:bg-green-900/20 dark:text-green-300", + }, + ]} + links={[ + { + href: `${app.socials.github}/herald-api`, + label: "Visit Repository", + type: "repository", + }, + ]} + alerts={[ + { + title: "Big Idea", + description: + "Own the transactional email path end to end: a multi-tenant send API in front of a self-hosted MTA with a dedicated sending reputation, suppression, and delivery feedback.", + }, + ]} +/> + +**Herald** is Omni's transactional email platform. It pairs a small multi-tenant +send API with a self-hosted [KumoMTA](https://kumomta.com) mail host on a +dedicated IP, so product emails (order receipts, notifications, verification) +flow over infrastructure Omni controls rather than a third-party ESP, while +keeping a third-party provider available as a fallback. + +## Key Features + +- **Multi-tenant send API**: API-key auth resolves a tenant; each tenant has its own suppression list and verified sending domains. +- **Self-hosted MTA**: KumoMTA on a dedicated Hetzner host with a dedicated mail IP, PTR, and DKIM signing, isolated from application egress to protect sending reputation. +- **Suppression built in**: hard bounces and spam complaints auto-suppress the recipient; suppressed addresses are rejected before any send. +- **Delivery feedback**: KumoMTA posts delivery/bounce/complaint events back to Herald, which advances message status and emits CloudEvents. +- **Idempotent sends**: an optional idempotency key dedupes retries so a redelivered request never sends twice. +- **Swappable engine**: the mail engine sits behind an interface, so the underlying MTA can change without touching callers. + +## How it fits + +Herald is consumed two ways: + +1. **Directly** by services that send their own mail (`POST /messages` with a tenant API key). +2. **Through [Vortex](/docs/core/vortex)** workflows, which render an email and hand it to Herald, falling back to a third-party provider on failure. + +Inter-service delivery events are published as [CloudEvents](https://cloudevents.io) +(`herald.message.delivered`, `herald.message.bounced`, `herald.message.complained`) +under the source `omni.herald`. + +## Sending domain + +Herald signs and sends from a dedicated sending subdomain (`send.omni.dev`) with +SPF, a `herald` DKIM selector, and DMARC configured, while the MTA host itself +(`mail.omni.dev`) carries the PTR and receives bounce/feedback mail. Keeping the +sending domain separate from the primary domain isolates reputation. + +## Status + +Herald is an internal platform service. See [Architecture](/docs/core/herald/architecture) +for how the API and MTA fit together. diff --git a/content/docs/core/herald/meta.json b/content/docs/core/herald/meta.json new file mode 100644 index 0000000..fa7c306 --- /dev/null +++ b/content/docs/core/herald/meta.json @@ -0,0 +1,14 @@ +{ + "title": "Herald", + "pages": [ + "index", + "quickstart", + "api-keys", + "domains", + "templates", + "tracking", + "webhooks", + "sdk", + "architecture" + ] +} diff --git a/content/docs/core/herald/quickstart.mdx b/content/docs/core/herald/quickstart.mdx new file mode 100644 index 0000000..dd571b0 --- /dev/null +++ b/content/docs/core/herald/quickstart.mdx @@ -0,0 +1,84 @@ +--- +title: Quickstart +description: Send your first email with Herald +--- + +This guide takes you from nothing to a delivered email: create an API key, +verify a sending domain, and send a message over REST or GraphQL. + +## 1. Create an API key + +Every request authenticates as a tenant with a bearer API key. Create one from +the dashboard (**API Keys → New key**) or with the `createApiKey` mutation. The +secret is shown **once**, so store it immediately. + +Keys carry a permission level: + +- `full` manages the tenant (keys, domains, templates, suppressions) and sends. +- `sending` may only send messages. + +See [API keys](/docs/core/herald/api-keys) for expiry and rotation. + +## 2. Verify a sending domain + +Herald only sends from a domain you have verified, which proves you control it +and lets Herald publish a DKIM key for signing. Add a domain (**Domains → Add +domain**), then publish the DKIM `TXT` record Herald shows you and click +**Verify**. See [Sending domains](/docs/core/herald/domains). + +## 3. Send a message + +### REST + +```bash +curl https://api.herald.omni.dev/messages \ + -H "authorization: Bearer $HERALD_API_KEY" \ + -H "content-type: application/json" \ + -d '{ + "to": "recipient@example.com", + "from": "hello@send.example.com", + "subject": "Welcome aboard", + "html": "

Thanks for signing up.

", + "idempotencyKey": "signup-12345" + }' +``` + +`from` must belong to a verified sending domain or the send is rejected. An +optional `idempotencyKey` dedupes retries so a redelivered request never sends +twice. Other fields: `text`, `cc`, `bcc`, `replyTo`, `attachments`, `sendAt` +(future ISO-8601 to schedule), `variables`, `trackOpens`, `trackClicks`. + +### GraphQL + +```graphql +mutation Send { + sendMessage( + input: { + to: "recipient@example.com" + from: "hello@send.example.com" + subject: "Welcome aboard" + html: "

Thanks for signing up.

" + } + ) { + messageId + status + } +} +``` + +The GraphQL endpoint is `https://api.herald.omni.dev/graphql`, authenticated with +the same bearer key. + +## 4. Track delivery + +Herald records every message and its delivery events (`delivered`, `bounced`, +`complained`, and, when enabled, `opened`/`clicked`). View them in the dashboard +**Messages** log, query the `messages` connection, or subscribe to +[webhooks](/docs/core/herald/webhooks). Hard bounces and complaints add the +recipient to your suppression list automatically. + +## Next steps + +- [Templates and variables](/docs/core/herald/templates) for reusable, personalized content +- [Open and click tracking](/docs/core/herald/tracking) +- [The TypeScript SDK and MCP server](/docs/core/herald/sdk) diff --git a/content/docs/core/herald/sdk.mdx b/content/docs/core/herald/sdk.mdx new file mode 100644 index 0000000..d35ae84 --- /dev/null +++ b/content/docs/core/herald/sdk.mdx @@ -0,0 +1,52 @@ +--- +title: SDK and MCP server +description: Typed client and an agent-ready tool surface +--- + +Herald ships a typed TypeScript client and a Model Context Protocol (MCP) server, +both generated from the same GraphQL schema so they stay in lockstep with the +API. + +## TypeScript SDK + +`@omnidotdev/herald` wraps the generated GraphQL client with an ergonomic facade: +API-key auth, automatic idempotency keys, and named methods. + +```ts +import { createHerald } from "@omnidotdev/herald"; + +const herald = createHerald({ apiKey: process.env.HERALD_API_KEY }); + +await herald.emails.send({ + to: "recipient@example.com", + from: "hello@send.example.com", + subject: "Welcome aboard", + html: "

Thanks for signing up.

", +}); +``` + +The client also exposes the messages, domains, and API-key operations. Because it +is generated from the schema, new fields and operations appear in the client as +the API grows. + +## MCP server + +`herald-mcp` is a standalone [Model Context Protocol](https://modelcontextprotocol.io) +server that exposes a curated, agent-friendly subset of Herald as tools, so an +assistant can send and inspect mail on a tenant's behalf: + +- `send_email` +- `list_messages` +- `get_message` +- `list_domains` +- `verify_domain` + +It speaks MCP over stdio and calls the Herald API with the tenant token supplied +in its environment, so the same permission model applies: the agent can only do +what its key allows. + +## Regenerating + +Both targets are generated with [ODK](/docs/armory/odk) from the committed Herald +schema and operation documents. Regenerate after the schema changes so the SDK, +its hooks, and the MCP tools reflect the current API. diff --git a/content/docs/core/herald/templates.mdx b/content/docs/core/herald/templates.mdx new file mode 100644 index 0000000..4b69978 --- /dev/null +++ b/content/docs/core/herald/templates.mdx @@ -0,0 +1,63 @@ +--- +title: Templates and variables +description: Reusable content with Handlebars merge fields +--- + +Templates store reusable subject, HTML, and text content. Combined with +variables, they let you send personalized mail without rebuilding the body on +every call. + +## Create a template + +From the dashboard (**Templates**) or `createTemplate`: + +```graphql +mutation { + createTemplate( + input: { + name: "welcome" + subject: "Welcome, {{name}}" + html: "

Hi {{name}}, thanks for joining {{company}}.

" + } + ) { + templateId + } +} +``` + +## Variables + +Herald renders [Handlebars](https://handlebarsjs.com) tags server-side before +the message is stored and injected, so the stored body and the delivered body +are identical. + +- `{{value}}` inserts an HTML-escaped value (the safe default). +- `{{{value}}}` inserts a raw, unescaped value. Use it only for content you + trust, since values are tenant-provided. + +Pass `variables` on a send to fill the tags: + +```json +{ + "to": "ada@example.com", + "from": "hello@send.example.com", + "subject": "Welcome, {{name}}", + "html": "

Hi {{name}}.

", + "variables": { "name": "Ada", "company": "Omni" } +} +``` + +When no `variables` are supplied the content is sent verbatim, so a body that +contains literal braces is never reinterpreted. + +## Preview + +Use the template preview (the editor's live preview, backed by the +`renderTemplate` query) to render a template against sample variables and see the +final subject and body before sending. + +## Broadcasts + +For audience sends, store per-contact attributes on each contact; a broadcast +renders the template with each recipient's own attributes, so one template +personalizes the whole send. diff --git a/content/docs/core/herald/tracking.mdx b/content/docs/core/herald/tracking.mdx new file mode 100644 index 0000000..5612a9f --- /dev/null +++ b/content/docs/core/herald/tracking.mdx @@ -0,0 +1,54 @@ +--- +title: Open and click tracking +description: Measure engagement on the messages you send +--- + +Herald can record when a recipient opens a message or clicks a link, surfacing +engagement in the message log and emitting `opened`/`clicked` events to your +webhooks. + +## Enable per send + +Set `trackOpens` and/or `trackClicks` on a send: + +```json +{ + "to": "recipient@example.com", + "from": "hello@send.example.com", + "subject": "Product update", + "html": "

See what's new.

", + "trackOpens": true, + "trackClicks": true +} +``` + +- **Opens** inject a 1x1 tracking pixel before ``. The pixel URL carries + an HMAC-signed token, so tokens cannot be forged or enumerated. +- **Clicks** rewrite each `` to a redirect endpoint that records the + click and then `302`s to the original URL. The original URL is stored, which + avoids an open redirect and yields per-link stats. + +Tracking only ever rewrites the delivered copy; the stored message body stays +clean, so a resend re-applies tracking fresh. + +## Tenant defaults + +Rather than setting flags on every send, set tenant-wide defaults in the +dashboard **Settings** page (`trackOpensDefault`, `trackClicksDefault`). A send +that omits `trackOpens`/`trackClicks` inherits the tenant default; an explicit +value on the send always wins. + +Transactional and verification mail is best left untracked for deliverability +and privacy; marketing broadcasts default to tracking on. + +## Tracking host + +Tracking links are served by Herald on a dedicated host so they sit on your +sending brand rather than the API host. A sending domain may also override the +host per domain with its own tracking subdomain. + +## Consuming engagement + +Opens and clicks appear on the message in the dashboard log and are delivered to +[webhook endpoints](/docs/core/herald/webhooks) subscribed to those event types, +alongside the standard delivery events. diff --git a/content/docs/core/herald/webhooks.mdx b/content/docs/core/herald/webhooks.mdx new file mode 100644 index 0000000..1d2bfdf --- /dev/null +++ b/content/docs/core/herald/webhooks.mdx @@ -0,0 +1,52 @@ +--- +title: Webhooks +description: Receive delivery and engagement events +--- + +Webhooks push message events to your own endpoint as they happen, so you can +react to deliveries, bounces, complaints, opens, and clicks without polling. + +## Register an endpoint + +From the dashboard (**Webhooks**) or `createWebhook`: + +```graphql +mutation { + createWebhook( + input: { url: "https://example.com/hooks/herald", eventTypes: ["bounced", "complained"] } + ) { + webhookId + secret + } +} +``` + +The `secret` is returned once and used to sign deliveries. Omit `eventTypes` +(or pass an empty list) to receive every event; otherwise you only receive the +types you list. + +## Event types + +- `delivered` +- `bounced` +- `complained` +- `opened` (requires [open tracking](/docs/core/herald/tracking)) +- `clicked` (requires click tracking) + +## Verifying signatures + +Each delivery is signed with an HMAC-SHA256 over the payload using your endpoint +secret, alongside a timestamp and version field. Recompute the signature with +your stored secret and compare before trusting a payload. Reject anything that +does not match. + +## Delivery, retries, and replay + +Deliveries are durable: each is enqueued and sent by a poller with exponential +backoff (escalating from about a minute up to several hours), and dead-lettered +after the maximum attempts rather than dropped. Recent deliveries appear in the +dashboard with their status, and a failed or dead delivery can be **replayed** +for an immediate retry once your endpoint is healthy again. + +Respond `2xx` promptly to acknowledge a delivery; a non-`2xx` or timeout is +treated as a failure and retried.