Email 1/7 (PR 1): explicit provider config, the provider contract and job identity #419

Part of #418. Read the parent first. Its Design sections Layers, Config, Provider contract and Job identity are the contract for this work. Step 1 of 7. server/ is short for apps/hocuspocus.server/.

Goal

Make the email provider an explicit choice from .env, behind one small contract, with one stable identity per job. Production behavior stays the same: the provider is still SMTP after this PR.

Owner and gate

• Builds it: an agent or engineer.
• Before merge (maintainer): set EMAIL_PROVIDER and EMAIL_FROM in both host env files, /opt/projects/stage.docs.plus/.env and /opt/projects/prod.docs.plus/.env.
  • Use the value that live GET /api/email/health reports in its provider field. Do not guess: today a leftover key can silently win.
  • Setting it early is safe on the old code, which honors an explicit EMAIL_PROVIDER (server/src/lib/email/providers/index.ts:18-23).
  • Post a comment here when done: "Env set in stage and prod: EMAIL_PROVIDER=".
• The PR may be built before that comment exists. It must not merge before it.

Steps

1. Env parse (server/src/config/env.schema.ts).
   • Keep EMAIL_PROVIDER as z.string(). Keep SENDGRID_API_KEY, only so a leftover key resolves to invalid.
   • Make SMTP_SECURE z.enum(['true', 'false']).optional() with no default.
   • Delete SMTP_FROM_NAME. Rewrite the comment at :108-110: providers now read config.email.delivery.
2. Email config (server/src/config/email.ts, create). Pure. Import the Env type only.
   • resolveEmailConfig(env): EmailConfig, exactly as the parent's Config table.
   • describeEmailEnv(env): EmailEnvFacts, booleans and non-secret values only.
   • In server/src/config/env.ts, call both once and expose config.email.delivery and config.email.facts. Delete the dead config.email.smtp, fromEmail and fromName (env.ts:88-101).
3. Contract (server/src/lib/email/providers/types.ts). Write the parent's Provider contract as given. Rename the old types as the parent says, and update the re-export at server/src/lib/email/index.ts:25.
4. Adapters (providers/resend.ts, providers/smtp.ts).
   • createResendProvider(cfg) and createSmtpProvider(cfg) take their values from cfg and never read process.env.
   • Resend passes options.signal to fetch and sends Idempotency-Key. SMTP ignores the signal and does not race it.
   • In this PR, every failure is thrown as new EmailSendError('transient', code, message, { cause, responseCode }). code is the provider's own code, verbatim (EAUTH, or the Resend name). That keeps today's retry-everything behavior and the live incident-email-broken query. Email 3/7 (PR 2): typed failures, delivery and a safe DLQ drain #421 (PR 2) adds the real classification.
   • Rename verify to checkConnection. Add close.
5. Factory (providers/index.ts). Replace auto-detect with createEmailProvider(cfg), a plain switch, and getEmailProvider(), which memoizes it from config.email.delivery. Delete getProviderStatus, isAnyProviderConfigured and verifyProvider. Their callers in server/src/lib/email/service.ts (:16, :26, :34, :92, :96, :105, :109) and server/src/api/email.ts (/health at :109-117, /status at :121) read config.email.delivery instead.
   • /api/email/health keeps its response keys, because the admin Notifications page reads provider and smtp_configured (apps/admin-dashboard/src/services/api.ts:476, :493-495). provider comes from the config. smtp_configured now means status === 'ready'.
6. Delete SendGrid: providers/sendgrid.ts and every reference.
7. From address. server/src/lib/email/sender.ts:79 reads config.email.delivery.from. No fallback.
8. Job identity (server/src/lib/email/jobIdentity.ts, create). Pure. Export EMAIL_QUEUE_NAME, EMAIL_DLQ_NAME, sentLogKey(jobId) and providerIdempotencyKey(namespace, id), per the parent.
   • queueEmail (server/src/lib/email/queue.ts:239-243) always sets a job id: the caller's id, else randomUUID().
   • In queue.ts, rename the local idempotencyKey (:105, :110) to sentLogKey and build it with sentLogKey(job.id).
   • The worker passes providerIdempotencyKey(namespace, job.id) to the provider. Inline sends use <namespace>/inline/<uuid>.
9. Same body on every retry. sender.ts:64: pass nowSeconds: Math.floor(Date.parse(data.created_at) / 1000) to signUnsubscribeToken. packages/email-templates/src/engine.ts:25: take year from a time the caller passes in, and pass the job's created_at. Search the templates for any other clock read and fix it the same way.
10. Status behavior in EmailGatewayService (service.ts), per the parent:
    • ready: log email config ready with provider.
    • off: the pgmq consumer and the BullMQ worker both settle rows skipped with email not configured.
    • invalid: start neither the pgmq consumer (server/src/lib/email/pgmqConsumer.ts:375) nor the BullMQ email worker. Log email config invalid with the problems at error level, at boot and every 5 min.
    • In server/src/hocuspocus.worker.ts, the health check (:173-180) treats email as healthy when the status is not ready. Add workers.email.configStatus beside pending, failed and provider (:182-200).
11. Age limit. In processMessage (pgmqConsumer.ts:365), a message whose enqueued_at is more than 24 h old settles skipped with stale and returns true. Name the constant EMAIL_MAX_AGE_MS. Apply it to notification and digest messages alike.
12. Deploy gate.
    • In .github/workflows/prod.docs.plus.yml, next to the loop at :543, check that EMAIL_PROVIDER and EMAIL_FROM are set in ${ENV_FILE}. When EMAIL_PROVIDER is set, also check EMAIL_UNSUBSCRIBE_SECRET and PUBLIC_RESTAPI_URL: without them every unsubscribe link is rejected and the one-click header is dropped (server/src/lib/email/sender.ts:58-61, :74). The error names the host env file. Do not reuse the GlitchTip message.
    • Add the same check after .github/workflows/stage.docs.plus.yml:230.
13. Alert. In scripts/observability/grafana/provisioning/alerting/rules-app-logs.yml, add incident-email-config-invalid, per the parent's alert table. Leave incident-email-broken as it is.
14. Templates and dev mail.
    • server/docker/config-templates/env.{development,staging,production}: add EMAIL_PROVIDER, drop SMTP_FROM_NAME.
    • In env.development, set SMTP_PORT=54325 and leave SMTP_USER and SMTP_PASS empty.
    • In packages/supabase/config.toml, uncomment smtp_port = 54325 (:105). Dev mail then shows in the Supabase catcher at http://localhost:54324.
    • Change the commented SendGrid example at config.toml:217-224 to Resend: host smtp.resend.com, port 465, user resend.
    • Root .env.example: the two blocks from the parent's Self-hoster .env, with blank values.

Test (one file)

server/src/config/__tests__/email.test.ts for resolveEmailConfig. It is dense branching that can hold all mail. Cover:

• each row of the parent's Config table;
• SMTP_SECURE unset with port 465 and 587, and an explicit false on 465;
• SMTP_USER unset gives no auth.

Docs (same PR)

• server/ENV.md §Email: explicit provider, required EMAIL_FROM, the off and invalid rules, optional SMTP auth, the SMTP_SECURE rule, no SendGrid, no SMTP_FROM_NAME.
• docs/self-hosting/install.md §3: add EMAIL_PROVIDER, EMAIL_FROM, EMAIL_UNSUBSCRIBE_SECRET and PUBLIC_RESTAPI_URL to "the values you cannot skip". Or say plainly that leaving email unset means off.
• docs/self-hosting/configuration.md and docs/self-hosting/README.md: an "Email in 5 minutes" section. Cover the variables, DNS (SPF, DKIM, DMARC p=none), and the fact that users must opt in to email.
• docs/self-hosting/configuration.md, a short "Your privacy duties" paragraph:
  • you are the controller of your users' data;
  • sign your email provider's DPA;
  • mail carries short extracts of document and chat text;
  • edit apps/webapp/src/components/pages/legal/legalMetadata.ts and /privacy before going live.
• AGENTS.md:386 and server/Readme.md: "SMTP/Resend/SendGrid" becomes "SMTP or Resend".
• server/src/modules/openapi/domain/paths/email.ts: keep the send-generic description true.
• server/CHANGELOG.md under ## [Unreleased]:
  • ### Breaking: EMAIL_PROVIDER and EMAIL_FROM are required.
  • ### Migration: the before and after .env lines. SendGrid users switch to the SendGrid SMTP relay with EMAIL_PROVIDER=smtp.
  • ### Removed: SendGrid, SMTP_FROM_NAME.
  • ### Fixed: jobs queued without an id could be skipped as already sent after a Redis reset.

Do not

• Do not change retry counts or add error classification. Those belong to Email 3/7 (PR 2): typed failures, delivery and a safe DLQ drain #421 (PR 2).
• Do not import server/src/lib/email/queue.ts or the lib/email index from a new pure file. queue.ts opens Redis at import (queue.ts:17).

Checks before push

• bun run check at the repo root.
• bun test in server/.
• Import each new pure file (config/email.ts, jobIdentity.ts) with REDIS_HOST, REDIS_PORT and DATABASE_URL unset.
• Locally: send one mail with EMAIL_PROVIDER=smtp and see it in the Supabase catcher.

Deploy

The deploy subject is exactly (build): back observability.

Acceptance

• Dev mail reaches the catcher at http://localhost:54324.
• Set SMTP_HOST and leave EMAIL_PROVIDER unset. The worker logs email config invalid, /health says healthy with workers.email.configStatus: "invalid", documents still save, and pgmq email messages stay unread.
• With EMAIL_PROVIDER=sendgrid, the problem text names the SMTP relay.
• A staging deploy with EMAIL_PROVIDER removed from the host env file stops at the gate. Restore it after.
• A message enqueued more than 24 h ago settles skipped with stale.
• After the prod deploy, one send-generic mail settles sent, and the worker logs its messageId.
• bun test is green, including the new file.
• Every doc listed above is updated in this PR.

Exit evidence

Post as a comment here:

• bun run check and bun test output.
• A screenshot of a dev mail in the catcher.
• The staging worker line email config ready with provider.
• The staging gate failure log from the acceptance check.

Revert

git revert the merge, then redeploy. The old code honors the explicit EMAIL_PROVIDER that the gate step set.