Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/decisions/0012-observability.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,3 +140,7 @@ So **we are far under the limit**, even in a bad case. Two things the alerts do


**Correction (2026-10-03):** the rule for a silent certificate-renewal timer waited 2 days. The renewal timers are daily with a random delay of up to 24 hours and an accuracy of 4 hours, so two runs can be almost 48 hours apart and the rule could fire without a fault. It now waits 3 days.

## Update 2026-10-15: the relay login is an option

The module logged in to the relay as `alerts@<domain>`, which the lab never exercised (a sink takes any login). Brevo's SMTP login is an id of its own (`xxxxxx001@smtp-brevo.com`, shown on its SMTP page), so the first real mail of the deployment would have failed with an authentication error. `tidepool.mail.smtpLogin` sets it; `null` (the default) keeps `alerts@<domain>`. Checked by evaluating the Alertmanager configuration of the real host with and without the option; the real mail is still tried at runbook step 1.8, with a deliberately failed backup.
2 changes: 1 addition & 1 deletion nixos/modules/observability.nix
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ let
mailFrom = "${config.tidepool.brand.name} alerts <alerts@${d}>"; mailTo = "admin@${d}";
smtp = if cfg.lab
then { smarthost = "127.0.0.1:1025"; require_tls = false; }
else { smarthost = "smtp-relay.brevo.com:587"; require_tls = true; auth_username = "alerts@${d}"; auth_password_file = "/run/credentials/alertmanager.service/smtp-password"; };
else { smarthost = "smtp-relay.brevo.com:587"; require_tls = true; auth_username = if cfg.mail.smtpLogin != null then cfg.mail.smtpLogin else "alerts@${d}"; auth_password_file = "/run/credentials/alertmanager.service/smtp-password"; };
heartbeat = if cfg.lab then { url = "http://127.0.0.1:8112/ping"; } else { url_file = "/run/credentials/alertmanager.service/heartbeat-url"; };
blackboxCfg = pkgs.writeText "blackbox.yml" (builtins.toJSON { modules = {
https = { prober = "http"; timeout = "10s"; http = { valid_status_codes = [ 200 301 302 401 403 404 ]; tls_config.insecure_skip_verify = cfg.lab; }; }; # production verifies the chain: a placeholder certificate must fail
Expand Down
1 change: 1 addition & 0 deletions nixos/modules/options.nix
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ in
big2tb = mkOption { type = types.str; description = "The second disk: the offsite repository and the VM pool."; };
};
arcMaxMiB = mkOption { type = types.int; default = 3072; description = "Cap of the ZFS cache."; };
mail.smtpLogin = mkOption { type = types.nullOr types.str; default = null; description = "The login Alertmanager presents to the SMTP relay (smtp-relay.brevo.com). Brevo's login is an id of its own (of the form `xxxxxx001@smtp-brevo.com`, on its SMTP page), not an address of your domain; the password is the sops secret `smtp-password`. null keeps `alerts@<domain>`, which only a relay that accepts that login will take."; };
acmeDns.apiBase = mkOption { type = types.str; default = "https://auth.acme-dns.io"; description = "The acme-dns server that holds the certificates' challenge records (ADR 0008): the public instance. The registrations' credentials are the sops secret `acme-dns-credentials`."; };
wireguard.peers = mkOption { type = types.listOf types.attrs; default = [ ]; };
services = {
Expand Down
1 change: 1 addition & 0 deletions private-repo-template/host.nix
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
};
# The network. Until the Ethernet cable is connected the machine is on WiFi: the LAN interface is then the WiFi one (set by tidepool.wifi). With the cable: remove `wifi`, set lanInterface to the Ethernet interface's name.
wifi = { enable = true; interface = "REPLACE-wifi-interface"; ssid = "REPLACE-with-your-network-name"; }; # the secret `wifi-psk` in secrets.yaml holds one line: psk_home=<64 hex digits from `wpa_passphrase 'SSID' 'passphrase'`>
# mail.smtpLogin = "<your SMTP login>"; # Brevo's login is an id of its own (of the form xxxxxx001@smtp-brevo.com, on its SMTP page), not an address of your domain: without this line Alertmanager logs in as alerts@<domain>, which Brevo refuses. The password is the secret `smtp-password`.
wireguard.peers = import ./vpn-peers.nix; # one entry per device: tools/add-peer.sh (docs/vpn-clients.md); the server's own public key is in vpn-server.pub (written by tools/secrets-init.sh)
nas.enable = true;
nas.timeMachine.path = "/mnt/timemachine"; # the Time Machine partition of the 16 TB disk; Linux cannot write a journaled HFS+, so make it ext4 and mount it below. Remove both if you have no Mac
Expand Down
Loading