From 4ad394204563cfdfb127a6360ae90437759cab29 Mon Sep 17 00:00:00 2001 From: S'Bussiso Dube <80188685+Sbussiso@users.noreply.github.com> Date: Wed, 9 Sep 2026 14:19:22 -0700 Subject: [PATCH 1/3] =?UTF-8?q?Deploy=20from=20CI=20=E2=80=94=20fly.toml?= =?UTF-8?q?=20was=20a=20file=20that=20did=20nothing?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Deploy automation was deferred so a human could watch the first-ever deploy of new infrastructure, with a note to automate once phase 1 was stable. Phase 1 is stable, and leaving it deferred turned out worse than what it avoided: fly.toml only took effect if someone remembered to run `fly deploy` by hand. That failed silently on 2026-09-09. A scale-to-zero change merged with CI fully green and never reached Fly — the app kept running always-on, and the config looked applied because the commit was on master. It was only caught by checking the machine state directly. Config that silently doesn't apply is more dangerous than no config, because it reads as done. Deploy runs on push to master only, after BOTH matrix legs (sqlite and postgres) pass — this service runs one codebase against either dialect, so a green sqlite leg alone is not evidence a deploy is safe. Two flags, each for a concrete reason: --strategy immediate this app mounts sentinel_license_data, and the default rolling strategy stands up a parallel machine first, which errors on the volume's single attachment slot. --ha=false Fly otherwise provisions two machines. It did exactly that on the manual deploy of the sibling Sync service and the extra had to be scaled away by hand; the volume could not serve two anyway. Co-Authored-By: Claude Opus 5 --- .github/workflows/test.yml | 50 +++++++++++++++++++++++++++++++++----- 1 file changed, 44 insertions(+), 6 deletions(-) diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 5e1247a..5d25613 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -1,10 +1,15 @@ -name: Test +name: Test & Deploy -# Deploy automation is deferred deliberately (see the Sentinel Command -# plan doc, Phase 1: "fine to start with a manual `fly deploy` for the -# very first cut given a human should be watching the first-ever deploy -# of new infrastructure anyway; automate once phase 1 is stable"). This -# workflow only covers lint + dependency audit + tests. +# Tests on every push and PR; deploys to Fly on pushes to master. +# +# Deploy automation was deferred so a human could watch the first-ever +# deploy of new infrastructure, with a note to "automate once phase 1 is +# stable". Phase 1 is stable, and leaving it deferred created a worse +# problem than it avoided: fly.toml became a file that did nothing. A +# scale-to-zero change merged with CI green on 2026-09-09 and never +# reached Fly — the app kept running always-on until someone noticed and +# deployed by hand. Config that silently doesn't apply is more dangerous +# than no config. on: push: @@ -68,3 +73,36 @@ jobs: env: TEST_DATABASE_URL: ${{ matrix.test_database_url }} run: uv run pytest -v + + deploy: + name: Deploy to Fly.io + runs-on: ubuntu-latest + # Waits on BOTH matrix legs (sqlite and postgres) — this service runs + # the same code against either dialect, so a green sqlite run alone + # is not evidence the deploy is safe. + needs: test + # Push-only: a PR runs the tests above but never ships. + if: github.event_name == 'push' + # Serialize so two quick pushes don't race on the machine update. + concurrency: + group: deploy-sentinel-license + cancel-in-progress: true + steps: + - uses: actions/checkout@v7 + + - uses: superfly/flyctl-actions/setup-flyctl@master + + # --strategy immediate because this app mounts a volume + # (sentinel_license_data). The default rolling strategy tries to + # stand up a parallel machine first and errors on the volume's + # single attachment slot. Sentinel-Sync has no volume and so needs + # no override; Command Center has the same constraint and the same + # flag. + # + # --ha=false because Fly otherwise provisions TWO machines, which + # this service does not need and which the volume cannot serve + # anyway. + - name: flyctl deploy + run: flyctl deploy --remote-only --strategy immediate --ha=false --yes + env: + FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }} From 708be52045ab00d4876d632cad5aa7ad5907ce9a Mon Sep 17 00:00:00 2001 From: S'Bussiso Dube <80188685+Sbussiso@users.noreply.github.com> Date: Wed, 9 Sep 2026 14:29:29 -0700 Subject: [PATCH 2/3] =?UTF-8?q?Watch=20dependencies=20=E2=80=94=20this=20r?= =?UTF-8?q?epo=20had=20nothing=20watching=20them?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit No Dependabot config existed here. That is the same gap that produced the Sentinel AI agent's three production breakages in two days: mcp 2.x renaming a symbol, Starlette removing on_startup, and a ceiling added to stop the second one silently pinning the service to a Starlette line carrying 7 advisories. All three surfaced on a rebuild of a repo nobody had rebuilt since June. `pip-audit --strict` in CI catches a known advisory against what is already pinned. It does nothing about a lockfile quietly ageing out of support, which is the failure that actually happened. No auto-merge workflow, deliberately. This repo now deploys to Fly on push to master, and GitHub does not trigger `on: push` workflows for commits pushed with GITHUB_TOKEN — so an auto-merge would land a bump on master WITHOUT deploying it, and nothing would report an error. That is an observed bug in Sentinel-Command, where four frontend bumps merged and never shipped. A human merge triggers the deploy normally. Co-Authored-By: Claude Opus 5 --- .github/dependabot.yml | 47 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) create mode 100644 .github/dependabot.yml diff --git a/.github/dependabot.yml b/.github/dependabot.yml new file mode 100644 index 0000000..6dbd2dd --- /dev/null +++ b/.github/dependabot.yml @@ -0,0 +1,47 @@ +# Dependabot configuration. +# +# This repo had NO dependency watching until 2026-09-09, which is the +# same gap that produced the Sentinel AI agent's three production +# breakages in two days: mcp 2.x renaming a symbol, Starlette removing +# on_startup, and a ceiling added to stop the second one silently +# pinning the service to a Starlette line with 7 advisories. All three +# surfaced on a rebuild of a repo nobody had rebuilt since June. +# +# `pip-audit --strict` in the Test workflow catches a *known advisory* +# in what is already pinned. It does nothing about a lockfile quietly +# ageing out of support. That is what this is for. +# +# NO AUTO-MERGE WORKFLOW HERE, DELIBERATELY. +# +# This repo now deploys to Fly on push to master. GitHub does not +# trigger `on: push` workflows for commits pushed with GITHUB_TOKEN, so +# an auto-merge action would land a bump on master WITHOUT deploying it +# — master and production drift apart, and nothing reports an error. +# That is a real, observed bug in Sentinel-Command (four frontend bumps +# merged and never shipped on 2026-09-09). A human merge triggers the +# deploy normally, so these PRs are reviewed and merged by hand until +# that repo's PAT fix is proven. +version: 2 +updates: + # Python (uv) — pyproject.toml + uv.lock live at the repo root. + - package-ecosystem: "uv" + directory: "/" + schedule: + interval: "weekly" + day: "monday" + open-pull-requests-limit: 5 + commit-message: + prefix: "build(deps)" + labels: + - "dependencies" + + # GitHub Actions — pinned action versions age out the same way. + - package-ecosystem: "github-actions" + directory: "/" + schedule: + interval: "weekly" + day: "monday" + open-pull-requests-limit: 5 + labels: + - "dependencies" + - "ci" From a656ef6f4f09aed5baf77ef1207b8b6bf8198f0a Mon Sep 17 00:00:00 2001 From: S'Bussiso Dube <80188685+Sbussiso@users.noreply.github.com> Date: Wed, 9 Sep 2026 18:21:32 -0700 Subject: [PATCH 3/3] README: this service deploys from CI and scales to zero MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two claims were false. "First deploy is manual by design — deploy automation is deferred" stopped being true when the deploy job landed; leaving it deferred had made fly.toml a file that did nothing, and a scale-to-zero change merged green and never reached Fly. And nothing documented that the service now sleeps between check-ins, or why that is safe (4s boot inside Fly's ~8s proxy window, 10s caller timeout, and a 72-hour grace window on the Command Center side). Also records the condition that would reverse it: a licence check moving onto a user-blocking path turns a 4s cold start into a 4s page stall. Co-Authored-By: Claude Opus 5 --- README.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index bd12f56..19d501a 100644 --- a/README.md +++ b/README.md @@ -49,7 +49,13 @@ uv run pytest ## Deploy -Single-stage `Dockerfile` (no frontend build — this service has no UI), `fly.toml` targets a much smaller VM than Command Center's (no video workload, tiny check-in traffic). First deploy is manual (`fly deploy`) by design — see `.github/workflows/test.yml`'s comment for why deploy automation is deferred. +Single-stage `Dockerfile` (no frontend build — this service has no UI). `fly.toml` targets a much smaller VM than Command Center's: 256 MB, no video workload, tiny check-in traffic. + +**Deploys from CI.** Every push to `master` runs the tests against both SQLite and Postgres, then `flyctl deploy`. Deploy automation was deferred while this was new infrastructure; that turned out worse than what it avoided, because `fly.toml` became a file that did nothing — a scale-to-zero change merged with CI fully green on 2026-09-09 and never reached Fly, and it *looked* applied because the commit was on master. Config that silently doesn't apply is more dangerous than no config. + +Two flags, each for a reason: `--strategy immediate` because this app mounts `sentinel_license_data` and the default rolling strategy errors on the volume's single attachment slot; `--ha=false` because Fly otherwise provisions two machines, which one volume can't serve anyway. + +**Scales to zero.** Self-hosted installs check in on a ~15-minute background tick, so this is idle by default. Safe because boot is ~4s — inside the ~8s Fly's proxy waits for an auto-started machine to bind — the caller's timeout is 10s, and a missed check-in is a *designed* path: Command Center treats network/5xx as "unreachable" and applies a 72-hour grace window. This flips if a licence check ever moves onto a user-blocking path; then a 4s cold start becomes a 4s page stall. ## Status