Close the direct run.app door with a shared-secret origin gate

.env.example

@@ -57,6 +57,13 @@ PORT=8000
 # requests with a valid JWT but an unlisted email return 403.
 # ADMIN_ALLOWED_EMAILS=alice@example.com,bob@example.com
 
+# Shared secret a Cloudflare Transform Rule stamps as X-Origin-Secret on every
+# request it proxies for api.anyplot.ai; api/origin_gate.py refuses anything
+# without it, which closes the direct *.run.app door. LEAVE THIS UNSET locally
+# and in tests — unset means the gate is off, and that is also the production
+# rollback. Set only on the Cloud Run service, from Secret Manager.
+# ORIGIN_SECRET=
+
 # ============================================================================
 # AI Services (optional)
 # ============================================================================

CHANGELOG.md

@@ -28,6 +28,41 @@ aggregate instead: an italic *Catalog* line at the end of the version section an
 
 ### Added
 
+- **A shared-secret origin gate closes the direct `*.run.app` door, and the apex Worker's
+  source moves into the repository** — the API runs on Cloud Run with `ingress=all`, so it
+  answers on two addresses: `api.anyplot.ai`, which Cloudflare proxies, and the raw
+  `*.run.app` URL, which it does not. Everything the edge enforces — the bot challenge, the
+  WAF, the cache that makes the `max-age=300` reads free — was one URL away from being
+  bypassed, and `api/request_context.py` already documented callers doing it. A Cloudflare
+  Transform Rule stamps `X-Origin-Secret` on everything it proxies for the API host, and
+  `api/origin_gate.py` refuses anything without it with 403 before the request costs
+  anything. **Unset means off**, which is what makes the rollback a single variable and
+  keeps local development and the test suite untouched: the code can ship long before the
+  rule and the secret exist. `/health` reports `origin_gate` (`off` · `off-seen` · `ok` ·
+  `missing` · `mismatch`) for the request it was asked with — never the value — so every
+  route into the service can be measured *before* the switch is thrown; `off-seen` is the
+  state every path that must keep working has to reach first. Exempt, as exact paths with no
+  prefixes: `/health` (the deploy smoke reaches the candidate on its `run.app` tag URL, which
+  never passes the edge) and `/debug/cache/invalidate` (`sync-postgres.yml` posts to the
+  direct URL by design, because Cloudflare's bot challenge answers an unauthenticated curl
+  POST with a 403 HTML page; that endpoint carries its own constant-time token), plus
+  `OPTIONS`, which a browser cannot attach a custom header to. `/seo-proxy/…` is deliberately
+  **not** exempt although the sibling repo exempts it: the site's nginx fetches those pages
+  over `api.anyplot.ai` and so carries the header, while an exemption would leave the API's
+  most expensive reads open on the direct URL — a cache miss or an unknown id queries the
+  repositories, and a crawler user agent schedules an outbound Plausible event per request.
+  Every header secret is now compared through one byte-wise comparator
+  (`api/secret_compare.py`, used by the gate and by both `/debug/*` locks), because
+  `secrets.compare_digest` raises `TypeError` on a non-ASCII `str` while a header arrives
+  latin-1-decoded from the wire: comparing strings handed any caller a one-byte way to turn a
+  cheap 401 or 403 into an unhandled, logged 500 — including on `/debug/cache/invalidate`,
+  which is exempt from the gate precisely because it has its own lock. The Cloudflare Worker
+  behind `anyplot.ai/api/*` now has
+  its source in `infra/cloudflare/`, because a Worker subrequest to a host in the same zone
+  bypasses that zone's Transform Rules — so the Worker stamps the header itself, deleting
+  any inbound one first so a caller cannot supply it. The pre-traffic smoke reads the secret
+  at run time and sends it, accepting `off`/`off-seen` so the pipeline keeps working before
+  the gate is armed and after a rollback. (#11208)
 - **The bot-serving monitor now raises an alarm instead of only turning a tab red, and
   derives what it asserts from the repo** — `bot-serving-check.yml` gained `issues: write`:
   a failing run opens the fixed-title issue **Bot serving check is red** (or comments on it,
@@ -54,7 +89,6 @@ aggregate instead: an italic *Catalog* line at the end of the version section an
   branch, so a dispatch from a feature branch can never raise or close a production
   incident. Timeout recomputed to 62 min by the file's own formula (36 checks x 90 s + five
   non-retried probes). (#11209)
-
 - **The API image is built and its container smoke-tested before merge, not after** — the
   first build attempt of a changed Dockerfile used to happen in Cloud Build, once the PR was
   already on `main`; that is how the deploy-api trigger sat red from 2026-08-30 until #10821
@@ -70,7 +104,6 @@ aggregate instead: an italic *Catalog* line at the end of the version section an
   excludes `plots/**`: the plot pipeline's PRs touch nothing the image serves. Adopted from the
   sibling repo kurrentschrift, which added the same job after its `pyproject.toml` fell out
   of the runtime stage. (#11205)
-
 - **IndexNow: changed pages are pushed to Bing, Yandex, Seznam, Naver and Yep instead of
   waiting for a crawl** — Bing Webmaster Tools' first recommendation for the site. A public
   key file (`app/public/<key>.txt`, served by an explicit nginx `location` so crawler UAs
@@ -238,6 +271,25 @@ aggregate instead: an italic *Catalog* line at the end of the version section an
   idle window — so the instance is in practice never reclaimed and visitors keep the
   same time to first byte. `anyplot-api` keeps `min-instances=1`: its cold start is
   ~11.6 s and its traffic does leave gaps over 15 minutes. (#10812)
+- **The API deploy configures the revision additively — `--update-secrets` and
+  `--update-env-vars`, not the `--set-` forms** — both `--set-` flags replace their whole set,
+  so anything attached to the service out of band is stripped from every revision the pipeline
+  creates. `ORIGIN_SECRET` is exactly such a binding — attached by hand to arm the origin gate,
+  removed by hand to roll back — and a secret-backed variable lives in the same revision
+  environment as a literal one, so either flag was a way to silently disarm the gate on the
+  next deploy. It cannot simply be listed in the flag instead: Cloud Run refuses a deploy
+  naming a secret that does not exist, which would break every build until the rollout creates
+  it. The cost is that a variable dropped from either line is no longer removed automatically.
+  (#11208)
+- **The analytics middleware moves inside `CORSMiddleware`** — a consequence of where the
+  origin gate has to sit. The gate belongs inside CORS, so its 403 still carries the headers
+  a browser needs to read it as a 403 rather than as an opaque network error, and outside
+  the bot counter, so a refused request can never fire an outbound Plausible event —
+  `track_asset_fetch` fires per request for anything with a crawler user agent, so a caller
+  on the direct URL could otherwise turn each of its own refusals into one. Those two are
+  only simultaneously possible with the counter inside CORS. The cache-header middleware
+  stays outside CORS, where its `setdefault` for the /og/ cards depends on being. `api/main.py`
+  now carries the stack order and the reason for each position. (#11208)
 
 ## [3.2.0] — 2026-08-29 — Findable by assistants
 

agentic/docs/project-guide.md

@@ -202,6 +202,8 @@ Example: `plots/scatter-basic/` contains everything for the basic scatter plot.
   - **`agentic/workflows/`**: Click CLI scripts (plan, build, test, review + orchestrators)
   - **`agentic/commands/`**: Markdown prompt templates
 - **`automation/`**: CI/CD helper scripts (workflow_cli, label_manager, sync_to_postgres)
+- **`infra/`**: Infrastructure that would otherwise live only in a dashboard
+  - **`infra/cloudflare/`**: Source of the apex `anyplot.ai/api/*` Worker, plus the origin gate's rollout and measuring procedure
 - **`tests/`**: Unit, integration, and e2e tests mirroring source structure
 - **`docs/`**: Architecture and workflow documentation
 

api/cloudbuild.yaml

@@ -68,10 +68,31 @@ steps:
        #   /insights/visitors on the public stats page). The Secret Manager
        #   entry must exist before the first deploy that includes this line —
        #   create it with: gcloud secrets create PLAUSIBLE_API_KEY --data-file=-
-      - "--set-secrets=DATABASE_URL=DATABASE_URL:latest,CACHE_INVALIDATE_TOKEN=CACHE_INVALIDATE_TOKEN:latest,ADMIN_TOKEN=ADMIN_TOKEN:latest,PLAUSIBLE_API_KEY=PLAUSIBLE_API_KEY:latest"
+      #
+      # `--update-secrets`, NOT `--set-secrets`: the latter replaces the whole
+      # binding set, so it would strip any secret attached out of band from
+      # every revision this pipeline creates. ORIGIN_SECRET (api/origin_gate.py)
+      # is exactly such a binding — it is attached by hand when the gate is
+      # armed and removed by hand to roll back, and `--set-secrets` would
+      # silently disarm the gate on the next deploy. It cannot be listed here
+      # instead: Cloud Run refuses a deploy that names a secret which does not
+      # exist, which would break every build until the rollout reaches the step
+      # that creates it. The cost of `--update-secrets` is that a binding
+      # dropped from this line is no longer removed automatically — worth it
+      # against a gate that turns itself off.
+      - "--update-secrets=DATABASE_URL=DATABASE_URL:latest,CACHE_INVALIDATE_TOKEN=CACHE_INVALIDATE_TOKEN:latest,ADMIN_TOKEN=ADMIN_TOKEN:latest,PLAUSIBLE_API_KEY=PLAUSIBLE_API_KEY:latest"
       - "--execution-environment=gen2"
       # ^|^ alt delimiter: values contain @ (emails) and may contain , (multi-email lists)
-      - "--set-env-vars=^|^ENVIRONMENT=production|GOOGLE_CLOUD_PROJECT=$PROJECT_ID|GCS_BUCKET=anyplot-images|CF_ACCESS_TEAM_DOMAIN=${_CF_ACCESS_TEAM_DOMAIN}|CF_ACCESS_AUD=${_CF_ACCESS_AUD}|ADMIN_ALLOWED_EMAILS=${_ADMIN_ALLOWED_EMAILS}"
+      #
+      # `--update-env-vars` for the same reason as `--update-secrets` above, and
+      # belt and braces on top of it: a secret-backed variable lives in the same
+      # revision environment as a literal one, so a destructive `--set-env-vars`
+      # is a second way this pipeline could drop the hand-attached
+      # ORIGIN_SECRET and silently disarm the gate (Copilot review). Additive on
+      # both flags means one deploy cannot undo an out-of-band change; the cost
+      # is that a variable dropped from this line is no longer removed by the
+      # next deploy, which is the trade already accepted for the secrets.
+      - "--update-env-vars=^|^ENVIRONMENT=production|GOOGLE_CLOUD_PROJECT=$PROJECT_ID|GCS_BUCKET=anyplot-images|CF_ACCESS_TEAM_DOMAIN=${_CF_ACCESS_TEAM_DOMAIN}|CF_ACCESS_AUD=${_CF_ACCESS_AUD}|ADMIN_ALLOWED_EMAILS=${_ADMIN_ALLOWED_EMAILS}"
       - "--cpu-throttling"
       - "--concurrency=15"
       - "--timeout=600"
@@ -112,18 +133,51 @@ steps:
         # _MIN_INSTANCES says, so the first call that touches the DB may be the
         # first real DB request of that container's life and can fail once.
         RETRY="--retry 5 --retry-delay 5 --retry-all-errors"
+        # The candidate is probed on its `run.app` tag URL, which by definition
+        # never passes the Cloudflare edge — so once ORIGIN_SECRET is set on the
+        # service (api/origin_gate.py) every probe but /health needs the header
+        # the edge would have stamped. Read here rather than through
+        # `availableSecrets` on purpose: that resolves at build start and would
+        # fail every build until the secret exists, which is precisely the first
+        # step of the rollout. Missing secret or missing permission => empty =>
+        # the probes run bare, which is correct while the gate is off and fails
+        # loudly at /libraries once it is on. The value is captured, never
+        # echoed; the step runs without `set -x`.
+        ORIGIN_SECRET=$$(gcloud secrets versions access latest --secret=ORIGIN_SECRET 2>/dev/null || true)
+        HDR=()
+        if [ -n "$$ORIGIN_SECRET" ]; then HDR=(-H "X-Origin-Secret: $$ORIGIN_SECRET"); fi
+        # /health stays bare: it is exempt from the gate, and that is what makes
+        # it the probe that always reaches a cold candidate.
         curl -fsS $$RETRY "$$URL/health" | grep -q '"healthy"'
+        # …and that the secret this BUILD can read is the one the SERVICE was
+        # given. /health reports the verdict for the request it was asked with
+        # (never the value), so a rotation applied to only one of the two shows
+        # up here instead of as a mysterious 403 after the promote.
+        # `off`/`off-seen` are ACCEPTED, not failures: they are the gate before
+        # it is armed and after a rollback, and a build that refused to run then
+        # would take the deploy pipeline down exactly when it is needed most.
+        # Only `mismatch` is a real disagreement.
+        if [ -n "$$ORIGIN_SECRET" ]; then
+          gate=$$(curl -fsS $$RETRY "$${HDR[@]}" "$$URL/health" | python3 -c "import json,sys; print(json.load(sys.stdin).get('origin_gate'))")
+          case "$$gate" in
+            ok) echo "origin gate: armed, and this build's secret matches" ;;
+            off|off-seen) echo "origin gate: $$gate (not armed on this revision)" ;;
+            *) echo "origin gate says '$$gate' for this build's secret — service and build disagree"; exit 1 ;;
+          esac
+        fi
         # /libraries and /languages fall back to static metadata when the DB is
         # unreachable (optional_db), so they prove the app serves but not the
         # database. /plots/filter takes require_db — it is the probe that fails
         # when the Cloud SQL connection is broken.
-        curl -fsS $$RETRY "$$URL/libraries" | grep -q '"libraries"'
-        curl -fsS $$RETRY "$$URL/languages" | grep -q '"languages"'
-        curl -fsS $$RETRY "$$URL/plots/filter" >/dev/null
+        curl -fsS $$RETRY "$${HDR[@]}" "$$URL/libraries" | grep -q '"libraries"'
+        curl -fsS $$RETRY "$${HDR[@]}" "$$URL/languages" | grep -q '"languages"'
+        curl -fsS $$RETRY "$${HDR[@]}" "$$URL/plots/filter" >/dev/null
         # Fail-closed admin gate. 401 is the answer with ADMIN_TOKEN present and
         # no header sent; a 503 here would mean the secret never arrived, which
-        # is exactly the misconfiguration worth failing the build over.
-        code=$$(curl -s $$RETRY -o /dev/null -w '%{http_code}' "$$URL/debug/status")
+        # is exactly the misconfiguration worth failing the build over. With the
+        # gate armed the origin header has to be sent too, or this reads 403 and
+        # says "admin gate" about something that never reached it.
+        code=$$(curl -s $$RETRY "$${HDR[@]}" -o /dev/null -w '%{http_code}' "$$URL/debug/status")
         test "$$code" = "401" || { echo "admin gate expected 401, got $$code"; exit 1; }
         echo "smoke OK"
     id: "smoke"

api/main.py

@@ -25,6 +25,7 @@
     http_exception_handler,
 )
 from api.mcp.server import mcp_server  # noqa: E402
+from api.origin_gate import OriginSecretMiddleware  # noqa: E402
 from api.routers import (  # noqa: E402
     debug_router,
     download_router,
@@ -161,26 +162,35 @@ async def lifespan(app: FastAPI):
 app.add_exception_handler(HTTPException, http_exception_handler)
 app.add_exception_handler(Exception, generic_exception_handler)
 
+# The middleware stack, written innermost-first because `add_middleware` and
+# `@app.middleware` both wrap what is already there — so reading this file from
+# here down gives the order a request actually travels, in reverse:
+#
+#   cache headers → CORS → origin gate → bot counter → gzip → router
+#
+# (`HeadAsGetMiddleware` and `MCPTrailingSlashMiddleware` wrap the whole app
+# further out still; both only rewrite the scope.)
+#
+# Two of those positions are load-bearing:
+#
+# * The origin gate directly inside CORS, so a 403 from it still carries the
+#   headers a browser needs to read it as a 403 rather than as an opaque
+#   network error — and OUTSIDE the bot counter, so a refused request can never
+#   fire an outbound Plausible event. That second one is why the counter moved
+#   in here from outside CORS: `track_asset_fetch` fires per request for
+#   anything with a crawler user agent, so a caller on the direct `run.app` URL
+#   could otherwise turn each of its own refusals into one.
+# * The cache-header middleware stays OUTSIDE CORS, because its `setdefault`
+#   for the /og/ cards is what keeps CORSMiddleware's own header when the
+#   request came from an allowlisted origin — it has to run after CORS on the
+#   way out.
+
 # Enable GZip compression for responses > 500 bytes
 # This significantly reduces payload size for JSON API responses
 # (e.g., /plots/filter: 301KB -> ~40KB with gzip)
 # Note: GZip must be added before CORS so compression happens before CORS headers are added
 app.add_middleware(GZipMiddleware, minimum_size=500)
 
-# Configure CORS. Origins come from settings.cors_origins (single source of
-# truth — a hardcoded list here previously left https://www.anyplot.ai out
-# even though config promised it); the regex additionally allows any
-# localhost port for local dev servers.
-app.add_middleware(
-    CORSMiddleware,
-    allow_origins=settings.cors_origins,
-    allow_origin_regex=r"http://localhost:\d+",
-    allow_credentials=True,
-    allow_methods=["*"],
-    allow_headers=["*"],
-    expose_headers=["Mcp-Session-Id"],  # MCP session tracking
-)
-
 
 # Record which AI or search agent requested which catalogue page.
 #
@@ -214,6 +224,26 @@ async def record_bot_fetch(request: Request, call_next):
     return response
 
 
+# Close the direct `*.run.app` door: require the header the Cloudflare edge
+# stamps. Dormant until ORIGIN_SECRET is set on the service, which is both the
+# rollout order and the rollback (api/origin_gate.py).
+app.add_middleware(OriginSecretMiddleware)
+
+# Configure CORS. Origins come from settings.cors_origins (single source of
+# truth — a hardcoded list here previously left https://www.anyplot.ai out
+# even though config promised it); the regex additionally allows any
+# localhost port for local dev servers.
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=settings.cors_origins,
+    allow_origin_regex=r"http://localhost:\d+",
+    allow_credentials=True,
+    allow_methods=["*"],
+    allow_headers=["*"],
+    expose_headers=["Mcp-Session-Id"],  # MCP session tracking
+)
+
+
 # Add cache headers middleware
 @app.middleware("http")
 async def add_cache_headers(request: Request, call_next):