From 3b1a1cd05b0811b0d89738cce4941ebe00319798 Mon Sep 17 00:00:00 2001 From: Jeff Larson Date: Sat, 7 Nov 2026 20:45:43 -0800 Subject: [PATCH] feat(dashboard): accept an internal cluster SA-token on the read API (ADR-0043) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add a mesh-internal auth lane to the dashboard read surface, ALONGSIDE the edge Cloudflare Access path (ADR-0030): an in-cluster caller authorized by its Kubernetes ServiceAccount-token JWT (cluster issuer, RS256, a required `aud`, and a `sub` allowlist) can read `/api/*.json` without an edge Access JWT it cannot mint in-cluster — the lane the cluster repo's `fixer` needs. Pure addition, read-only, fail-closed: - reuses the ADR-0030 `Verifier` (alg pinned to RS256 — the alg-confusion close; iss/aud/exp/nbf/sub all required) pointed at the cluster JWKS via a new `ClusterJwksFetcher` that trusts the cluster CA and presents protector's own projected SA token, read fresh on every fetch, to the authenticated `openid/v1/jwks` discovery endpoint; - one gate, two lanes tried in order: a verified + allowlisted SA token is admitted read-only at the floor `Redacted` tier; a verified-but-non-allowlisted one is a hard 401 (never a 302, never a fall-through); an unverifiable one falls through to the UNCHANGED edge path (the internal lane never fails open); - fails closed everywhere: an empty allowlist, an unreadable cluster CA, or an unreachable cluster JWKS all admit no internal caller, and the edge path and every mutating/admission path are untouched. Config via `PROTECTOR_DASHBOARD_INTERNAL_*` (aud enables the lane; allowed subs; issuer/jwks/token/ca default to the in-pod projected-token locations). The lane rides on the Oidc-mode Enforcer, so it is active only when the edge OIDC issuer is configured. New unit + integration tests mirror watcher's internal-lane suite (allowlisted admit; valid-non-allowlisted 401; unverifiable fall-through; wrong aud/iss/alg/ expiry reject; JWKS-unreachable fail-closed) and prove the edge Bearer path and no-token 401 still hold with the lane configured. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_013cYVqzH7Xfwea7fAozdQK7 --- ...iceaccount-token-auth-on-dashboard-read.md | 110 +++++ docs/adr/README.md | 1 + engine/src/engine/dashboard/auth/enforce.rs | 72 ++- .../engine/dashboard/auth/enforce_tests.rs | 175 ++++++- engine/src/engine/dashboard/auth/internal.rs | 455 ++++++++++++++++++ engine/src/engine/dashboard/auth/jwks.rs | 92 +++- engine/src/engine/dashboard/auth/mod.rs | 12 + engine/src/engine/run_loop.rs | 43 ++ engine/src/engine/run_loop/tests.rs | 70 +++ 9 files changed, 1020 insertions(+), 10 deletions(-) create mode 100644 docs/adr/0043-internal-serviceaccount-token-auth-on-dashboard-read.md create mode 100644 engine/src/engine/dashboard/auth/internal.rs diff --git a/docs/adr/0043-internal-serviceaccount-token-auth-on-dashboard-read.md b/docs/adr/0043-internal-serviceaccount-token-auth-on-dashboard-read.md new file mode 100644 index 00000000..9224a8c5 --- /dev/null +++ b/docs/adr/0043-internal-serviceaccount-token-auth-on-dashboard-read.md @@ -0,0 +1,110 @@ +# 0043. Accept an internal Kubernetes ServiceAccount-token on the dashboard read surface + +- Status: Accepted +- Date: 2026-11-07 +- Relates to: [0030](0030-app-level-oidc-verification-supersedes-edge-trust.md) (the app-level + OIDC verifier + fail-closed enforcement this extends), [0031](0031-read-only-mcp-server-tiered-redaction.md) + (the same read-only, tiered-redaction surface — the internal lane is read-only too), + [0016](0016-severity-vs-urgency.md) (presentation is a view, never a decision gate — this gates + *viewing*, never the verdict/actuation path) + +## Context + +[0030](0030-app-level-oidc-verification-supersedes-edge-trust.md) made protector a **resource +server**: the dashboard read surface (the shell + the `/api/*.json` snapshots) verifies the +Cloudflare Access JWT the edge issues (the `Cf-Access-Jwt-Assertion` header / `CF_Authorization` +cookie) and fails closed. That is the right gate for a *browser* reaching protector through the +Cloudflare tunnel. It is the wrong gate for an *in-cluster* caller. + +The cluster repo's **`fixer`** runs in the same Kubernetes cluster as protector and needs to read +`/api/*.json` (its new `protector_read` tool). Routing it out to the Cloudflare edge and back to +obtain an Access assertion is absurd for a Service-to-Service call on the pod network — slower, +coupling an internal control path to the public edge, and `fixer` has no browser to carry the +Access cookie and cannot mint an edge Access JWT in-cluster. What `fixer` *does* already have is a +projected Kubernetes ServiceAccount token. + +Those tokens are OIDC JWTs the cluster itself issues and can verify: + +- issuer `https://kubernetes.default.svc.cluster.local`, signed **RS256**; +- JWKS published in-pod at `https://kubernetes.default.svc.cluster.local/openid/v1/jwks`; +- a projected token carries `aud=["protector"]` (the audience we require), `exp`, `nbf`, and + `sub="system:serviceaccount::"`. + +Reading that JWKS is itself authenticated: the `system:service-account-issuer-discovery` +ClusterRole gates it, so the fetch must present protector's own projected token as +`Authorization: Bearer` and trust the cluster CA (the apiserver's serving cert chains to it, not a +public root). + +## Decision + +Accept an internal ServiceAccount-token JWT on the dashboard read surface **in addition to** the +Cloudflare Access assertion. This is **pure addition** — the edge/Access path is unchanged, and the +lane is **read-only** (it mounts on the same read gate, never on a mutating or admission path). + +- **Reuse the existing verifier.** `dashboard::auth::Verifier` already checks RS256 signature + + `iss` + `aud` + `exp` + `nbf` + `sub`, pins the algorithm from config (never the token's own + header — the alg-confusion close of 0030 §1), and is issuer/audience-agnostic. A new + `jwks::ClusterJwksFetcher` points it at the cluster issuer/JWKS, trusts the cluster CA + (`reqwest::Certificate::from_pem`), and reads protector's SA token **fresh on every JWKS fetch** + (projected tokens rotate) to authenticate the discovery request. The same bounded-body / + single-flight / rate-limited `JwksStore` (0030 §5) protects this fetch too. +- **Require a `sub` allowlist.** A verified internal token is admitted only when its `sub` is in + `PROTECTOR_DASHBOARD_INTERNAL_ALLOWED_SUBS` (e.g. `system:serviceaccount:fixer:fixer`). A valid + token with a non-allowlisted `sub` is a hard `401` — a presented-but-unauthorized identity is a + reject, never a fall-through, and never a browser `302` (a machine identity is not re-challenged + at a login page). If the lane is configured (`PROTECTOR_DASHBOARD_INTERNAL_AUD` set) but the + allowlist is empty, the lane is treated as **off** (logged at ERROR) — fail closed, and the edge + path is left untouched. +- **One gate, two lanes, tried in order.** The dashboard enforce gate first tries the internal lane + when an `Authorization: Bearer` token is present; on a verify failure it **falls through** to the + edge path (it never fails *open* on the internal lane). An admitted internal identity is inserted + and subjected to the **same** minimum-tier gate as the edge path — it authorizes read exactly as + the edge path authorizes read today. +- **Floor tier, read-only.** A cluster SA token carries no `tier` claim, so an admitted internal + identity resolves to the most-restricted `Redacted` tier (0030 §6 — a missing tier is never + permissive). `fixer` reads the redacted view, which is the appropriate disclosure for an + automated reader. + +### Posture: fail-closed internal, fail-open-nowhere + +The edge path remains defense-in-depth exactly as 0030 left it. The internal lane fails **closed**: +an internal caller sends no `Cf-Access-Jwt-Assertion`, so when its token can't be verified it falls +through to the edge path, hits "no token," and gets `401`/`302` per the route class. A cluster-JWKS +outage (`AuthError::JwksUnreachable`) therefore admits no unverified internal caller — unlike the +Cloudflare path, there is no warm-cache fail-open here, because the internal lane is the *only* +thing that could admit a tokenless-at-the-edge caller. + +### Coupling to the edge gate + +The internal lane rides on `dashboard::auth::enforce::Enforcer`, which `run_loop` mounts only when +the edge OIDC issuer is configured (the dashboard's `Oidc` mode). An edge-trust-only deployment (no +issuer) mounts no gate at all, so the `PROTECTOR_DASHBOARD_INTERNAL_*` vars are inert there — but +that mode already serves the dashboard unauthenticated in-cluster, so there is nothing for the +internal lane to add. The cluster deploys protector with the edge issuer set, so the lane is active. + +### Configuration (env vars) + +| Variable | Default | Meaning | +| --- | --- | --- | +| `PROTECTOR_DASHBOARD_INTERNAL_AUD` | *(unset ⇒ lane off)* | Required audience; enables the lane when set and non-empty. | +| `PROTECTOR_DASHBOARD_INTERNAL_ALLOWED_SUBS` | *(empty ⇒ lane off)* | Comma-separated exact `sub` allowlist. | +| `PROTECTOR_DASHBOARD_INTERNAL_ISSUER` | `https://kubernetes.default.svc.cluster.local` | Expected `iss`. | +| `PROTECTOR_DASHBOARD_INTERNAL_JWKS_URL` | `/openid/v1/jwks` | JWKS endpoint (direct, no discovery). | +| `PROTECTOR_DASHBOARD_INTERNAL_TOKEN_PATH` | `/var/run/secrets/kubernetes.io/serviceaccount/token` | protector's own SA token, read fresh per fetch. | +| `PROTECTOR_DASHBOARD_INTERNAL_CA_PATH` | `/var/run/secrets/kubernetes.io/serviceaccount/ca.crt` | Cluster CA trusted for the JWKS fetch. | + +## Consequences + +- `fixer` (and any future in-cluster `/api` reader) authenticates with its own SA token — no + Cloudflare round-trip, scoped to a named identity, read-only at the redacted tier. The cluster + repo's protector chart sets `PROTECTOR_DASHBOARD_INTERNAL_AUD=protector` and the allowlist to + `system:serviceaccount:fixer:fixer`; the `fixer` chart mounts a projected `aud=protector` SA token + and presents it as the bearer. +- The attack surface stays narrow: RS256 only (no alg-confusion regression), the exact cluster `iss` + and the required `aud`, a non-expired/active token, and an explicit `sub` allowlist. Token + contents are never logged — only the `sub` on a reject. +- The edge path is untouched; existing Access/enforce tests and behavior stand. The cost is a second + verifier and a slightly more involved gate (two ordered lanes instead of one), plus the operational + duty to keep the allowlist correct — an empty or wrong allowlist fails closed, the safe direction. +- Unset `PROTECTOR_DASHBOARD_INTERNAL_AUD` leaves the lane entirely out of the gate, so local dev and + non-cluster deploys are unchanged. diff --git a/docs/adr/README.md b/docs/adr/README.md index ddaa6886..a8ecb52a 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -48,5 +48,6 @@ Copy [`0000-template.md`](0000-template.md) to start one. | [0040](0040-node-scoped-containment-mechanism-escalation.md) | Node-scoped containment is a **deterministic mechanism escalation** of a model-decided target: when typed evidence proves the adversary broke the *pod* boundary (host-cred read; root + escape edge; ptrace/module-load; or ≥2 model-confirmed actively-exploited pods co-resident), the resolver escalates the model-named workload's cut from a pod NetworkPolicy to `ContainNode` (cordon + co-resident default-deny) — the model keeps *what*, determinism owns *how* (rejects a model-selectable node menu line: highest blast on the weakest measured axis). Own `node` action class at ladder rung 3 (above quarantine), propose-first by construction (a node always has alive collateral), reversible + ownership-marked + one-node-cap + control-plane-excluded + worker-floor; posture-derived `nodes` RBAC; ships shadow-first (trigger/proposal, then actuator). Also records: adversary-reach is a **presentation annotation, never judge context** (context-not-evidence enforced by absence from the prompt), and off-path auto-containment is a **won't-build** (reaffirms 0032 §6, strengthened by 0038). Refines 0032/0034; reuses 0010/0017/0021/0035/0036 rails | Proposed | | [0041](0041-narrow-blanket-notable-exec-corroboration.md) | Narrow the blanket notable-exec corroboration arm to shapes: a bare interactive-shell/pkg-mgr `ProcessExec` stops blanket-corroborating every objective (`Behavior::ProcessExec → false` in the flat arm — symmetric with PrivilegeChange/PtraceAttach/ModuleLoad, restoring the 0011 on-call-engineer FP guard), and exec-based corroboration re-homes in an entry-scoped `reverse_shell_on_foothold` shape (interactive-shell exec + internet NetworkConnection within a symmetric 60s window, foothold-gated) — unmasking the deferred reverse-shell shape per 0024. Corrects the framing: the model never sees `corroborated` (removed from the prompt by 0034); narrowing buys deterministic auto-apply-gate precision + honest latent/live presentation + 0024 unmasking, NOT model-input precision. Safe **now** because Falco's `Alert` arm still backstops (`falco.enabled: true`), pre-arming in `audit`. One real behavior change (inert under audit): a shell-only + decisive-attack incident becomes a proposal not an auto-cut in enforce (0011-correct). Package managers excluded (always egress); `is_alarming_now`/menu seeding untouched; a narrowing-delta counter feeds the retire-Falco parity bake (0037). Amends 0009's action-bar input | Proposed | | [0042](0042-flag-remaining-consumer-count-on-consumer-removal.md) | Flag a shared layer's remaining live-consumer count when a PR removes a consumer: a consumer-removal PR states the layer's remaining live-consumer count in one line, and a drop-to-one is flagged in review as a refactor smell rather than merged silently — the discipline PR #189's shared model-completion cache and PR #190's hypothesis-stage deletion needed (their conjunction, invisible to either single-PR review, left a redundant, `Uncertain`-pinning cache that PR #191 had to revert). Extends 0024's single-PR redundant-by-construction rule across a two-PR conjunction | Accepted | +| [0043](0043-internal-serviceaccount-token-auth-on-dashboard-read.md) | Accept an internal Kubernetes ServiceAccount-token on the dashboard read surface, IN ADDITION TO the edge Cloudflare Access path (0030): an in-cluster caller (the cluster repo's `fixer`), authorized by its SA identity, reads `/api/*.json` with its projected SA-token JWT (cluster issuer, RS256, required `aud`, `sub` allowlist) instead of an edge Access JWT it cannot mint in-cluster. Pure addition + read-only: reuses the 0030 `Verifier` (alg-pinned, iss/aud/exp/nbf/sub) pointed at the cluster JWKS via a `ClusterJwksFetcher` (trusts the cluster CA, presents protector's own SA token read fresh per fetch); one gate, two lanes tried in order — a verified+allowlisted token admits read-only at the floor `Redacted` tier, a verified-but-non-allowlisted one is a hard 401 (never a 302 or fall-through), an unverifiable one falls through to the unchanged edge path. FAILS CLOSED (empty allowlist / unreadable CA / JWKS-unreachable ⇒ the lane admits no one; edge untouched). Rides the Oidc-mode Enforcer, so it needs the edge issuer set. Extends 0030; read-only sibling of 0031 | Accepted | See also [`../VISION.md`](../VISION.md) for the longer-form narrative this ADR realizes. diff --git a/engine/src/engine/dashboard/auth/enforce.rs b/engine/src/engine/dashboard/auth/enforce.rs index 52a4929b..d8b9ae4e 100644 --- a/engine/src/engine/dashboard/auth/enforce.rs +++ b/engine/src/engine/dashboard/auth/enforce.rs @@ -30,6 +30,7 @@ use axum::middleware::Next; use axum::response::{IntoResponse, Response}; use super::claims::Tier; +use super::internal::{InternalAuth, InternalOutcome}; use super::{AuthError, ConfigError, Identity, OidcConfig, Verifier, authenticate, non_empty_env}; /// The tiny JSON body for a `401` — no graph data, no stack, no which-check-failed detail (ADR-0030 @@ -60,6 +61,12 @@ pub struct Enforcer { login_redirect: String, /// The minimum authorization tier a verified identity must hold to view (default `Redacted`). min_tier: Tier, + /// The additive internal Kubernetes SA-token lane (ADR-0043), or `None` when it is not + /// configured. When set, it is tried BEFORE the edge path and ONLY for an + /// `Authorization: Bearer` token: a verified, allowlisted SA token is admitted read-only; a + /// non-allowlisted one is a hard `401`; an unverifiable one falls through to the edge path. It + /// never weakens the edge path and never fails open. + internal: Option, } impl Enforcer { @@ -67,7 +74,8 @@ impl Enforcer { /// issuer's keys over HTTPS, the login redirect (configured URL, else the issuer), and the /// minimum tier (env escape hatch, else the most-restricted default). A MISTYPED minimum tier is /// a loud [`ConfigError`] — fail-closed exactly like a bad audience/algorithm, so `run_loop` - /// refuses to serve rather than silently degrade the gate to allow-all. + /// refuses to serve rather than silently degrade the gate to allow-all. The internal lane is + /// NOT attached here — the caller adds it with [`with_internal`](Self::with_internal). pub fn new(config: OidcConfig) -> Result { let login_redirect = login_redirect(&config); let min_tier = configured_min_tier()?; @@ -75,17 +83,27 @@ impl Enforcer { verifier: Verifier::from_config(config), login_redirect, min_tier, + internal: None, }) } + /// Attach (or clear) the additive internal SA-token lane (ADR-0043). Pure addition and + /// reversible: `None` leaves the enforcer the edge-only gate it was. + pub fn with_internal(mut self, internal: Option) -> Self { + self.internal = internal; + self + } + /// Assemble a gate from explicit parts — the seam tests use to inject a [`Verifier`] built over /// an in-memory JWKS (no egress) and pin the login redirect / minimum tier deterministically. + /// The internal lane starts unattached; a test adds it with [`with_internal`](Self::with_internal). #[cfg(test)] pub(crate) fn from_parts(verifier: Verifier, login_redirect: String, min_tier: Tier) -> Self { Self { verifier, login_redirect, min_tier, + internal: None, } } @@ -107,6 +125,46 @@ pub async fn enforce( next: Next, ) -> Response { let class = route_class(request.uri(), request.headers()); + + // 1. Internal Kubernetes SA-token lane (ADR-0043) — additive, read-only, tried FIRST. It engages + // only when configured AND an `Authorization: Bearer` token is present, and it NEVER fails + // open: an unverifiable internal token falls through to the unchanged edge path below. + if let Some(internal) = &enforcer.internal + && let Some(token) = super::bearer_from_headers(request.headers()) + { + match internal.authorize(&token).await { + InternalOutcome::Admit(identity) => { + // Authorize READ-ONLY exactly as the edge path does: insert the verified + // Identity and apply the SAME minimum-tier gate. + if enforcer.identity_permitted(&identity) { + request.extensions_mut().insert(identity); + return next.run(request).await; + } + tracing::warn!( + subject = %identity.subject, + "dashboard: internal SA identity is below the minimum tier (forbidden)" + ); + return deny(Denial::Forbidden, class, &enforcer.login_redirect); + } + InternalOutcome::NotAllowed(subject) => { + // A validly-signed cluster token whose `sub` is NOT allowlisted is a hard reject + // — never a fall-through, and never a browser redirect (a machine identity is + // not re-challenged with a login page). + tracing::warn!(%subject, "dashboard: internal SA-token sub not in allowlist"); + return deny_internal_unauthorized(class); + } + InternalOutcome::Unverifiable(error) => { + // Do NOT fail open: fall through to the edge path. An in-cluster caller carries + // no Cloudflare Access assertion, so the edge path then rejects it fail-closed. + tracing::debug!( + %error, + "dashboard: internal bearer did not verify; falling through to edge auth" + ); + } + } + } + + // 2. Edge Cloudflare Access lane (ADR-0030) — UNCHANGED. match authenticate(&enforcer.verifier, request.headers()).await { Ok(identity) if enforcer.identity_permitted(&identity) => { request.extensions_mut().insert(identity); @@ -187,6 +245,18 @@ fn deny(denial: Denial, class: RouteClass, login_redirect: &str) -> Response { no_store(response) } +/// A hard `401` for a validly-signed internal SA token whose `sub` is NOT allowlisted (ADR-0043). +/// NEVER a `302` — a machine identity is not a browser to re-challenge at a login page — and never +/// a fall-through. It carries the same tiny JSON body (for the `/api` class) + `no-store` as the +/// edge `401`, so a rejected internal caller leaks no more than an edge one. +fn deny_internal_unauthorized(class: RouteClass) -> Response { + let response = match class { + RouteClass::Api => json(StatusCode::UNAUTHORIZED, BODY_UNAUTHENTICATED), + RouteClass::Document => StatusCode::UNAUTHORIZED.into_response(), + }; + no_store(response) +} + /// A tiny JSON error response with the given status — no graph data, no error detail. fn json(status: StatusCode, body: &'static str) -> Response { ( diff --git a/engine/src/engine/dashboard/auth/enforce_tests.rs b/engine/src/engine/dashboard/auth/enforce_tests.rs index cd82f810..9bed24f4 100644 --- a/engine/src/engine/dashboard/auth/enforce_tests.rs +++ b/engine/src/engine/dashboard/auth/enforce_tests.rs @@ -11,12 +11,13 @@ use axum::body::{Body, to_bytes}; use axum::http::{Request, StatusCode, header}; use tower::ServiceExt; -use super::Verifier; -use super::claims::Tier; +use super::claims::{Tier, TierGrants}; use super::enforce::Enforcer; +use super::internal::InternalAuth; use super::test_support::{ KEY_A_N, KEY_A_PEM, KID_A, TestFetcher, base_claims, jwk_set, now, sign, test_config, }; +use super::{OidcConfig, SigningAlgorithm, Verifier}; use crate::engine::dashboard::{AuthMode, DashboardState, router}; use crate::engine::journal::DecisionJournal; use crate::engine::policy_log::PolicyDecisionLog; @@ -60,6 +61,57 @@ fn enforcer_jwks_down() -> Arc { )) } +// --- the additive internal Kubernetes SA-token lane (ADR-0043) ---------------------------------- + +const INTERNAL_ISSUER: &str = "https://kubernetes.default.svc.cluster.local"; +const INTERNAL_AUD: &str = "protector"; +const FIXER_SUB: &str = "system:serviceaccount:fixer:fixer"; + +/// The cluster OIDC config the internal lane's verifier enforces (RS256, cluster issuer, internal +/// audience). DISTINCT from `test_config`'s edge issuer, so an internal token verifies on the +/// internal lane and an edge token on the edge lane. +fn internal_config() -> OidcConfig { + OidcConfig { + issuer: INTERNAL_ISSUER.into(), + audience: INTERNAL_AUD.into(), + tier_claim: "tier".into(), + algorithm: SigningAlgorithm::Rs256, + tier_grants: TierGrants::default(), + } +} + +/// An enforcer with BOTH the edge lane (`test_config` issuer) and the additive internal lane +/// (cluster issuer/aud), each served key-A in-memory (zero egress), allowlisting `allowed`. +fn enforcer_with_internal(min_tier: Tier, allowed: &[&str]) -> Arc { + let edge = Verifier::with_fetcher( + test_config(), + Arc::new(TestFetcher::new(jwk_set(&[(KID_A, KEY_A_N)]))), + ); + let internal_verifier = Verifier::with_fetcher( + internal_config(), + Arc::new(TestFetcher::new(jwk_set(&[(KID_A, KEY_A_N)]))), + ); + let internal = InternalAuth::with_verifier( + internal_verifier, + allowed.iter().map(|s| s.to_string()).collect(), + ); + Arc::new(Enforcer::from_parts(edge, LOGIN_URL.into(), min_tier).with_internal(Some(internal))) +} + +/// A cluster-SA-token-shaped JWT: cluster issuer, internal aud, the given `sub`, in-window exp/nbf, +/// and — like a real projected token — NO `tier` claim. +fn internal_token(sub: &str) -> String { + let claims = serde_json::json!({ + "iss": INTERNAL_ISSUER, + "aud": INTERNAL_AUD, + "sub": sub, + "iat": now() - 10, + "nbf": now() - 10, + "exp": now() + 3600, + }); + sign(KEY_A_PEM, KID_A, &claims) +} + /// Send a request through the configured router and return the response. async fn send(auth: Option>, request: Request) -> axum::response::Response { router(empty_state(AuthMode::Oidc), auth) @@ -415,3 +467,122 @@ async fn unconfigured_router_serves_without_rejecting() { "an unconfigured dashboard honestly reports edge-only" ); } + +// ------------------------------------------------------------------------------------------------- +// The additive internal Kubernetes SA-token lane (ADR-0043): an allowlisted in-cluster SA token +// reads `/api` read-only WITHOUT an edge Access JWT; a non-allowlisted one is a hard 401; the edge +// path is untouched; and the lane never fails open. +// ------------------------------------------------------------------------------------------------- + +#[tokio::test] +async fn api_with_allowlisted_internal_sa_token_is_200_read_only() { + // fixer's own cluster SA token (cluster issuer, internal aud, allowlisted sub, no tier claim) + // authorizes the read-only JSON — the whole point of the lane. No edge Access JWT involved. + let response = send( + Some(enforcer_with_internal(Tier::Redacted, &[FIXER_SUB])), + get( + "/api/findings.json", + Some(&internal_token(FIXER_SUB)), + Some("application/json"), + ), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); + let value: serde_json::Value = serde_json::from_str(&body_string(response).await).unwrap(); + assert!( + value.get("strip").is_some(), + "the internal lane authorizes the read-only view-model" + ); +} + +#[tokio::test] +async fn api_with_valid_but_non_allowlisted_internal_sa_token_is_401_never_302() { + // A validly-signed cluster token whose `sub` is NOT allowlisted is a hard 401 — never a 302 + // (a machine token is not a browser to redirect) and never the graph. + let response = send( + Some(enforcer_with_internal(Tier::Redacted, &[FIXER_SUB])), + get( + "/api/findings.json", + Some(&internal_token("system:serviceaccount:other:intruder")), + Some("application/json"), + ), + ) + .await; + assert_eq!(response.status(), StatusCode::UNAUTHORIZED); + assert!( + response.headers().get(header::LOCATION).is_none(), + "a non-allowlisted machine token is a hard 401, never a redirect" + ); + assert_eq!( + body_string(response).await, + r#"{"error":"unauthenticated"}"# + ); +} + +#[tokio::test] +async fn internal_lane_leaves_the_edge_bearer_path_working_via_fall_through() { + // An EDGE token offered on Bearer fails the internal verifier (wrong issuer → Unverifiable) and + // FALLS THROUGH to the unchanged edge path, which accepts it → 200. Proves the lane is additive + // (the existing machine/ID-JAG Bearer path still works) and that the fall-through reaches edge. + let response = send( + Some(enforcer_with_internal(Tier::Redacted, &[FIXER_SUB])), + get( + "/api/findings.json", + Some(&valid_token()), + Some("application/json"), + ), + ) + .await; + assert_eq!(response.status(), StatusCode::OK); +} + +#[tokio::test] +async fn internal_unverifiable_bearer_falls_through_and_edge_rejects_401() { + // A tampered cluster token fails the internal verifier (Unverifiable, NOT a hard reject) → falls + // through to the edge path, which also rejects → 401. The internal lane never fails OPEN. + let token = internal_token(FIXER_SUB); + let mut segments: Vec = token.split('.').map(String::from).collect(); + let sig = &segments[2]; + let first = sig.chars().next().unwrap(); + segments[2] = format!("{}{}", if first == 'A' { 'B' } else { 'A' }, &sig[1..]); + let tampered = segments.join("."); + let response = send( + Some(enforcer_with_internal(Tier::Redacted, &[FIXER_SUB])), + get( + "/api/findings.json", + Some(&tampered), + Some("application/json"), + ), + ) + .await; + assert_eq!(response.status(), StatusCode::UNAUTHORIZED); +} + +#[tokio::test] +async fn internal_configured_still_401s_an_unauthenticated_api_request() { + // With the internal lane configured but NO bearer present, the lane does not engage and the edge + // path rejects fail-closed — the additive lane never opens an unauthenticated hole. + let response = send( + Some(enforcer_with_internal(Tier::Redacted, &[FIXER_SUB])), + get("/api/findings.json", None, Some("application/json")), + ) + .await; + assert_eq!(response.status(), StatusCode::UNAUTHORIZED); +} + +#[tokio::test] +async fn internal_sa_token_below_the_minimum_tier_is_403() { + // A cluster SA token resolves to the floor (Redacted). If the operator raised the viewing + // minimum to Raw, the internal identity is verified-but-forbidden → 403, exactly as the edge + // path treats a below-tier identity (the lane authorizes "exactly as the edge path does"). + let response = send( + Some(enforcer_with_internal(Tier::Raw, &[FIXER_SUB])), + get( + "/api/findings.json", + Some(&internal_token(FIXER_SUB)), + Some("application/json"), + ), + ) + .await; + assert_eq!(response.status(), StatusCode::FORBIDDEN); +} diff --git a/engine/src/engine/dashboard/auth/internal.rs b/engine/src/engine/dashboard/auth/internal.rs new file mode 100644 index 00000000..64a92de6 --- /dev/null +++ b/engine/src/engine/dashboard/auth/internal.rs @@ -0,0 +1,455 @@ +//! The additive, read-only **internal** authentication lane (ADR-0043): accept an in-cluster +//! Kubernetes ServiceAccount-token JWT on the dashboard read surface **alongside** the edge +//! Cloudflare Access path (ADR-0030), so an in-mesh caller (the cluster repo's `fixer`) authorized +//! by its own SA identity can read `/api/*.json` WITHOUT minting an edge Access JWT it structurally +//! cannot obtain in-cluster. +//! +//! This lane is **pure addition**. It never weakens the edge path (which stays exactly as ADR-0030 +//! left it), it is **read-only** (it rides the same read-surface gate and touches no mutating / +//! admission path), and it **fails closed**: +//! +//! - An `Authorization: Bearer` token that **verifies** (RS256 against the cluster JWKS, cluster +//! `iss`, the required internal `aud`, in-window `exp`/`nbf`) **and** whose `sub` is in the +//! allowlist is admitted — read-only, at the most-restricted [`Tier::Redacted`] (a cluster SA +//! token carries no `tier` claim). +//! - A token that verifies but whose `sub` is **not** allowlisted is a hard reject +//! ([`InternalOutcome::NotAllowed`]) — a presented-but-unauthorized identity, never a +//! fall-through. +//! - A token that does **not** verify is [`InternalOutcome::Unverifiable`]: the caller falls +//! through to the unchanged edge path. The internal lane itself never fails *open* — an +//! in-cluster caller sends no `Cf-Access-Jwt-Assertion`, so the edge path then rejects it. +//! +//! The verification primitive is the SAME [`Verifier`] the edge path uses (RS256 pinned — the +//! alg-confusion close of ADR-0030 §1 — with `iss`/`aud`/`exp`/`nbf`/`sub` all required), pointed at +//! the cluster issuer/JWKS via a [`ClusterJwksFetcher`]. Only the key source and the required +//! issuer/audience differ; the fail-closed checks are identical. +//! +//! **Coupling.** The lane rides on [`super::enforce::Enforcer`], which `run_loop` mounts only when +//! the edge OIDC issuer is configured (the dashboard's `Oidc` mode). An edge-trust-only deployment +//! (no issuer) mounts no gate at all, so these env vars are inert there — but that mode already +//! serves the dashboard unauthenticated in-cluster, so there is nothing for the internal lane to +//! add. +//! +//! **Env namespace.** `PROTECTOR_DASHBOARD_INTERNAL_*` — distinct from the edge +//! `PROTECTOR_DASHBOARD_OIDC_*` and from the sigstore `PROTECTOR_OIDC_ISSUER`. + +use std::sync::Arc; + +use super::claims::TierGrants; +use super::jwks::ClusterJwksFetcher; +use super::{AuthError, Identity, OidcConfig, SigningAlgorithm, Verifier, non_empty_env}; + +/// `PROTECTOR_DASHBOARD_INTERNAL_AUD` — the audience the cluster SA-token MUST carry. Its PRESENCE +/// (non-empty) is what enables the internal lane; absent ⇒ the lane is off (edge-only), exactly as +/// an unset edge issuer leaves the edge path off. +const ENV_AUD: &str = "PROTECTOR_DASHBOARD_INTERNAL_AUD"; +/// `PROTECTOR_DASHBOARD_INTERNAL_ALLOWED_SUBS` — comma-separated EXACT `sub` allowlist, e.g. +/// `system:serviceaccount:fixer:fixer`. Empty ⇒ the lane admits nobody (fail closed; the binary +/// logs loudly and leaves the edge path untouched). +const ENV_ALLOWED_SUBS: &str = "PROTECTOR_DASHBOARD_INTERNAL_ALLOWED_SUBS"; +/// `PROTECTOR_DASHBOARD_INTERNAL_ISSUER` — the expected `iss` (the cluster OIDC issuer). +const ENV_ISSUER: &str = "PROTECTOR_DASHBOARD_INTERNAL_ISSUER"; +/// `PROTECTOR_DASHBOARD_INTERNAL_JWKS_URL` — the cluster JWKS endpoint (direct, no discovery). +const ENV_JWKS_URL: &str = "PROTECTOR_DASHBOARD_INTERNAL_JWKS_URL"; +/// `PROTECTOR_DASHBOARD_INTERNAL_TOKEN_PATH` — this pod's own projected SA token, read fresh per +/// JWKS fetch to authenticate the discovery request. +const ENV_TOKEN_PATH: &str = "PROTECTOR_DASHBOARD_INTERNAL_TOKEN_PATH"; +/// `PROTECTOR_DASHBOARD_INTERNAL_CA_PATH` — the cluster CA trusted for the JWKS fetch. +const ENV_CA_PATH: &str = "PROTECTOR_DASHBOARD_INTERNAL_CA_PATH"; + +/// Defaults matching an in-pod projected SA token at the standard mount. +const DEFAULT_ISSUER: &str = "https://kubernetes.default.svc.cluster.local"; +const DEFAULT_TOKEN_PATH: &str = "/var/run/secrets/kubernetes.io/serviceaccount/token"; +const DEFAULT_CA_PATH: &str = "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"; + +/// Configuration for the internal lane, built from the environment via [`InternalConfig::from_env`]. +/// `from_env` models the lane's OFF state (audience absent) as `None`, so the binary simply does not +/// attach the lane. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct InternalConfig { + /// Expected `iss` — the cluster OIDC issuer. + pub issuer: String, + /// The cluster JWKS endpoint the keys are fetched from (authenticated; cluster-CA-signed). + pub jwks_url: String, + /// The required `aud` the SA token must carry. + pub audience: String, + /// This pod's own projected SA token, read fresh per JWKS fetch. + pub token_path: String, + /// The cluster CA trusted for the JWKS fetch. + pub ca_path: String, + /// The exact `sub` allowlist. Empty ⇒ the lane admits nobody (fail closed). + pub allowed_subs: Vec, +} + +impl InternalConfig { + /// Build from the environment, or `None` when `PROTECTOR_DASHBOARD_INTERNAL_AUD` is unset/empty + /// (the lane is off). The issuer/JWKS/token/CA fall back to the in-pod defaults; the allowlist + /// is parsed from `PROTECTOR_DASHBOARD_INTERNAL_ALLOWED_SUBS` and MAY be empty here — the binary + /// treats an empty allowlist as a fail-closed disable (it never admits an un-allowlisted token). + pub fn from_env() -> Option { + let audience = non_empty_env(ENV_AUD)?; + let issuer = non_empty_env(ENV_ISSUER).unwrap_or_else(|| DEFAULT_ISSUER.to_string()); + let jwks_url = non_empty_env(ENV_JWKS_URL) + .unwrap_or_else(|| format!("{}/openid/v1/jwks", issuer.trim_end_matches('/'))); + let token_path = + non_empty_env(ENV_TOKEN_PATH).unwrap_or_else(|| DEFAULT_TOKEN_PATH.to_string()); + let ca_path = non_empty_env(ENV_CA_PATH).unwrap_or_else(|| DEFAULT_CA_PATH.to_string()); + let allowed_subs = non_empty_env(ENV_ALLOWED_SUBS) + .map(|value| { + value + .split(',') + .map(str::trim) + .filter(|sub| !sub.is_empty()) + .map(str::to_string) + .collect() + }) + .unwrap_or_default(); + Some(InternalConfig { + issuer, + jwks_url, + audience, + token_path, + ca_path, + allowed_subs, + }) + } +} + +/// The outcome of offering a bearer token to the internal lane. Only [`InternalOutcome::Admit`] +/// authorizes; the other two deny (a hard reject, or a fall-through to the edge path) — there is no +/// "allow / skip" variant. +#[derive(Debug)] +pub enum InternalOutcome { + /// A verified token with an allowlisted `sub` — admit, read-only, carrying the normalized + /// [`Identity`] (tier resolves to the floor [`super::claims::Tier::Redacted`], no `tier` claim). + Admit(Identity), + /// A verified token whose `sub` is NOT allowlisted — a hard reject (carries the `sub` for the + /// guard's warn log). Never a fall-through: a presented-but-unauthorized identity is a deny. + NotAllowed(String), + /// The token did not verify. The guard falls through to the edge path (the internal lane never + /// fails *open*); carries the distinct [`AuthError`] for the debug log. + Unverifiable(AuthError), +} + +/// The internal lane: a cluster-pointed [`Verifier`] plus the `sub` allowlist. Cheaply cloneable +/// parts; held by the [`super::enforce::Enforcer`] as an `Option`. +pub struct InternalAuth { + verifier: Verifier, + allowed_subs: Arc>, +} + +impl InternalAuth { + /// Production constructor: a [`Verifier`] over a [`ClusterJwksFetcher`] (trusts `ca_pem`, + /// presents the pod's SA token). RS256 is pinned and the tier machinery is inert (no `tier` + /// claim, no grants), so every admitted internal identity is read-only at the floor tier. + pub fn from_config(config: &InternalConfig, ca_pem: &[u8]) -> Self { + let fetcher = Arc::new(ClusterJwksFetcher::new( + config.jwks_url.clone(), + config.token_path.clone(), + ca_pem, + )); + Self { + verifier: Verifier::with_fetcher(cluster_oidc_config(config), fetcher), + allowed_subs: Arc::new(config.allowed_subs.clone()), + } + } + + /// Assemble from an explicit [`Verifier`] + allowlist — the seam tests use to inject an + /// in-memory JWKS (no egress). + #[cfg(test)] + pub(crate) fn with_verifier(verifier: Verifier, allowed_subs: Vec) -> Self { + Self { + verifier, + allowed_subs: Arc::new(allowed_subs), + } + } + + /// The expected issuer the internal verifier enforces (for the binary's startup log). + pub fn issuer(&self) -> &str { + self.verifier.config().issuer.as_str() + } + + /// The required audience the internal verifier enforces (for the binary's startup log). + pub fn audience(&self) -> &str { + self.verifier.config().audience.as_str() + } + + /// The `sub` allowlist (for the binary's startup log). + pub fn allowed_subs(&self) -> &[String] { + &self.allowed_subs + } + + /// Offer `token` to the internal lane: verify it, then gate on the allowlist. See + /// [`InternalOutcome`] for the three results. + pub async fn authorize(&self, token: &str) -> InternalOutcome { + match self.verifier.verify(token).await { + Ok(identity) => { + if self.allowed_subs.iter().any(|sub| sub == &identity.subject) { + InternalOutcome::Admit(identity) + } else { + InternalOutcome::NotAllowed(identity.subject) + } + } + Err(error) => InternalOutcome::Unverifiable(error), + } + } +} + +/// The cluster OIDC config the internal [`Verifier`] enforces: RS256 (cluster SA tokens are RS256), +/// the cluster `iss` + the required internal `aud`, and NO tier claim/grants — a cluster SA token +/// carries no tier, so it resolves to the most-restricted [`super::claims::Tier::Redacted`] +/// (read-only, floor redaction). +fn cluster_oidc_config(config: &InternalConfig) -> OidcConfig { + OidcConfig { + issuer: config.issuer.clone(), + audience: config.audience.clone(), + tier_claim: super::DEFAULT_TIER_CLAIM.to_string(), + algorithm: SigningAlgorithm::Rs256, + tier_grants: TierGrants::default(), + } +} + +#[cfg(test)] +mod tests { + //! Mirrors watcher's `access_jwt` internal-lane suite, adapted to protector's verifier: an + //! allowlisted `sub` is admitted; a valid-but-non-allowlisted token is a hard reject; an + //! unverifiable token falls through; a wrong `aud`/`iss`/alg/expiry rejects; and the lane fails + //! closed when the JWKS is unreachable. Every token is minted and verified through the in-memory + //! `test_support` JWKS — zero egress. + + use std::sync::Arc; + + use serde_json::{Value, json}; + + use super::super::claims::Tier; + use super::super::test_support::{ + ENV_LOCK, KEY_A_N, KEY_A_PEM, KID_A, TestFetcher, jwk_set, now, sign, + }; + use super::super::{AuthError, SigningAlgorithm, Verifier, claims::TierGrants}; + use super::*; + + const CLUSTER_ISSUER: &str = "https://kubernetes.default.svc.cluster.local"; + const INTERNAL_AUD: &str = "protector"; + const FIXER_SUB: &str = "system:serviceaccount:fixer:fixer"; + + /// The cluster OIDC config a test verifier enforces (RS256, cluster issuer, internal aud). + fn cluster_config() -> OidcConfig { + OidcConfig { + issuer: CLUSTER_ISSUER.into(), + audience: INTERNAL_AUD.into(), + tier_claim: "tier".into(), + algorithm: SigningAlgorithm::Rs256, + tier_grants: TierGrants::default(), + } + } + + /// An internal lane over an in-memory JWKS serving key-A, allowlisting `allowed`. + fn internal_auth(allowed: &[&str]) -> InternalAuth { + let fetcher = Arc::new(TestFetcher::new(jwk_set(&[(KID_A, KEY_A_N)]))); + let verifier = Verifier::with_fetcher(cluster_config(), fetcher); + InternalAuth::with_verifier(verifier, allowed.iter().map(|s| s.to_string()).collect()) + } + + /// An internal lane whose JWKS fetch always fails — the unreachable-cluster-JWKS condition. + fn internal_auth_jwks_down(allowed: &[&str]) -> InternalAuth { + let verifier = Verifier::with_fetcher(cluster_config(), Arc::new(TestFetcher::failing())); + InternalAuth::with_verifier(verifier, allowed.iter().map(|s| s.to_string()).collect()) + } + + /// A cluster-SA-token-shaped claim set: cluster issuer, internal aud, in-window exp/nbf, and — + /// like a real projected token — NO `tier` claim. + fn cluster_claims(sub: &str) -> Value { + json!({ + "iss": CLUSTER_ISSUER, + "aud": INTERNAL_AUD, + "sub": sub, + "iat": now() - 10, + "nbf": now() - 10, + "exp": now() + 3600, + }) + } + + #[tokio::test] + async fn allowlisted_sub_is_admitted_read_only_at_the_floor_tier() { + let auth = internal_auth(&[FIXER_SUB]); + let token = sign(KEY_A_PEM, KID_A, &cluster_claims(FIXER_SUB)); + match auth.authorize(&token).await { + InternalOutcome::Admit(identity) => { + assert_eq!(identity.subject, FIXER_SUB); + assert_eq!( + identity.tier, + Tier::Redacted, + "a cluster SA token has no tier claim → floor (read-only redacted)" + ); + } + other => panic!("expected Admit, got {other:?}"), + } + } + + #[tokio::test] + async fn valid_but_non_allowlisted_sub_is_a_hard_reject() { + // Verifies fine, but its `sub` is not allowlisted → NotAllowed (a hard reject, never a + // fall-through). The guard turns this into a 401. + let auth = internal_auth(&[FIXER_SUB]); + let token = sign( + KEY_A_PEM, + KID_A, + &cluster_claims("system:serviceaccount:other:intruder"), + ); + match auth.authorize(&token).await { + InternalOutcome::NotAllowed(sub) => { + assert_eq!(sub, "system:serviceaccount:other:intruder") + } + other => panic!("expected NotAllowed, got {other:?}"), + } + } + + #[tokio::test] + async fn tampered_signature_is_unverifiable_falls_through() { + let auth = internal_auth(&[FIXER_SUB]); + let token = sign(KEY_A_PEM, KID_A, &cluster_claims(FIXER_SUB)); + let mut segments: Vec = token.split('.').map(String::from).collect(); + let sig = &segments[2]; + let first = sig.chars().next().unwrap(); + segments[2] = format!("{}{}", if first == 'A' { 'B' } else { 'A' }, &sig[1..]); + let tampered = segments.join("."); + assert!(matches!( + auth.authorize(&tampered).await, + InternalOutcome::Unverifiable(AuthError::InvalidSignature) + )); + } + + #[tokio::test] + async fn wrong_audience_is_unverifiable() { + let auth = internal_auth(&[FIXER_SUB]); + let mut claims = cluster_claims(FIXER_SUB); + claims["aud"] = json!("not-protector"); + let token = sign(KEY_A_PEM, KID_A, &claims); + assert!(matches!( + auth.authorize(&token).await, + InternalOutcome::Unverifiable(AuthError::InvalidAudience) + )); + } + + #[tokio::test] + async fn wrong_issuer_is_unverifiable() { + // An edge-issuer token offered on Bearer does NOT verify on the cluster lane → falls through + // to the edge path (the fall-through the guard relies on to keep the ID-JAG lane working). + let auth = internal_auth(&[FIXER_SUB]); + let mut claims = cluster_claims(FIXER_SUB); + claims["iss"] = json!("https://issuer.example"); + let token = sign(KEY_A_PEM, KID_A, &claims); + assert!(matches!( + auth.authorize(&token).await, + InternalOutcome::Unverifiable(AuthError::InvalidIssuer) + )); + } + + #[tokio::test] + async fn expired_token_is_unverifiable() { + let auth = internal_auth(&[FIXER_SUB]); + let mut claims = cluster_claims(FIXER_SUB); + claims["nbf"] = json!(now() - 7200); + claims["exp"] = json!(now() - 3600); + let token = sign(KEY_A_PEM, KID_A, &claims); + assert!(matches!( + auth.authorize(&token).await, + InternalOutcome::Unverifiable(AuthError::Expired) + )); + } + + #[tokio::test] + async fn alg_confusion_hs256_is_unverifiable() { + // RS256 is pinned from config; a token whose header claims HS256 is rejected BEFORE any + // signature check — the same alg-confusion close as the edge path. + use base64::Engine as _; + use base64::engine::general_purpose::URL_SAFE_NO_PAD; + let auth = internal_auth(&[FIXER_SUB]); + let header = json!({ "alg": "HS256", "typ": "JWT", "kid": KID_A }); + let token = format!( + "{}.{}.{}", + URL_SAFE_NO_PAD.encode(serde_json::to_vec(&header).unwrap()), + URL_SAFE_NO_PAD.encode(serde_json::to_vec(&cluster_claims(FIXER_SUB)).unwrap()), + URL_SAFE_NO_PAD.encode(b"not-a-real-signature"), + ); + assert!(matches!( + auth.authorize(&token).await, + InternalOutcome::Unverifiable(AuthError::InvalidAlgorithm) + )); + } + + #[tokio::test] + async fn fails_closed_when_cluster_jwks_unreachable() { + // An unreachable cluster JWKS is Unverifiable(JwksUnreachable) — never an admit. The guard + // then falls through to the edge path, which rejects an internal caller (no edge header). + let auth = internal_auth_jwks_down(&[FIXER_SUB]); + let token = sign(KEY_A_PEM, KID_A, &cluster_claims(FIXER_SUB)); + assert!(matches!( + auth.authorize(&token).await, + InternalOutcome::Unverifiable(AuthError::JwksUnreachable) + )); + } + + #[test] + fn from_env_models_off_defaults_and_allowlist() { + let _env = ENV_LOCK + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + let keys = [ + ENV_AUD, + ENV_ALLOWED_SUBS, + ENV_ISSUER, + ENV_JWKS_URL, + ENV_TOKEN_PATH, + ENV_CA_PATH, + ]; + let clear = || unsafe { + for key in keys { + std::env::remove_var(key); + } + }; + clear(); + + // Audience absent ⇒ the lane is off. + assert_eq!(InternalConfig::from_env(), None); + + // Audience set ⇒ Some, with the in-pod defaults and a parsed (here empty) allowlist. + unsafe { std::env::set_var(ENV_AUD, "protector") }; + let config = InternalConfig::from_env().expect("audience set ⇒ configured"); + assert_eq!(config.audience, "protector"); + assert_eq!(config.issuer, DEFAULT_ISSUER); + assert_eq!( + config.jwks_url, + "https://kubernetes.default.svc.cluster.local/openid/v1/jwks" + ); + assert_eq!(config.token_path, DEFAULT_TOKEN_PATH); + assert_eq!(config.ca_path, DEFAULT_CA_PATH); + assert!( + config.allowed_subs.is_empty(), + "an unset allowlist parses empty (the binary fails closed on it)" + ); + + // The allowlist parses + trims; an explicit issuer re-derives the default JWKS URL. + unsafe { + std::env::set_var( + ENV_ALLOWED_SUBS, + " system:serviceaccount:fixer:fixer , system:serviceaccount:ops:reader ", + ); + std::env::set_var(ENV_ISSUER, "https://kubernetes.default.svc.cluster.local/"); + } + let config = InternalConfig::from_env().unwrap(); + assert_eq!( + config.allowed_subs, + vec![ + "system:serviceaccount:fixer:fixer".to_string(), + "system:serviceaccount:ops:reader".to_string(), + ] + ); + assert_eq!( + config.jwks_url, "https://kubernetes.default.svc.cluster.local/openid/v1/jwks", + "a trailing slash on the issuer is trimmed before deriving the JWKS URL" + ); + + clear(); + } +} diff --git a/engine/src/engine/dashboard/auth/jwks.rs b/engine/src/engine/dashboard/auth/jwks.rs index 8cf69212..e3452ba4 100644 --- a/engine/src/engine/dashboard/auth/jwks.rs +++ b/engine/src/engine/dashboard/auth/jwks.rs @@ -111,20 +111,98 @@ impl JwksFetcher for HttpJwksFetcher { "{}/.well-known/openid-configuration", self.issuer.trim_end_matches('/') ); - let discovery: Discovery = get_json(client, &discovery_url).await?; - get_json(client, &discovery.jwks_uri).await + let discovery: Discovery = get_json(client, &discovery_url, None).await?; + get_json(client, &discovery.jwks_uri, None).await } } -/// GET `url` and deserialize a 2xx JSON body **under a hard byte budget**. Any failure — -/// unreachable, non-2xx, an over-large body, or an unparsable one — collapses to -/// [`AuthError::JwksUnreachable`] so a fetch failure denies, never bypasses. +/// Fetches the **cluster's** OIDC JWKS for the additive internal ServiceAccount-token lane +/// (ADR-0043). Unlike [`HttpJwksFetcher`], the Kubernetes apiserver's discovery endpoint +/// (`/openid/v1/jwks`) is itself **authenticated** — the +/// `system:service-account-issuer-discovery` ClusterRole gates it — and its serving cert chains to +/// the **cluster CA**, not a public root. So this fetcher trusts the cluster CA and presents this +/// pod's own projected ServiceAccount token as `Authorization: Bearer`, **read fresh on every +/// fetch** (projected tokens rotate; a token cached at construction would go stale). It hits the +/// JWKS URL directly — no discovery round-trip. Any failure is [`AuthError::JwksUnreachable`], so a +/// fetch failure denies rather than bypasses (ADR-0030 §6 posture, carried into the internal lane). +pub struct ClusterJwksFetcher { + /// `None` when a bounded client trusting the cluster CA could not be built (unreadable/unparsable + /// CA, TLS backend init failure). We fail closed at fetch rather than fall back to a client that + /// skips CA verification or drops the timeout. + client: Option, + jwks_url: String, + /// Path to this pod's projected SA token, re-read on every fetch. + token_path: String, +} + +impl ClusterJwksFetcher { + /// Build a fetcher for `jwks_url`, trusting `ca_pem` (the cluster CA) and presenting the token + /// at `token_path` on each fetch. The timeout honors `PROTECTOR_OIDC_JWKS_TIMEOUT_SECS` (shared + /// with [`HttpJwksFetcher`]). + pub fn new(jwks_url: impl Into, token_path: impl Into, ca_pem: &[u8]) -> Self { + let timeout = std::env::var("PROTECTOR_OIDC_JWKS_TIMEOUT_SECS") + .ok() + .and_then(|s| s.trim().parse().ok()) + .unwrap_or(DEFAULT_FETCH_TIMEOUT_SECS); + Self { + client: build_cluster_client(timeout, ca_pem), + jwks_url: jwks_url.into(), + token_path: token_path.into(), + } + } +} + +/// A bounded-timeout client that trusts ONLY the supplied cluster CA. `None` on any failure — we +/// never fall back to the default trust store or an unbounded client (either would defeat the +/// fail-closed posture), so the fetch simply denies. +fn build_cluster_client(timeout: u64, ca_pem: &[u8]) -> Option { + let cert = match reqwest::Certificate::from_pem(ca_pem) { + Ok(cert) => cert, + Err(error) => { + tracing::error!(%error, "OIDC(internal): cluster CA PEM did not parse; internal JWKS fetch will fail closed"); + return None; + } + }; + match reqwest::Client::builder() + .timeout(Duration::from_secs(timeout)) + .add_root_certificate(cert) + .build() + { + Ok(client) => Some(client), + Err(error) => { + tracing::error!(%error, "OIDC(internal): could not build a bounded cluster JWKS client; internal JWKS fetch will fail closed"); + None + } + } +} + +#[async_trait] +impl JwksFetcher for ClusterJwksFetcher { + async fn fetch(&self) -> Result { + let client = self.client.as_ref().ok_or(AuthError::JwksUnreachable)?; + // Re-read the projected token each fetch (it rotates). A tiny local file — a blocking read + // is sub-millisecond and matches the rest of the engine's `std::fs` file reads. + let token = + std::fs::read_to_string(&self.token_path).map_err(|_| AuthError::JwksUnreachable)?; + get_json(client, &self.jwks_url, Some(token.trim())).await + } +} + +/// GET `url` and deserialize a 2xx JSON body **under a hard byte budget**, optionally presenting a +/// `Bearer` credential (the authenticated cluster discovery endpoint requires it; the public +/// Cloudflare path passes `None`). Any failure — unreachable, non-2xx, an over-large body, or an +/// unparsable one — collapses to [`AuthError::JwksUnreachable`] so a fetch failure denies, never +/// bypasses. async fn get_json( client: &reqwest::Client, url: &str, + bearer: Option<&str>, ) -> Result { - let mut response = client - .get(url) + let mut request = client.get(url); + if let Some(token) = bearer { + request = request.bearer_auth(token); + } + let mut response = request .send() .await .and_then(reqwest::Response::error_for_status) diff --git a/engine/src/engine/dashboard/auth/mod.rs b/engine/src/engine/dashboard/auth/mod.rs index bd578c19..a2a708cc 100644 --- a/engine/src/engine/dashboard/auth/mod.rs +++ b/engine/src/engine/dashboard/auth/mod.rs @@ -25,6 +25,7 @@ pub mod claims; pub mod enforce; +pub mod internal; pub mod jwks; #[cfg(test)] @@ -485,6 +486,17 @@ fn bearer_token(value: &str) -> Option<&str> { (!rest.is_empty()).then_some(rest) } +/// The `Authorization: Bearer` token from a request, if present. The internal SA-token lane +/// (ADR-0043) reads ONLY this header — a cluster SA token is presented as a bearer, never as a +/// Cloudflare Access assertion — so it does not reuse [`extract_token`]'s edge-lane fallbacks. +pub(super) fn bearer_from_headers(headers: &HeaderMap) -> Option { + headers + .get(header::AUTHORIZATION) + .and_then(|value| value.to_str().ok()) + .and_then(bearer_token) + .map(str::to_string) +} + /// The value of a named cookie from the `Cookie` header, if present and non-empty. fn cookie_value(headers: &HeaderMap, name: &str) -> Option { let cookies = headers.get(header::COOKIE)?.to_str().ok()?; diff --git a/engine/src/engine/run_loop.rs b/engine/src/engine/run_loop.rs index 4435306a..1c64f68c 100644 --- a/engine/src/engine/run_loop.rs +++ b/engine/src/engine/run_loop.rs @@ -75,6 +75,11 @@ fn build_dashboard_auth() -> Option<( // gate that silently degrades to allow-all. match dashboard::auth::enforce::Enforcer::new(config) { Ok(enforcer) => { + // Additive internal SA-token lane (ADR-0043): attach it when configured so an + // in-cluster caller (fixer) authorized by its SA identity can read `/api` + // without an edge Access JWT. It rides on the Enforcer — so it is active only in + // this Oidc mode — and never weakens the edge path (a `None` here is edge-only). + let enforcer = enforcer.with_internal(build_dashboard_internal_auth()); tracing::info!( %issuer, "dashboard OIDC enforcement ENABLED (fail-closed, ADR-0030)" @@ -112,6 +117,44 @@ fn build_dashboard_auth() -> Option<( } } +/// Build the additive internal Kubernetes SA-token lane for the dashboard read surface (ADR-0043), +/// or `None` when it is off / misconfigured. `None` leaves the gate edge-only — the lane is pure +/// addition and never weakens the edge path. It is **fail-closed**: an empty allowlist or an +/// unreadable cluster CA DISABLES the lane (admitting no internal token), logged loudly, rather than +/// widening access. Enabled only when `PROTECTOR_DASHBOARD_INTERNAL_AUD` is set. +fn build_dashboard_internal_auth() -> Option { + let config = dashboard::auth::internal::InternalConfig::from_env()?; + if config.allowed_subs.is_empty() { + tracing::error!( + aud = %config.audience, + "dashboard internal SA-token auth: PROTECTOR_DASHBOARD_INTERNAL_AUD is set but \ + PROTECTOR_DASHBOARD_INTERNAL_ALLOWED_SUBS is empty — lane DISABLED (fail closed, \ + admitting no internal token); edge auth is unaffected" + ); + return None; + } + let ca_pem = match std::fs::read(&config.ca_path) { + Ok(pem) => pem, + Err(error) => { + tracing::error!( + %error, + ca_path = %config.ca_path, + "dashboard internal SA-token auth: cannot read the cluster CA — lane DISABLED \ + (fail closed); edge auth is unaffected" + ); + return None; + } + }; + let auth = dashboard::auth::internal::InternalAuth::from_config(&config, &ca_pem); + tracing::info!( + issuer = %auth.issuer(), + aud = %auth.audience(), + allowed_subs = ?auth.allowed_subs(), + "dashboard internal SA-token auth ENABLED (additive, read-only, fail-closed, ADR-0043)" + ); + Some(auth) +} + /// Build the read-only MCP server's OIDC verifier from the environment (ADR-0031). The /// MCP surface REQUIRES app-level auth — its tier ceiling + audit line bind to a verified human /// subject, so there is NO edge-only bypass here (unlike the dashboard, ADR-0030 §6). Returns the diff --git a/engine/src/engine/run_loop/tests.rs b/engine/src/engine/run_loop/tests.rs index 1a426817..23fab3d4 100644 --- a/engine/src/engine/run_loop/tests.rs +++ b/engine/src/engine/run_loop/tests.rs @@ -118,6 +118,76 @@ fn tier_grants_config_fails_loud_but_serves_on_valid_or_absent() { clear(); } +/// The additive internal SA-token lane (ADR-0043) is off unless `PROTECTOR_DASHBOARD_INTERNAL_AUD` +/// is set, FAILS CLOSED (lane disabled, never admit-all) on an empty allowlist, and builds once a +/// non-empty allowlist and a readable cluster CA are present. Drives the real +/// `build_dashboard_internal_auth` over the env. +#[test] +fn dashboard_internal_auth_off_unless_aud_set_and_fails_closed_on_empty_allowlist() { + let _env = crate::engine::dashboard::auth::test_support::ENV_LOCK + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + + const AUD: &str = "PROTECTOR_DASHBOARD_INTERNAL_AUD"; + const SUBS: &str = "PROTECTOR_DASHBOARD_INTERNAL_ALLOWED_SUBS"; + const CA: &str = "PROTECTOR_DASHBOARD_INTERNAL_CA_PATH"; + let clear = || unsafe { + for key in [ + AUD, + SUBS, + CA, + "PROTECTOR_DASHBOARD_INTERNAL_ISSUER", + "PROTECTOR_DASHBOARD_INTERNAL_JWKS_URL", + "PROTECTOR_DASHBOARD_INTERNAL_TOKEN_PATH", + ] { + std::env::remove_var(key); + } + }; + clear(); + + // Audience unset ⇒ the lane is off (edge-only). + assert!( + super::build_dashboard_internal_auth().is_none(), + "no PROTECTOR_DASHBOARD_INTERNAL_AUD ⇒ no internal lane" + ); + + // Audience set but the allowlist empty ⇒ fail closed: the lane is DISABLED, never admit-all. + unsafe { std::env::set_var(AUD, "protector") }; + assert!( + super::build_dashboard_internal_auth().is_none(), + "an empty allowlist disables the lane (fail closed), never admits anyone" + ); + + // A non-empty allowlist + a readable CA ⇒ the lane builds. (A bad CA PEM still builds the lane; + // it just fails closed at fetch — so any readable CA file suffices to exercise the wiring.) + let ca_file = std::env::temp_dir().join(format!( + "protector-internal-ca-{}-{:?}", + std::process::id(), + std::thread::current().id() + )); + std::fs::write( + &ca_file, + b"-----BEGIN CERTIFICATE-----\nnot-a-real-cert\n-----END CERTIFICATE-----\n", + ) + .unwrap(); + unsafe { + std::env::set_var(SUBS, "system:serviceaccount:fixer:fixer"); + std::env::set_var(CA, &ca_file); + } + let auth = + super::build_dashboard_internal_auth().expect("aud + allowlist + readable CA ⇒ lane built"); + assert_eq!(auth.allowed_subs().len(), 1); + assert_eq!(auth.allowed_subs()[0], "system:serviceaccount:fixer:fixer"); + assert_eq!(auth.audience(), "protector"); + assert_eq!( + auth.issuer(), + "https://kubernetes.default.svc.cluster.local" + ); + + let _ = std::fs::remove_file(&ca_file); + clear(); +} + // an operational incident reported the protector container exiting cleanly // (exit 0) and restart-looping, never reaching Ready — read from the tail of its logs as // "the engine is doing real work, then the process just stops" and hypothesized as a