Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion devenv.lock
Original file line number Diff line number Diff line change
Expand Up @@ -426,7 +426,8 @@
"go-overlay": "go-overlay",
"hk": "hk",
"nix2container": "nix2container",
"nixpkgs": "nixpkgs"
"nixpkgs": "nixpkgs",
"secretspec-nixpkgs": "secretspec-nixpkgs"
}
},
"rust-overlay": {
Expand All @@ -450,6 +451,22 @@
"type": "github"
}
},
"secretspec-nixpkgs": {
"locked": {
"lastModified": 1788549839,
"narHash": "sha256-kOrCcSIA6w9J1hX5DqHy2k9pDTJymExTsbV74U9UtCA=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "17de0b976395537756f30a3e78f2f06e5cec89ed",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixpkgs-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"systems": {
"locked": {
"lastModified": 1681028828,
Expand Down
15 changes: 15 additions & 0 deletions devenv.nix
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,21 @@ in
pkgs.cloud-hypervisor
pkgs.virtiofsd
pkgs.passt
]
# secretspec: the CLI the Go secrets write path spawns BY NAME for
# `set`/`delete` (go/internal/secrets/resolver.go's `cli` default), so the
# write path is unreachable unless this shell puts one on PATH. Resolved from
# the `secretspec-nixpkgs` input rather than this shell's own nixpkgs because
# that channel's rev still carries 0.14.0, which has no `age` provider
# compiled in — the encrypted-at-rest default the server-secret resolver
# writes through. This input's version matches the Go SDK pin in go/go.mod, so
# the read path (SDK + native lib) and the write path (this CLI) advance
# together; `internal/secrets` asserts both halves rather than assuming them.
# A dotted input reference, so it is appended OUTSIDE the parsed `with pkgs`
# literal (same reason as skopeo-nix2container: the toolchain-parity gate
# resolves every bare attr in that literal, including on macOS).
++ [
inputs.secretspec-nixpkgs.legacyPackages.${pkgs.stdenv.system}.secretspec
];

env = {
Expand Down
17 changes: 17 additions & 0 deletions devenv.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,3 +54,20 @@ inputs:
inputs:
nixpkgs:
follows: nixpkgs
# secretspec-nixpkgs: a SECOND nixpkgs, pinned by rev in devenv.lock, solely
# for the `secretspec` CLI the Go secrets write path spawns by name. The
# `age://` provider it needs to write encrypted-at-rest secrets only exists
# from 0.15 on (it is a default-on cargo feature), and the rolling channel
# this shell's own nixpkgs is locked to still resolves 0.14.0 — a build with
# no `age` backend compiled in at all, which fails a write with `Provider
# backend 'age' not found` rather than degrading. This input tracks the
# channel that carries a version matching the Go SDK pin (go/go.mod's
# secretspec module), so the read path (SDK) and the write path (CLI) move
# together instead of skewing across an independent seam. It deliberately
# does NOT `follows: nixpkgs` — following would defeat the entire purpose by
# collapsing it back onto the rev that lacks the provider. Consumed as a
# dotted attr in devenv.nix, OUTSIDE the parsed `with pkgs` packages literal,
# because the toolchain-parity gate resolves every bare attr in that literal
# (the same reason skopeo-nix2container sits outside it).
secretspec-nixpkgs:
url: github:NixOS/nixpkgs/nixpkgs-unstable
10 changes: 5 additions & 5 deletions docs/designs/DECISIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,13 +72,13 @@ check enforces the mechanical half. Full rationale:
| DL-018 | The frozen `RunnerService` transport recommendation #2 is socket-only, superseding the earlier off-stdio carrier clause | Active (Matt, 2026-07-22) | [agent comms tools §The frozen transport](agent/compass-agent-comms-tools/design.md#the-frozen-transport-this-rides-was-the-keystone-fork-now-decided) |
| DL-313 | NATS is the single eventing substrate — run as a standalone stack service (alongside Postgres and the OTel collector) in every deployment, reached over `nats://`, with no embedded/in-process mode and no transport phase; single-node or clustered NATS is selected by connection string, never by application code. JetStream is the durable comms-delivery transport; core NATS carries routing/binding invalidation; queue groups partition delivery work. Connect stays the synchronous RPC edge; the agent↔Runner hop stays vsock (RIG-2394); no LISTEN/NOTIFY phase. Supersedes DL-014 and DL-021 (RIG-2861 OQ-1) | Active (Matt, 2026-08-31) | [multi-tenancy & NATS substrate](infra/runtime/compass-managed-multitenancy/design.md#q3--the-eventing-substrate-one-nats-eventfabric-a-standalone-stack-service-jetstream-as-the-delivery-transport) |
| DL-316 | Server↔Runner transport is TWO-PLANE (amends DL-013's Runner↔Server clause, RIG-2861 OQ-5 Variant B): async command-push + Runner event fan-in ride NATS (per-Runner command subjects + queue-group fan-in — `Sessions`/`PublishEvents` reshaped to pub/sub for the non-sticky-wake fabric); the typed request/reply legs (enrollment, unary `Relay*Call`s/`CommitConversationFrame`/`FetchSecrets`, bulk `FetchAgentConfig`) stay on the reduced Connect/gRPC edge (deadline propagation, typed proto errors, generated stubs); the per-Runner provisioned token is retained as the NATS-credentials seed via auth-callout. Client↔Server stays Connect; Runner↔Agent stays vsock. Supersedes DL-013 | Active (Matt, 2026-08-31) | [multi-tenancy & NATS substrate](infra/runtime/compass-managed-multitenancy/design.md#resolved-decisions-freeze--matt-2026-08-31) |
| DL-327 | The delivery work-queue consumer's JetStream ack for a HELD deliver (agent-authored message held until its author's session settles) is ACK-ON-RECEIVE (OQ-1): the fabric callback acks as soon as the message is classified and held/dispatched, not at fire — the held registry stays in-RAM and a crash between hold and fire recovers via the Postgres delivery-cursor sweep, exactly as today's in-process bus. Ack-on-fire is rejected (AckWait=30s << an agent turn ⇒ healthy held messages redeliver mid-turn and DLQ-park after MaxDeliver=5, absent per-message InProgress heartbeats) | Active (Matt, 2026-09-05) | [delivery cutover §OQ-1](infra/runtime/compass-managed-delivery-cutover/design.md#oq-1-load-bearing-jetstream-ack-timing-for-held-delivers) |
| DL-328 | The migrated delivery consumer runs `onEventRef` (re-read + classify + hold/dispatch) DIRECTLY on the fabric callback goroutine under `c.mu` + the per-session gates (OQ-2), concurrent with the settle/start drain loop — NOT enqueued onto a single goroutine. The relaxed cross-channel ordering is one today's non-deterministic `select` never actually guaranteed; the cursor sweep remains the no-loss floor. Two Option-A obligations are invariants, not forks: `scanMissedMentions`'s held-check and `MarkMentionsRouted` run under one critical section (scan-vs-hold), and per-callback dispatch work stays bounded so the ack does not block past AckWait behind a long `sweepSession` | Active (Matt, 2026-09-05) | [delivery cutover §OQ-2](infra/runtime/compass-managed-delivery-cutover/design.md#oq-2-load-bearing-concurrency-model--callback-direct-vs-loop-enqueue) |
| DL-329 | The delivery consumer splits its DB role once each event carries an explicit `ref.Tenant` (OQ-3 part 1): the inherently cross-tenant background sweeps/drains keep `WithSystemRole` (BYPASSRLS), but per-event processing runs under `store.WithTenant(baseCtx, ref.Tenant)` for the `MessageByID` re-read and the whole `onMessagePosted` chain — and `heldEntry` gains a `tenant` field so the `fireHeld` re-read is tenant-scoped too. Fail-closed: a forged/corrupted ref whose row belongs to another tenant reads zero rows under RLS instead of cross-tenant-delivering under BYPASSRLS. Whole-loop system-role (ref.Tenant routing-only) rejected — forfeits the stamped ref's isolation dividend | Active (Matt, 2026-09-05) | [delivery cutover §OQ-3](infra/runtime/compass-managed-delivery-cutover/design.md#oq-3-load-bearing-rls-scope-split--lag-recovery-replacement) |
| DL-330 | Recovery after a publish-SIDE fabric failure (commit ok, publish fails — the loss mode the infallible bus never had) is triggered by a FABRIC-RECONNECT HOOK plus a minutes-scale PERIODIC FLOOR TICK, each running `sweepAllLive` + `scanMissedMentions` (OQ-3 part 2). This replaces the deleted `sub.Lagged()` bus-ring branch. The draft's original mapping was falsified by the design-critic red-team: `scanMissedMentions` routes only mentions/ask-answers (never plain delivers) and NATS auto-reconnect (`MaxReconnects(-1)`) keeps the ConsumeContext alive so a "re-subscribe" trigger never fires across an outage — leaving a publish-failed plain deliver to an always-live agent silently undelivered until its next session restart. `sweepAllLive` (not just the scan) is the load-bearing plain-deliver recovery path; publisher-side bounded retry MAY be added but is not sufficient alone | Active (Matt, 2026-09-05) | [delivery cutover §OQ-3 part 2](infra/runtime/compass-managed-delivery-cutover/design.md#oq-3-load-bearing-rls-scope-split--lag-recovery-replacement) |
| DL-334 | The delivery work-queue consumer's JetStream ack for a HELD deliver (agent-authored message held until its author's session settles) is ACK-ON-RECEIVE (OQ-1): the fabric callback acks as soon as the message is classified and held/dispatched, not at fire — the held registry stays in-RAM and a crash between hold and fire recovers via the Postgres delivery-cursor sweep, exactly as today's in-process bus. Ack-on-fire is rejected (AckWait=30s << an agent turn ⇒ healthy held messages redeliver mid-turn and DLQ-park after MaxDeliver=5, absent per-message InProgress heartbeats) | Active (Matt, 2026-09-05) | [delivery cutover §OQ-1](infra/runtime/compass-managed-delivery-cutover/design.md#oq-1-load-bearing-jetstream-ack-timing-for-held-delivers) |
| DL-335 | The migrated delivery consumer runs `onEventRef` (re-read + classify + hold/dispatch) DIRECTLY on the fabric callback goroutine under `c.mu` + the per-session gates (OQ-2), concurrent with the settle/start drain loop — NOT enqueued onto a single goroutine. The relaxed cross-channel ordering is one today's non-deterministic `select` never actually guaranteed; the cursor sweep remains the no-loss floor. Two Option-A obligations are invariants, not forks: `scanMissedMentions`'s held-check and `MarkMentionsRouted` run under one critical section (scan-vs-hold), and per-callback dispatch work stays bounded so the ack does not block past AckWait behind a long `sweepSession` | Active (Matt, 2026-09-05) | [delivery cutover §OQ-2](infra/runtime/compass-managed-delivery-cutover/design.md#oq-2-load-bearing-concurrency-model--callback-direct-vs-loop-enqueue) |
| DL-336 | The delivery consumer splits its DB role once each event carries an explicit `ref.Tenant` (OQ-3 part 1): the inherently cross-tenant background sweeps/drains keep `WithSystemRole` (BYPASSRLS), but per-event processing runs under `store.WithTenant(baseCtx, ref.Tenant)` for the `MessageByID` re-read and the whole `onMessagePosted` chain — and `heldEntry` gains a `tenant` field so the `fireHeld` re-read is tenant-scoped too. Fail-closed: a forged/corrupted ref whose row belongs to another tenant reads zero rows under RLS instead of cross-tenant-delivering under BYPASSRLS. Whole-loop system-role (ref.Tenant routing-only) rejected — forfeits the stamped ref's isolation dividend | Active (Matt, 2026-09-05) | [delivery cutover §OQ-3](infra/runtime/compass-managed-delivery-cutover/design.md#oq-3-load-bearing-rls-scope-split--lag-recovery-replacement) |
| DL-337 | Recovery after a publish-SIDE fabric failure (commit ok, publish fails — the loss mode the infallible bus never had) is triggered by a FABRIC-RECONNECT HOOK plus a minutes-scale PERIODIC FLOOR TICK, each running `sweepAllLive` + `scanMissedMentions` (OQ-3 part 2). This replaces the deleted `sub.Lagged()` bus-ring branch. The draft's original mapping was falsified by the design-critic red-team: `scanMissedMentions` routes only mentions/ask-answers (never plain delivers) and NATS auto-reconnect (`MaxReconnects(-1)`) keeps the ConsumeContext alive so a "re-subscribe" trigger never fires across an outage — leaving a publish-failed plain deliver to an always-live agent silently undelivered until its next session restart. `sweepAllLive` (not just the scan) is the load-bearing plain-deliver recovery path; publisher-side bounded retry MAY be added but is not sufficient alone | Active (Matt, 2026-09-05) | [delivery cutover §OQ-3 part 2](infra/runtime/compass-managed-delivery-cutover/design.md#oq-3-load-bearing-rls-scope-split--lag-recovery-replacement) |
| DL-331 | SINGLE-INSTANCE is a transitional deployment constraint for the delivery cutover (OQ-4, design-critic HIGH): the cutover moves the delivery TRIGGER cross-instance (durable queue-group single-claim) but the dispatch plane it feeds — `SessionForAccount`/`LiveAgentSessions`, the held registry, settle edges, per-session gates — is instance-local hub RAM, so a two-instance deploy breaks hold/settle for a message claimed by the instance not hosting the author's session (immediate fire from partial mid-turn blocks, late-block mention loss, undelivered settled suffix). A single Server is assumed until the parent record's durable session bindings land (parent T4, sequenced after this cutover); this record's two-instance integration proof is scoped to fabric TRANSPORT claim semantics only, not multi-instance delivery correctness | Active (Matt, 2026-09-05) | [delivery cutover §OQ-4](infra/runtime/compass-managed-delivery-cutover/design.md#oq-4-load-bearing-cross-instance-session-locality-single-instance-transitional-constraint) |
| DL-332 | Publishing `message_posted` on BOTH the in-process `events.Bus` (client gRPC stream + presence, until the client edge migrates) and the fabric (delivery) during the phased, multi-step migration is NOT a violation of DL-313's "one eventing substrate — NATS only": that constraint bans a second SWAPPABLE `EventFabric` implementation (an in-process channel impl of the seam), not the pre-existing bus coexisting during the phased cutover. The two publishes serve disjoint consumer sets (no consumer reads both), so no double-handling occurs; the transitional shape ends when the client edge migrates and the bus retires. Interpretation frozen here (per the red-team) so the bus-retirement inherits it explicitly | Active (Matt, 2026-09-05) | [delivery cutover §double-publish](infra/runtime/compass-managed-delivery-cutover/design.md#why-the-double-publish-is-not-a-global-constraint-violation) |
| DL-333 | The OQ-3-part-2 recovery trigger's fabric-reconnect hook (DL-330) needs a seam the frozen 3-method `EventFabric` (DL-316/DL-313 — `Publish`/`Subscribe`/`SubscribeKind`) does not expose; the fabric's own `ReconnectHandler` is log-only, set once at `New()`, and REPLACED (not chained) via `Config.Options` (a replacing caller loses the fabric's outage diagnostics). Ruling: ADD an `EventFabric` method `OnReconnect(fn func()) (Unsubscribe, error)` on the interface and `*Fabric`, chained onto the fabric's existing reconnect handler so its log survives, reached by the delivery consumer through the interface value it holds. Grows the seam to four methods (accepted, consistent with the `SubscribeKind` precedent). Assembly-side wiring through `Config.Options` rejected — splits the trigger across packages and re-implements the fabric's diagnostics | Active (Matt, 2026-09-05) | [delivery cutover §OQ-3](infra/runtime/compass-managed-delivery-cutover/design.md#oq-3-load-bearing-rls-scope-split--lag-recovery-replacement) |
| DL-333 | The OQ-3-part-2 recovery trigger's fabric-reconnect hook (DL-337) needs a seam the frozen 3-method `EventFabric` (DL-316/DL-313 — `Publish`/`Subscribe`/`SubscribeKind`) does not expose; the fabric's own `ReconnectHandler` is log-only, set once at `New()`, and REPLACED (not chained) via `Config.Options` (a replacing caller loses the fabric's outage diagnostics). Ruling: ADD an `EventFabric` method `OnReconnect(fn func()) (Unsubscribe, error)` on the interface and `*Fabric`, chained onto the fabric's existing reconnect handler so its log survives, reached by the delivery consumer through the interface value it holds. Grows the seam to four methods (accepted, consistent with the `SubscribeKind` precedent). Assembly-side wiring through `Config.Options` rejected — splits the trigger across packages and re-implements the fabric's diagnostics | Active (Matt, 2026-09-05) | [delivery cutover §OQ-3](infra/runtime/compass-managed-delivery-cutover/design.md#oq-3-load-bearing-rls-scope-split--lag-recovery-replacement) |

## Storage

Expand Down
2 changes: 1 addition & 1 deletion docs/designs/agent/compass-agent-container-runtime.md
Original file line number Diff line number Diff line change
Expand Up @@ -711,7 +711,7 @@ repo manifest, no grants table). All types redact like `Credentials`
generic channels; `SecretGH` rows carry `Host string` (default
`github.com`) so T5 routes them to `GHCredentials.SetupScript`
(Decision 3's gh placement), never the generic file path.
- `type Resolver interface { Resolve(ctx context.Context, reason string) ([]ResolvedSecret, error); Set(ctx context.Context, name, value string) error; Delete(ctx context.Context, name string) error }`
- `type Resolver interface { Resolve(ctx context.Context, reason string) ([]ResolvedSecret, error); Set(ctx context.Context, name, value, reason string) error; Delete(ctx context.Context, name string) error }`
— `Resolve` resolves the **whole registry** (inject-all; a
`names []string` parameter returns with the future grants seam);
`Set`/`Delete` are the provider **write** path T7's
Expand Down
Loading
Loading