From 0256afe64ca547f6bc7736815002938cf0cf40cc Mon Sep 17 00:00:00 2001 From: mintaka Date: Sun, 6 Sep 2026 01:12:58 -0400 Subject: [PATCH 1/2] fix(design): renumber colliding ledger ids DL-327..330 to DL-334..337 (RIG-3107) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A concurrent-merge window landed five records claiming overlapping ledger ids. This record (#905, RIG-3107 delivery cutover) allocated DL-327..333 off a main whose tail was DL-326, but four earlier-dated records merged first and already held the low four: - DL-327 — SubjectService token principal (#868, compass-server) - DL-328 — gateway-credentials at-rest encryption (#859, compass-server) - DL-329 — compass-stack supervision (#872, platform) - DL-330 — apple-container macOS runner (platform) The double-booked ids fail design-ledger-gate:check on main ("duplicate ledger id" x4), which blocks every lane's pre-push moon ci until it lands. Renumber this record's four rows to the next free ids (max on main is DL-333): DL-327 to DL-334 (OQ-1 held-deliver ack timing), DL-328 to DL-335 (OQ-2 callback-direct dispatch), DL-329 to DL-336 (OQ-3 part 1 DB-role split), DL-330 to DL-337 (OQ-3 part 2 publish-side recovery trigger). DL-331/332/333 are uniquely this record's already and stay put. The earlier claimants' ids do NOT move, per the DL-319/320 precedent (fab4a67f, #851): the later claimant renumbers. Those ids are cited by merged frozen records other lanes implement from (DL-327 in 6 files, DL-328 in 4), and both the stack-supervision and apple-container records carry prose deriving their own id from main's then-current tail ("main's ledger tail was DL-328 ... so DL-329 is the first free number") — reasoning that stays correct precisely because only this record's rows move. DL-333's Decision cell cites the reconnect hook by id, so its internal DL-330 reference is retargeted to DL-337 in the same pass; the ledger's decision prose is otherwise byte-identical (verified: 304 rows before and after, diff empty once ids are masked). Annotates the frozen record header with the mapping and restates its Ledger-impact line as DL-331..337 (annotate-don't-rewrite house rule); the in-body citations read as point-in-time, with the ledger rows canonical. Standalone on purpose: main is red for every lane until this lands, so it is not bundled into the T3 implementation stack. Co-authored-by: Matt Wilkinson --- docs/designs/DECISIONS.md | 10 +++++----- .../compass-managed-delivery-cutover/design.md | 12 +++++++++++- 2 files changed, 16 insertions(+), 6 deletions(-) diff --git a/docs/designs/DECISIONS.md b/docs/designs/DECISIONS.md index 408c520d..cdfe672e 100644 --- a/docs/designs/DECISIONS.md +++ b/docs/designs/DECISIONS.md @@ -72,13 +72,13 @@ check enforces the mechanical half. Full rationale: | DL-018 | The frozen `RunnerService` transport recommendation #2 is socket-only, superseding the earlier off-stdio carrier clause | Active (Matt, 2026-07-22) | [agent comms tools §The frozen transport](agent/compass-agent-comms-tools/design.md#the-frozen-transport-this-rides-was-the-keystone-fork-now-decided) | | DL-313 | NATS is the single eventing substrate — run as a standalone stack service (alongside Postgres and the OTel collector) in every deployment, reached over `nats://`, with no embedded/in-process mode and no transport phase; single-node or clustered NATS is selected by connection string, never by application code. JetStream is the durable comms-delivery transport; core NATS carries routing/binding invalidation; queue groups partition delivery work. Connect stays the synchronous RPC edge; the agent↔Runner hop stays vsock (RIG-2394); no LISTEN/NOTIFY phase. Supersedes DL-014 and DL-021 (RIG-2861 OQ-1) | Active (Matt, 2026-08-31) | [multi-tenancy & NATS substrate](infra/runtime/compass-managed-multitenancy/design.md#q3--the-eventing-substrate-one-nats-eventfabric-a-standalone-stack-service-jetstream-as-the-delivery-transport) | | DL-316 | Server↔Runner transport is TWO-PLANE (amends DL-013's Runner↔Server clause, RIG-2861 OQ-5 Variant B): async command-push + Runner event fan-in ride NATS (per-Runner command subjects + queue-group fan-in — `Sessions`/`PublishEvents` reshaped to pub/sub for the non-sticky-wake fabric); the typed request/reply legs (enrollment, unary `Relay*Call`s/`CommitConversationFrame`/`FetchSecrets`, bulk `FetchAgentConfig`) stay on the reduced Connect/gRPC edge (deadline propagation, typed proto errors, generated stubs); the per-Runner provisioned token is retained as the NATS-credentials seed via auth-callout. Client↔Server stays Connect; Runner↔Agent stays vsock. Supersedes DL-013 | Active (Matt, 2026-08-31) | [multi-tenancy & NATS substrate](infra/runtime/compass-managed-multitenancy/design.md#resolved-decisions-freeze--matt-2026-08-31) | -| DL-327 | The delivery work-queue consumer's JetStream ack for a HELD deliver (agent-authored message held until its author's session settles) is ACK-ON-RECEIVE (OQ-1): the fabric callback acks as soon as the message is classified and held/dispatched, not at fire — the held registry stays in-RAM and a crash between hold and fire recovers via the Postgres delivery-cursor sweep, exactly as today's in-process bus. Ack-on-fire is rejected (AckWait=30s << an agent turn ⇒ healthy held messages redeliver mid-turn and DLQ-park after MaxDeliver=5, absent per-message InProgress heartbeats) | Active (Matt, 2026-09-05) | [delivery cutover §OQ-1](infra/runtime/compass-managed-delivery-cutover/design.md#oq-1-load-bearing-jetstream-ack-timing-for-held-delivers) | -| DL-328 | The migrated delivery consumer runs `onEventRef` (re-read + classify + hold/dispatch) DIRECTLY on the fabric callback goroutine under `c.mu` + the per-session gates (OQ-2), concurrent with the settle/start drain loop — NOT enqueued onto a single goroutine. The relaxed cross-channel ordering is one today's non-deterministic `select` never actually guaranteed; the cursor sweep remains the no-loss floor. Two Option-A obligations are invariants, not forks: `scanMissedMentions`'s held-check and `MarkMentionsRouted` run under one critical section (scan-vs-hold), and per-callback dispatch work stays bounded so the ack does not block past AckWait behind a long `sweepSession` | Active (Matt, 2026-09-05) | [delivery cutover §OQ-2](infra/runtime/compass-managed-delivery-cutover/design.md#oq-2-load-bearing-concurrency-model--callback-direct-vs-loop-enqueue) | -| DL-329 | The delivery consumer splits its DB role once each event carries an explicit `ref.Tenant` (OQ-3 part 1): the inherently cross-tenant background sweeps/drains keep `WithSystemRole` (BYPASSRLS), but per-event processing runs under `store.WithTenant(baseCtx, ref.Tenant)` for the `MessageByID` re-read and the whole `onMessagePosted` chain — and `heldEntry` gains a `tenant` field so the `fireHeld` re-read is tenant-scoped too. Fail-closed: a forged/corrupted ref whose row belongs to another tenant reads zero rows under RLS instead of cross-tenant-delivering under BYPASSRLS. Whole-loop system-role (ref.Tenant routing-only) rejected — forfeits the stamped ref's isolation dividend | Active (Matt, 2026-09-05) | [delivery cutover §OQ-3](infra/runtime/compass-managed-delivery-cutover/design.md#oq-3-load-bearing-rls-scope-split--lag-recovery-replacement) | -| DL-330 | Recovery after a publish-SIDE fabric failure (commit ok, publish fails — the loss mode the infallible bus never had) is triggered by a FABRIC-RECONNECT HOOK plus a minutes-scale PERIODIC FLOOR TICK, each running `sweepAllLive` + `scanMissedMentions` (OQ-3 part 2). This replaces the deleted `sub.Lagged()` bus-ring branch. The draft's original mapping was falsified by the design-critic red-team: `scanMissedMentions` routes only mentions/ask-answers (never plain delivers) and NATS auto-reconnect (`MaxReconnects(-1)`) keeps the ConsumeContext alive so a "re-subscribe" trigger never fires across an outage — leaving a publish-failed plain deliver to an always-live agent silently undelivered until its next session restart. `sweepAllLive` (not just the scan) is the load-bearing plain-deliver recovery path; publisher-side bounded retry MAY be added but is not sufficient alone | Active (Matt, 2026-09-05) | [delivery cutover §OQ-3 part 2](infra/runtime/compass-managed-delivery-cutover/design.md#oq-3-load-bearing-rls-scope-split--lag-recovery-replacement) | +| DL-334 | The delivery work-queue consumer's JetStream ack for a HELD deliver (agent-authored message held until its author's session settles) is ACK-ON-RECEIVE (OQ-1): the fabric callback acks as soon as the message is classified and held/dispatched, not at fire — the held registry stays in-RAM and a crash between hold and fire recovers via the Postgres delivery-cursor sweep, exactly as today's in-process bus. Ack-on-fire is rejected (AckWait=30s << an agent turn ⇒ healthy held messages redeliver mid-turn and DLQ-park after MaxDeliver=5, absent per-message InProgress heartbeats) | Active (Matt, 2026-09-05) | [delivery cutover §OQ-1](infra/runtime/compass-managed-delivery-cutover/design.md#oq-1-load-bearing-jetstream-ack-timing-for-held-delivers) | +| DL-335 | The migrated delivery consumer runs `onEventRef` (re-read + classify + hold/dispatch) DIRECTLY on the fabric callback goroutine under `c.mu` + the per-session gates (OQ-2), concurrent with the settle/start drain loop — NOT enqueued onto a single goroutine. The relaxed cross-channel ordering is one today's non-deterministic `select` never actually guaranteed; the cursor sweep remains the no-loss floor. Two Option-A obligations are invariants, not forks: `scanMissedMentions`'s held-check and `MarkMentionsRouted` run under one critical section (scan-vs-hold), and per-callback dispatch work stays bounded so the ack does not block past AckWait behind a long `sweepSession` | Active (Matt, 2026-09-05) | [delivery cutover §OQ-2](infra/runtime/compass-managed-delivery-cutover/design.md#oq-2-load-bearing-concurrency-model--callback-direct-vs-loop-enqueue) | +| DL-336 | The delivery consumer splits its DB role once each event carries an explicit `ref.Tenant` (OQ-3 part 1): the inherently cross-tenant background sweeps/drains keep `WithSystemRole` (BYPASSRLS), but per-event processing runs under `store.WithTenant(baseCtx, ref.Tenant)` for the `MessageByID` re-read and the whole `onMessagePosted` chain — and `heldEntry` gains a `tenant` field so the `fireHeld` re-read is tenant-scoped too. Fail-closed: a forged/corrupted ref whose row belongs to another tenant reads zero rows under RLS instead of cross-tenant-delivering under BYPASSRLS. Whole-loop system-role (ref.Tenant routing-only) rejected — forfeits the stamped ref's isolation dividend | Active (Matt, 2026-09-05) | [delivery cutover §OQ-3](infra/runtime/compass-managed-delivery-cutover/design.md#oq-3-load-bearing-rls-scope-split--lag-recovery-replacement) | +| DL-337 | Recovery after a publish-SIDE fabric failure (commit ok, publish fails — the loss mode the infallible bus never had) is triggered by a FABRIC-RECONNECT HOOK plus a minutes-scale PERIODIC FLOOR TICK, each running `sweepAllLive` + `scanMissedMentions` (OQ-3 part 2). This replaces the deleted `sub.Lagged()` bus-ring branch. The draft's original mapping was falsified by the design-critic red-team: `scanMissedMentions` routes only mentions/ask-answers (never plain delivers) and NATS auto-reconnect (`MaxReconnects(-1)`) keeps the ConsumeContext alive so a "re-subscribe" trigger never fires across an outage — leaving a publish-failed plain deliver to an always-live agent silently undelivered until its next session restart. `sweepAllLive` (not just the scan) is the load-bearing plain-deliver recovery path; publisher-side bounded retry MAY be added but is not sufficient alone | Active (Matt, 2026-09-05) | [delivery cutover §OQ-3 part 2](infra/runtime/compass-managed-delivery-cutover/design.md#oq-3-load-bearing-rls-scope-split--lag-recovery-replacement) | | DL-331 | SINGLE-INSTANCE is a transitional deployment constraint for the delivery cutover (OQ-4, design-critic HIGH): the cutover moves the delivery TRIGGER cross-instance (durable queue-group single-claim) but the dispatch plane it feeds — `SessionForAccount`/`LiveAgentSessions`, the held registry, settle edges, per-session gates — is instance-local hub RAM, so a two-instance deploy breaks hold/settle for a message claimed by the instance not hosting the author's session (immediate fire from partial mid-turn blocks, late-block mention loss, undelivered settled suffix). A single Server is assumed until the parent record's durable session bindings land (parent T4, sequenced after this cutover); this record's two-instance integration proof is scoped to fabric TRANSPORT claim semantics only, not multi-instance delivery correctness | Active (Matt, 2026-09-05) | [delivery cutover §OQ-4](infra/runtime/compass-managed-delivery-cutover/design.md#oq-4-load-bearing-cross-instance-session-locality-single-instance-transitional-constraint) | | DL-332 | Publishing `message_posted` on BOTH the in-process `events.Bus` (client gRPC stream + presence, until the client edge migrates) and the fabric (delivery) during the phased, multi-step migration is NOT a violation of DL-313's "one eventing substrate — NATS only": that constraint bans a second SWAPPABLE `EventFabric` implementation (an in-process channel impl of the seam), not the pre-existing bus coexisting during the phased cutover. The two publishes serve disjoint consumer sets (no consumer reads both), so no double-handling occurs; the transitional shape ends when the client edge migrates and the bus retires. Interpretation frozen here (per the red-team) so the bus-retirement inherits it explicitly | Active (Matt, 2026-09-05) | [delivery cutover §double-publish](infra/runtime/compass-managed-delivery-cutover/design.md#why-the-double-publish-is-not-a-global-constraint-violation) | -| DL-333 | The OQ-3-part-2 recovery trigger's fabric-reconnect hook (DL-330) needs a seam the frozen 3-method `EventFabric` (DL-316/DL-313 — `Publish`/`Subscribe`/`SubscribeKind`) does not expose; the fabric's own `ReconnectHandler` is log-only, set once at `New()`, and REPLACED (not chained) via `Config.Options` (a replacing caller loses the fabric's outage diagnostics). Ruling: ADD an `EventFabric` method `OnReconnect(fn func()) (Unsubscribe, error)` on the interface and `*Fabric`, chained onto the fabric's existing reconnect handler so its log survives, reached by the delivery consumer through the interface value it holds. Grows the seam to four methods (accepted, consistent with the `SubscribeKind` precedent). Assembly-side wiring through `Config.Options` rejected — splits the trigger across packages and re-implements the fabric's diagnostics | Active (Matt, 2026-09-05) | [delivery cutover §OQ-3](infra/runtime/compass-managed-delivery-cutover/design.md#oq-3-load-bearing-rls-scope-split--lag-recovery-replacement) | +| DL-333 | The OQ-3-part-2 recovery trigger's fabric-reconnect hook (DL-337) needs a seam the frozen 3-method `EventFabric` (DL-316/DL-313 — `Publish`/`Subscribe`/`SubscribeKind`) does not expose; the fabric's own `ReconnectHandler` is log-only, set once at `New()`, and REPLACED (not chained) via `Config.Options` (a replacing caller loses the fabric's outage diagnostics). Ruling: ADD an `EventFabric` method `OnReconnect(fn func()) (Unsubscribe, error)` on the interface and `*Fabric`, chained onto the fabric's existing reconnect handler so its log survives, reached by the delivery consumer through the interface value it holds. Grows the seam to four methods (accepted, consistent with the `SubscribeKind` precedent). Assembly-side wiring through `Config.Options` rejected — splits the trigger across packages and re-implements the fabric's diagnostics | Active (Matt, 2026-09-05) | [delivery cutover §OQ-3](infra/runtime/compass-managed-delivery-cutover/design.md#oq-3-load-bearing-rls-scope-split--lag-recovery-replacement) | ## Storage diff --git a/docs/designs/infra/runtime/compass-managed-delivery-cutover/design.md b/docs/designs/infra/runtime/compass-managed-delivery-cutover/design.md index 76c33008..218cee27 100644 --- a/docs/designs/infra/runtime/compass-managed-delivery-cutover/design.md +++ b/docs/designs/infra/runtime/compass-managed-delivery-cutover/design.md @@ -3,7 +3,17 @@ Status: Active Ratified: OQ-1..OQ-4 decided by Matt (2026-09-05, see Resolved decisions); frozen on merge Parent: `docs/designs/infra/runtime/compass-managed-multitenancy/design.md` (frozen), T3 -Ledger-impact: appends DL-327..333 for the OQ-1/OQ-2/OQ-3/OQ-4 rulings, the reconnect-seam shape, and the double-publish interpretation (design-ledger-gate) +Ledger-impact: appends DL-331..337 for the OQ-1/OQ-2/OQ-3/OQ-4 rulings, the reconnect-seam shape, and the double-publish interpretation (design-ledger-gate) + +> Ledger-id note (post-freeze, annotate-don't-rewrite): a concurrent-merge +> window allocated four of this record's rows the same DL ids as +> earlier-merged records (the token-subject, gateway-credentials, +> stack-supervision, and apple-container rows). They were renumbered on +> `DECISIONS.md` to keep ids unique: **DL-327 → DL-334** (OQ-1 held-deliver ack +> timing), **DL-328 → DL-335** (OQ-2 callback-direct dispatch), **DL-329 → +> DL-336** (OQ-3 part 1 DB-role split), **DL-330 → DL-337** (OQ-3 part 2 +> publish-side recovery trigger). DL-331/DL-332/DL-333 are unchanged. The +> in-body citations below read as point-in-time; the ledger rows are canonical. ## Problem / Intent From 1e0f76c61ac05e0d3f0c1072dc8923fab032b60e Mon Sep 17 00:00:00 2001 From: mintaka Date: Sun, 6 Sep 2026 01:38:39 -0400 Subject: [PATCH 2/2] docs(design): state the DL-333 citation retarget in the ledger-id note (RIG-3107) --- .../compass-managed-delivery-cutover/design.md | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/docs/designs/infra/runtime/compass-managed-delivery-cutover/design.md b/docs/designs/infra/runtime/compass-managed-delivery-cutover/design.md index 218cee27..e1c0764c 100644 --- a/docs/designs/infra/runtime/compass-managed-delivery-cutover/design.md +++ b/docs/designs/infra/runtime/compass-managed-delivery-cutover/design.md @@ -12,8 +12,18 @@ Ledger-impact: appends DL-331..337 for the OQ-1/OQ-2/OQ-3/OQ-4 rulings, the reco > `DECISIONS.md` to keep ids unique: **DL-327 → DL-334** (OQ-1 held-deliver ack > timing), **DL-328 → DL-335** (OQ-2 callback-direct dispatch), **DL-329 → > DL-336** (OQ-3 part 1 DB-role split), **DL-330 → DL-337** (OQ-3 part 2 -> publish-side recovery trigger). DL-331/DL-332/DL-333 are unchanged. The -> in-body citations below read as point-in-time; the ledger rows are canonical. +> publish-side recovery trigger). DL-331/DL-332/DL-333 keep their ids; DL-333's +> Decision cell has its internal reconnect-hook citation retargeted DL-330 → +> DL-337 in the same pass, so the cell keeps pointing at this record's own +> recovery-trigger row rather than at the apple-container row that now holds +> DL-330. That retarget is a deliberate, narrow exception to the ledger's +> "Decision — a one-line paraphrase, immutable after append" rule +> (`DECISIONS.md` §Conventions): the rule bans an in-place REWORD so a cell +> cannot silently drift as a truth surface, and here leaving the old id in +> place is exactly what would have made it silently wrong. An id-only fix that +> preserves the sentence is the minimal way to keep the cell true; it is not +> licence to reword a landed Decision. The in-body citations below read as +> point-in-time; the ledger rows are canonical. ## Problem / Intent