docs(platform): record T-1 spike findings on real hardware (RIG-3238) - #923
Draft
rigel-mintaka wants to merge 1 commit into
Draft
docs(platform): record T-1 spike findings on real hardware (RIG-3238)#923rigel-mintaka wants to merge 1 commit into
rigel-mintaka wants to merge 1 commit into
Conversation
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
rigel-mintaka
changed the title
rigel-mintaka
force-pushed
the
compass-native/rig-3238-t1-spike-findings
branch
from
September 6, 2026 04:03
b30a827 to
ebbedd5
Compare
|
Compass engineering docs preview: https://compass-native-rig-3238-t1-s.compass-eng-docs.pages.dev Deployed from Changed pages: |
T-1 is the gating spike for the apple-container macOS runner: it proves the
load-bearing hardware unknowns before any backend code lands. Ran all six
probes (a-f) on the committed mac mini and landed the record's required
`spike-findings.md` deliverable with a per-OQ verdict and measured numbers.
Verdict: the apple-container DIRECTION is green and every probe passed, but
the ruled vsock TRANSPORT is red and returns to Matt. design.md:612-613 routes
exactly this outcome ("If the vsock leg is NOT reachable through the CLI, the
transport question ... returns to Matt with the finding") and OQ-11
(design.md:712-714) is his ruling, so the working substitute is recorded as a
recommendation awaiting his call, not a decision this record makes. T-2 holds
the transport leg until he rules; nothing else in T-2 is blocked by it.
- OQ-2/OQ-11 (transport, the load-bearing one) — GREEN by a DIFFERENT
mechanism than the record assumed. A raw AF_UNIX virtiofs bind-mount is RED
(`ls: Not supported`), confirming the compass-local-dev limitation holds
here. AF_VSOCK is present in the guest (CONFIG_VIRTIO_VSOCKETS=y, /dev/vsock)
and socket() succeeds, but the CLI exposes no host-side vsock attach point,
so the guestd unix->vsock forwarder cannot be ported as written. The CLI's
own `--publish-socket` carries it instead, proven with a full bidirectional
round-trip as uid 1000, including an identical host/container path (the
invariant postgres needs). It runs guest-listens/host-dials — the INVERSE of
today's gateway ordering.
- OQ-1 (uid mapping) — GREEN, simpler than podman: virtiofs translates
identity automatically (guest uid 1000 write lands host-owned as the
invoking user), so the `--userns=keep-id` port is unnecessary.
- OQ-3 (egress arming) — GREEN. Capabilities are silently dropped at any
non-zero uid (CapEff=0, CapAmb=0, even with --cap-add ALL), so podman's
uid-1000 arming identity cannot exist here; but arm-as-root-then-drop is
what `internal/runtime/egress.go` already documents, and it preserves the
whole property: default-deny blocks non-allowlisted 443/80, allowlist
carve-outs pass, and the dropped user can neither read nor flush the
ruleset. The OQ-2/OQ-3 coupling stays dissolved — the published socket
survives default-deny with no carve-out.
- OQ-4 (streaming exec) — GREEN: incremental stdout, stdin delivery, exit-code
passthrough, SIGTERM -> 143. Matches the ChildHandle kill/wait contract.
- OQ-5 (timings) — GREEN: ~720-950ms warm start, 2.6-2.9 MiB idle per
container VM, 4 vCPU / 1 GiB defaults. Substantiates the sub-second-boot and
cheap-per-agent-VM premises; these are T-5's flip-brief numbers.
- OQ-12 (runner-on-darwin) — GREEN: compass-runner cross-builds for
darwin/arm64 and runs natively on macOS 26, confirming the ruled host-side
topology. The darwin sun_path budget is measured at 34 on the host, matching
socket.go:138-139 exactly. The podman-host-capability-preflight leg is NOT
exercised by a --help run (help returns at main.go:84, preflight runs at
:106-112) and carries to T-4.
- OQ-1 is GREEN only in part: the ownership round-trip is proven, but the
/nix + $HOME leg is unprobed because ghcr.io/rigelbuild/compass-agent has no
linux/arm64 manifest and cannot run on this backend at all — itself a T-2/T-3
prerequisite (a multi-arch agent image).
Also drops the record's manual-.pkg provisioning assumption: nixpkgs-unstable
packages apple/container for aarch64-darwin at 1.1.0, above the >=1.0.0 floor,
and the host is nix-darwin managed from the fleet flake whose macRunnerOverlays
already has the pull-from-unstable pattern. The pinned nixpkgs-darwin has only
0.12.3, below the floor, so it must come from unstable. Filed RIG-3352 for the
owning repo-infra lane; nothing in T-2..T-5 blocks on it.
Two consequences the review surfaced and this record now carries: T-2's
postgres + collector port (design.md:444-450) relies on the host-socket-dir
bind-mount shape the spike proved RED (postgres_container.go:45-48), so it
needs a new socket plan; and arming as root is not a one-line identity swap,
since armEgress (agent.go:318-326) is shared with podman and takes no user
parameter, leaving T-2 a choice between widening that seam and the
inGuestEgressArmer marker design.md:455 declines.
Spec-impact: OQ-11's vsock transport ruling is materially changed; escalated
to Matt per design.md:612-613 rather than decided here.
Ledger-impact: none.
Refs RIG-3238
Co-authored-by: Matt Wilkinson <matt@rigel.build>
rigel-mintaka
force-pushed
the
compass-native/rig-3238-t1-spike-findings
branch
from
September 6, 2026 04:47
ebbedd5 to
6127172
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
T-1 is the gating spike for the apple-container macOS runner: it proves the
load-bearing hardware unknowns before any backend code lands. Ran all six
probes (a-f) on the committed mac mini and landed the record's required
spike-findings.mddeliverable with a per-OQ verdict and measured numbers.Verdict: the gate is GREEN and the ruled direction holds. Every probe passed,
with one real contract change and two simplifications.
mechanism than the record assumed. A raw AF_UNIX virtiofs bind-mount is RED
(
ls: Not supported), confirming the compass-local-dev limitation holdshere. AF_VSOCK is present in the guest (CONFIG_VIRTIO_VSOCKETS=y, /dev/vsock)
and socket() succeeds, but the CLI exposes no host-side vsock attach point,
so the guestd unix->vsock forwarder cannot be ported as written. The CLI's
own
--publish-socketcarries it instead, proven with a full bidirectionalround-trip as uid 1000. It runs guest-listens/host-dials — the INVERSE of
today's gateway ordering, which is the one real contract change for T-2.
identity automatically (guest uid 1000 write lands host-owned as the
invoking user), so the
--userns=keep-idport is unnecessary.non-zero uid (CapEff=0, CapAmb=0, even with --cap-add ALL), so podman's
uid-1000 arming identity cannot exist here; but arm-as-root-then-drop is
what
internal/runtime/egress.goalready documents, and it preserves thewhole property: default-deny blocks non-allowlisted 443/80, allowlist
carve-outs pass, and the dropped user can neither read nor flush the
ruleset. The OQ-2/OQ-3 coupling stays dissolved — the published socket
survives default-deny with no carve-out.
passthrough, SIGTERM -> 143. Matches the ChildHandle kill/wait contract.
container VM, 4 vCPU / 1 GiB defaults. Substantiates the sub-second-boot and
cheap-per-agent-VM premises; these are T-5's flip-brief numbers.
darwin/arm64 and runs natively on macOS 26, confirming the ruled host-side
topology.
Also drops the record's manual-.pkg provisioning assumption: nixpkgs-unstable
packages apple/container for aarch64-darwin at 1.1.0, above the >=1.0.0 floor,
and the host is nix-darwin managed from the fleet flake whose macRunnerOverlays
already has the pull-from-unstable pattern. The pinned nixpkgs-darwin has only
0.12.3, below the floor, so it must come from unstable. Filed RIG-3352 for the
owning repo-infra lane; nothing in T-2..T-5 blocks on it.
Spec-impact: none — records spike results; no frozen ruling reopened.
Ledger-impact: none.
Refs RIG-3238
Co-authored-by: Matt Wilkinson matt@rigel.build