These modules are packaged as node-install.pyz, which Web serves at /node-install/. They install a sandbox node on a Linux host that runs sandboxes for one Core; they never install Core. The bootstrap is one file that runs with the host's python3.
| Module | Role |
|---|---|
node_install.py, node_spec.py, node_generations.py |
Node installer and generation helper. node_spec.py is generated by go run ./services/core/cmd/specification-contract -write |
distribution.py |
Verified artifacts, private temporary files, release metadata and Docker image identity |
install_display.py, node_output.py |
Terminal output |
provider_assets.py |
Node provider artifact names, generated by go run ./services/core/cmd/provider-artifacts |
node_payload.py |
Publish a release's node files without replacing bytes already installed, then activate a release |
The node installer runs as root and prepares the host for one node per installation.
- It creates or adopts the
oac-nodesystem user, adds it to thedockerorkvmgroup (no other group), and installs one root-owned system service per installation that runs the node program asUser=oac-node. Nodes on a host share that account, so a host serves one Core. - Docker group membership makes that user, and so the node, root-equivalent on the host; that is inherent to Docker sandboxes. microsandbox needs only
kvm, user KVM access and the Linux runtime libraries. - Node configuration and identity live under
~/.oac/nodes/<installation-id>/in the node account's home (/var/lib/oac-node); microsandbox uses a separate short private Runtime home. - The node service owns its provider processes outside the Core container.
KillMode=processkeeps resident microVM and helper processes across a service restart. The service restarts after failures with no start limit, so a node outlasts a Core outage, and stops restarting when the node program exits 78 because Core answered 401 to its credential (a removed node). - It never installs Docker, KVM or packages and never changes device permissions. It refuses SELinux-enforcing hosts and changes nothing when a check fails. The enrollment token comes only on standard input, never in arguments or the environment.
- Files the service account owns are read, written and deleted only with that account's credentials. The one exception is root removing the account's home after
userdel, when no process can still run as that account. That work runs in a child that starts its own session with/dev/nullas input, joins a new session keyring and dies with its parent; root shows its output only as plain text (terminal controls become?). SIGINT, SIGHUP and SIGTERM stop that child and what it started. - Root never runs a file the service account can write, opens a URL it wrote, or follows a link in its home. Capture the trusted bootstrap bytes before dropping to the service account and pass them through the fork; the service account writes its own retained generation helper. Never open the caller's private download directory to it or let root write into service-owned state.
- The generated bootstrap passes only the six standard HTTP/HTTPS proxy and bypass variables through sudo and gives both spellings the lowercase value when present, even if empty, so curl, urllib and the Go registration command follow the same rules. The installation child keeps just those names beside its fixed environment. Proxy values stay out of arguments, unit files and diagnostics; never use broad sudo environment inheritance. The installation child uses them for downloads. The node service keeps the settings of the latest run in
/etc/oac-node/<installation-id>.env(root, mode 0600, read throughEnvironmentFile=; removed when a run has no proxy and by uninstall), with loopback added toNO_PROXY. --uninstallremoves a node only after Core rejects its credential, except for a node that never registered and with--force, which Web offers when the old Core address no longer responds. It never touches sandboxes, volumes or Docker images (the Runtime image stays, with the command that removes it). Once Core confirmed the removal it also deletes the node's microsandbox store, which--forcenever does (it prints the command instead). It deletes the account only when the installer created it and no node remains, and otherwise removes only the groups it added.- Refuse resources of an older product name for the same installation ID; never adopt them or remove another installation's resources.
The generated command uses ~/.oac/node-bootstrap in the invoking account's home. Under download.lock, it discards an unfinished download.partial, downloads and verifies the bootstrap, then publishes <sha256>.pyz. The lock is released before invoking sudo; verified files stay available to running installers. Node file writes share distribution.temporary_file: under the caller's installation lock, remove the previous temporary file, write privately, and publish atomically. The same lifecycle covers payloads, extraction, helpers, settings and enrollment tokens. Registration tokens are removed even when a rerun finds the node already registered.
Core records a node when its registration completes, and the installer marks that with registered.json. Until then, an earlier attempt's configuration, identity and cached files that no longer match are disposable: the next command replaces them and keeps the files that match. A registered node's files are never replaced; a rerun that finds a different node program stops and names the remove, uninstall and add steps.
Before downloading, the installer refuses when the node account's disk lacks the space the missing files and the Runtime import need. It deletes the Runtime archive and its unpacked copy once the image is imported.
Configuration files are published only after their complete contents are written; a retry preserves an existing matching file and refuses conflicting contents. Generation leases are durable locks, not disposable temporary files: interruption between creating a lease and recording its inode identity requires operator inspection. A retry refuses that unrecorded lease instead of replacing an inode that a Runtime helper could still hold.
The distribution manifest is the one download contract for nodes: flat versioned file names, and the compressed and unpacked size and SHA-256 of the Runtime.
- A Compose installation keeps only the node metadata from its release archive; Core's image never acquires execution-only payloads.
- A node obtains bootstrap metadata from the console that generated its command. Web serves artifacts it has locally and redirects missing declared execution artifacts to the versioned HTTPS release base in the verified manifest. Web never downloads or caches those bytes.
- Only artifact requests may follow HTTPS redirects, and only without credentials or cookies. Metadata and enrollment requests stay on the configured console. The console publishes only fixed non-secret files and declared artifact names.
- Download into private temporary files, verify size and SHA-256 before an atomic rename, resume a leftover partial artifact download with an HTTP range (a server that cannot resume, or bytes that fail verification, restart the file) and clear leftover extracted files before retrying, and reuse only verified cache entries or exact image identities. Never select a release other than the pinned one.
- Release downloads are anonymous. Never add repository credentials to installed node or Runtime configuration.
- Manual builds use the
build-<full SHA>release tag and tag builds thev*tag. The manifest's download base must match the release tag; artifact file names and source provenance keep the full source SHA.
A node stores the deployment generations it prepares under its installation state and keeps, collects and repairs them on its own. The Sandbox node protocol defines what crosses the wire: generation observations, the retention exchange and the readiness classes.
A Core drop grant is necessary but not sufficient for collection. Queued and running provider calls, preparation, the local target and the serving pin all keep references; collection rechecks them and refuses on an already canceled connection. A canceled provider caller does not prove that its native helper stopped: the node counts a helper until its actual Wait returns.
Every generation owns a permanent private lease file, state/node/generations/<generation>.lease. Before starting a native helper, the node takes a shared flock on it, validates the durable lease identity under that lock, and requires the published final provider configuration and no preparing, collecting or dropped journal. The helper inherits the descriptor; the node closes its own copy only after Wait and never unlocks the shared open-file description, so a canceled caller or a node exit does not release a live helper's reference. Native helpers set the descriptor close-on-exec before calling the SDK, so VM and daemon descendants do not inherit it.
Collection takes the exclusive nonblocking flock before it inspects references, removes shared images or release files, or publishes the dropped marker, and holds it through those changes. Lease files belong to stable node state and are never removed or replaced during collection; symlinks, multiply linked files, foreign ownership, unsafe permissions and replaced lock paths are refused. Before the first helper can start, the installer creates the lease exclusively and fsyncs it, then atomically persists and fsyncs a private .lease-identity record and its directory. The record binds installation, generation, specification digest, device and inode. The Python collector and the Go helper opener validate the same record on every open, including after a restart; neither adopts a missing identity, replaces its inode or erases it after collection. An installation interrupted before the identity is durable refuses re-adoption and is kept for inspection. A removed identity or a replaced lease refuses even when its current metadata agree.
A dropped generation can never be prepared or used again. A helper's exit is evidence about local files only, not proof that a remote mutation or an uncertain provider receipt has been released; Core's durable allocation and placement retention stays independent.
The host program release and Core's selected Runtime release are independent. A fresh node gets its executable and private preparer from the console's current release, and reads the exact Runtime source, image identities and native runtime and firmware digests from its authenticated configuration. When that Runtime is older, the console still serves its immutable releases/<source>/ manifest, checksums and allowlisted artifacts: the Runtime helper, firmware, seccomp profile and image bytes come from the selected release, and artifact URLs stay pinned to their verified manifest even if the console's current release changes during a download.
A missing retained release refuses the installation rather than substituting the current Runtime, and so does a local bundle that holds only a different Runtime. These refusals happen before the installer writes the node identity, imports the Runtime, registers the node or starts its service.
A published console release keeps its metadata and artifact bytes. Publishing it again first validates all metadata and every existing declared artifact, then may atomically add only missing, checksum-matched declared artifacts; any conflict prevents every addition, and nothing is overwritten.
Missing Runtime bytes never move a pinned placement to the current Runtime. The node keeps the original generation and specification digest as unready and asks Core's authenticated configuration route for that exact generation before recovery. Missing seccomp bytes may leave an unready provider placeholder; missing image or native artifacts found by a provider probe queue a repair without advertising readiness.
Preparation and repair are serialized. Target and serving generations come first, with bounded progress on the others. Each attempt has a 30-minute deadline, and failures back off for 1, 2, 5 and 10 minutes, then at most 30. Nothing is prepared before the connection has delivered deployment facts. Repair keeps existing configurations and paths, verifies the selected release and every existing sibling checksum, and downloads only missing immutable files. Conflicting bytes or a different retained specification refuse repair, and a missing provider configuration without an exact preparation plan is refused rather than reconstructed.
Repair takes the same exclusive generation lease and installation lock as collection, so a live helper or a running collector keeps ownership and repair retries later. After the bytes are restored, the node still runs the provider's readiness probe; file presence and executable capability never establish readiness.
New preparation writes two distinct records. Before downloads, .preparing holds the installation, generation and specification identity, the private provider paths and import_started: false; it is a recovery and collection plan, not a provider. Before the importer runs, the plan records import_started: true. Python retention discovery and Go restart recovery both recognize pending-only plans but never build, probe or acquire a provider from them, and recovery or collection still needs authorization on the current connection.
Only a successful preparation publishes the final .json provider configuration, which is write-once. Docker records the immutable local image ID its resolver returns; either the builder's config identity or the manifest identity can be valid for the same specification. Publication is durable before the preparation journal is cleared. An interruption between the two revalidates the same plan and final identity; plan, specification or path drift refuses. A canceled or failed import stays visible to Core's retention exchange without becoming a serving generation.
Before any native or release deletion, the node persists a private collection journal bound to its installation, generation and specification digest. After a restart an unfinished journal is only a retention-exchange candidate: it cannot prepare, probe, acquire or advertise that generation, and a fresh correlated Core drop grant is needed to resume. Native completion is persisted before release files are removed, so a retry can finish a partly removed release without running an already removed helper. The digest-bound dropped marker follows durable file cleanup and prevents re-adoption; the small configuration and ownership journals stay as local identity records.
For the installation-private microsandbox store, a successful, complete native image inventory distinguishes absence from a CLI failure; a failed query, malformed inventory, native in-use refusal or unknown ownership keeps the bytes. Shared microsandbox images and private releases stay until their last local reference. Docker images belong to the host's shared daemon: automatic collection never removes or prunes them, and only the host administrator can remove them after confirming that no installation on the host needs them.
A fresh installation also records the verified checksums of its Runtime files separately from the host program. Collecting the original generation removes only the exact private Runtime helper, executable, firmware, seccomp and import-cache files that no retained configuration references. Every remaining file is checked before the first deletion; an unknown hash, changed bytes, links or missing ownership metadata refuse cleanup. The node executable, preparer, identity, base provider configuration and manifests stay, so a restarted node can still read its enrolled identity and construct a newer retained provider. Shared native paths are compared across all retained configurations before removal.
An interrupted download repairs only missing bytes at the original paths. When collection comes before any import attempt, the preparation journal proves that the generation has no imported native image. A generation whose native executable is missing and whose import may have started stays retained: missing files never prove native absence, and an empty native inventory never erases receipt or store history.
The manifest's images records each exported image's config digest, and image_manifest_digests its OCI manifest or index digest. Derive and verify both from the same archive, including its referenced config and layer bytes, and require the build host's selected image ID to match one of them.
Docker's classic image store identifies images by config digest, and its containerd store by the OCI descriptor. The build therefore takes the digest the local store resolves from BuildKit's build metadata, never the --iidfile config digest alone, and disables provenance attestations so each image and archive holds one platform manifest in both stores.
The node installer confirms Linux amd64 and the returned immutable local ID, and service and provider configuration and Runtime launches use that ID. Tags never replace identity verification. The microsandbox runtime_ref is independent of Docker's local store identity.