[olcedar] d8 platform olcedar node add for static immutable nodes - #450
Draft
borg-z wants to merge 4 commits into
Draft
[olcedar] d8 platform olcedar node add for static immutable nodes#450borg-z wants to merge 4 commits into
d8 platform olcedar node add for static immutable nodes#450borg-z wants to merge 4 commits into
Conversation
Adding a static node running the immutable OS took five steps by hand: read the NodeGroup template, curl the machine inventory, uncomment a disk, merge the two documents with yq, and PUT the result back. Neither side could do it alone — the cluster does not know the machine's disks, the machine does not know the cluster's bootstrap token — but the CLI sees both, and unlike a file it can ask. The command reads the aggregated NodeConfigTemplate of the group and the machine's /inventory.json, asks which disk, which network and which node name, and pushes the assembled NodeConfig to the machine's maintenance port. Nothing is decided silently. The single blank disk is offered as a default but still confirmed; a disk that already holds a system is refused until the operator confirms the erase (or passes --wipe); without a terminal and without --yes the command refuses rather than choosing. A machine whose port is held by the node agent is refused by node name: a second configuration would replace the one a working node runs on. The template carries a live bootstrap token, the registry auth and the registry-packages-proxy token on every read, so the document is built in memory, never logged, and --dry-run prints it with those three fields redacted. The machine-facing HTTP contract (/whoami, /inventory.json, /config) mirrors dhctl/pkg/immutable of the deckhouse repository rather than importing it: the dhctl module would pull its whole dependency tree in for some 250 lines. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: borg-z <me@zykov.tatar>
borg-z
force-pushed
the
olcedar-node-add
branch
from
August 21, 2026 17:58
96a8488 to
c9f8ebb
Compare
…sing it
The first cut read `system-layout` as "an OS is here, the install will erase
it" and wrote `spec.storage.wipe: true` once the operator confirmed. On a
machine whose disk was cloned from an image — every DVP node — that is not a
confirmation, it is a self-destruct.
init decides for itself whether anything is installed:
`resolveSystemDisk` (images/init/src/0.1/disk.go) finds one disk carrying
BOOT/CONFIG/DATA and answers `install=false`, and initmode.go computes
`provision := install || cfg.Spec.Storage.Wipe`. So `wipe: true` is the one
thing that can force a provision where none is needed — and the provision is
destructive first and sourced second: `provisionDisk` erases, and only then
`copy-boot-assets` looks for the UKI and the rootfs under /run/media. A machine
booted from its own disk has no such media, so the copy fails after the erase
and there is nothing left to boot from.
So:
- a disk that already carries the layout is adopted, not installed onto, and
the document names no storage at all: init identifies that disk itself, and
a selector disagreeing with the pin recorded at install (writeDiskPin) is
itself a reason for it to reinstall;
- `wipe` reaches the document only from an explicit --wipe, which now reads as
"reinstall, from installation media", and says what it costs without it;
- a second disk carrying the layout is called out: the node identifies its
disk by that layout before it reads any selector, so it would take that one
and leave the named disk alone.
What the notes warn about cannot be settled from an inventory: `diskState`
spells installer media and an installed system the same way (inventory.go:101),
and only the machine knows which disk it booted from. Adding that fact to the
inventory belongs to the initramfs repository, where bootedDisk() already
computes it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: borg-z <me@zykov.tatar>
A live run on zykov-st read as one paragraph: the answer to a question, the
note about the disk and the next question all ran together, and the wait that
followed said nothing for minutes.
So the prompt sets its blocks apart itself — a blank line after a list, before
a question, and around a note — rather than leaving each caller to remember a
newline. A machine with one disk now offers it as the default: there is no
choice to make, and the list still shows what the disk is.
The wait says it is still waiting every 30 seconds, and the command closes with
what the addition cost:
Timings
read the cluster and the machine 1.2s
pushed the configuration 0.3s
node registered 1m24s
machine time in total 1m24s
The operator's own thinking time sits between the read and the push and is
deliberately not counted: the number worth knowing is how long the machine
takes, not how long someone stared at a disk list.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: borg-z <me@zykov.tatar>
A run against 10.12.0.40 hung with nothing on the screen at all. The machine was fine — it had already answered /whoami — and the command was sitting in the template read: nodeconfigtemplates is served by an aggregated API, so the kube-apiserver proxies it to node-controller, and client-go carries no timeout of its own. A backend that never answers hangs the command forever. So the rest config now bounds every cluster read at 30 seconds, and each network step says what it is about to do before it does it. Two silent minutes become two lines and, at worst, one bounded failure. That failure names where the answer comes from, because the kube-apiserver is almost never the part that is broken: read the node configuration template of worker: <err>. It is served by an aggregated API, so this read is proxied by the kube-apiserver to node-controller of node-manager: a node-controller that is down, unreachable or unregistered fails it. Check it with: d8 k get apiservice v1alpha1.templates.internal.deckhouse.io Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: borg-z <me@zykov.tatar>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
d8 platform olcedar node add <address> --group <nodegroup>— one command in place of the five-step manual flow for introducing a static node running the immutable OS (olcedar):Neither side can do this alone: the cluster does not know the machine's disks, the machine does not know the cluster's bootstrap token. The CLI sees both — and, unlike a file, it can ask.
Flow
GET /whoami— a machine already held by the node agent is refused by node name, resolved from its InternalIP.nodeconfigtemplates.templates.internal.deckhouse.io/v1alpha1for the group. A 404 is explained against the NodeGroup itself (missing / wrongnodeType/ wrongsystemType), never shown as "not found".GET /inventory.json. An image too old to serve it is refused — there is nothing to pick a disk out of.internal.deckhouse.io/v1alpha1NodeConfig,PUT /config, wait for the node to register.Adopting a disk, and never erasing one by accident
initdecides for itself whether anything is installed.resolveSystemDisk(images/init/src/0.1/disk.go) finds one disk carrying BOOT/CONFIG/DATA and answersinstall=false;initmode.gothen computesprovision := install || cfg.Spec.Storage.Wipe. Sowipe: trueis the one thing that can force a provision where none is needed — and that provision is destructive first and sourced second:provisionDiskerases, and only afterwardscopy-boot-assetslooks for the UKI and the rootfs under/run/media. A machine booted from its own disk (every disk cloned from an image, so every DVP node) has no such media: the erase succeeds, the copy fails, and nothing is left to boot from.This command therefore:
storageat all, becauseinitidentifies that disk itself, and a selector disagreeing with the pinwriteDiskPinrecorded at install is itself a reason for it to reinstall;wipeonly from an explicit--wipe, which reads as "reinstall, from installation media" and says what it costs without one;What those notes warn about cannot be settled from an inventory —
diskStatespells installer media and an installed system the same way (inventory.go:101), and only the machine knows which disk it booted from. Teaching the inventory that fact belongs to the initramfs repository, wherebootedDisk()already computes it.The three documents this produces:
The rest of what is never decided silently
--disk-selector.wwid→serial→busPath→name), so the document survivessdabecomingsdb.--network-interface.--network dhcp|staticdecides it up front.--yesthe command refuses instead of choosing.Secrets
The template carries a live bootstrap token,
registry.authand the registry-packages-proxy token on every read. The document is assembled in memory, is never logged or written to a file, and--dry-runprints it with those three fields redacted. There is a test asserting the fixture token value does not appear in the redacted output.Non-interactive
--groupcompletes to Static+Immutable NodeGroups only — a group that cannot take a machine is never offered.On not importing dhctl
The machine-facing HTTP contract (
/whoami,/inventory.json,/config) and the inventory types mirrordhctl/pkg/immutableof the deckhouse repository rather than importing it.CheckDocumentAgainstInventorythere reachesdhctl/pkg/configanddhctl/pkg/global, so importing it means pulling dhctl's whole dependency tree into the CLI for some 250 lines ofnet/http. The copies carry a mirror comment naming the original.Its disk-vs-inventory check is not ported wholesale either:
TemplateStorage.renderblanksspec.storageandspec.networkbefore serving, so both halves are filled in by this command out of the inventory it just read — checking them back against it checks the CLI against itself. What is ported is the part that catches a live mistake: a--disk-selectorfrom a human must resolve to exactly one disk.Tests
go test ./internal/olcedar/... ./internal/system/...→ 222 passed in 44 packages. Among them:agent,--yes,worker-0taken →worker-1offered (and hole-filling, and other groups ignored),system-layoutdisk adopted with nostoragein the document,wipepresent only when--wipewas passed,--dry-runoutput.Still to do
A live run on a stand — machine in
installer→ command → node registers,d8 k get nodeconfig <name>showing the chosen network and disk — is not part of this PR; that is what the draft state is for.🤖 Generated with Claude Code