From 5d4562568a1560d191bc40afc623abb2d8e3666c Mon Sep 17 00:00:00 2001 From: PhysShell <45852143+PhysShell@users.noreply.github.com> Date: Tue, 8 Sep 2026 10:03:27 +0500 Subject: [PATCH 1/5] Give xterm a font that has Cyrillic glyphs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Typing Russian over RDP produced boxes instead of characters, while switching the host back to a Latin layout worked fine. That reads like a keyboard problem and is not one: the input path is correct end to end. The guest runs `services.xserver.xkb.layout = "us"` and never sees a Cyrillic keysym of its own - the host owns the layout, and xrdp translates through its own keymaps (km-00000419.ini for Russian ships with the image). The characters arrive intact; xterm simply cannot draw them. xterm's compiled-in default is the bitmap "fixed" font in ISO-8859-1, which contains no Cyrillic glyphs at all, so every one of them renders as a box. The fonts themselves were never missing - the image carries 75 Cyrillic-capable faces (DejaVu, Liberation) and the locale is en_US.UTF-8. X resources would be the usual fix, but nothing in this image loads them: there is no display manager session and no xrdb call anywhere, and ~/.Xresources would live on the persistent home disk rather than in Nix. So the defaults go into the binary that the Openbox menu actually invokes, with hiPrio so the wrapper wins the name collision against xterm in the system profile. The rest of the package (resize, uxterm) is untouched. Verified on the live VM by running both variants side by side in the same xrdp session: the stock xterm shows boxes for "Кириллица: АБВГДЕ абвгде", the wrapped one renders it correctly. Co-Authored-By: Claude Opus 5 --- profiles/gui/openbox.nix | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/profiles/gui/openbox.nix b/profiles/gui/openbox.nix index 6fd5c74..45d5048 100644 --- a/profiles/gui/openbox.nix +++ b/profiles/gui/openbox.nix @@ -11,7 +11,17 @@ lib.mkIf (config.qubix.gui == "openbox") { # Remote sessions get a bare Openbox unless an app profile overrides this. qubix.session.command = lib.mkDefault "${pkgs.openbox}/bin/openbox-session"; + # xterm's compiled-in default is the bitmap "fixed" font in ISO-8859-1, which + # carries no Cyrillic (or any non-latin) glyphs, so typing Russian over RDP + # renders as boxes even though 75 Cyrillic-capable fonts are installed and the + # locale is UTF-8. Normally X resources would fix this, but nothing in this + # image loads them: there is no display manager session and no xrdb call, and + # ~/.Xresources would live on the persistent home disk rather than in Nix. + # So the defaults are baked into the binary the Openbox menu actually invokes. environment.systemPackages = with pkgs; [ + (lib.hiPrio (writeShellScriptBin "xterm" '' + exec ${xterm}/bin/xterm -fa 'DejaVu Sans Mono' -fs 11 -u8 "$@" + '')) openbox xterm ]; From 517dbd1a3f4ee0f3c031d50057d1bbe4a8a306ff Mon Sep 17 00:00:00 2001 From: PhysShell <45852143+PhysShell@users.noreply.github.com> Date: Tue, 8 Sep 2026 10:50:19 +0500 Subject: [PATCH 2/5] Keep a Latin keyboard group in remote sessions, with a toggle Changing the keyboard layout on the Windows side has no effect on a running session: xrdp applies the client's layout once, from the Client Info PDU at connect time, and sends bare scancodes afterwards. Measured in a live session whose guest config says `layout = "us"`: $ setxkbmap -query layout: ru So the guest is not ignoring the client - it was told exactly once, and a later switch on the host never reaches it. Reconnecting is currently the only way. Wrap the session so that, after xrdp has applied the client's layout, a Latin group is placed next to it together with a switch key. The layout list is deliberately not hardcoded: whatever this particular client negotiated is what gets the companion group, so a German client gets "us,de" and a Russian one "us,ru", while a Latin-only client keeps its single group and sees no change. Pinning "us,ru" into the image would have tied a redistributable appliance to one person's keyboard. Both ends are configurable via qubix.keyboard.latinGroup and .toggle. The default toggle is grp:win_space_toggle to match the Windows shortcut. Verified by hand in the live session: setxkbmap with these arguments produces `layout: us,ru` / `options: grp:win_space_toggle` and switching works without reconnecting or touching the host layout. The generated wrapper passes `sh -n` and `nix flake check`, but has not yet run inside a rebuilt image -- that comes with the next image build. Two limits are documented in the README rather than papered over: Win keys only reach the guest when mstsc is full screen, and reconnecting to an existing session does not re-run the wrapper. Co-Authored-By: Claude Opus 5 --- README.md | 24 ++++++++++++++++++++++++ modules/qubix-options.nix | 23 +++++++++++++++++++++++ profiles/remote/xrdp.nix | 35 +++++++++++++++++++++++++++++++++-- 3 files changed, 80 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 938c1c0..71bfb29 100644 --- a/README.md +++ b/README.md @@ -332,6 +332,30 @@ PipeWire -> disabled EasyEffects is intentionally not the active DSP baseline here. It is PipeWire-oriented, while this xrdp audio path expects PulseAudio. +### Keyboard groups in remote sessions + +xrdp pins the guest's XKB layout to whatever the client had **at connect time** +and never revisits it: RDP carries the layout once, in the Client Info PDU, and +sends bare scancodes afterwards. Switching the layout on the Windows side does +nothing in the guest until you reconnect - which reads as "the VM ignores my +keyboard" and is really "the VM was told once and never again". + +`profiles/remote/xrdp.nix` wraps the session so that, once xrdp has applied the +client's layout, a Latin group is added next to it plus a toggle. The list is +not hardcoded: whatever the client negotiated is what gets a companion group, so +a German client gets `us,de` and a Russian one `us,ru`, while a Latin-only +client keeps its single group and notices nothing. Tunable through +`qubix.keyboard.latinGroup` and `qubix.keyboard.toggle`. + +Two caveats worth knowing: + +- The default toggle is `grp:win_space_toggle`, and **Win keys only reach the + guest when mstsc runs full screen** (`Ctrl+Alt+Break` toggles that). In a + windowed session Windows keeps Win+Space for itself. +- On *reconnect* to an existing session the wrapper does not run again, so the + groups can collapse back to the client's single layout. Fixing that properly + belongs in xrdp, not here. + ### Why PCM-only audio nixpkgs builds xrdp with `--enable-mp3lame` and `--enable-opus`. With those diff --git a/modules/qubix-options.nix b/modules/qubix-options.nix index 3f8bf70..8d10565 100644 --- a/modules/qubix-options.nix +++ b/modules/qubix-options.nix @@ -49,6 +49,29 @@ ''; }; + keyboard = { + latinGroup = lib.mkOption { + type = lib.types.str; + default = "us"; + description = '' + XKB layout kept as the first group in remote sessions, so a Latin + keyboard is always available for shell commands regardless of what + the connecting client uses. + ''; + }; + + toggle = lib.mkOption { + type = lib.types.str; + default = "grp:win_space_toggle"; + example = "grp:alt_shift_toggle"; + description = '' + XKB option that switches between the Latin group and the client's + own layout. Win+Space matches the Windows shortcut; note that Win + keys only reach the guest when mstsc runs full screen. + ''; + }; + }; + homeDisk = { enable = lib.mkOption { type = lib.types.bool; diff --git a/profiles/remote/xrdp.nix b/profiles/remote/xrdp.nix index 6ce3488..bbdcb62 100644 --- a/profiles/remote/xrdp.nix +++ b/profiles/remote/xrdp.nix @@ -1,13 +1,44 @@ -{ config, ... }: +{ config, lib, pkgs, ... }: +let + kb = config.qubix.keyboard; + + # xrdp pins the guest's XKB layout to whatever the client had at connect time + # and never revisits it: RDP carries the layout once, in the Client Info PDU, + # and sends bare scancodes from then on. Switching the layout on the Windows + # side therefore changes nothing in the guest until you reconnect, which is + # exactly what it looks like from the user's chair. + # + # Hardcoding a layout list here would tie the image to one person's keyboard. + # Instead, take whatever xrdp negotiated with *this* client and keep a Latin + # group alongside it, plus a toggle: a German client gets "us,de", a Russian + # one "us,ru", and a Latin-only client keeps its single group untouched. + session = pkgs.writeShellScript "qubix-xrdp-session" '' + layout=$(${pkgs.xorg.setxkbmap}/bin/setxkbmap -query \ + | ${pkgs.gawk}/bin/awk '/^layout:/ { print $2 }') + case "$layout" in + "" | ${kb.latinGroup} | ${kb.latinGroup},*) : ;; + *) layout="${kb.latinGroup},$layout" ;; + esac + ${pkgs.xorg.setxkbmap}/bin/setxkbmap \ + -layout "''${layout:-${kb.latinGroup}}" \ + -option "" \ + -option "${config.services.xserver.xkb.options}" \ + -option "${kb.toggle}" || true + + exec ${config.qubix.session.command} + ''; +in { # xrdp is the appliance's window to the Windows host. Whatever the active # profiles put into qubix.session.command becomes the session: a plain # window manager for generic images, or a single-app kiosk session such as # Spotify. When that command exits, xrdp ends the session and mstsc closes. + # + # The session is wrapped so keyboard groups are fixed up first; see above. services.xrdp = { enable = true; - defaultWindowManager = config.qubix.session.command; + defaultWindowManager = "${session}"; openFirewall = true; }; } From 947e628f3e3f3f992f22d95ae7819cab1dd11f0c Mon Sep 17 00:00:00 2001 From: PhysShell <45852143+PhysShell@users.noreply.github.com> Date: Tue, 8 Sep 2026 11:37:04 +0500 Subject: [PATCH 3/5] Do not arm Ctrl+Alt+Backspace in the kiosk session The previous commit forwarded services.xserver.xkb.options into the session's setxkbmap call. Its NixOS default is terminate:ctrl_alt_bksp, which xrdp never applies by itself -- so passing it through would newly arm Ctrl+Alt+Backspace to kill the X server. In a kiosk session that is not a debugging convenience: the RDP window simply disappears on a stray keypress, mid-track. Caught on the live VM: pressing the combination closed the session while the option was set by hand, and stopped doing anything after reconnecting, when xrdp had reset the keymap without it. Only the group toggle is set now. Co-Authored-By: Claude Opus 5 --- profiles/remote/xrdp.nix | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/profiles/remote/xrdp.nix b/profiles/remote/xrdp.nix index bbdcb62..1d583e6 100644 --- a/profiles/remote/xrdp.nix +++ b/profiles/remote/xrdp.nix @@ -13,6 +13,11 @@ let # Instead, take whatever xrdp negotiated with *this* client and keep a Latin # group alongside it, plus a toggle: a German client gets "us,de", a Russian # one "us,ru", and a Latin-only client keeps its single group untouched. + # Note that services.xserver.xkb.options is deliberately NOT forwarded here. + # Its NixOS default is terminate:ctrl_alt_bksp, which xrdp never applies on + # its own; passing it through would newly arm Ctrl+Alt+Backspace to kill the + # X server, and in a kiosk session that means the RDP window vanishes on a + # stray keypress. Only the group toggle is set. session = pkgs.writeShellScript "qubix-xrdp-session" '' layout=$(${pkgs.xorg.setxkbmap}/bin/setxkbmap -query \ | ${pkgs.gawk}/bin/awk '/^layout:/ { print $2 }') @@ -23,7 +28,6 @@ let ${pkgs.xorg.setxkbmap}/bin/setxkbmap \ -layout "''${layout:-${kb.latinGroup}}" \ -option "" \ - -option "${config.services.xserver.xkb.options}" \ -option "${kb.toggle}" || true exec ${config.qubix.session.command} From dae0e940b2dbedbaae1b4af806b0ded49255b089 Mon Sep 17 00:00:00 2001 From: PhysShell <45852143+PhysShell@users.noreply.github.com> Date: Tue, 8 Sep 2026 12:36:40 +0500 Subject: [PATCH 4/5] Actually apply the keyboard groups, and let layouts be named explicitly The previous attempt did not work on a rebuilt image, and hid why: it called setxkbmap once from startwm.sh and ended the line with `|| true`, so the failure went nowhere. On the live VM the result was visible as a missing options line entirely: $ setxkbmap -query rules: base model: pc104 layout: us No grp:win_space_toggle at all, although running the same setxkbmap by hand in that very session worked. X is simply not guaranteed to accept connections yet when startwm.sh runs, and xrdp may also apply the client's layout after the session script has started. So do it from a background loop that checks the result and re-applies while the toggle is missing, for a dozen rounds, and report failures on stderr instead of swallowing them. The second half of the problem was a design mistake. The dynamic default adds a Latin group beside whatever xrdp negotiated, which is right for keeping the image client-agnostic -- but RDP reports only the client's *active* layout, so connecting from a US-layout Windows meant a single group and nothing to toggle to, no matter how well the code ran. Add qubix.keyboard.layouts for naming the list outright, keep the dynamic behaviour as the default, and set "us,ru" on spotibox, which is a specific machine with a fixed address and a lab password rather than a redistributable template. Co-Authored-By: Claude Opus 5 --- machines/spotibox.nix | 5 +++ modules/qubix-options.nix | 16 +++++++++ profiles/remote/xrdp.nix | 71 +++++++++++++++++++++++++++------------ 3 files changed, 70 insertions(+), 22 deletions(-) diff --git a/machines/spotibox.nix b/machines/spotibox.nix index ba92da9..9d96d89 100644 --- a/machines/spotibox.nix +++ b/machines/spotibox.nix @@ -26,6 +26,11 @@ audio = "pulseaudio-xrdp"; app = "spotify"; kernel = "default"; + # RDP only reports the client's *active* layout, so a Russian typist + # connecting while Windows sits on the US layout would otherwise get a + # single Latin group and nothing to toggle to. Name both explicitly. + keyboard.layouts = "us,ru"; + homeDisk.sizeMiB = 16 * 1024; network = { staticIp = "192.168.250.10"; gateway = "192.168.250.1"; }; }; diff --git a/modules/qubix-options.nix b/modules/qubix-options.nix index 8d10565..4ecb9ba 100644 --- a/modules/qubix-options.nix +++ b/modules/qubix-options.nix @@ -60,6 +60,22 @@ ''; }; + layouts = lib.mkOption { + type = lib.types.str; + default = ""; + example = "us,ru"; + description = '' + Explicit XKB layout list for remote sessions. Empty (the default) + keeps whatever layout xrdp negotiated with the client and places + `latinGroup` beside it, which keeps the image client-agnostic. + + Set this when the people using the appliance need a layout their RDP + client does not announce - a Russian typist connecting while the + Windows side happens to sit on the US layout, for example. RDP only + reports the client's *active* layout, so that case cannot be guessed. + ''; + }; + toggle = lib.mkOption { type = lib.types.str; default = "grp:win_space_toggle"; diff --git a/profiles/remote/xrdp.nix b/profiles/remote/xrdp.nix index 1d583e6..01c0163 100644 --- a/profiles/remote/xrdp.nix +++ b/profiles/remote/xrdp.nix @@ -3,32 +3,56 @@ let kb = config.qubix.keyboard; + setxkbmap = "${pkgs.xorg.setxkbmap}/bin/setxkbmap"; + awk = "${pkgs.gawk}/bin/awk"; + # xrdp pins the guest's XKB layout to whatever the client had at connect time # and never revisits it: RDP carries the layout once, in the Client Info PDU, # and sends bare scancodes from then on. Switching the layout on the Windows - # side therefore changes nothing in the guest until you reconnect, which is - # exactly what it looks like from the user's chair. + # side therefore changes nothing in the guest until you reconnect. + # + # Two things make this awkward to fix from startwm.sh: # - # Hardcoding a layout list here would tie the image to one person's keyboard. - # Instead, take whatever xrdp negotiated with *this* client and keep a Latin - # group alongside it, plus a toggle: a German client gets "us,de", a Russian - # one "us,ru", and a Latin-only client keeps its single group untouched. - # Note that services.xserver.xkb.options is deliberately NOT forwarded here. - # Its NixOS default is terminate:ctrl_alt_bksp, which xrdp never applies on - # its own; passing it through would newly arm Ctrl+Alt+Backspace to kill the - # X server, and in a kiosk session that means the RDP window vanishes on a - # stray keypress. Only the group toggle is set. + # * X is not necessarily accepting connections yet when startwm.sh runs, so + # a single setxkbmap call can silently do nothing; + # * xrdp may apply the client's layout *after* the session script starts, + # overwriting whatever was set. + # + # So run in the background and keep checking for a while, re-applying if the + # toggle disappeared. Failures go to stderr (and thus the session log) rather + # than being swallowed, which is how the first version of this hid its own + # breakage. session = pkgs.writeShellScript "qubix-xrdp-session" '' - layout=$(${pkgs.xorg.setxkbmap}/bin/setxkbmap -query \ - | ${pkgs.gawk}/bin/awk '/^layout:/ { print $2 }') - case "$layout" in - "" | ${kb.latinGroup} | ${kb.latinGroup},*) : ;; - *) layout="${kb.latinGroup},$layout" ;; - esac - ${pkgs.xorg.setxkbmap}/bin/setxkbmap \ - -layout "''${layout:-${kb.latinGroup}}" \ - -option "" \ - -option "${kb.toggle}" || true + ( + attempts=12 + while [ "$attempts" -gt 0 ]; do + attempts=$((attempts - 1)) + sleep 1 + + query=$(${setxkbmap} -query 2>/dev/null) || continue + + # Already carrying our toggle: nothing to do this round. + case "$query" in + *${kb.toggle}*) continue ;; + esac + + wanted='${kb.layouts}' + if [ -z "$wanted" ]; then + # Client-agnostic default: keep the negotiated layout and put a Latin + # group next to it, so shell commands stay typable either way. + current=$(printf '%s\n' "$query" | ${awk} '/^layout:/ { print $2 }') + case "$current" in + "" | ${kb.latinGroup} | ${kb.latinGroup},*) wanted="$current" ;; + *) wanted="${kb.latinGroup},$current" ;; + esac + [ -n "$wanted" ] || wanted='${kb.latinGroup}' + fi + + if ! ${setxkbmap} -layout "$wanted" -option "" -option '${kb.toggle}'; then + echo "qubix: setxkbmap -layout $wanted failed" >&2 + fi + done + ) & exec ${config.qubix.session.command} ''; @@ -39,7 +63,10 @@ in # window manager for generic images, or a single-app kiosk session such as # Spotify. When that command exits, xrdp ends the session and mstsc closes. # - # The session is wrapped so keyboard groups are fixed up first; see above. + # The session is wrapped so keyboard groups are fixed up alongside it; see + # above. services.xserver.xkb.options is deliberately NOT forwarded: its + # NixOS default is terminate:ctrl_alt_bksp, which xrdp never applies on its + # own, and arming it would let a stray keypress kill the kiosk session. services.xrdp = { enable = true; defaultWindowManager = "${session}"; From 8d6a4b2f5befe358055eb508465fdc57747e56cc Mon Sep 17 00:00:00 2001 From: PhysShell <45852143+PhysShell@users.noreply.github.com> Date: Tue, 8 Sep 2026 13:05:11 +0500 Subject: [PATCH 5/5] Add a gc command for the image cache Every `up` or `recreate` from a .gz leaves an unpacked copy behind, and release downloads keep a directory per tag, so the cache grows without anything ever clearing it. On the machine this was written on it had reached 5.89 GB of images that no longer corresponded to anything installed. qubixctl -Command gc # dry run: what would go, and how much qubixctl -Command gc -Force # delete it qubixctl -Command gc -Force -All # the installed image's cache too Nothing is deleted without -Force, and the dry run reads only, so it needs no elevation. The directory matching image-version.txt is kept by default, because re-fetching a release means downloading the assets again; `local/` is always dropped, since it only holds a copy unpacked from a file the caller still has. Non-release markers ('file:...', 'wsl:...') cannot name a cache directory, so they protect nothing -- which the unit checks pin down. The VM directory is never a candidate: not the home disk, which is the only state in the system, and not the live system disk. Images staged by hand under vmRoot are reported and never deleted. Cleaning up after the controller is one thing; deleting files a person placed themselves is another, and the useful half is simply saying that 18 GB is sitting there. The decision half is a pure function taking directory names, so it is unit checked without a disk; sizing and the foreign-image scan are checked against a temporary tree. 55 checks pass under both Windows PowerShell 5.1 and pwsh 7, PSScriptAnalyzer is clean, and the dry run was exercised against the real cache. Co-Authored-By: Claude Opus 5 --- README.md | 31 +++++++++ tests/qubixctl.Tests.ps1 | 41 ++++++++++++ tools/qubixctl.ps1 | 140 ++++++++++++++++++++++++++++++++++++++- 3 files changed, 210 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 71bfb29..62f73dc 100644 --- a/README.md +++ b/README.md @@ -95,6 +95,7 @@ From a console the same thing is: | `destroy` | remove VM + system disk; `-Purge` also deletes the home disk | | `fetch` | download release images into the cache without touching the VM | | `build` | build images in WSL (developer path) | +| `gc` | report unused cached images; `-Force` deletes them | | `manifest` | print the resolved machine config | Useful switches: `-Release v0.2.0` (pin a release), `-VmRoot D:\vms`, @@ -115,6 +116,36 @@ public repository needs no token. Private repositories are not supported by the `release` source yet; use `fetch` from a machine that can reach the assets, or the `wsl` / `file` sources. +### Reclaiming Disk Space + +Every `up` or `recreate` from a `.gz` leaves an unpacked copy in the cache, and +release downloads keep one directory per tag. `gc` clears what is no longer +needed: + +```bash +qubixctl -Command gc # dry run: what would go, and how much +qubixctl -Command gc -Force # delete it +qubixctl -Command gc -Force -All # drop the installed image's cache as well +``` + +Nothing is deleted without `-Force`, and the dry run needs no elevation. The +cache directory matching `image-version.txt` is kept by default, since +re-fetching a release means downloading the assets again; `-All` is the +`nix-collect-garbage -d` of this command. `local/` is always dropped - it only +ever holds a copy unpacked from a file the caller already has. + +The VM directory is never touched: not the home disk, not the live system disk. +Images staged by hand under `vmRoot` (for `-ImageSource file`) are **reported +but never deleted** - tidying up after the controller is one thing, deleting +what a person put there is another: + +```text +Disk images under C:\HyperV\Qubix that qubixctl did not create (18.67 GB): + C:\HyperV\Qubix\src\spotibox-kb.vhdx (5.57 GB) + ... +These were staged by hand; delete them yourself if they are no longer needed. +``` + ## Persistence Model ```text diff --git a/tests/qubixctl.Tests.ps1 b/tests/qubixctl.Tests.ps1 index 3dd5aa2..f374686 100755 --- a/tests/qubixctl.Tests.ps1 +++ b/tests/qubixctl.Tests.ps1 @@ -130,6 +130,47 @@ try { Assert-True ((Get-QubixAddress -Config $config -Explicit '10.0.0.5') -eq '10.0.0.5') 'explicit address wins' Assert-True ((Get-QubixAddress -Config $config -Explicit '') -eq (Get-Prop $config 'staticIp')) 'static IP is used without Hyper-V lookups' Assert-True (-not (Test-TcpPort -TargetHost '127.0.0.1' -Port 1 -TimeoutMs 500)) 'closed ports are reported as closed' + + # --- gc ---------------------------------------------------------------- + # 'local' is a copy of something the caller already has, so it always goes. + Assert-True ((Select-QubixGarbage -CacheNames @('local') -InstalledVersion 'v1.0.0' -All $false) -contains 'local') 'gc always drops the local cache' + + # A release tag matching image-version.txt is worth keeping: re-fetching it + # would mean downloading the assets again. + $g = @(Select-QubixGarbage -CacheNames @('v1.0.0', 'v0.9.0', 'local') -InstalledVersion 'v1.0.0' -All $false) + Assert-True ($g -notcontains 'v1.0.0') 'gc keeps the installed release cache' + Assert-True ($g -contains 'v0.9.0') 'gc drops release caches that are not installed' + Assert-True ($g -contains 'local') 'gc drops local alongside stale tags' + + # -All is the nix-collect-garbage -d equivalent. + $gAll = @(Select-QubixGarbage -CacheNames @('v1.0.0', 'local') -InstalledVersion 'v1.0.0' -All $true) + Assert-True ($gAll -contains 'v1.0.0') '-All drops the installed release cache too' + + # image-version.txt carries 'file:...' / 'wsl:...' for non-release images, + # and those can never name a cache directory. + $gFile = @(Select-QubixGarbage -CacheNames @('v1.0.0') -InstalledVersion 'file:spotibox.vhdx' -All $false) + Assert-True ($gFile -contains 'v1.0.0') 'a file: marker protects no cache directory' + Assert-True (@(Select-QubixGarbage -CacheNames @() -InstalledVersion '' -All $false).Count -eq 0) 'an empty cache yields no garbage' + + # Sizes and the foreign-image report work off a real directory tree. + $gcRoot = Join-Path $tmp 'gcroot' + $vmDir = Join-Path $gcRoot 'qubix-box' + $cache = Join-Path (Join-Path $gcRoot 'images') 'box' + New-Item -ItemType Directory -Force -Path (Join-Path $cache 'local') | Out-Null + New-Item -ItemType Directory -Force -Path $vmDir | Out-Null + [System.IO.File]::WriteAllBytes((Join-Path (Join-Path $cache 'local') 'sys.vhdx'), (New-Object byte[] 2048)) + [System.IO.File]::WriteAllBytes((Join-Path $vmDir 'qubix-box.vhdx'), (New-Object byte[] 512)) + [System.IO.File]::WriteAllBytes((Join-Path $gcRoot 'staged.vhdx'), (New-Object byte[] 1024)) + + Assert-True ((Get-QubixPathSize -Path (Join-Path $cache 'local')) -eq 2048) 'cache size is measured recursively' + Assert-True ((Get-QubixPathSize -Path (Join-Path $tmp 'missing')) -eq 0) 'a missing path measures as zero' + + $gcPaths = [PSCustomObject]@{ VmRoot = $gcRoot; VmDir = $vmDir; ImageCache = $cache } + $foreign = @(Get-QubixForeignImage -Paths $gcPaths) + Assert-True ($foreign.Count -eq 1) 'only hand-staged images are reported as foreign' + Assert-True ($foreign[0].Name -eq 'staged.vhdx') 'the staged image is the one reported' + Assert-True (-not ($foreign.Name -contains 'qubix-box.vhdx')) 'the live system disk is never reported as foreign' + Assert-True (-not ($foreign.Name -contains 'sys.vhdx')) 'the controller cache is not reported as foreign' } finally { Remove-Item -LiteralPath $tmp -Recurse -Force -ErrorAction SilentlyContinue } diff --git a/tools/qubixctl.ps1 b/tools/qubixctl.ps1 index a13899b..f66c85d 100644 --- a/tools/qubixctl.ps1 +++ b/tools/qubixctl.ps1 @@ -26,6 +26,7 @@ destroy remove VM + system disk (add -Purge to delete the home disk too) fetch download release images into the cache without touching the VM build build images in WSL (developer path) + gc report and (with -Force) delete unused cached images manifest print the resolved machine config .PARAMETER ImageSource @@ -43,7 +44,7 @@ Justification = 'Script parameters are consumed by Invoke-QubixMain; the analyzer does not follow that.')] [CmdletBinding()] param( - [ValidateSet('up', 'connect', 'start', 'stop', 'status', 'recreate', 'destroy', 'fetch', 'build', 'manifest')] + [ValidateSet('up', 'connect', 'start', 'stop', 'status', 'recreate', 'destroy', 'fetch', 'build', 'gc', 'manifest')] [string]$Command = 'up', [string]$Machine = 'spotibox', @@ -73,7 +74,13 @@ param( [int]$TimeoutSeconds = 300, [switch]$NoConnect, [switch]$NoSavedCredential, - [switch]$Purge + [switch]$Purge, + + # gc only. Without -Force nothing is deleted, which is the default so + # that a stray `gc` cannot cost anyone their images. -All additionally + # drops the cache of the image that is currently installed. + [switch]$Force, + [switch]$All ) Set-StrictMode -Version Latest @@ -765,6 +772,78 @@ function Initialize-QubixVm { Set-Content -LiteralPath $Paths.VersionFile -Value $Version } +# -------------------------------------------------------------------------- +# Image cache garbage collection +# -------------------------------------------------------------------------- + +function Select-QubixGarbage { + # Pure decision half of `gc`, so it can be unit-checked without a disk. + # + # Cache layout is \images\\, where is either a + # release tag or the literal 'local'. A tag directory is worth keeping: it + # saves re-downloading a couple of gigabytes. 'local' never is - it only + # ever holds a copy unpacked from a file the caller already has. + param( + [string[]]$CacheNames, + [string]$InstalledVersion, + [bool]$All + ) + + # image-version.txt holds a bare tag for release images, and a prefixed + # marker ('file:...', 'wsl:...') otherwise, so only a colon-free value can + # name a cache directory worth preserving. + $keep = '' + if (-not $All -and $InstalledVersion -and ($InstalledVersion -notmatch ':')) { + $keep = $InstalledVersion.Trim() + } + + $garbage = @() + foreach ($name in $CacheNames) { + if ($name -eq 'local') { $garbage += $name; continue } + if ($keep -and ($name -eq $keep)) { continue } + $garbage += $name + } + return $garbage +} + +function Get-QubixPathSize { + param([string]$Path) + + if (-not (Test-Path -LiteralPath $Path)) { return [int64]0 } + $sum = Get-ChildItem -LiteralPath $Path -Recurse -File -ErrorAction SilentlyContinue | + Measure-Object -Property Length -Sum + if ($null -eq $sum.Sum) { return [int64]0 } + return [int64]$sum.Sum +} + +function Format-QubixSize { + param([int64]$Bytes) + return ('{0:N2} GB' -f ($Bytes / 1GB)) +} + +function Get-QubixForeignImage { + # Disk images sitting under vmRoot that this controller did not put there - + # typically staged by hand for -ImageSource file. They are reported and + # never deleted: cleaning up after the machine is one thing, deleting what + # a person placed themselves is another. + param([object]$Paths) + + if (-not (Test-Path -LiteralPath $Paths.VmRoot)) { return @() } + + $skip = @($Paths.VmDir, (Join-QubixPath $Paths.VmRoot 'images')) + return @( + Get-ChildItem -LiteralPath $Paths.VmRoot -Recurse -File -ErrorAction SilentlyContinue | + Where-Object { + # $_ is rebound by the inner Where-Object, so hold on to the file. + $file = $_ + ($file.Name -like '*.vhdx' -or $file.Name -like '*.vhdx.gz') -and + -not ($skip | Where-Object { + $_ -and $file.FullName.StartsWith($_, [StringComparison]::OrdinalIgnoreCase) + }) + } + ) +} + function Get-QubixVm { param([object]$Paths) return Get-VM -Name $Paths.VmName -ErrorAction SilentlyContinue @@ -1062,6 +1141,62 @@ function Invoke-QubixRecreate { Invoke-QubixUp -Config $Config -Paths $Paths -Ctx $Ctx } +function Invoke-QubixGc { + param( + [object]$Paths, + [bool]$DeleteThem, + [bool]$All + ) + + $installed = '' + if (Test-Path -LiteralPath $Paths.VersionFile) { + $installed = (Get-Content -LiteralPath $Paths.VersionFile -Raw).Trim() + } + Write-Host "Installed image: $(if ($installed) { $installed } else { '(none)' })" + + $names = @() + if (Test-Path -LiteralPath $Paths.ImageCache) { + $names = @(Get-ChildItem -LiteralPath $Paths.ImageCache -Directory | ForEach-Object { $_.Name }) + } + $garbage = @(Select-QubixGarbage -CacheNames $names -InstalledVersion $installed -All $All) + $kept = @($names | Where-Object { $garbage -notcontains $_ }) + + $total = [int64]0 + foreach ($name in $garbage) { + $dir = Join-QubixPath $Paths.ImageCache $name + $size = Get-QubixPathSize -Path $dir + $total += $size + if ($DeleteThem) { + Write-Host "Removing $dir ($(Format-QubixSize $size))" + Remove-Item -LiteralPath $dir -Recurse -Force + } else { + Write-Host "Would remove $dir ($(Format-QubixSize $size))" + } + } + + foreach ($name in $kept) { + Write-Host "Keeping $(Join-QubixPath $Paths.ImageCache $name) (installed image; -All removes it too)" + } + + if ($garbage.Count -eq 0) { Write-Host 'Cache is already clean.' } + Write-Host "$(if ($DeleteThem) { 'Reclaimed' } else { 'Reclaimable' }): $(Format-QubixSize $total)" + + # Never deleted, only surfaced: see Get-QubixForeignImage. + $foreign = @(Get-QubixForeignImage -Paths $Paths) + if ($foreign.Count -gt 0) { + $foreignBytes = [int64]($foreign | Measure-Object -Property Length -Sum).Sum + Write-Host '' + Write-Host "Disk images under $($Paths.VmRoot) that qubixctl did not create ($(Format-QubixSize $foreignBytes)):" + foreach ($f in $foreign) { Write-Host " $($f.FullName) ($(Format-QubixSize $f.Length))" } + Write-Host 'These were staged by hand; delete them yourself if they are no longer needed.' + } + + if (-not $DeleteThem -and $garbage.Count -gt 0) { + Write-Host '' + Write-Host 'Nothing was deleted. Re-run with -Force to actually remove the entries above.' + } +} + function Invoke-QubixDestroy { param( [object]$Paths, @@ -1131,6 +1266,7 @@ function Invoke-QubixMain { 'build' { Build-QubixImagesInWsl -Config $config -Distro $ctx.WslDistro -RepoPath $ctx.RepoLinuxPath | Out-Null } + 'gc' { Invoke-QubixGc -Paths $paths -DeleteThem ([bool]$Force) -All ([bool]$All) } 'manifest' { $config | ConvertTo-Json -Depth 8 } } }