Skip to content

feat(vm): vm-encrypt - move the backup on Drive under rclone crypt - #40

Merged
mbeczynski merged 9 commits into
mainfrom
feat/drive-encryption
Nov 4, 2026
Merged

mbeczynski merged 9 commits into
mainfrom
feat/drive-encryption

Conversation

@mbeczynski

@mbeczynski mbeczynski commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Client-side encryption of the backup on Google Drive, without a new full Time Machine backup. Prepared, not run anywhere: no release, nothing touched on the live machine.

Why rclone crypt, not Time Machine's "Encrypt backups"

  • TM encrypts the disk image itself. vm-prepare-bundle must re-band TM's new image (875 MB → 32 MB bands); for an encrypted image that means hdiutil convert of an encrypted sparse bundle, which answered Permission denied on macOS 27 (scratch image, 03.11) - and hdiutil convert -format is deprecated.
  • Whether TM still finds its keychain password for a converted image cannot be tried without the live destination (Time Machine is global).
  • TM encryption can only be turned on for a new backup: ~215 GiB again plus losing the history.
  • crypt sits below everything TM sees (image, Samba, punchfs unchanged). The existing bands are copied once, encrypted, next to the plain ones; TM history is kept. Overhead measured: 8 224 B per 32 MiB band (0.025 %). Names are encrypted too.

Commands (cloudmachine-agent vm-encrypt ...)

step does refuses unless
setup adds gdrive-crypt (over gdrive:CloudMachine/<folder>-crypt) to rclone.conf on the Mac and in the machine; password + random salt into the Keychain; prints the salt the remote exists nowhere yet, password ≥ 12 chars
copy plain → encrypted (bands + metadata) in the background (systemd-run, --drive-stop-on-upload-limit); backups go on not already switched / running
switch stops Samba, last sync, cryptcheck of every file, repoints vm.env, deletes the old remote's buffer, checks the band count; any failure puts vm.env back (trap). Writes the Mac marker last. The plain folder stays. TM stopped, guard holds no pause, 0 unsent / in progress / errored, copy finished (exited/success), machine on the template and mounting exactly the plain folder
delete-plain purge --drive-use-trash=true of the plain folder and its metadata (Drive bin, 30 days) a TM backup finished after the switch, metadata copied after it, nothing unsent, --secrets-saved-elsewhere and --restore-checked
rollback switch the other way, decrypting what changed before delete-plain

Without the marker nothing changes: VMLayout.remote stays gdrive:CloudMachine/<folder>-tm (test), so this code is dormant on existing installations.

Tests

  • 15 new tests (DriveEncryptionTests): folder derivation, marker, config section, copy state parsing, script order (queue check before vm.env, trap armed before units stop, only the old buffer deleted, preflight before Samba stops), every switch/delete gate refusing on each bad fact, delete goes to the bin and never touches the encrypted folder.
  • Canary: weakening two gates (<= → < for "backup after switch", dropping erroredFiles) fails testDeleteGate and testSwitchGate; restored.
  • Generated scripts pass bash -n; the nested quoting of copy checked with a stubbed systemd-run.
  • Full suite: 324 tests green, swift-format lint --strict clean.

Not tested yet - required before merge

The scripts have not run in a Linux machine. Before merging: one pass of setup → copy → switch → rollback → switch → delete-plain on a throwaway machine (CLOUDMACHINE_VM_INSTANCE=cmtest and CLOUDMACHINE_VM_REMOTE=gdrive:CloudMachine/cmtest-tm, plus RCLONE_CONFIG pointing at a copy of rclone.conf), see the plan in renacode-infra. Every step refuses a non-default instance without CLOUDMACHINE_VM_REMOTE, and the marker is per instance (drive-encryption-cmtest), so the test cannot move or delete the live backup (second commit, testThrowawayMachineIsIsolated, canary checked). Note: switch reads Time Machine's global state; on a throwaway machine TM must simply be idle.

Merging releases: this touches mac-app/Sources/**, so the merge publishes a new version through release.yml.

🤖 Generated with Claude Code

https://claude.ai/code/session_01AU1Mhva84gi4TY9YBib3e4

mbeczynski and others added 3 commits November 3, 2026 09:39
Client-side encryption without a new full backup. rclone `crypt` sits
below everything Time Machine sees (image, Samba, punchfs unchanged), so
the existing bands are copied once, encrypted, into
gdrive:CloudMachine/<folder>-crypt and the machine then mounts that
folder. Time Machine's own "Encrypt backups" was not taken: it encrypts
the image itself, vm-prepare-bundle would have to re-band an encrypted
image (hdiutil convert of an encrypted sparse bundle: "Permission
denied" on macOS 27), and whether Time Machine finds its password for a
converted image cannot be tried without the live destination.

vm-encrypt setup | copy | switch | rollback | delete-plain | status:
- setup: the `gdrive-crypt` remote on the Mac and in the machine,
  password and a random salt in the Keychain; refuses if it exists.
- copy: plain -> encrypted in the background (systemd-run), backups go on.
- switch: refuses unless Time Machine is stopped, the guard holds no
  pause, nothing is left to upload and the copy finished; checks the
  machine is on the template and mounts the plain folder; stops Samba,
  syncs the rest, cryptchecks every file, then repoints vm.env (trap puts
  it back on any failure). The plain folder stays.
- delete-plain: only after a backup finished after the switch, the
  metadata was copied since, nothing is unsent, and the owner confirms
  the secrets are kept outside the Mac and a restore worked; to Drive's bin.
- rollback: the switch the other way, decrypting what changed.

Without the marker written by switch nothing changes: VMLayout.remote is
still gdrive:CloudMachine/<folder>-tm (tested).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AU1Mhva84gi4TY9YBib3e4
The marker and every remote derived from DriveFolder.name were global to
the Mac: `switch` tried on cmtest would have moved cmsmb's agents to the
encrypted folder, and `delete-plain` there would have deleted the live
plain backup. Now the marker is per instance (drive-encryption-<name>),
CLOUDMACHINE_VM_REMOTE replaces the plain folder (and the encrypted root
follows it, <x>-tm -> <x>-crypt), and every step refuses a non-default
instance without CLOUDMACHINE_VM_REMOTE.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AU1Mhva84gi4TY9YBib3e4
Found by running the generated guest scripts in an Ubuntu 24.04 container
(systemd, rclone v1.75.1, punchfs, a local alias remote as "Drive"):

- A wrong password or salt in the machine's rclone.conf made `switch`
  succeed: `gdrive-crypt:tm` encrypts to another folder name, the last
  `sync` uploaded the whole backup again under that key and `cryptcheck`
  agreed with itself. The last sync and the background copy now list the
  crypt root with `--crypt-strict-names` first (the copy creates the root
  before, `lsf` fails on a missing one), and `copy`/`switch`/`rollback`
  compare the machine's crypt section (password, salt, folder) with the
  Mac's - the only check that sees a key wrong from the first copy on.
  A listing failure explains what to compare or remove.
- `move` stops smbd before the last sync; a repoint failure before its
  restore trap (no bands, a preflight miss) or no answer at all left it
  stopped. The script now starts it on ERR from its first line, and
  `move` starts it after any failed repoint.
- All guest scripts are checked with `bash -n`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AU1Mhxpy65J75YYTkmt7NF
@mbeczynski

Copy link
Copy Markdown
Contributor Author

Test skryptów vm-encrypt w kontenerze (jednorazowa maszyna)

Środowisko: Ubuntu 24.04 arm64 z systemd jako PID 1 (jak vm/cmsmb.yaml), rclone v1.75.1 (suma z cmsmb.yaml), punchfs zbudowany z vm/punchfs, jednostki z vm/systemd (bez smb-throttle), Samba. „Google Drive” = rclone alias na katalog w kontenerze; fikcyjny sparsebundle (48–248 losowych pasm 0,25–4 MiB + metadane). Hasło i sól losowe, tylko w kontenerze. Skrypty gościa wygenerowane z tego kodu Swift (copyScript, finalSyncScript, repointScript, preflightScript, deletePlainScript, configSection) i uruchomione bash -c jak przez limactl shell … sudo bash -c. Logikę strony Maca z copy()/move() (bramki, kolejność, znacznik) odtworzył skrypt sterujący. Nie uruchamiałem cloudmachine-agent (pęk kluczy, prawdziwy rclone.conf), tmutil ani bramek delete-plain (mają testy jednostkowe).

Krok Wynik
status na czystym stanie notRun ✅
switch bez setup odmowa (brak gdrive-crypt w maszynie) ✅
setup (połowa w maszynie) / drugi raz sekcja dopisana / odmowa ✅
copy → stan done, CM-COPY-DONE w logu, nazwy w cmtest-crypt nieczytelne, cryptcheck 48/48 ✅
copy drugi raz w trakcie odmowa „already running” ✅
copy zabity w połowie (kill -9, 42/88 plików) failed(signal); switch odmawia ✅
copy wznowiony done, dociągnął resztę, cryptcheck OK ✅
switch CM-REPOINTED 248, vm.env → gdrive-crypt:tm, pasma przez udział == oryginał (sha256), smbd aktywny ✅
switch / copy drugi raz po przełączeniu odmowa ✅
tm-meta-sync po przełączeniu sukces, pisze do gdrive-crypt:tm-meta ✅
zapis + usunięcie pasma po switch, rollback CM-REPOINTED, vm.env jawny, udział == stan przed rollbackiem, nowe pasmo jest w jawnym folderze, usunięte zniknęło ✅
rollback przy niewysłanej kolejce odmowa „unsent” (write-back 600 s) ✅
rollback drugi raz odmowa ✅
ponowny switch CM-REPOINTED, udział zgodny ✅
delete-plain (skrypt gościa) jawne foldery usunięte, udział nadal zgodny ✅
remote niedostępny w trakcie ostatniego sync rc=3, maszyna nie przełączona, smbd wrócił ✅
brak miejsca (tmpfs 30 MB) copy → failed(exit-code) („file too big for remaining disk space”), switch odmawia ✅
złe hasło w rclone.conf maszyny ❌ przed poprawką: switch się udał — gdrive-crypt:tm szyfruje się do innej nazwy folderu, sync wysłał całą kopię drugi raz złym kluczem (182 pliki), cryptcheck zgodny sam ze sobą. Po poprawce: odmowa „undecryptable dir name”, maszyna na starym folderze ✅
copy wznowiony ze złym hasłem po poprawce: failed, „undecryptable dir name detected” ✅
repoint pada przed trap (0 pasm widocznych) ❌ przed poprawką smbd zostawał zatrzymany; po poprawce smbd aktywny, vm.env bez zmian ✅

Poprawki (f57b104)

  • Ostatni sync i copy najpierw listują korzeń crypt z --crypt-strict-names (folder z innym kluczem = błąd); sync wymaga tm/ i tm-meta/; copy najpierw robi mkdir korzenia (pierwsza wersja tej poprawki w kontenerze psuła każdy pierwszy copy, bo lsf pada na nieistniejący folder).
  • copy/switch/rollback porównują sekcję gdrive-crypt maszyny (hasło, sól, folder) z tą na Macu; tylko to wykrywa klucz zły od pierwszego copy. Przy błędzie listingu komunikat mówi, co porównać albo co usunąć.
  • repointScript: trap 'systemctl start smbd' ERR od pierwszej linii; move startuje smbd po każdym nieudanym repoincie (także bez odpowiedzi z limactl).
  • Testy: kolejność sprawdzeń przed sync, porównanie kluczy, bash -n wszystkich skryptów gościa. swift test: 329/329.

Zostaje do próby na cmtest

  • Pęk kluczy (cloudmachine-drive-crypt) i rclone.conf Maca są wspólne dla wszystkich instancji. setup na cmtest bez RCLONE_CONFIG dopisze sekcję do prawdziwego pliku i zablokuje późniejszy prawdziwy setup; próba powtórzona po prawdziwym setup nadpisze w pęku kluczy prawdziwe hasło i sól. Kod tego nie wymusza (plan to tylko zaleca) — do decyzji, czy dodać odmowę albo osobne konto w pęku kluczy dla instancji innej niż cmsmb.
  • Nie przetestowane tutaj: cloudmachine-agent end-to-end (getpass, pęk kluczy, limactl copy), tmutil, bramki delete-plain na żywo, prawdziwy Drive (--drive-stop-on-upload-limit, kosz), Time Machine na udziale, fsck_apfs.

🤖 Generated with Claude Code

mbeczynski and others added 4 commits November 4, 2026 00:26
Decision 04.11: a machine other than cmsmb never writes the Mac's
rclone.conf or the live Keychain entry. Every vm-encrypt step there
refuses unless RCLONE_CONFIG names a file other than
~/.config/rclone/rclone.conf (compared after standardizing the path and
resolving links), and says what to set. Its password and salt go to its
own Keychain service, cloudmachine-drive-crypt-<machine>, so a repeated
trial cannot overwrite the live password and salt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AU1Mhxpy65J75YYTkmt7NF
The 58 vm-encrypt strings in the 20 languages added by #42, one table per
language (L10nTranslations+Encryption-<code>.swift) merged into the main
one, with each language's existing terms (Keychain, Full Disk Access, the
machine, the buffer guard). L10nTests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AU1Mhxpy65J75YYTkmt7NF
@mbeczynski

Copy link
Copy Markdown
Contributor Author

Decyzja 04.11: maszyna jednorazowa nie dotyka konfiguracji Maca

Zmiana Commit
Na maszynie innej niż cmsmb każdy krok vm-encrypt odmawia bez RCLONE_CONFIG wskazującego plik inny niż ~/.config/rclone/rclone.conf (ścieżka ujednolicona, dowiązania rozwiązane); komunikat mówi, co ustawić 5dc5adf
Hasło i sól maszyny jednorazowej trafiają do osobnej usługi w pęku kluczy: cloudmachine-drive-crypt-<maszyna>. Prawdziwego wpisu cloudmachine-drive-crypt nie da się nadpisać próbą 5dc5adf
Testy: odmowa bez RCLONE_CONFIG, przy pustym, przy domyślnym i przy domyślnym zapisanym inaczej; nazwy usług w pęku kluczy. Kanarek: osłabiona odmowa daje czerwone testy 5dc5adf
Scalenie origin/main (bez force-push) 69f15ef
58 napisów vm-encrypt w 20 językach z #42 (osobne tabele L10nTranslations+Encryption-<kod>.swift, terminy jak w istniejących tabelach) e6c1b9c

swift test 340/340, swift-format lint --strict czysty, CI (push i pull_request) zielone.

Na próbie na cmtest ustaw: CLOUDMACHINE_VM_INSTANCE=cmtest, CLOUDMACHINE_VM_REMOTE=gdrive:CloudMachine/cmtest-tm, RCLONE_CONFIG=<kopia rclone.conf>. Nadal DO NOT MERGE przed tą próbą.

🤖 Generated with Claude Code

mbeczynski and others added 2 commits November 4, 2026 08:32
An independent review against the existing tables: the same word for an
rclone remote as elsewhere (ar, he, th), the Mac as the Arabic table
writes it, template and pause as in the Thai table, "move the machine to
another folder" instead of "swap the machine" (th), and wording a reader
would stumble on (ar: enable vs restart, disable vs stop; he: the guard
sentence, "next step").

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AU1Mhxpy65J75YYTkmt7NF
@mbeczynski mbeczynski changed the title feat(vm): vm-encrypt - move the backup on Drive under rclone crypt (DO NOT MERGE before a throwaway-machine test) feat(vm): vm-encrypt - move the backup on Drive under rclone crypt Nov 4, 2026
@mbeczynski
mbeczynski marked this pull request as ready for review November 4, 2026 10:14
@mbeczynski
mbeczynski merged commit 0325c11 into main Nov 4, 2026
1 check passed
@mbeczynski
mbeczynski deleted the feat/drive-encryption branch November 4, 2026 10:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant