Skip to content

[vm-repair] Fix ADE OS disk unlock: select root and boot partitions by role instead of size - #10198

Open
Mauricio (msaenzbosupport) wants to merge 1 commit into
Azure:mainfrom
msaenzbosupport:vm-repair-ade-partition-role-selection
Open

[vm-repair] Fix ADE OS disk unlock: select root and boot partitions by role instead of size#10198
Mauricio (msaenzbosupport) wants to merge 1 commit into
Azure:mainfrom
msaenzbosupport:vm-repair-ade-partition-role-selection

Conversation

@msaenzbosupport

@msaenzbosupport Mauricio (msaenzbosupport) commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

🤖 PR Validation — ️✔️ All clear

Breaking Changes
️✔️ None

Description

az vm repair create --unlock-encrypted-vm fails on ADE single-pass Linux VMs whose OS disk contains a partition larger than the root partition.

INFO: Unlocking attached copied disk...
WARNING: Encryption unlock script error was generated:
Cannot use device /dev/sda3 which is in use (already mapped or mounted).
ERROR: Unexpected error occured while unlocking encrypted disk.
ERROR: An unexpected error occurred. Try running again with the --debug flag to debug.
INFO: Cleaning up resources by deleting repair resource group 'repair-ubu2204adeext-...'
{
  "error_message": "Unexpected error occured while unlocking encrypted disk.",
  "message": "An unexpected error occurred. Try running again with the --debug flag to debug.",
  "status": "ERROR"
}

Repro layout (Ubuntu 22.04, ADE single pass, detached LUKS header): sda1 root 126.6G (encrypted), sda2 /boot 256M ext2, sda3 129G ext4 mounted at /512G, sda14 BIOS boot, sda15 EFI.

Root cause

linux-mount-encrypted-disk.sh picks the root partition by size:

root_part=$(lsblk "${data_disk}" -l -n -p -b | grep -w -v "${data_disk}" | sort -k4 -rn | awk 'NR==1{print $1}')

"root is always the largest partition" does not hold. sda3 (129G) wins over sda1 (126.6G), and cryptsetup luksOpen is then run against a partition that locate_mount_data_boot has already mounted under /tmp, hence the "already mapped or mounted" error.

Two related issues in the same code path:

  • LVM is detected with fdisk -l | grep -i lvm, which reads the partition table type code. Under ADE the LVM lives inside the LUKS container, so the partition type never reports LVM.
  • The /boot partition is located by mounting every partition read-write under /tmp and running find /tmp -name osluksheader. That replays the journal of a filesystem belonging to a VM that already failed to boot, and find /tmp also scans the repair VM's own /tmp.

Note on the repro layout

We are aware that an extra data partition on the OS disk is not part of any Azure Marketplace image, and that this is a customer-customized layout. The point of the fix is not that specific layout, though: the assumption "root is the largest partition" has no ground truth behind it, so the script is guessing where it could be checking. Any of these produce a wrong pick with the current code, and none of them require an exotic disk:

  • an oversized /boot — a plain marketplace-derived Ubuntu 22.04 with /boot grown to 36G and root at 31G already selects /boot as the root;
  • a customer-added partition on the OS disk, as in the repro above;
  • any future image whose partition ordering or sizing changes.

Selecting by role and confirming the choice by actually opening the device costs a handful of lsblk/blkid calls and removes the guess entirely, so it is the safer behaviour even on stock images where the current heuristic happens to work.

Fix

Classify partitions by role, then verify instead of guessing:

  • Skip BIOS boot (21686148-6449-6e6f-744e-656564454649) and EFI (c12a7328-f81f-11d2-ba4b-00a0c93ec93b) by GPT type GUID, with a blkid fallback for older lsblk that lacks PARTTYPE.
  • A partition with a readable Linux filesystem cannot be the ADE root (the detached header leaves no signature on the encrypted partition) so it is a /boot candidate. A partition with empty FSTYPE or crypto_LUKS is a root candidate.
  • /boot is the candidate that actually contains luks/osluksheader, probed read-only (ro,noload then ro,norecovery then ro) one at a time. No more mass read-write mounts and no find /tmp.
  • The root is identified by opening it: candidates are tried in turn and accepted only if the decrypted device is a filesystem or an LVM PV, otherwise the mapper is closed and the next candidate is tried.
  • LVM is now detected from that decrypted content instead of from the partition type, so the existing mount_lvm path is reached exactly as before.
  • Error paths use exit 1 instead of bare exit, which returned status 0 and let the caller report success after mounting nothing.

No parameter, API or dependency changes. mount_lvm, check_local_lvm, mount_cmd and install_required_packages are untouched.

Testing

Verified end to end with the patched script installed in the local extension, running az vm repair create --unlock-encrypted-vm on its own:

Scenario Before After
Ubuntu 22.04, extra partition larger than root (sda3 129G > sda1 126.6G) Cannot use device /dev/sda3 which is in use root sda1 unlocked, /investigateroot on osencrypt, /investigateroot/boot on sda2
Ubuntu 22.04, extra partition smaller than root OK OK, no regression
RHEL 9.4 Gen1, LVM on crypt, two partitions with blank FSTYPE OK OK, sdb4 decrypts to LVM2_member, rootvg activated, all 5 LVs mounted

Log from the fixed run on the failing layout:

/dev/sda14 skipped, BIOS boot or EFI system partition
/dev/sda15 skipped, BIOS boot or EFI system partition
Boot candidates on the data drive:  /dev/sda2 /dev/sda3
Root candidates on the data drive:  /dev/sda1
Probing /dev/sda2 for luks/osluksheader
The boot partition on the data disk is /dev/sda2
unlocking root with command: cryptsetup luksOpen --key-file /mnt/azure_bek_disk/LinuxPassPhraseFileName --header /investigateboot/luks/osluksheader /dev/sda1 osencrypt
/dev/sda1 opened, decrypted content is 'ext4'
LVM not found on the data disk
The OS partition on the data drive is /dev/sda1
Mounting /dev/mapper/osencrypt on /investigateroot
Mounting the boot partition /dev/sda2 on /investigateroot/boot

Resulting state in the repair VM, matching the guest's own /etc/fstab (UUID=4e109fbd... /, UUID=da920a35... /boot, UUID=6e0aaa26... /512G):

/investigateroot        /dev/mapper/osencrypt  ext4
/investigateroot/boot   /dev/sda2              ext2

sda3 is left untouched, and no partition remains mounted under /tmp.

LVM run:

/dev/sdb4 opened, decrypted content is 'LVM2_member'
LVM found on the data disk
  VG     #PV #LV #SN Attr   VSize  VFree
  rootvg   1   5   0 wz--n- 63.31g 40.31g
/investigateroot        /dev/mapper/rootvg-rootlv xfs
|-/investigateroot/boot /dev/sdb2                 xfs
|-/investigateroot/var  /dev/mapper/rootvg-varlv  xfs
|-/investigateroot/home /dev/mapper/rootvg-homelv xfs
|-/investigateroot/usr  /dev/mapper/rootvg-usrlv  xfs
`-/investigateroot/tmp  /dev/mapper/rootvg-tmplv  xfs

Known issues not addressed here

Kept out of scope to limit the blast radius of this change:

  • mount_lvm hardcodes rootvg and the LV set rootlv/varlv/homelv/usrlv/tmplv; layouts with other names or an extra LV are not fully mounted.
  • mount_cmd's probe (mount -o nouuid with no device or mountpoint) validates nothing and always logs Trapped error code 1.
  • check_local_lvm renames the repair VM's own VG instead of importing the source VG with vgimportclone, and only when exactly one VG is present.

Copilot AI lite review requested due to automatic review settings August 10, 2026 15:47
@azure-client-tools-bot-prd

Copy link
Copy Markdown

Hi Mauricio (@msaenzbosupport),
Please write the description of changes which can be perceived by customers into HISTORY.rst.
If you want to release a new extension version, please update the version in pyproject.toml (or setup.py, if the extension has not migrated yet) as well.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@microsoft-github-policy-service microsoft-github-policy-service Bot added the customer-reported Issues that are reported by GitHub users external to the Azure organization. label Aug 10, 2026
@microsoft-github-policy-service

Copy link
Copy Markdown
Contributor

Thank you for your contribution Mauricio (@msaenzbosupport)! We will review the pull request and get back to you soon.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes az vm repair create --unlock-encrypted-vm for ADE-encrypted Linux VMs by changing the unlock/mount script to identify /boot and the encrypted root partition by partition role/signature (and verifying by actually opening LUKS), instead of guessing the root partition by “largest size”.

Changes:

  • Update linux-mount-encrypted-disk.sh to classify boot/root candidates by GPT type GUID + filesystem signature, probe /boot read-only for the detached LUKS header, and verify the root by attempting cryptsetup luksOpen and checking decrypted content.
  • Fix failure-path exit codes (use exit 1) to avoid reporting success after failures.
  • Bump extension version to 2.2.4 and document the fix in release notes.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
src/vm-repair/azext_vm_repair/scripts/linux-mount-encrypted-disk.sh Reworks boot/root detection and unlock verification; improves error-path exit codes.
src/vm-repair/setup.py Bumps extension version to 2.2.4.
src/vm-repair/HISTORY.rst Adds 2.2.4 release note describing the ADE unlock fix.
Suppressed comments (1)

src/vm-repair/azext_vm_repair/scripts/linux-mount-encrypted-disk.sh:57

  • The -z check is unquoted ([ -z ${data_disk} ]). If data_disk is empty/unset, this expands to [ -z ] and can error out instead of entering the intended failure branch. Quote the variable in the test.
	if [ -z ${data_disk} ]
	then
		echo "`date` OS disk attached as data disk was not found, cannot continue" >> ${logpath}/${logfile}
		exit 1

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@@ -31,7 +31,7 @@ locatebekvol () {
if [ -z ${bekdisk} ]
Comment on lines +203 to +204
echo "`date` unlocking root with command: cryptsetup luksOpen --key-file /mnt/azure_bek_disk/LinuxPassPhraseFileName --header /investigateboot/luks/osluksheader ${part} osencrypt" >> ${logpath}/${logfile}
cryptsetup luksOpen --key-file /mnt/azure_bek_disk/LinuxPassPhraseFileName --header /investigateboot/luks/osluksheader ${part} osencrypt >> ${logpath}/${logfile} 2>&1 || continue
@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 2 pipeline(s).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

act-observability-squad Auto-Assign Auto assign by bot Compute customer-reported Issues that are reported by GitHub users external to the Azure organization.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants