Skip to content

UKI Addons Support - #2448

Open
Johan-Liebert1 wants to merge 19 commits into
bootc-dev:mainfrom
Johan-Liebert1:uki-addon-full-support
Open

Johan-Liebert1 wants to merge 19 commits into
bootc-dev:mainfrom
Johan-Liebert1:uki-addon-full-support

Conversation

@Johan-Liebert1

Copy link
Copy Markdown
Member

uki-addon: Update addons on update/switch

Introduce a function to gather all currently installed addons, scoped
and global. On upgrade/switch, gather all installed addons and if an
addon with the same name is found in the upgrade image, update that
particular addon automatically


uki/addon: Support global addons

We were partially supporting global addons, but they were lumped in with
scoped/local addons. Add a new cli option to composefs installs called
--global-uki-addon which would determine which global addon to
install.


cfs/upgrade/switch: Handle UKI Addons

Here is what we do now with UKI Addons

  • Accept --uki-addon and --global-uki-addon cli options for bootc
    switch/upgrade commands

  • If we find an installed addon with the same name as the one in the new
    image, we update it


global-uki-addons: Add prefix to name

Prefix global addon filenames with the bootc identifier in the ESP so
we can distinguish bootc-managed global addons from third-party ones


uki-addon: Add CLI for managing UKI Addons

Add bootc uki-addon subcommand with three operations:

  • bootc uki-addon list: List installed UKI addons
    Supports --json for JSON output

  • bootc uki-addon add <name> <global|scoped>: Install an addon from
    the booted image onto the ESP

  • bootc uki-addon remove <name> [deployment_id]: Remove an addon

Add Display and Serialize to UkiAddonType/UkiAddonsList

@github-actions github-actions Bot added area/install Issues related to `bootc install` area/documentation Updates to the documentation labels Sep 10, 2026
@Johan-Liebert1
Johan-Liebert1 force-pushed the uki-addon-full-support branch 2 times, most recently from 2579e09 to e31ce45 Compare September 11, 2026 07:44
@supakeen

Copy link
Copy Markdown
Contributor

Can we install UKI addons as part of a deployment with bootc install? These are (sometimes) used to have addons for mount options so the UKI isn't tightly coupled to a specific disk layout.

Comment thread crates/lib/src/bootc_composefs/boot.rs
Comment thread crates/lib/src/bootc_composefs/boot.rs
Comment thread crates/lib/src/bootc_composefs/uki_addon.rs Outdated
Comment thread crates/lib/src/bootc_composefs/uki_addon.rs Outdated
Comment thread crates/lib/src/cli.rs
pub(crate) enum UkiAddonCliOpts {
/// List all installed UKI Addons
List {
/// Output in JSON format

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc #522

Comment thread docs/src/man/bootc-uki-addon-add.8.md
Comment thread docs/src/experimental-composefs.md Outdated
Comment thread crates/lib/src/install.rs
@cgwalters

Copy link
Copy Markdown
Collaborator

Can we install UKI addons as part of a deployment with bootc install?

That's in the docs here right?

@cgwalters

Copy link
Copy Markdown
Collaborator

Or well actually, I think what you may be asking about is external UKI addons (i.e. not included in the container image), in which case it would totally make sense to do.

@supakeen

Copy link
Copy Markdown
Contributor

Or well actually, I think what you may be asking about is external UKI addons (i.e. not included in the container image), in which case it would totally make sense to do.

Yea I wasn't super clear but that was what I was thinking about 🙂

@Johan-Liebert1
Johan-Liebert1 force-pushed the uki-addon-full-support branch 3 times, most recently from 3422e57 to 4b63be2 Compare September 15, 2026 12:58

@cgwalters cgwalters left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about bootc install --included-uki-addon foo --external-uki-addon /path/to/external.efi i.e. we always disambiguate between internal and external?

Comment thread docs/src/experimental-composefs.md Outdated
Comment thread docs/src/bootc-experimental-composefs.7.md Outdated
@Johan-Liebert1

Copy link
Copy Markdown
Member Author

How about bootc install --included-uki-addon foo --external-uki-addon /path/to/external.efi i.e. we always disambiguate between internal and external?

I don't understand what we mean by external here? I'm guessing it doesn't mean "global" addons

@cgwalters

Copy link
Copy Markdown
Collaborator

external ➡️ #2448 (comment)

@Johan-Liebert1
Johan-Liebert1 force-pushed the uki-addon-full-support branch 2 times, most recently from bcafaf3 to 77db443 Compare September 16, 2026 02:28
@Johan-Liebert1
Johan-Liebert1 force-pushed the uki-addon-full-support branch 2 times, most recently from b3b9d76 to 3ca9715 Compare September 16, 2026 11:40
@cgwalters-bot cgwalters-bot moved this to Todo in Workstream Sep 23, 2026
@cgwalters-bot cgwalters-bot moved this from Todo to In Review in Workstream Sep 23, 2026
@cgwalters-bot cgwalters-bot moved this from In Review to Draft in Workstream Sep 23, 2026
@Johan-Liebert1
Johan-Liebert1 force-pushed the uki-addon-full-support branch 2 times, most recently from c11883b to ec66b3d Compare September 25, 2026 09:12

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Backend auto-detection, stale addon cleanup, legacy migration, and silently ignored option paths introduce blocking correctness regressions.

14 open findings
What changed in this PR

Adds composefs UKI addon lifecycle management across installation, upgrades, garbage collection, CLI operations, documentation, and integration testing.

Changes:

  • Adds scoped/global addon installation and automatic upgrade handling.
  • Adds bootc uki-addon management commands and JSON output.
  • Adds addon-aware garbage collection, documentation, and integration tests.
File Description
tmt/​tests/​tests.fmf Registers addon integration test.
tmt/​tests/​booted/​test-composefs-uki-addons.nu Tests addon lifecycle and CLI.
tmt/​tests/​booted/​test-composefs-gc-uki.nu Extends UKI GC coverage.
tmt/​tests/​booted/​tap.nu Supports building test addons.
tmt/​plans/​integration.fmf Adds addon test plan.
docs/​src/​SUMMARY.md Links addon manuals.
docs/​src/​man/​bootc.8.md Documents addon subcommand.
docs/​src/​man/​bootc-upgrade.8.md Documents upgrade options.
docs/​src/​man/​bootc-uki-addon.8.md Adds command overview.
docs/​src/​man/​bootc-uki-addon-remove.8.md Documents removal.
docs/​src/​man/​bootc-uki-addon-list.8.md Documents listing.
docs/​src/​man/​bootc-uki-addon-list-referenced.8.md Documents referenced-addon listing.
docs/​src/​man/​bootc-uki-addon-add.8.md Documents addition.
docs/​src/​man/​bootc-switch.8.md Documents switch options.
docs/​src/​man/​bootc-install-to-filesystem.8.md Documents install options.
docs/​src/​man/​bootc-install-to-existing-root.8.md Documents existing-root options.
docs/​src/​man/​bootc-install-to-disk.8.md Documents disk-install options.
crates/​lib/​src/​install.rs Adds scoped/global install options.
crates/​lib/​src/​composefs_consts.rs Clarifies addon naming.
crates/​lib/​src/​cli.rs Adds CLI commands and options.
crates/​lib/​src/​bootc_composefs/​update.rs Passes addons into upgrades.
crates/​lib/​src/​bootc_composefs/​uki_addons_cli.rs Implements addon CLI operations.
crates/​lib/​src/​bootc_composefs/​uki_addon.rs Implements addon discovery.
crates/​lib/​src/​bootc_composefs/​switch.rs Passes switch addon options.
crates/​lib/​src/​bootc_composefs/​mod.rs Exposes addon modules.
crates/​lib/​src/​bootc_composefs/​gc.rs Collects unreferenced global addons.
crates/​lib/​src/​bootc_composefs/​boot.rs Selects, validates, and writes addons.
contrib/​packaging/​finalize-uki Copies generated addons into images.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread contrib/packaging/finalize-uki
Comment thread crates/lib/src/bootc_composefs/uki_addon.rs Outdated
Comment on lines 338 to 342
let (boot_digest, deploy_id) = match boot_type {
BootType::Bls => (
setup_composefs_bls_boot(
BootSetupType::Upgrade((storage, booted_cfs, &host)),
BootSetupType::Upgrade((storage, booted_cfs, &host, None)),
&repo,
Comment thread crates/lib/src/cli.rs
Comment thread crates/lib/src/cli.rs
Comment thread crates/lib/src/cli.rs Outdated
Comment thread crates/lib/src/install.rs Outdated
Comment thread docs/src/man/bootc-uki-addon.8.md Outdated
Comment thread docs/src/man/bootc-uki-addon.8.md Outdated
Comment thread docs/src/man/bootc-uki-addon.8.md

@jmarrero-bot jmarrero-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are findings from @jmarrero's review of 2aea32e that aren't already raised in existing threads. Inline comments carry their severity; additions to existing threads are posted as replies there. The rest, which isn't on diff lines:

Tests

  • No unit tests for pure logic. Besides gather_addons_from_dir (already asked in the uki_addon.rs:177 thread), good table-driven candidates are parse_uki_cmdline (generic over Read + Seek), the get_*_addon_name helpers, the unreferenced-global-addon calculation (inline in composefs_gc; extract it), and the CLI and carry-over selection (inline in setup_composefs_uki_boot).

Docs

  • docs/src/bootc-composefs.7.md:126 still says addons "are only installed by bootc install: they aren't updated on upgrade, garbage collected, or reverted on rollback". This PR changes the first two.
  • The conceptual docs were lost: 86b4d23 "Add docs for UKI Addons" now contains only a comment tweak, and the PR no longer touches any docs besides the man pages and SUMMARY.md. The how-to (building addons, ESP layout) is gone, and so is the Secure Boot signing note requested in #2448 (comment).

Commit hygiene

  • db64bbf fixes a clap panic introduced in c429341 in this same PR; please squash it. Its message also says the options are ignored on ostree, which 2aea32e then reverses.
  • 6ad99fa "Remove addon dir if empty" has no body and is a follow-up to 87b533f; please squash it.
  • 86b4d23 "Add docs for UKI Addons" is misnamed (see Docs) and has no body.
  • 01d573e "tmt: Add tests" also changes the shipped contrib/packaging/finalize-uki without mentioning it, and has no body.
  • 2aea32e bundles three unrelated changes: the ostree bail, a behavior change (a missing addon becomes an error), and the SUMMARY.md man page links. Please split it.
  • f9ee463 is not rustfmt-clean on its own (match setup_type and { split across lines in boot.rs; later commits fix it).

Generated-by: https://github.com/jmarrero-bot#llms

Comment thread crates/lib/src/bootc_composefs/gc.rs Outdated
..
} = uki_info;

for entry in entries_to_write {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Must fix. Staging an upgrade rewrites the global addons the booted deployment uses.

On upgrade/switch every installed global addon is carried forward (line 1906, UkiAddonType::Global => true). Any of them the new image ships are then overwritten in place in the shared loader/addons/ here, at stage time, before the boot entry and deployment state are written.

Scenario: A is booted with global addon foo v1, and image B ships foo v2. bootc upgrade --download-only to B replaces bootc_composefs-foo.addon.efi immediately. If the user reboots without applying B, A boots with foo v2. The same happens if the staged deployment is discarded, or if write_systemd_uki_config / write_composefs_state fails after this loop.

The TODO on GLOBAL_UKI_ADDONS_DIR covers rollback but not this. At minimum the global addon write should not happen for a deployment that may never be applied. Ideally globals are written at finalize time, or changing global addons only happens through an explicit bootc uki-addon add.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Staging an upgrade rewrites the global addons the booted deployment uses.

yeah, that is the very nature of global UKI Addons. We have two options here

  1. Have a .staged entry per addon to be updated, examples loader/addons/bootc-composefs_global.efi.addon.staged which we would rename exchange on finalize. This does require more syscalls depending on the number of addons being updated which means more points of failures

  2. Have loader/addons.staged similar to what we do with BLS config directory. This will require a copy of all existing UKI addons but is effectively less syscalls and less points of failures, but requires extra disk space

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or changing global addons only happens through an explicit bootc uki-addon add.

this will require at least two reboots to take effect, so not a good plan imo

Comment thread crates/lib/src/bootc_composefs/gc.rs Outdated
}
// We only prefix global UKI Addons for identification
UkiAddonType::Global => {
tracing::info!("Global UKI Addon not managed by bootc found: {addon_name}")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs a design decision. Compatibility with global addons installed by released versions.

v1.16.11 through v1.17.1 already install global addons through --uki-addon. That flag matched scoped and global addons by name, and wrote global ones without the bootc_composefs- prefix. After this PR:

  • Existing unprefixed global addons count as third-party (here), so they are never updated or GC'd.
  • --global-uki-addon foo writes bootc_composefs-foo.addon.efi next to the old foo.addon.efi. systemd-stub loads both, so the kernel arguments are duplicated or conflict.
  • --uki-addon foo, where foo is only a global addon, now fails with "Requested scoped UKI addon 'foo' was not found in the image" (boot.rs:1971), which breaks existing install scripts.

The feature is experimental, so a break may be acceptable. It still needs either a migration (adopt unprefixed files that match an image's global addons) or an explicit release note.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jmarrero wdyt about this? It is true that now global addon installations explicitly require --global-uki-addon cli opt instead of just --uki-addon


// An addon requested explicitly on the CLI must exist in the image; bail
// immediately if one is missing, before we touch the ESP. Addons carried
// forward on upgrade/switch may be absent from the new image, which is fine.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should fix. Carried-over scoped addons that are missing from the new image are dropped without a message. At least log at info level so the admin can see an addon disappeared.

.reopen_as_ownedfd()
.context("Reopening as owned fd")?,
)
.context("fsync")?;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: bootc uki-addon add prints nothing on success, while list and remove do.

}

let final_pe_path = pe_output_dir(&pe_type, mounted_efi.as_ref(), file_path, uki_id);
validate_uki_candidates(&composefs_candidates, boot_ids, Some(uki_name.into()))?;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: when composefs= came from a scoped addon, uki_name here is the addon's file name, so print_uki_dumpfile_diff (line 210) looks for <addon>.addon.dump instead of ${kver}.dump and silently skips the diff.

bootc switch --transport containers-storage --global-uki-addon global-cfs-cmdline localhost/bootc-uki-addons-2
} | complete)

assert ($result_global.exit_code != 0) "Global addon with composefs= should be rejected"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests. The negative tests only check exit_code != 0 (here, and lines 128, 134, 212). "Duplicate composefs=" can't be told apart from a digest mismatch or a broken test image. The --addon-cmds heredoc also has no set -e, so a failing bootc compute-composefs-digest still yields a failing switch. Please match on the error message.

"

let cmds = "
ukify build --cmdline 'johan=liebert' --output /out/${kver}.efi.extra.d/monster-cmdline.addon.efi

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests. Unlike the later boots, this writes into ${kver}.efi.extra.d without mkdir -p. If ukify doesn't create the directory, no scoped addon is built (the heredoc has no set -e) and the "no addons included" check in second_boot passes trivially.

assert ($failed.exit_code != 0)

# Adding an addon that carries a composefs= cmdline should be rejected.
let cfs_add = (do { bootc uki-addon add local-cfs-cmdline scoped } | complete)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests. A successful bootc uki-addon add ... scoped is never tested; only global add and this rejected scoped add are.

@jmarrero jmarrero left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a few comments, and also passed down the review I did with Claude to my bot to double check before posting. @Johan-Liebert1 I am happy to work on these together in a meeting to iterate quickly.

Introduce a function to gather all currently installed addons, scoped
and global. On upgrade/switch, gather all installed addons and if an
addon with the same name is found in the upgrade image, update that
particular addon automatically

Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
We were partially supporting global addons, but they were lumped in with
scoped/local addons. Add a new cli option to composefs installs called
`--global-uki-addon` which would determine which global addon to
install.

Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
Here is what we do now with UKI Addons

- Accept `--uki-addon` and `--global-uki-addon` cli options for bootc
  switch/upgrade commands

- If we find an installed addon with the same name as the one in the new
  image, we update it

Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
Assisted-by: AI

Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
Prefix global addon filenames with the bootc identifier in the ESP so
we can distinguish bootc-managed global addons from third-party ones

Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
Add `bootc uki-addon` subcommand with three operations:

- `bootc uki-addon list`: List installed UKI addons
  Supports `--json` for JSON output

- `bootc uki-addon add <name> <global|scoped>`: Install an addon from
  the booted image onto the ESP

- `bootc uki-addon remove <name> [deployment_id]`: Remove an addon

Add Display and Serialize to UkiAddonType/UkiAddonsList

Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
- Add GC tests for Addons
- Add CLI tests

Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
Mark UKI Addons as experimental

Assisted-by: Claude-Code (Opus)
Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
UkiAddonOpts is now flattened into both upgrade and switch commands,
which don't have a --composefs-backend flag. The
`requires = "composefs_backend"` constraint causes a panic
at clap validation time because the referenced argument doesn't exist
in those command contexts.

This is generally safe as the options are ignored for ostree installs
anyway

Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
Extract cmdline parsing from write_pe_to_esp into parse_uki_cmdline so
it runs for both the UKI and UKI addons. The `composefs=` parameter can
now be found in the main UKI or a scoped addon

- At most one composefs= cmdline across all PE binaries (UKI + addons).
  A second one is rejected even if the digest matches.

- Global UKI addons must never contain `composefs=` cmdline

- At least one `composefs=` cmdline must be found or the install fails

Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
Before removing a scoped addon, parse its PE binary and check for a
composefs= kernel parameter. If found, early exit as removing that addon
would make the system unbootable

Global addons are not checked because `composefs=` is rejected at
install time for global addons

Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
Test that bootc rejects composefs= in the wrong places

- Build a global addon containing composefs= from
  `bootc compute-composefs-digest`, attempt switch with
  --global-uki-addon, assert failure

- Build a scoped addon containing composefs= alongside
  the UKI (which already has it), attempt switch with --uki-addon,
  assert failure due to duplicate composefs=

It's a shame that we can't test UKI Addon only cmdline without piling on
a bunch of hacks since `bootc container ukify` unconditionally puts the
cmdline inside the UKI

Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
This option lists all the UKI Addons across all deployments. Also, it
associates global addons with the deployments that reference them. It is
useful for GC-ing global UKI Addons if no deployments refer to them

Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
Similar to how we GC UKIs and scoped UKI Addons, GC Global UKI addons if
we have no EROFS images remaining that hold a reference to them, which
means that the deployments that depended on the Global addons have been
removed

Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
Reject an addon whose cmdline embeds a `composefs=` or
`composefs.digest=` argument before copying it. An addon added
this way must not be able to override the composefs digest.

Write the addon with `atomic_replace_with` and fsync the destination
directory, so a crash mid add can't leave a truncated PE on the
ESP.

Add a tmt test to verify.

Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
`--uki-addon` was a no-op on ostree backend, now we bail if it's passed
during ostree installs/upgrades

Also, bail if we fail to find an addon in the image instead of just
logging as info

Update manpage links

Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
We get referenced global UKI Addons by mounting non-bootable EROFS
images and collecting all the addons referenced. This is a pretty
expensive operation, so now we only do that if:

- The system is booted via UKI, and
- Global Addons are actually installed

Also, update docs on how scoped and global addons are updated

Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
@Johan-Liebert1

Copy link
Copy Markdown
Member Author

@jmarrero A live chat would be helpful

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/documentation Updates to the documentation area/install Issues related to `bootc install`

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants