config: Add versioned operator configuration through a CRD by HarshwardhanPatil07 · Pull Request #203 · bootc-dev/bootc-operator

HarshwardhanPatil07 · 2026-09-29T10:19:43Z

Administrators currently need to edit operator workload arguments to change settings and preserve those edits during upgrades. Add an optional, administrator-owned, cluster-scoped BootcOperatorConfig using the versioned node.bootc.dev/v1alpha1 API. The resource may have any valid Kubernetes name, but the controller and daemon require at most one instance at startup.

The API exposes controller insecure-registry policy and image-tag resolution period, plus the daemon fallback status-poll period. Kubernetes validates types and positive whole-second periods. Both processes make one bounded, uncached list request at startup: zero instances use existing defaults, one supplies settings, and multiple instances or API errors stop startup with a contextual error. Explicit CLI flags take precedence, including false and fractional durations. Changes require a manual restart of the affected workload. Both service accounts receive only list permission for this resource.

The bink workflow now installs the CRD, waits for it, creates or updates its local-registry configuration, and only then deploys workloads. Install and Helm manifests include the CRD and permissions without creating or overwriting administrator configuration. Releases publish an optional, editable operator-config.yaml separately from install.yaml. Documentation covers precedence, restarts, revert and upgrade behavior, and Helm CRD upgrade ordering. Live updates, operator status reporting, and feature gates from #181 remain deferred.

Closes #92.

alicefr · 2026-09-29T12:48:05Z

I personally prefer a new CRD over a configmap. Mostly for 2 reasons, first it is versioned, secondly eventually we could use the crd status for reporting the operator status. The second point right now isn't used so the motivation isn't that strong.

I'd love also @cheesesashimi's opinion

HarshwardhanPatil07 · 2026-09-30T13:11:36Z

updated PTAL

alicefr · 2026-10-02T12:48:02Z

+
+// TestOperatorConfig changes shared operator configuration and must remain
+// serial. Run it on a dedicated test cluster, like the other lifecycle tests.
+func TestOperatorConfig(t *testing.T) {


Is this test really necessary, it adds a lot of complexity for little gain imo. Cannot we cover with unit test?

alicefr · 2026-10-02T12:59:52Z

We should also probably generate a cr as part of the release with some good defaults. This is at least how kubevirt is doing it

HarshwardhanPatil07 · 2026-10-06T09:34:51Z

PTAL

alicefr · 2026-10-06T11:44:56Z

+  resources:
+  - bootcoperatorconfigs
+  verbs:
+  - list


isn't get also required?

this is what it ends up doing: Both processes use the shared startup loader, which performs a single List request and reads the complete configuration from its response.

alicefr · 2026-10-06T12:12:29Z

@HarshwardhanPatil07 it looks great many thanks for doing this. Few last comment and this requires a rebase

Parallel E2E tests create daemon pods on other managed nodes. Restrict the membership lookup to its own node so unrelated pods do not cause the exactly-one assertion to time out. Refs: bootc-dev#226 Refs: bootc-dev#203 Assisted-by: AI Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>

Parallel E2E tests create daemon pods on other managed nodes. Restrict the membership lookup to its own node so unrelated pods do not cause the exactly-one assertion to time out. Refs: #226 Refs: #203 Assisted-by: AI Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>

Give administrators a typed cluster-scoped API for controller and daemon settings without editing workloads. Keep naming flexible; startup selection enforces that only one instance is used. Refs: bootc-dev#92 Assisted-by: AI Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>

Prove API-server defaulting and rejection of invalid period, type, and field values before the startup path depends on the resource. Refs: bootc-dev#92 Assisted-by: AI Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>

Preserve existing CLI behavior while allowing administrators to supply defaults through the resource. Explicit false values and fractional CLI durations continue to take precedence. Refs: bootc-dev#92 Assisted-by: AI Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>

Read the administrator resource before informer caches exist. Treat missing CRDs, permission failures, API errors, and multiple instances as actionable startup errors while an absent instance keeps defaults. Refs: bootc-dev#92 Assisted-by: AI Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>

Exercise absent, custom-named, updated, conflicting, and missing-CRD resources through envtest so the loader behavior is covered beyond mocked HTTP responses. Refs: bootc-dev#92 Assisted-by: AI Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>

Make registry policy and tag resolution configurable on controller startup without changing explicit CLI overrides. Grant its service account only the list permission needed to select the singleton. Refs: bootc-dev#92 Assisted-by: AI Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>

Let each daemon load the same administrator-owned resource at startup while preserving explicit CLI overrides. Give the daemon service account the list permission needed for the read. Refs: bootc-dev#92 Assisted-by: AI Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>

Bink needs local registry settings at the controller first startup. Wait for the CRD, create or update the one resource, then deploy; preserve the redeploy restart that refreshes the latest image. Refs: bootc-dev#92 Assisted-by: AI Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>

The release upgrade test temporarily removes current CRDs, which also deletes configuration. Snapshot and restore the optional resource during cleanup and restart workloads so later tests observe the original settings. Refs: bootc-dev#92 Assisted-by: AI Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>

Give administrators a concrete example and document the single-instance rule, CLI precedence, manual restarts, and upgrade-safe ownership before they adopt the new API. Refs: bootc-dev#92 Assisted-by: AI Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>

Helm does not upgrade CRDs in the chart automatically. Explain how to establish the new CRD before workloads restart and read the optional configuration. Refs: bootc-dev#92 Assisted-by: AI Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>

Provide an editable example without putting an administrator-owned instance into install.yaml, so routine installs and upgrades cannot overwrite configuration. Refs: bootc-dev#92 Assisted-by: AI Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>

Keep the downloadable example aligned with API defaults and ensure install.yaml ships the CRD without creating a configuration instance. Run the checks on every PR. Refs: bootc-dev#92 Assisted-by: AI Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>

Make the versioned API discoverable alongside each release. Validate both assets before publishing and direct administrators to check for an existing instance before applying the example. Refs: bootc-dev#92 Assisted-by: AI Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>

alicefr · 2026-10-12T15:27:20Z

+Logging, leader election, health-probe binding, kubeconfig, and node identity
+use their existing command-line flags or environment variables. Run the
+relevant binary with `--help` for its process options. The stock Deployment
+explicitly enables leader election. Changing the health-probe port requires
+matching changes to its readiness and liveness probes.


Is this relevant?

alicefr · 2026-10-12T15:28:13Z

+existing configuration:
+
+```shell
+kubectl wait --for=condition=Established crd/bootcoperatorconfigs.node.bootc.dev --timeout=1m


Who is setting this condition?

alicefr · 2026-10-12T15:32:17Z

+Both binaries use their existing service account or kubeconfig to read
+configuration directly from the API with a 10-second deadline for that read.
+The controller and daemon service accounts need only `list` access to


Isn't this an implementation detail?

alicefr · 2026-10-12T15:32:59Z

+A running process keeps its loaded settings during a later API outage.
+`--help` does not require Kubernetes API access.
+
+## Configuration flow


Is this whole section relevant for the end-user?

alicefr · 2026-10-12T15:35:34Z

+Reapplying installation manifests during an ordinary upgrade preserves the
+configuration resource. Uninstalling the operator's CRDs deletes their
+instances too. Export the configuration before uninstalling if you intend to
+restore it after reinstalling:
+
+```shell
+kubectl get bootcoperatorconfig cluster -o yaml > operator-config-backup.yaml


when should one use this flow? I would keep this simply and remove this

alicefr · 2026-10-12T15:36:34Z

+Replace `cluster` here too when using another name.
+Restore its `spec` in a fresh resource, omitting server-managed metadata such
+as `uid`, `resourceVersion`, and `managedFields`.
+
+The API currently serves `v1alpha1`. Versioning defines the configuration
+schema; it does not retain configuration history or automatically roll back
+settings. Operator status, live configuration reload, and feature gates are
+not provided by this resource.


This part as well. It isn't relevant for the end users

alicefr · 2026-10-12T15:38:15Z

+
+```bash
+make helm
+kubectl apply -f chart/bootc-operator/crds/


Why is this necessary? Doesn't helm install the crd before the deployment and the daemon set?

alicefr · 2026-10-12T15:40:38Z

I'm still not quite sure about the helm manifest generation, I will give it a try tomorrow. IMO, helm should install the resources in the right order. If not, we might have a bug in our charts

HarshwardhanPatil07 requested review from Johan-Liebert1, alicefr, cheesesashimi, jbtrystram and ptalgulk01 September 29, 2026 10:38

HarshwardhanPatil07 force-pushed the issue-92-configmap-config branch from 3f7612f to abb529e Compare September 30, 2026 09:52

HarshwardhanPatil07 changed the title ~~config: Support operator configuration through an optional ConfigMap~~ config: Add versioned operator configuration through a CRD Sep 30, 2026

HarshwardhanPatil07 force-pushed the issue-92-configmap-config branch 5 times, most recently from 16f3e66 to 19361d9 Compare September 30, 2026 11:24

alicefr reviewed Oct 2, 2026

View reviewed changes

Comment thread api/v1alpha1/bootcoperatorconfig_types.go Outdated

Comment thread internal/config/config.go Outdated

Comment thread Makefile Outdated

alicefr reviewed Oct 2, 2026

View reviewed changes

Comment thread test/e2e/config_test.go Outdated

alicefr reviewed Oct 2, 2026

View reviewed changes

HarshwardhanPatil07 force-pushed the issue-92-configmap-config branch 6 times, most recently from bfbeb93 to e2740b9 Compare October 6, 2026 07:48

alicefr reviewed Oct 6, 2026

View reviewed changes

HarshwardhanPatil07 force-pushed the issue-92-configmap-config branch from e2740b9 to 7a4ebda Compare October 9, 2026 10:15

HarshwardhanPatil07 added 14 commits October 12, 2026 17:14

api: Test configuration defaults and schema validation

f7b6327

Prove API-server defaulting and rejection of invalid period, type, and field values before the startup path depends on the resource. Refs: bootc-dev#92 Assisted-by: AI Signed-off-by: HarshwardhanPatil07 <harshpat@redhat.com>

HarshwardhanPatil07 force-pushed the issue-92-configmap-config branch from 7a4ebda to 898117e Compare October 12, 2026 11:44

alicefr reviewed Oct 12, 2026

View reviewed changes

Conversation

HarshwardhanPatil07 commented Sep 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alicefr commented Sep 29, 2026

Uh oh!

HarshwardhanPatil07 commented Sep 30, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alicefr commented Oct 2, 2026

Uh oh!

HarshwardhanPatil07 commented Oct 6, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

alicefr commented Oct 6, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alicefr commented Oct 12, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

HarshwardhanPatil07 commented Sep 29, 2026 •

edited

Loading