next (v5.7.0; do not merge)

.github/workflows/build.yml

@@ -23,7 +23,7 @@ jobs:
   httpclient-tests:
     strategy:
       matrix:
-        httpclient: [ 'vertx', 'jdk', 'jetty' ]
+        httpclient: [ 'vertx-5', 'jdk', 'jetty' ]
     uses: ./.github/workflows/integration-tests.yml
     with:
       java-version: 25

.github/workflows/integration-tests.yml

@@ -12,7 +12,7 @@ on:
       http-client:
         type: string
         required: false
-        default: 'vertx'
+        default: 'vertx-5'
       experimental:
         type: boolean
         required: false

bootstrapper-maven-plugin/pom.xml

@@ -22,7 +22,7 @@
   <parent>
     <groupId>io.javaoperatorsdk</groupId>
     <artifactId>java-operator-sdk</artifactId>
-    <version>5.6.4-SNAPSHOT</version>
+    <version>999-SNAPSHOT</version>
   </parent>
 
   <artifactId>bootstrapper</artifactId>

caffeine-bounded-cache-support/pom.xml

@@ -21,7 +21,7 @@
   <parent>
     <groupId>io.javaoperatorsdk</groupId>
     <artifactId>java-operator-sdk</artifactId>
-    <version>5.6.4-SNAPSHOT</version>
+    <version>999-SNAPSHOT</version>
   </parent>
 
   <artifactId>caffeine-bounded-cache-support</artifactId>

docs/content/en/docs/documentation/dependent-resource-and-workflows/dependent-resources.md

@@ -531,6 +531,34 @@ samples [here](https://github.com/java-operator-sdk/java-operator-sdk/tree/main/
    in [related integration test](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework/src/test/java/io/javaoperatorsdk/operator/workflow/orderedmanageddependent/ConfigMapDependentResource2.java)
    .
 
+### Adding Common Metadata to All Managed Resources (Desired State Aspects)
+
+Operators often need to mark every resource they manage in a uniform way, for example with a
+`app.kubernetes.io/managed-by` label, so that these resources can easily be identified, selected or
+garbage-collected later on. Instead of repeating that logic in every `desired()` implementation, a
+`DesiredStateAspect` can be registered once, at the operator level, and is then applied to the
+desired state of every Kubernetes dependent resource managed by the operator:
+
+```java
+Operator operator = new Operator(overrider -> overrider
+    .withDesiredStateAspects(List.of(
+        (desired, dependentResource, context) -> desired.getMetadata().getLabels()
+            .put("app.kubernetes.io/managed-by", "my-operator"))));
+```
+
+Aspects are applied, in registration order, right after the desired state has been computed and
+before the desired state is matched against the actual resource, created or updated. As a
+consequence, the metadata added by an aspect is part of the desired state proper: if it is removed
+from the actual resource, or if the aspect itself changes, the associated secondary resources are
+updated accordingly on the next reconciliation.
+
+Since the desired state is computed at most once per reconciliation and cached in the `Context`,
+aspects are called at most once per dependent resource and reconciliation. They are only called for
+dependent resources whose desired state is a `HasMetadata`, meaning that external (non-Kubernetes)
+dependent resources are left untouched. Implementations are expected to modify the provided desired
+state in place and need to be thread-safe as they can be called concurrently for different primary
+resources.
+
 ## "Read-only" Dependent Resources vs. Event Source
 
 See Integration test for a read-only

docs/content/en/docs/documentation/eventing.md

@@ -222,6 +222,37 @@ is similar to `PerResourcePollingEventSource` except that, contrary to that even
 doesn't poll a specific API separately per resource, but periodically and independently of
 actually observed primary resources.
 
+#### Threading of the polling event sources
+
+Each polling event source polls on a scheduled executor of its own, so a slow poll of one event
+source never delays the polls of the others. Unless you provide one, the event source creates a
+single daemon thread executor when it is started and shuts it down when it is stopped. With a single
+thread the polls of a source run one after the other. A `PollingEventSource` runs a single periodic
+task, so more threads would not help it. A `PerResourcePollingEventSource` however polls each
+registered resource on its own, so if it has many registered resources or fetches slowly, it can be
+given an executor with more threads using the `withExecutorService` method of its configuration
+builder:
+
+```java
+var config =
+    new PerResourcePollingConfigurationBuilder<>(this::fetchResources, Duration.ofSeconds(5))
+        .withExecutorService(Executors.newScheduledThreadPool(4))
+        .build();
+```
+
+An executor provided that way is not managed by the operator: it is your responsibility to shut it
+down. It is only ever used as is, the virtual threads flag has no effect on it. You can also share
+it between multiple event sources deliberately.
+
+With virtual threads enabled, see [virtual threads](../operations/configuration/#virtual-threads),
+the executor the event source creates runs the polls on virtual threads, only keeping one platform
+thread to do the timing. The number of polls running at the same time is then not limited.
+
+Retried and rescheduled reconciliations are triggered on a separate executor, so a slow poll can
+never delay them. It is sized with `withConcurrentRetryAndRescheduleThreads` (2 threads by
+default); few threads are needed there since triggering a reconciliation only enqueues an event
+for one of the reconciliation threads to pick up.
+
 #### Inbound event sources
 
 [SimpleInboundEventSource](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/inbound/SimpleInboundEventSource.java)
@@ -348,6 +379,30 @@ See
 also [CaffeineBoundedItemStores](https://github.com/operator-framework/java-operator-sdk/blob/main/caffeine-bounded-cache-support/src/main/java/io/javaoperatorsdk/operator/processing/event/source/cache/CaffeineBoundedItemStores.java)
 for more details.
 
+### Removing the Namespace Index
+
+Informers keep an index from namespace to the resources cached for it. JOSDK never reads that index,
+only the ones registered explicitly through `IndexerResourceCache.addIndexers(..)`, so it can be
+removed to save an entry per cached resource:
+
+```java
+@ControllerConfiguration(informer = @Informer(withoutNamespaceIndex = true))
+public class MyReconciler implements Reconciler<MyCustomResource> { }
+```
+
+The same option is available on `InformerEventSourceConfiguration.Builder` for event sources, and on
+`InformerConfiguration.Builder`.
+
+This matters most for informers that cache a large number of resources, and in particular together
+with a custom [item store](#bounded-caches-for-informers) that keeps only a reduced form of each
+resource: the index is keyed independently of what the store does with the resource itself, so
+shrinking what is cached does not shrink the index.
+
+Note that this takes part in the informer pool identity described below: an event source that
+removes the index does not share an informer with one that keeps it. If two event sources watch the
+same resource type and only one of them sets the option, they end up with two informers, and two
+caches of that resource type, which can cost far more memory than the index ever did.
+
 ### Sharing Informers Between Controllers (Informer Pool)
 
 {{% alert title="Experimental" color="warning" %}}
@@ -374,7 +429,10 @@ Two event sources share an informer when their effective informer configuration
 - the resource type (or the group/version/kind for generic resources),
 - the watched namespace,
 - the label, field and shard selectors,
-- the configured [item store](#bounded-caches-for-informers).
+- the configured [item store](#bounded-caches-for-informers),
+- whether the [namespace index is removed](#removing-the-namespace-index): it cannot be present for
+  one event source and absent for another on one shared informer, so event sources that disagree on
+  it are backed by separate informers.
 
 The `informerListLimit` is intentionally *not* part of this identity: if two otherwise-equivalent
 event sources request a different list limit, the existing informer is reused (a warning is logged

docs/content/en/docs/documentation/operations/configuration.md

@@ -23,6 +23,60 @@ Operator operator = new Operator( override -> override
         .withLeaderElectionConfiguration(new LeaderElectionConfiguration("bar", "barNS")));
 ```
 
+### Virtual Threads
+
+Reconciliation is mostly about blocking: talking to the Kubernetes API server or to external
+systems. Virtual threads make such blocking calls much cheaper than platform threads, and the
+framework can be switched over to them with a single flag:
+
+```java
+Operator operator = new Operator(override -> override.withUseVirtualThreads(true));
+```
+
+When enabled, reconciliations, dependent resource workflows and the framework's internal
+housekeeping (starting the informers, for example) all run on virtual threads.
+
+Enabling virtual threads does **not** remove the concurrency limits, parallelism is configured
+exactly as before: `withConcurrentReconciliationThreads(int)` still caps how many reconciliations
+run at the same time and `withConcurrentWorkflowExecutorThreads(int)` how many dependent resources
+of a workflow are processed concurrently. Only the threads backing those limits change. Since
+virtual threads are cheap, these limits can usually be raised significantly compared to what is
+reasonable with platform threads.
+
+Things to keep in mind:
+
+- Virtual threads are only supported on **Java 25 or later**. Before Java 25 a virtual thread pins
+  its carrier thread while it is inside a `synchronized` block, which can starve the carrier pool
+  and even deadlock the operator on startup. [JEP 491](https://openjdk.org/jeps/491), delivered in
+  Java 25, removed that pinning. This is checked at runtime: on an older JVM, including the ones
+  where virtual threads exist (Java 21 to 24), a warning is logged and platform threads are used
+  instead, so the same configuration works on any Java version the operator runs on.
+- The scheduled tasks (the polls of the polling event sources, which each create an executor of
+  their own unless one is provided, and the retried and rescheduled reconciliations) run on virtual
+  threads too. The JDK has no scheduled executor backed by virtual threads, so the default ones keep
+  a single platform thread for the timing only and run each task they fire on a virtual thread. The
+  number of those tasks running at the same time is not limited, so
+  `withConcurrentRetryAndRescheduleThreads(int)` only applies to platform threads.
+- Virtual threads are always daemon threads, so unlike the threads of the platform thread pools they
+  don't keep the JVM running: if the `main` method of your application returns after starting the
+  operator and nothing else keeps the JVM alive, call `Operator.keepAlive()`, see
+  [Keeping the JVM Alive](#keeping-the-jvm-alive).
+- A custom `ExecutorService` provided through `withExecutorService(...)` or
+  `withWorkflowExecutorService(...)` is always used as is, the flag has no effect on it. The same
+  goes for `withRetryAndRescheduleExecutorService(...)`. Note however that overriding the flag, or
+  `withConcurrentRetryAndRescheduleThreads(int)`, of a configuration that provides its own retry and
+  reschedule executor replaces that executor with the default one, so provide it again with
+  `withRetryAndRescheduleExecutorService(...)` in that case.
+- The Kubernetes client the framework creates when none is provided also switches its internal
+  task executor (used to dispatch informer events to their handlers and to deliver watch events)
+  to virtual threads. A client you provide through `withKubernetesClient(...)` is used as is: to
+  get the same behavior, configure it yourself, e.g. with
+  `new KubernetesClientBuilder().withTaskExecutor(Executors.newVirtualThreadPerTaskExecutor())`
+  (the client doesn't shut down an executor passed that way, use `withTaskExecutorSupplier(...)`
+  if it should be shut down when the client is closed).
+  Either way, the blocking calls your reconciler makes through the client work well on virtual
+  threads without any client-side change.
+
 ## Reconciler-Level Configuration
 
 While reconcilers are typically configured using the `@ControllerConfiguration` annotation, you can also override configuration at runtime when registering the reconciler with the operator. You can either:
@@ -101,6 +155,24 @@ See also
 the [integration test](https://github.com/operator-framework/java-operator-sdk/tree/main/operator-framework/src/test/java/io/javaoperatorsdk/operator/baseapi/changenamespace)
 for this feature.
 
+## Keeping the JVM Alive
+
+The operator normally keeps the JVM running through the non-daemon threads of its thread pools. If
+nothing else does, for example because all the threads of the operator are daemon threads as it is
+the case with virtual threads, the JVM exits as soon as the `main` method returns after starting
+the operator. Call `Operator.keepAlive()` to have a non-daemon thread keep the JVM running until the
+operator is stopped:
+
+```java
+public static void main(String[] args) {
+    Operator operator = new Operator();
+    operator.register(new MyReconciler());
+    operator.installShutdownHook();
+    operator.start();
+    operator.keepAlive();
+}
+```
+
 ## DependentResource-level configuration
 
 It is possible to define custom annotations to configure custom `DependentResource` implementations. In order to provide
@@ -267,6 +339,7 @@ All operator-level keys are prefixed with `josdk.`.
 |---|---|---|
 | `josdk.check-crd` | `Boolean` | Validate CRDs against local model on startup |
 | `josdk.close-client-on-stop` | `Boolean` | Close the Kubernetes client when the operator stops |
+| `josdk.use-virtual-threads` | `Boolean` | Run the framework's concurrent work on virtual threads (only supported on Java 25+, platform threads are used on older versions) |
 | `josdk.use-ssa-to-patch-primary-resource` | `Boolean` | Use Server-Side Apply to patch the primary resource |
 | `josdk.clone-secondary-resources-when-getting-from-cache` | `Boolean` | Clone secondary resources on cache reads |
 
@@ -283,6 +356,12 @@ All operator-level keys are prefixed with `josdk.`.
 |---|---|---|
 | `josdk.workflow.executor-threads` | `Integer` | Thread pool size for workflow execution |
 
+#### Retry and Reschedule
+
+| Key | Type | Description |
+|---|---|---|
+| `josdk.retry-and-reschedule.concurrent-threads` | `Integer` | Thread pool size for triggering retried and rescheduled reconciliations |
+
 #### Informer
 
 | Key | Type | Description |
@@ -300,7 +379,7 @@ All operator-level keys are prefixed with `josdk.`.
 
 | Key | Type | Description |
 |---|---|---|
-| `josdk.events.cluster-scoped-namespace` | `String` | Namespace to record events about cluster scoped resources in (defaults to `default`) |
+| `josdk.events.cluster-scoped-namespace` | `String` | Namespace to record events about cluster scoped resources in (defaults to `default`); the API server only accepts `default` and `kube-system` |
 
 Recording events requires the operator's service account to be allowed `get`, `create` and `patch` on
 `events` in the core (`""`) API group. All three are used: an event that does not exist yet is
@@ -316,6 +395,11 @@ therefore has to grant the permission in every namespace events are recorded in
 `default`, or whatever `josdk.events.cluster-scoped-namespace` is set to, if the operator reconciles
 cluster scoped resources. A `ClusterRole` covers all of them at once.
 
+The API server only accepts events about cluster scoped objects in the `default` and `kube-system`
+namespaces, so those are the only values `josdk.events.cluster-scoped-namespace` can usefully be set
+to. In particular, the namespace the operator itself runs in is rejected, and the event dropped with
+a warning in the operator log.
+
 #### Leader Election
 
 Leader election is activated when at least one `josdk.leader-election.*` key is present.
@@ -385,6 +469,7 @@ josdk.controller.mycontroller.namespaces=team-a,team-b
 | `josdk.controller.<name>.informer.shard-selector` | `String` | Shard selector for the primary resource informer (alias for `shard-selector`) |
 | `josdk.controller.<name>.informer.list-limit` | `Long` | Page size for paginated informer list requests; omit for no pagination |
 | `josdk.controller.<name>.informer.comparable-resource-versions` | `Boolean` | Whether the resource versions of the primary resource can be treated as integers and thus compared |
+| `josdk.controller.<name>.informer.without-namespace-index` | `Boolean` | Removes the namespace index the informer maintains; defaults to `false`. See [Removing the Namespace Index](../eventing#removing-the-namespace-index) |
 
 #### Retry