Queueing-time profiler aborts the whole instrumentation install under a JDK 24+ AOT cache (zero spans); disabling that one feature is enough

[gibKim]

Tracer Version(s)

1.65.1, 1.66.0

Java Version(s)

25

JVM Vendor

Eclipse Adoptium / Temurin

Bug Report

TL;DR: with a JDK 25 AOT cache, the queueing-time profiler's failed retransformation batch silently aborts the entire instrumentation install (zero spans). Disabling that single feature — -Ddd.profiling.queueing.time.enabled=false — restores all spans and keeps every other profiling capability. Fully disabling the profiler is not necessary.

Behavior

With -XX:AOTCache enabled (jib containerizingMode=packaged, Spring Boot, JDK 25):

• The tracer starts normally, prints its configuration, reports agent_error: false.
• Not a single span is produced. No warnings at default log level.
• Only -Ddd.trace.debug=true reveals: Exception while retransforming 574 classes.

Identical behaviour on 1.65.1 and 1.66.0, so not a recent regression.

Root cause

UnwrappingVisitor (queueing-time profiling) adds the TaskWrapper interface to the classes it instruments. Adding an interface is a structural change, which the JVM rejects on retransformation. Normally this works because the agent installs before those classes load — but a JDK 24+ AOT cache (JEP 483) materializes them before premain.

Since no BatchAllocator is configured, that is the single batch holding every class, and RedefinitionStrategy aborts the whole install. One unusable profiler feature takes all instrumentation down with it.

Why this is filed separately from #10479

#10479 covers partial span loss via the context-store weak-map fallback (addressed by #12105). This is a different mechanism with a total, silent loss — and a much narrower fix surface.

Expected Behavior

A retransformation failure of one optional profiling feature should not abort the entire instrumentation install, and the degradation should be visible at default log level.

Workaround

-Ddd.profiling.queueing.time.enabled=false — all spans return, all other profiling features (CPU, allocation, lock, IO, …) keep working.

The failing setup is exactly the one from the docs, with profiling enabled. Training with the explicit -javaagent:dd-java-agent.jar=aot_training argument fails the same way.

-XX:-AOTClassLinking also avoids it, but gives up the startup gain entirely: on our Spring service startup was 20.5s without a cache, 12.8s with it, and 21.0s with -XX:-AOTClassLinking.

Fix

#12506 retries the failed batch without the interface change. Queueing-time profiling itself keeps working — TaskWrapper is only read by QueueTimeEvent.setTask() and is guarded by instanceof, so the events, durations, scheduler, queue type and span ids are all still recorded; only the task field loses its type resolution (wrapper class instead of the real one). JFR table in #12506. The degradation is logged once at info. With that fix the workaround flag is no longer needed.

Reproduction Code

1. Spring Boot app, jib containerizingMode=packaged, JDK 25 (Temurin).
2. Training run: -XX:AOTCacheOutput=app.aot with the agent attached, then run with -XX:AOTCache=app.aot -javaagent:dd-java-agent.jar.
3. Send requests: zero spans. Add -Ddd.profiling.queueing.time.enabled=false: all spans appear (245 spans in our 20-request measurement, identical to no-AOT baseline — table in Fix instrumentation being dropped when a JDK AOT cache is used #12506).

[mcculls]

Thanks for the report @gibKim and the suggested fix - we'll look into this.

[mcculls]

Hi @gibKim btw, do you have a zip of your reproducer you could share?

[gibKim]

Sure, here's a minimal reproducer: dd-aot-queueing-repro.zip

No framework and no Datadog agent needed. A single class submits 20 tasks to a thread pool, and a -Ddd.trace.methods span is counted from LoggingWriter output. ./run.sh builds it, downloads the agent from Maven Central, does the AOT training run and prints the span count per run. It also runs in a container: docker run --rm -v "$PWD":/w -w /w eclipse-temurin:25-jdk ./run.sh

Agent	no AOT	AOT	AOT + queueing.time.enabled=false
1.65.1	20	0	20
1.66.0	20	0	20
#12506 (d1be92e0)	20	20	20

Temurin 25.0.4 arm64, same result on macOS and Linux. AGENT_JAR=/path/to/agent.jar ./run.sh runs it against a local build. On Linux the stock agent logs no WARN or ERROR line on the AOT run; #12506 logs one INFO line (Disabling task unwrapping for queueing time profiling ...).

So Spring Boot isn't needed: anything that loads FutureTask & co. during training is enough.

[mcculls]

Hi @gibKim - I went with a more targeted approach in #12610 and confirmed that your reproducer now passes with that build