Onboarding ADOT Java to v2.x - with otel v2.10.0 (#975)

*Description of changes:*
Onboarding ADOT Java to v2.x, with the latest
`opentelemetry-java-instrumentation` dependency version `v2.10.0`.
Update the workflows, adot core implementation codes, contract tests and
smoke tests with following change:
**A.  Workflows:**
1. Update `.github/patches/opentelemetry-java-instrumentation.patch`
file with the same code change as previous patch based on upstream
v2.10.0.
2. Upgrade contract testing build env with JDK 23.

**B. Core repo:**
1. Upgrade `com.github.johnrengelman.shadow` to `com.gradleup.shadow` as
it has been marked under maintenance mode:
https://github.com/GradleUp/shadow?tab=readme-ov-file#gradle-shadow
2. Upgrade to new ktlint version 1.4.0, and apply latest spotless
format.
3. Upgrade gradle to 8.10
4. Upgrade `com.diffplug.spotless` to newer version 6.25.0
5. Upgrade `com.google.cloud.tools.jib` to newer version 3.4.4
6. Update the unit test to check the migrated semantic conventions.
7. AWS Resource detector is [set as disabled by
default](open-telemetry/opentelemetry-java-instrumentation#10754),
enable it by setting environment variable
`"otel.resource.providers.aws.enabled"` to be `true`.

**C. Contract tests:**
1. Use the latest `kotlin("jvm")` version to be compatible if the new
Java class from upstream.
2. Add `OTEL_EXPORTER_OTLP_PROTOCOL` to `grpc` for instrumentation as in
[version
v2.0.0](https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/tag/v2.0.0):
> The default OTLP protocol has been changed from grpc to http/protobuf
in orderto align with the specification.
3. Update tomcat contract test `TOMCAT_THREADS` metric threshold to
`-2`, as the value can be negative by [tomcat
design](https://github.com/apache/tomcat/blob/1afe41491f0e56ec0a776db5ff84607f87ce6640/java/org/apache/tomcat/util/net/AbstractEndpoint.java#L1204).
4. Update the test classes to migrate to the new semantic conventions to
match with [upstream latest
change](open-telemetry/opentelemetry-java-instrumentation@7cd705b):
a. All the change made in
#972:
      `http.url` -> `url.full`
      `http.method` -> `http.request.method`
      `http.status_code` -> `http.response.status_code`
      `net.peer.name` -> `server.address`
      `net.peer.port` -> `server.port`
   b. extra semantic conventions included in contract tests:
      `http.scheme` -> `url.scheme`
      `http.target` -> `url.query`
      `http.url` -> `url.full`
      `net.host.name` -> `server.address`
      `net.host.port` -> `server.port`
      `net.peer.name` -> `server.address`
      `net.peer.port` -> `server.port`
      `net.protocol.name` -> `network.protocol.name`
      `net.protocol.version` -> `network.protocol.version`
      `net.sock.host.addr` -> `network.local.address`
      `net.sock.host.port` -> `network.local.port`
      `net.sock.peer.addr` -> `network.peer.address`
      `net.sock.peer.name` -> no replacement, removed
`messaging.kafka.destination.partition` ->
`messaging.destination.partition.id`
`messaging.message.payload_size_bytes` -> `messaging.message.body.size`
Remove `network.protocol.name` attribute check as it has marked as
Conditionally required if not http and network.protocol.version is set:
https://opentelemetry.io/blog/2023/http-conventions-declared-stable/#summary-of-changes.
Conditionally check `peer.service` for http client. Match with PR:
https://github.com/open-telemetry/opentelemetry-java-instrumentation/pull/12083/files
Remove local socket attributes from http server span check as it is not
extracted from HttpServerAttributesExtractor
[code](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/5ebb81b8a8ac0e5b3c9f2e175b847a3d0b12251f/instrumentation-api/src/main/java/io/opentelemetry/instrumentation/api/semconv/http/HttpServerAttributesExtractorBuilder.java#L141),
Remove `http.response.header.content_length` as it need an explicit
configuration:
https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-response-header

**D. Smoke tests**
Enable controller telemetry using env variable
`OTEL_INSTRUMENTATION_COMMON_EXPERIMENTAL_CONTROLLER_TELEMETRY_ENABLED`,
which is disabled by default.

https://opentelemetry.io/docs/zero-code/java/agent/disable/#suppressing-controller-andor-view-spans


**Testing:**
1. All contract tests and smoke tests passed.
2. All workflow passed.
3. Tested all E2E tests (This E2E test PR need to be merged after ADOT
Java v2.x released):
- EC2:
https://github.com/aws-observability/aws-application-signals-test-framework/actions/runs/12378967612
- EKS:
https://github.com/aws-observability/aws-application-signals-test-framework/actions/runs/12382416662
- K8S:
https://github.com/aws-observability/aws-application-signals-test-framework/actions/runs/12378972156
- ECS:
https://github.com/aws-observability/aws-application-signals-test-framework/actions/runs/12378968843

4. Audit all upstream v2.0.0 - v2.10.0 [change
logs](https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases)
and manual did E2E test and compared the traces, logs, metrics for Java
v1.x vs Java v2.x:
 
![Screenshot 2024-12-16 at 3 27
02 PM](https://github.com/user-attachments/assets/52bb9106-b4a1-4223-9df5-74f1f1b10089)


By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.

Author: zzhlogin

.github/patches/opentelemetry-java-instrumentation.patch

@@ -1 +1 @@
-OTEL_JAVA_INSTRUMENTATION_VERSION=v1.33.6
+OTEL_JAVA_INSTRUMENTATION_VERSION=v2.10.0

.github/patches/versions

@@ -25,6 +25,19 @@ jobs:
         with:
           java-version: 17
           distribution: 'temurin'
+        # cache local patch outputs
+      - name: Cache local Maven repository
+        uses: actions/cache@v3
+        with:
+          path: |
+            ~/.m2/repository/io/opentelemetry/
+          key: ${{ runner.os }}-maven-local-${{ hashFiles('.github/patches/opentelemetry-java*.patch') }}
+
+      - name: Publish patched dependencies to maven local
+        uses: ./.github/actions/patch-dependencies
+        with:
+          gpg_private_key: ${{ secrets.GPG_PRIVATE_KEY }}
+          gpg_password: ${{ secrets.GPG_PASSPHRASE }}
       - uses: gradle/wrapper-validation-action@v1
       - name: Configure AWS Credentials
         uses: aws-actions/configure-aws-credentials@v4

.github/workflows/docker-build-smoke-tests-fake-backend.yml

@@ -194,7 +194,7 @@ jobs:
           fetch-depth: 0
       - uses: actions/setup-java@v4
         with:
-          java-version: 21
+          java-version: 23
           distribution: 'temurin'
       - uses: gradle/wrapper-validation-action@v1
 
@@ -219,7 +219,7 @@ jobs:
           key: ${{ runner.os }}-maven-local-${{ hashFiles('.github/patches/opentelemetry-java*.patch') }}
 
       - name: Pull base image of Contract Tests Sample Apps
-        run: docker pull public.ecr.aws/docker/library/amazoncorretto:21-alpine
+        run: docker pull public.ecr.aws/docker/library/amazoncorretto:23-alpine
 
       - name: Build snapshot with Gradle
         uses: gradle/gradle-build-action@v3

.github/workflows/main-build.yml

@@ -32,6 +32,20 @@ jobs:
           java-version: 17
           distribution: 'temurin'
 
+        # cache local patch outputs
+      - name: Cache local Maven repository
+        uses: actions/cache@v3
+        with:
+          path: |
+            ~/.m2/repository/io/opentelemetry/
+          key: ${{ runner.os }}-maven-local-${{ hashFiles('.github/patches/opentelemetry-java*.patch') }}
+
+      - name: Publish patched dependencies to maven local
+        uses: ./.github/actions/patch-dependencies
+        with:
+          gpg_private_key: ${{ secrets.GPG_PRIVATE_KEY }}
+          gpg_password: ${{ secrets.GPG_PASSPHRASE }}
+
       - uses: gradle/wrapper-validation-action@v1
 
       - name: Configure AWS Credentials
@@ -120,7 +134,7 @@ jobs:
           fetch-depth: 0
       - uses: actions/setup-java@v4
         with:
-          java-version: 21
+          java-version: 23
           distribution: 'temurin'
       - uses: gradle/wrapper-validation-action@v1
 
@@ -136,7 +150,7 @@ jobs:
           registry: public.ecr.aws
 
       - name: Pull base image of Contract Tests Sample Apps
-        run: docker pull public.ecr.aws/docker/library/amazoncorretto:21-alpine
+        run: docker pull public.ecr.aws/docker/library/amazoncorretto:23-alpine
 
       - name: Build snapshot with Gradle
         uses: gradle/gradle-build-action@v3

.github/workflows/nightly-upstream-snapshot-build.yml

@@ -12,7 +12,7 @@ on:
 
 env:
   AWS_DEFAULT_REGION: us-east-1
-  TEST_TAG: public.ecr.aws/aws-observability/adot-autoinstrumentation-java:test
+  TEST_TAG: public.ecr.aws/aws-observability/adot-autoinstrumentation-java:test-v2
 
 permissions:
   id-token: write

.github/workflows/patch-release-build.yml

@@ -5,7 +5,7 @@ on:
       - main
       - "release/v*"
 env:
-  TEST_TAG: public.ecr.aws/aws-observability/adot-autoinstrumentation-java:test
+  TEST_TAG: public.ecr.aws/aws-observability/adot-autoinstrumentation-java:test-v2
 
 jobs:
   testpatch:
@@ -63,6 +63,13 @@ jobs:
 
       - uses: gradle/wrapper-validation-action@v1
 
+        # Cleanup directories before proceeding with setup
+      - name: Clean up old installations
+        if: ${{ matrix.os != 'windows-latest' }}
+        run: |
+          sudo rm -rf /usr/local/lib/android
+          sudo rm -rf /usr/share/dotnet
+
         # cache local patch outputs
       - name: Cache local Maven repository
         uses: actions/cache@v3
@@ -84,12 +91,12 @@ jobs:
       - name: Set up Java version for tests
         uses: actions/setup-java@v4
         with:
-          java-version: 21
+          java-version: 23
           distribution: temurin
 
       - name: Pull base image of Contract Tests Sample Apps
         if: ${{ matrix.os == 'ubuntu-latest' }}
-        run: docker pull public.ecr.aws/docker/library/amazoncorretto:21-alpine
+        run: docker pull public.ecr.aws/docker/library/amazoncorretto:23-alpine
 
       - name: Run contract tests
         uses: gradle/gradle-build-action@v3

.github/workflows/pr-build.yml

@@ -9,7 +9,7 @@ on:
 env:
   AWS_PUBLIC_ECR_REGION: us-east-1
   AWS_PRIVATE_ECR_REGION: us-west-2
-  TEST_TAG: public.ecr.aws/aws-observability/adot-autoinstrumentation-java:test
+  TEST_TAG: public.ecr.aws/aws-observability/adot-autoinstrumentation-java:test-v2
   PUBLIC_REPOSITORY: public.ecr.aws/aws-observability/adot-autoinstrumentation-java
   PRIVATE_REPOSITORY: 020628701572.dkr.ecr.us-west-2.amazonaws.com/adot-autoinstrumentation-java
   PRIVATE_REGISTRY: 020628701572.dkr.ecr.us-west-2.amazonaws.com

.github/workflows/release-build.yml

@@ -13,6 +13,11 @@ can be exported in a variety of formats. In addition, the agent and exporter can
 command line arguments or environment variables. The net result is the ability to gather telemetry
 data from a Java application without any code changes.
 
+Note: There are 2.x releases and 1.x releases. The 2.0 release included significant breaking changes from [OpenTelemetry Agent for Java](https://github.com/open-telemetry/opentelemetry-java-instrumentation), 
+the details of which can be found in the [release notes](https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases). 
+It is recommended to use the latest 2.x release which will have the latest features and improvements. 
+1.x will receive security patches for a limited time and will not include other bug fixes and enhancements.
+
 ## Getting Started
 
 Check out the [getting started documentation](https://aws-otel.github.io/docs/getting-started/java-sdk/auto-instr).
@@ -45,4 +50,4 @@ In addition to the sample apps in this repository, there are also a set of [stan
 Please note that as per policy, we're providing support via GitHub on a best effort basis. However, if you have AWS Enterprise Support you can create a ticket and we will provide direct support within the respective SLAs.
 
 ## Security issue notifications
-If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.
+If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.

README.md

@@ -33,7 +33,7 @@ import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
 
 plugins {
   java
-  kotlin("jvm") version "1.8.22"
+  kotlin("jvm") version "2.1.0-RC2"
 }
 
 java {