aws · iulianbudau · Aug 21, 2026
@@ -0,0 +1,157 @@
+# DynamoDB Performance Benchmarks
+
+Manual-first DynamoDB suite in `sdk-benchmarks`. New Tier C / Tier D classes are **not** selected by
+`BenchmarkRunner`, shards, baselines, or CI.
+
+Results across tiers answer different questions and must **not** be compared as one score.
+
+## Measurement tiers
+
+| Tier | What it measures | What it excludes |
+|---|---|---|
+| **A** — Pure micro / protocol | Marshalling, JSON protocol, schema/JSON conversion, cold start | Full client pipeline, network |
+| **B** — Mapper isolation | Enhanced mapping via stub `V2TestDynamoDb*Client` | Marshalling, signing, HTTP, network |
+| **C** — Mocked pipeline *(new)* | Full sync/async SDK client path with deterministic mock HTTP | Network and DynamoDB service latency |
+| **D** — Live DynamoDB *(new)* | End-to-end path: SDK + HTTP transport + **network latency** + **DynamoDB service latency** + response handling | Table provisioning, seeding, client/fixture construction (those stay in `@Setup` / `@TearDown`, outside the timed `@Benchmark` method) |
+
+Packages:
+
+- Tier C: `software.amazon.awssdk.benchmark.dynamodb.pipeline`
+- Tier D: `software.amazon.awssdk.benchmark.dynamodb.live`
+- Shared: `…dynamodb.fixture`, `…dynamodb.mock`, `DynamoDbBenchmarkConstant`
+
+## Client layers (LOW / DOCUMENT / TYPED)
+
+These labels are an analysis taxonomy for the suite, not official AWS client product names:
+
+| Layer | API surface | What the timed path emphasizes |
+|---|---|---|
+| **LOW** | `DynamoDbClient` / `DynamoDbAsyncClient` | Direct DynamoDB request/response (pre-built low-level requests in sync Get/Put) |
+| **DOCUMENT** | `DynamoDbTable<EnhancedDocument>` | Enhanced Document model over the same logical item |
+| **TYPED** | `DynamoDbTable<BenchmarkItem>` (bean mapping) | Enhanced typed mapping to/from the shared fixture bean |
+
+LOW, DOCUMENT, and TYPED reuse the same logical fixture and keys so cross-layer comparisons stay fair within a tier.
+
+## Inventory (implemented)
+
+| Tier | Layer | Sync/Async | Operations |
+|---|---|---|---|
+| A | Protocol / micro | Sync (existing) | Existing marshaller / protocol / cold-start benches |
+| B | TYPED mapper isolation | Sync (existing) | Get/Put/Query/Update/Delete/Scan via stub clients |
+| C | LOW | Sync | GetItem, PutItem |
+| C | DOCUMENT | Sync | GetItem, PutItem |
+| C | TYPED | Sync | GetItem, PutItem, Query (first page) |
+| C | LOW | Async | GetItem (`.join()`) |
+| C | TYPED | Async | GetItem (`.join()`) |
+| D | LOW | Sync | GetItem, PutItem |
+| D | TYPED | Sync | GetItem, PutItem, Query (first page) |
+
+Deferred by design: DOCUMENT async, LOW Query, async Put/Query, concurrency, Batch/Transact, CI/shards/baselines.
+
+## JMH modes
+
+| Family | Mode | Unit | Defaults |
+|---|---|---|---|
+| Tier C (`pipeline`) | `AverageTime` | µs/op | warmup 5 / measurement 5 / forks 2 |
+| Tier D (`live`) | `SampleTime` | ms/op | warmup 3 / measurement 5 / fork 1 |
+
+Reduced CLI overrides (`-wi 1 -i 1 -f 1`) are fine for smoke checks. Use class defaults for meaningful comparisons.
+
+## Manual build and run
+
+From the repository root:
+
+```bash
+mvn clean install -P quick -pl :sdk-benchmarks --am
+cd test/sdk-benchmarks
+```
+
+List DynamoDB-related benchmarks:
+
+```bash
+# Windows (cmd / PowerShell)
+java -jar target/benchmarks.jar -l | findstr /i dynamodb
+
+# Unix-like (macOS / Linux / Git Bash)
+java -jar target/benchmarks.jar -l | grep -i dynamodb
+```
+
+### Tier C (mocked — no AWS)
+
+```bash
+# All Tier C pipeline benches
+java -jar target/benchmarks.jar "software.amazon.awssdk.benchmark.dynamodb.pipeline"
+
+# LOW
+java -jar target/benchmarks.jar ".*pipeline.LowLevel"
+
+# DOCUMENT
+java -jar target/benchmarks.jar ".*pipeline.Document"
+
+# TYPED
+java -jar target/benchmarks.jar ".*pipeline.Typed"
+
+# Async Get only
+java -jar target/benchmarks.jar ".*pipeline.*Async"
+
+# Single class
+java -jar target/benchmarks.jar LowLevelGetItemBenchmark
+```
+
+### Tier D (live — opt-in required)
+
+```bash
+# PowerShell
+$env:DYNAMODB_BENCHMARK_LIVE="true"
+$env:AWS_REGION="us-east-1"   # or rely on the default AWS region chain / DYNAMODB_BENCHMARK_REGION
+
+java -jar target/benchmarks.jar "software.amazon.awssdk.benchmark.dynamodb.live"
+
+# Equivalent system property
+java -Ddynamodb.benchmark.live=true -jar target/benchmarks.jar "software.amazon.awssdk.benchmark.dynamodb.live"
+```
+
+Region override (optional):
+
+```bash
+$env:DYNAMODB_BENCHMARK_REGION="us-west-2"
+# or: -Ddynamodb.benchmark.region=us-west-2
+```
+
+### JSON output and profiling
+
+```bash
+java -jar target/benchmarks.jar LowLevelGetItemBenchmark -rf json -rff results.json
+java -jar target/benchmarks.jar LowLevelGetItemBenchmark -prof gc
+```
+
+## Live safety
+
+- AWS credentials are required (`DefaultCredentialsProvider`).
+- Opt-in is mandatory: `DYNAMODB_BENCHMARK_LIVE=true` or `-Ddynamodb.benchmark.live=true`.
+  Without it, `@Setup` aborts **before** credential resolution, client construction, or any AWS call.
+  (System property takes precedence over the environment variable if both are set.)
+- Each trial creates a **unique** table: `sdk-java-ddb-perf-{op}-{8hex}`.
+- Billing mode: **PAY_PER_REQUEST**.
+- Tables are tagged (`sdk-java-ddb-perf-benchmark=owned`, `Purpose=aws-sdk-java-v2-dynamodb-live-benchmark`) so orphans from interrupted runs can be identified.
+- Teardown deletes **only** the exact table created by that trial (`createdByThisTrial`).
+- Interrupted runs (kill/OOM) may leave orphaned tagged tables — clean up manually if needed.
+- Live runs incur DynamoDB request cost and service/network variance.
+
+Retry observation (default on): a lightweight `ExecutionInterceptor` counts attempts. Disable with
+`DYNAMODB_BENCHMARK_LIVE_RETRY_OBSERVE=false` / `-Ddynamodb.benchmark.live.retryObserve=false`.
+Default SDK retry policy is unchanged. Runs with retries print a warning — interpret scores carefully.
+
+## Interpretation
+
+- **Tier C** is deterministic SDK-side cost (mock HTTP). Best signal for SDK regressions.
+- **Tier D** includes network and DynamoDB latency. Use for directional E2E checks, not µs SDK diffs.
+- **LOW vs TYPED** under Tier D is directional only (service variance dominates).
+- **Async** single-op benches include mock/async executor scheduling and `.join()` completion.
+- Untimed smoke calls (DNS/TLS/pool init on live; pipeline warm on mocked) are **not** JMH warmup
+  iterations — JMH still runs its own warmup afterward.
+
+## Automation invariant
+
+Do **not** add these classes to `BenchmarkRunner`, shards, `baseline.json`, or CI without an explicit
+follow-up design. `mvn package` / `exec:exec` behavior for existing suites is unchanged.
@@ -8,6 +8,13 @@ JMH configurations tailored to SDK's build job and you might need to
 adjust them based on your test environment such as increasing warmup iterations
 or measurement time in order to get more reliable data.
 
+## DynamoDB suite (manual)
+
+New DynamoDB Tier C (mocked pipeline) and Tier D (live) benchmarks live under
+`software.amazon.awssdk.benchmark.dynamodb`. They are **manual-only** and are not
+selected by `BenchmarkRunner` / CI. See **[DYNAMODB.md](DYNAMODB.md)** for tiers,
+inventory, commands, live opt-in, and interpretation notes.
+
 There are three ways to run benchmarks.
 
 - Using the executable JAR (Preferred usage per JMH site)

@@ -272,6 +272,47 @@
             <version>${awsjavasdk.version}</version>
             <scope>compile</scope>
         </dependency>
+        <dependency>
+            <groupId>junit</groupId>
+            <artifactId>junit</artifactId>
+            <scope>test</scope>
+        </dependency>
+        <!-- MockAsyncHttpClient for Tier C async benchmarks; exclude unused test-only transitive jars -->
+        <dependency>
+            <groupId>software.amazon.awssdk</groupId>
+            <artifactId>service-test-utils</artifactId>
+            <version>${awsjavasdk.version}</version>
+            <exclusions>
+                <exclusion>
+                    <groupId>junit</groupId>
+                    <artifactId>junit</artifactId>
+                </exclusion>
+                <exclusion>
+                    <groupId>org.junit.jupiter</groupId>
+                    <artifactId>junit-jupiter</artifactId>
+                </exclusion>
+                <exclusion>
+                    <groupId>org.junit.vintage</groupId>
+                    <artifactId>junit-vintage-engine</artifactId>
+                </exclusion>
+                <exclusion>
+                    <groupId>org.hamcrest</groupId>
+                    <artifactId>hamcrest-core</artifactId>
+                </exclusion>
+                <exclusion>
+                    <groupId>io.projectreactor.tools</groupId>
+                    <artifactId>blockhound</artifactId>
+                </exclusion>
+                <exclusion>
+                    <groupId>io.projectreactor.tools</groupId>
+                    <artifactId>blockhound-junit-platform</artifactId>
+                </exclusion>
+                <exclusion>
+                    <groupId>software.amazon.awssdk</groupId>
+                    <artifactId>test-utils</artifactId>
+                </exclusion>
+            </exclusions>
+        </dependency>
     </dependencies>
 
     <dependencyManagement>

@@ -0,0 +1,43 @@
+/*
+ * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License").
+ * You may not use this file except in compliance with the License.
+ * A copy of the License is located at
+ *
+ *  http://aws.amazon.com/apache2.0
+ *
+ * or in the "license" file accompanying this file. This file is distributed
+ * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
+ * express or implied. See the License for the specific language governing
+ * permissions and limitations under the License.
+ */
+
+package software.amazon.awssdk.benchmark.dynamodb;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.regions.Region;
+
+/**
+ * Shared constants for DynamoDB performance benchmark infrastructure (mocked and live).
+ * Does not construct clients; factories and benchmarks consume these values.
+ */
+public final class DynamoDbBenchmarkConstant {
+
+    public static final Region REGION = Region.US_EAST_1;
+
+    public static final String TABLE_NAME = "sdk-java-ddb-perf-benchmark";
+
+    /**
+     * Deterministic fake credentials for mocked Tier C clients. Not used for live Tier D.
+     */
+    public static final AwsCredentialsProvider MOCK_CREDENTIALS_PROVIDER =
+        StaticCredentialsProvider.create(AwsBasicCredentials.create("akid", "skid"));
+
+    public static final String MOCK_ERROR_RESPONSE_BODY = "{}";
+
+    private DynamoDbBenchmarkConstant() {
+    }
+}
@@ -0,0 +1,70 @@
+/*
+ * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License").
+ * You may not use this file except in compliance with the License.
+ * A copy of the License is located at
+ *
+ *  http://aws.amazon.com/apache2.0
+ *
+ * or in the "license" file accompanying this file. This file is distributed
+ * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
+ * express or implied. See the License for the specific language governing
+ * permissions and limitations under the License.
+ */
+
+package software.amazon.awssdk.benchmark.dynamodb;
+
+import software.amazon.awssdk.utils.SystemSetting;
+
+/**
+ * Environment variables and system properties for DynamoDB Tier D live benchmarks.
+ *
+ * <p>System properties take precedence over environment variables (SDK {@link SystemSetting} rule).
+ */
+public enum DynamoDbBenchmarkSystemSetting implements SystemSetting {
+
+    /**
+     * Explicit opt-in for live DynamoDB benchmarks ({@code true} required).
+     */
+    LIVE_OPT_IN("dynamodb.benchmark.live", "DYNAMODB_BENCHMARK_LIVE"),
+
+    /**
+     * Optional region override for live benchmarks.
+     */
+    REGION("dynamodb.benchmark.region", "DYNAMODB_BENCHMARK_REGION"),
+
+    /**
+     * Whether to attach the lightweight retry observer interceptor (default {@code true}).
+     */
+    LIVE_RETRY_OBSERVE("dynamodb.benchmark.live.retryObserve", "DYNAMODB_BENCHMARK_LIVE_RETRY_OBSERVE", "true");
+
+    private final String property;
+    private final String environmentVariable;
+    private final String defaultValue;
+
+    DynamoDbBenchmarkSystemSetting(String property, String environmentVariable) {
+        this(property, environmentVariable, null);
+    }
+
+    DynamoDbBenchmarkSystemSetting(String property, String environmentVariable, String defaultValue) {
+        this.property = property;
+        this.environmentVariable = environmentVariable;
+        this.defaultValue = defaultValue;
+    }
+
+    @Override
+    public String property() {
+        return property;
+    }
+
+    @Override
+    public String environmentVariable() {
+        return environmentVariable;
+    }
+
+    @Override
+    public String defaultValue() {
+        return defaultValue;
+    }
+}