Skip to content

Fix DynamoDB Enhanced Client projection expressions for nested list-index paths - #7475

Open
iulianbudau wants to merge 2 commits into
aws:masterfrom
iulianbudau:feature/support-nested-list-references-in-projections
Open

iulianbudau wants to merge 2 commits into
aws:masterfrom
iulianbudau:feature/support-nested-list-references-in-projections

Conversation

@iulianbudau

Copy link
Copy Markdown
Contributor

Motivation and Context

Projecting a nested list path such as NestedAttributeName.create("phoneNumbers[1]", "type") produced an invalid ExpressionAttributeNames key (#AMZN_MAPPED_phoneNumbers[1]). DynamoDB placeholder keys may contain only # followed by [A-Za-z0-9_], so the service rejects the request.

DynamoDB document-path rules treat trailing [n] / [n][m] as list dereferences on the path, not as part of the attribute name token. The expected shape is a projection like #AMZN_MAPPED_phoneNumbers[1].#AMZN_MAPPED_type with ExpressionAttributeNames mapping only the base names (phoneNumbers, type).

Fixes #5603.

Modifications

  • Updated ProjectionExpression to peel trailing list-dereference suffixes ([digits]…) from each path element before building placeholders, then re-attach those suffixes in the projection expression string. Parsing is done once per element via Pair<base, suffix>.
  • Did not change EnhancedClientUtils.SPECIAL_CHARACTERS / cleanAttributeName (shared by other expression builders).
  • Expanded ProjectionExpressionTest for root/nested/multi-index list paths, special-character bases, placeholder reuse/collision, and malformed non-digit brackets.
  • Added Scan/Query generateRequest request-shape coverage for list projection and filter+projection name merge/collision (ScanOperationTest, QueryOperationTest).
  • Added DynamoDB Local functional tests: sync scan (filter + list projection) and sync query (projection only) in ListProjectionExpressionTest; async scan smoke in AsyncListProjectionExpressionTest (EnhancedDocument / documentSchemaBuilder).
  • Added changelog: .changes/next-release/bugfix-AmazonDynamoDBEnhancedClient-5603list.json.

Testing

  • Unit: ProjectionExpressionTest covers root list indexes, nested map/list paths, multi-index suffixes, special-character base names, placeholder reuse/collision, and malformed non-digit brackets.
  • Request-shape: ScanOperationTest and QueryOperationTest assert generated ScanRequest / QueryRequest projection expression and ExpressionAttributeNames for list-index paths, including filter+projection name merge and conflicting-alias failure.
  • Functional (DynamoDB Local): ListProjectionExpressionTest covers sync scan with filter + list-index projection and sync query with list-index projection; AsyncListProjectionExpressionTest covers async scan smoke with the same projection path.
  • Schema used in Local tests: EnhancedDocument via TableSchema.documentSchemaBuilder().
  • Build, unit, functional and integration tests executed and passed locally.

Test Coverage Matrix

Scenario Done Comments if Not Done
1. Different TableSchema Creation Methods
a. TableSchema.fromBean(Customer.class) [ ] Not applicable to this projection-path fix; projection generation is schema-agnostic and was exercised with document schema only.
b. TableSchema.fromImmutableClass(Customer.class) for immutable classes [ ] Not applicable to this projection-path fix; not tested.
c. TableSchema.documentSchemaBuilder().build() [x] Sync + async Local functional tests use documentSchemaBuilder / EnhancedDocument; targeted suite passed.
d. StaticTableSchema.builder(Customer.class) [ ] Not applicable to this projection-path fix; not used in the new Local functional coverage.
2. Nesting of Different TableSchema Types
a. @DynamoDbBean with nested @DynamoDbBean as NonNull [ ] Not applicable to this projection-path fix.
b. @DynamoDbBean with nested @DynamoDbImmutable as NonNull [ ] Not applicable to this projection-path fix.
c. @DynamoDbImmutable with nested @DynamoDbBean as NonNull [ ] Not applicable to this projection-path fix.
d. @DynamoDbBean with nested @DynamoDbBean as Null [ ] Not applicable to this projection-path fix.
e. @DynamoDbBean with nested @DynamoDbImmutable as Null [ ] Not applicable to this projection-path fix.
f. @DynamoDbImmutable with nested @DynamoDbBean as Null [ ] Not applicable to this projection-path fix.
3. CRUD Operations
a. scan() [x] Sync Local scan (filter + list projection) + async Local scan smoke; ScanOperationTest list/filter request-shape; targeted suite passed.
b. query() [x] Sync Local query (list projection) + QueryOperationTest list/filter request-shape; targeted suite passed. Async query not covered.
c. updateItem() [ ] Not applicable to this projection-path fix.
d. putItem() [ ] Not applicable as a projection scenario (Local tests only use put to seed data).
e. getItem() [ ] Not applicable to this projection-path fix.
f. deleteItem() [ ] Not applicable to this projection-path fix.
g. batchGetItem() [ ] Not applicable to this projection-path fix.
h. batchWriteItem() [ ] Not applicable to this projection-path fix.
i. transactGetItems() [ ] Not applicable to this projection-path fix.
j. transactWriteItems() [ ] Not applicable to this projection-path fix.
4. Data Types and Null Handling
a. top-level null attributes [ ] Not applicable to this projection-path fix.
b. collections with null elements [ ] Not applicable to this projection-path fix.
c. maps with null values [ ] Not applicable to this projection-path fix.
d. conversion between null Java values and AttributeValue [ ] Not applicable to this projection-path fix.
e. full serialization/deserialization cycle with null values [ ] Not applicable to this projection-path fix.
5. AsyncTable and SyncTable
a. DynamoDbAsyncTable Testing [x] AsyncListProjectionExpressionTest async scan smoke; passed in targeted suite.
b. DynamoDbTable Testing [x] ListProjectionExpressionTest sync scan + query; passed in targeted suite.
6. New/Modification in Extensions
a. Tables with Scenario in ScenarioSl No.1 (All table schemas are Must) [ ] Not applicable to this projection-path fix; extensions unchanged.
b. Test with Default Values in Annotations [ ] Not applicable to this projection-path fix.
c. Combination of Annotation and Builder passes extension [ ] Not applicable to this projection-path fix.
7. New/Modification in Converters
a. Tables with Scenario in ScenarioSl No.1 (All table schemas are Must) [ ] Not applicable to this projection-path fix; converters unchanged.
b. Test with Default Values in Annotations [ ] Not applicable to this projection-path fix.
c. Test All Scenarios from 1 to 5 [ ] Not applicable to this projection-path fix.
8. Shared Client Cross-Table Behavior
a. Single DynamoDbEnhancedClient used by supported schema table and unsupported/custom schema table [ ] Not applicable to this projection-path fix.
b. Extension-enabled shared client: supported schema behavior applied, unsupported schema safely skipped (no throw) [ ] Not applicable to this projection-path fix.
c. Cache isolation: prior operation on unsupported schema must not impact subsequent supported schema operations [ ] Not applicable to this projection-path fix.
d. Same scenarios for async shared client [ ] Not applicable to this projection-path fix.
9. Extension Interaction Matrix
a. Default extensions + custom extension chain ordering does not change correctness [ ] Not applicable to this projection-path fix.
b. Auto timestamp + versioning interaction on same item [ ] Not applicable to this projection-path fix.
c. Auto UUID + update behavior interaction (WRITE_ALWAYS, WRITE_IF_NOT_EXISTS) [ ] Not applicable to this projection-path fix.
d. Extension behavior with condition expressions (success/failure paths) [ ] Not applicable to this projection-path fix.
e. Extension behavior in batch/transact flows with mixed commands [ ] Not applicable to this projection-path fix.
10. Nested/Flattened Cross-Operation Invariants
a. Flattened attributes preserved across chained operations (put -> update -> scan -> batchGet) [ ] Not applicable to this projection-path fix; flatten not involved in ProjectionExpression.
b. Flattened map behavior under mixed scalar/map/list payloads [ ] Not applicable to this projection-path fix.
c. Nested projection + filter combinations (valid + invalid projection names) [ ] Partial only: valid sync Local scan filter + phoneNumbers[1].type projection covered and passed; invalid projection-name cases not added/verified for this fix.
d. Nested structures with tags (partition/sort/index tags) preserved correctly [ ] Not applicable to this projection-path fix.
e. Sync + async parity for nested/flattened invariants [ ] Not full parity: sync covers scan (filter+proj) and query; async covers scan smoke only; flatten not tested.
11. Mixed-Schema Multi-Table Request
a. batchGet with static + bean + immutable + custom schemas in one request [ ] Not applicable to this projection-path fix.
b. batchWrite with supported + unsupported schemas in one request [ ] Not applicable to this projection-path fix.
c. transactGet mixed schema tables under extension-enabled client [ ] Not applicable to this projection-path fix.
d. transactWrite mixed schema tables with conditions and response options [ ] Not applicable to this projection-path fix.
e. Validation of partial success/failure semantics and diagnostics [ ] Not applicable to this projection-path fix.

Screenshots (if appropriate)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

Checklist

  • I have read the CONTRIBUTING document
  • Local run of mvn install succeeds
  • My code follows the code style of this project
  • My change requires a change to the Javadoc documentation
  • I have updated the Javadoc documentation accordingly
  • I have added tests to cover my changes
  • All new and existing tests passed
  • I have added a changelog entry. Adding a new entry must be accomplished by running the scripts/new-change script and following the instructions. Commit the new file created by the script in .changes/next-release with your changes.
  • My change is to implement 1.11 parity feature and I have updated LaunchChangelog

License

  • I confirm that this pull request can be released under the Apache 2 license

@iulianbudau
iulianbudau requested a review from a team as a code owner October 22, 2026 15:02

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Projection expression with list item throws error using DynamoDB Enhanced client

1 participant