HDDS-13919. S3 Conditional Writes (PutObject) [2/2] - Reuse Atomic Rewrite at Commit Path

[peterxcli]

What changes were proposed in this pull request?

OM Create Phase
Validation is performed within the validateAndUpdateCache method to ensure atomicity within the Ratis state machine application.

1. Locking: The OM acquires the write lock for the bucket/key.
2. Key Lookup: Retrieve the existing key from KeyTable.
3. Validation:
   • Key Not Found: If the key does not exist, throw KEY_NOT_FOUND (maps to S3 412).
   • No ETag Metadata: If the existing key (e.g., uploaded via OFS) does not have an ETag property, throw ETAG_NOT_AVAILABLE (maps to S3 412). The precondition cannot be evaluated, so we must fail rather than silently proceed.
   • ETag Mismatch: Compare existingKey.ETag with expectedETag. If they do not match, throw ETAG_MISMATCH (maps to S3 412).
4. Extract Generation: If ETag matches, extract existingKey.updateID.
5. Create Open Key: Create open key entry with expectedDataGeneration = existingKey.updateID.

Sorry I forgot to follow the step 4 and 5 in my previous PR #9815 ...

What is the link to the Apache JIRA

issues.apache.org/jira/browse/HDDS-13919

How was this patch tested?

existing tests

[yandrey321]

it seems like the same logic ad in CommitRequest

[peterxcli]

correct, and that's actually unnecessary as the reason I wrote in the description.

4. Extract Generation: If ETag matches, extract existingKey.updateID.
5. Create Open Key: Create open key entry with expectedDataGeneration = existingKey.updateID.

[ivandika3]

Ok after thinking a bit more, I think using expectedDataGeneration ensure that only one PutObject can succeed even if two PutObject are using the same valid If-Match.

From the if-match semantic, if there are two concurrent clients that are calling PutObject with the same if-match, both should succeed since the ETag is the same although expectedDataGeneration are not the same.

So in the end, our implementation of if-match is more strict (not necessarily a bad thing). However, it might cause duplicated requests to be rejected. Let me know what you think.

[peterxcli]

From the if-match semantic, if there are two concurrent clients that are calling PutObject with the same if-match, both should succeed since the ETag is the same although expectedDataGeneration are not the same.

This would only be true if the ETag of the object didn't change upon a successful (Conditional) PutObject.

[ivandika3]

Yes, overall I'm OK with the current approach.

[peterxcli]

However, it might cause duplicated requests to be rejected

let me think about which applications built on top of the AWS S3 spec would be affected by this stricter semantics.

[peterxcli]

@ivandika3 The only thing I can think of is that the AWS behavior can suffer from the ABA problem if duplicate conditional PutObject requests are not rejected. In contrast, our version behaves more like a tagged pointer, which prevents the ABA scenario. So everything else stays the same; only concurrent duplicate or ABA-style requests gain a stronger safety guarantee.

[ivandika3]

@peterxcli Thanks for the patch.

[ivandika3]

Ok after thinking a bit more, I think using expectedDataGeneration ensure that only one PutObject can succeed even if two PutObject are using the same valid If-Match.

From the if-match semantic, if there are two concurrent clients that are calling PutObject with the same if-match, both should succeed since the ETag is the same although expectedDataGeneration are not the same.

So in the end, our implementation of if-match is more strict (not necessarily a bad thing). However, it might cause duplicated requests to be rejected. Let me know what you think.

[ivandika3]

LGTM +1.

[peterxcli]

Thanks @ivandika3 and @yandrey321 for the review!