Skip to content

Conversation

@KodaiD
Copy link
Contributor

@KodaiD KodaiD commented Nov 12, 2025

Description

This PR adds S3 support to the Object Storage adapter.

Related issues and/or PRs

Changes made

  • Added an ObjectStorageWrapper implementation for S3.
  • Added unit tests.
  • Added a GitHub workflow for integration tests.

Checklist

  • I have commented my code, particularly in hard-to-understand areas.
  • I have updated the documentation to reflect the changes.
  • I have considered whether similar issues could occur in other products, components, or modules if this PR is for bug fixes.
  • Any remaining open issues linked to this PR are documented and up-to-date (Jira, GitHub, etc.).
  • Tests (unit, integration, etc.) have been added for the changes.
  • My changes generate no new warnings.
  • Any dependent changes in other PRs have been merged and published.

Additional notes (optional)

N/A

Release notes

Added Amazon S3 adapter.

@KodaiD KodaiD self-assigned this Nov 12, 2025
@KodaiD KodaiD added the enhancement New feature or request label Nov 12, 2025
@KodaiD
Copy link
Contributor Author

KodaiD commented Nov 12, 2025

/gemini review

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces S3 support for the Object Storage adapter, which is a significant feature addition. The implementation is comprehensive, leveraging the AWS CRT-based S3 client for better performance and includes new configuration options, exception types, and both unit and integration tests. My review has identified a critical issue in the delete method's logic that needs to be addressed. Additionally, I've pointed out several areas for improvement regarding code duplication, potential memory inefficiencies when handling large numbers of objects, and making exception handling more robust. Overall, this is a solid contribution, and addressing these points will enhance its correctness and maintainability.

@KodaiD KodaiD marked this pull request as ready for review November 12, 2025 02:14
Copilot AI review requested due to automatic review settings November 12, 2025 02:14
Copilot finished reviewing on behalf of KodaiD November 12, 2025 02:16
@KodaiD KodaiD requested review from a team, Torch3333, brfrn169, feeblefakie and komamitsu and removed request for a team November 12, 2025 02:17
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds Amazon S3 support to the Object Storage adapter, enabling ScalarDB to use S3 as a backend storage option alongside the existing Azure Blob Storage adapter.

  • Implemented S3Wrapper class that provides S3-specific operations (get, insert, update, delete, etc.)
  • Added configuration, provider, and error handling classes for S3
  • Refactored shared exception constructors and interface methods to support both storage types

Reviewed Changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
S3Wrapper.java Core S3 implementation with async client operations and error handling
S3Config.java Configuration parser for S3-specific settings (region, bucket, parallelism, etc.)
S3Provider.java Service provider for S3 storage
S3ErrorCode.java Enum for S3 HTTP status codes
S3WrapperTest.java Comprehensive unit tests for S3Wrapper
S3ConfigTest.java Unit tests for S3Config
ObjectStorageWrapperFactory.java Added S3 instantiation logic
ObjectStorageUtils.java Added S3 config factory logic
ObjectStorageConfig.java Removed getEndpoint() method (S3-specific)
BlobStorageConfig.java Moved getEndpoint() from interface to implementation, fixed error message
PreconditionFailedException.java Removed unused single-argument constructor
ObjectStorageWrapperException.java Removed unused single-argument constructor
ConflictOccurredException.java New exception class for conflict errors
ObjectStorageWrapperIntegrationTest.java Fixed test signature and improved test cleanup
ObjectStorageEnv.java Added storage type configuration support
core/build.gradle Added AWS S3 SDK dependencies
DistributedStorageProvider Registered S3Provider service
object-storage-adapter-check.yaml New GitHub workflow for S3 integration tests

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@KodaiD KodaiD closed this Nov 13, 2025
@KodaiD KodaiD reopened this Nov 13, 2025
@KodaiD KodaiD requested a review from komamitsu November 13, 2025 07:59
Copy link
Collaborator

@brfrn169 brfrn169 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left several comments. PTAL!

@KodaiD KodaiD requested a review from brfrn169 November 14, 2025 00:43
Copy link
Contributor

@komamitsu komamitsu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 👍

Copy link
Collaborator

@brfrn169 brfrn169 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thank you!

Copy link
Contributor

@Torch3333 Torch3333 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you!

Copy link
Contributor

@feeblefakie feeblefakie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, looking good!
Left some suggestions. PTAL!

@KodaiD KodaiD requested a review from feeblefakie November 14, 2025 09:06
Copy link
Contributor

@feeblefakie feeblefakie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thank you!

@feeblefakie feeblefakie enabled auto-merge (squash) November 17, 2025 04:57
@feeblefakie feeblefakie merged commit 1e10da9 into master Nov 17, 2025
137 of 140 checks passed
@feeblefakie feeblefakie deleted the add-s3-support branch November 17, 2025 04:58
feeblefakie pushed a commit that referenced this pull request Nov 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants