feat(store): implement pluggable cloud storage for HStore

[vaijosh]

Purpose

• Closes: [Feature] Implement Cloud Storage Support for HugeGraph #3079

This PR introduces a cloud storage architecture for HugeGraph Store (HStore). In cloud-native environments, storage nodes are ephemeral, making reliance on local disk storage a single point of failure. This implementation decouples the storage layer from local disk dependencies, allowing SST files to be offloaded to durable, scalable cloud storage providers.

Key Changes

Pluggable Architecture

• Created the CloudStorageProvider SPI interface to enable extensible storage backends.
• Added CloudStorageProviderFactory for seamless provider discovery and lifecycle management.

S3 Provider Implementation

• Introduced the hg-store-cloud-s3 module, leveraging AWS SDK v2 for production-ready S3/S3 compatible storage interaction.

Lifecycle & Event Integration

• Integrated CloudStorageEventListener with RocksDBFactory. This hooks into critical SST lifecycle events (onTableFileCreated, onTableFileDeleted) to ensure synchronized state between local RocksDB and cloud storage.
• Implemented startup hydration (onDBCreated) to backfill pre-existing files and read-miss on-demand hydration to ensure data availability.

Infrastructure & Testing

• Added a comprehensive local development environment via docker/cloud-storage/ using MinIO.
• Included an integration test script (test-graph-queries-and-sst.sh) to verify end-to-end data durability and query consistency.

Implementation Highlights

• SPI Integration: Standardized service discovery via META-INF/services.
• RocksDB Hooking: Registered a singleton RocksdbEventListener within RocksDBFactory to intercept file operations without modifying core RocksDB logic.
• Read-Miss Protection: Added a guard window mechanism in onReadMiss to prevent redundant hydration requests during high-concurrency read scenarios.

Verifying These Changes

• Unit Tests: Coverage for config parsing (CloudStorageConfigTest), factory registration, and event listener logic.
• Integration Tests: Verified using the new docker-compose setup with MinIO.
• Configuration: Verified application.yml bindings for credentials, bucket management, and sync intervals.

Impact

• Dependencies: Added AWS SDK v2; updated LICENSE and NOTICE accordingly.
• Configurations: Added cloud.storage namespace to application.yml (disabled by default).
• Public API: No breaking changes to existing public-facing client APIs.

Documentation

• Architecture and configuration details are documented in hugegraph-store/docs/pluggable-cloud-storage-architecture.md.

[codecov]

Codecov Report

❌ Patch coverage is 64.06250% with 23 lines in your changes missing coverage. Please review.
✅ Project coverage is 30.62%. Comparing base (03e6b8e) to head (d5f64df).
⚠️ Report is 4 commits behind head on master.

Files with missing lines	Patch %	Lines
...graph/store/cloud/CloudStorageProviderFactory.java	59.18%	17 Missing and 3 partials ⚠️
...store/cloud/CloudStorageNonRetryableException.java	0.00%	2 Missing ⚠️
...ache/hugegraph/store/cloud/CloudStorageConfig.java	91.66%	1 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (03e6b8e) and HEAD (d5f64df). Click for more details.

HEAD has 2 uploads less than BASE

Flag	BASE (03e6b8e)	HEAD (d5f64df)
	3	1

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #3081      +/-   ##
============================================
- Coverage     36.80%   30.62%   -6.18%     
- Complexity      338      395      +57     
============================================
  Files           805      810       +5     
  Lines         68587    69157     +570     
  Branches       9029     9119      +90     
============================================
- Hits          25241    21180    -4061     
- Misses        40653    45555    +4902     
+ Partials       2693     2422     -271     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[VGalaxies]

Review summary

• Blocking: yes
• Summary: The change cannot currently guarantee recoverable RocksDB state, may silently disable cloud storage, and contains S3 retry and deployment defects.
• Evidence:
  • Static analysis of git diff origin/master...HEAD
  • shell syntax and POM XML checks passed
  • git diff --check reported whitespace-only errors

[VGalaxies]

High: S3 service failures bypass the provider retry contract

hugegraph-store/hg-store-cloud-s3/src/main/java/org/apache/hugegraph/store/cloud/s3/S3CloudStorageProvider.java:540

Evidence

• Provider operations catch only SdkClientException; S3 service responses use S3Exception/AwsServiceException, which are separate exception types. The outer part retry catches only IOException.

Impact

• 429/5xx responses bypass part-level retries, while callers can receive unchecked exceptions despite the provider’s IOException contract.

Requested fix

• Catch service exceptions, classify retryable versus permanent status codes, and translate them consistently into retryable or non-retryable cloud exceptions.