Skip to content

services/pcloud: add digest auth, recursive listing, and read-path caching#7617

Open
mro68 wants to merge 7 commits into
apache:mainfrom
mro68:feat/opendal-pcloud-20260518
Open

services/pcloud: add digest auth, recursive listing, and read-path caching#7617
mro68 wants to merge 7 commits into
apache:mainfrom
mro68:feat/opendal-pcloud-20260518

Conversation

@mro68
Copy link
Copy Markdown
Contributor

@mro68 mro68 commented May 25, 2026

Summary

This PR improves OpenDAL's pCloud service for real-world rustic backup workloads.

It addresses authentication compatibility issues and significantly reduces repository scan/read latency by using pCloud-native capabilities and lightweight caches.

Problem

  • Username/password auth failed for pCloud accounts that require digest-based login semantics.
  • Listing large repository trees was slow due to fallback traversal behavior.
  • Repeated reads were paying avoidable path-resolution and link-generation costs.

Root Cause

  • pCloud service auth path did not use digest authentication flow.
  • The service did not expose native recursive listing capability and needed robust recursive metadata flattening.
  • Read path lacked caching for stable object identifiers and short-lived direct links.

Changes

1) Digest authentication

  • Use pCloud digest flow (getdigest + passworddigest) in service auth requests.

2) Native recursive listing

  • Advertise recursive listing capability and implement recursive flattening for nested metadata.
  • Reconstruct missing child paths from parent metadata where pCloud omits path in recursive responses.
  • Keep absolute path propagation correct across recursive traversal.

3) Read-path performance caching

  • Cache path -> fileid from list/stat metadata.
  • Cache getfilelink results with expiration handling.
  • Prefer getfilelink(fileid=...) when available.
  • Invalidate caches on mutating operations (write/delete/rename/copy).

Validation

  • Unit tests:
    • cargo test -p opendal-service-pcloud --lib
  • Build:
    • cargo build --bin rustic
  • Real-world verification with pCloud-backed rustic repository:
    • repoinfo succeeds with digest-authenticated backend access.
    • --dry-run prune succeeds and no longer shows prior path/ID parsing failures.

Downstream Impact

  • rustic_core feature-enablement PR depends on these pCloud service improvements and should consume them via an OpenDAL release that includes this change set.

Notes

  • Changes are intentionally limited to the pCloud service implementation.
  • No API-breaking changes are introduced.

@mro68 mro68 requested a review from Xuanwo as a code owner May 25, 2026 04:42
@dosubot dosubot Bot added size:XL This PR changes 500-999 lines, ignoring generated files. releases-note/feat The PR implements a new feature or has a title that begins with "feat" labels May 25, 2026
pull Bot pushed a commit to ChecksumDev/rustic_core that referenced this pull request May 25, 2026
## Summary
Enable OpenDAL pCloud support in `rustic_core` so pCloud repositories
can be used natively through the OpenDAL backend.

## Motivation
`rustic_core` already supports multiple OpenDAL-backed services. This
change enables pCloud in the backend feature set to make pCloud
repositories usable without external workarounds.

## Changes
- Enable the pCloud service feature in backend OpenDAL dependency
configuration.

## Dependency / Release Note (Important)
- This PR depends on the corresponding OpenDAL pCloud fixes and
performance improvements:
  - apache/opendal#7617
- For production usage in rustic/rustic_core, this should be merged
together with (or after) an OpenDAL release that includes those changes.

## Validation
- Compiles and links with the updated OpenDAL pCloud implementation.
- Rebased onto current `main` and conflict-resolved in
`crates/backend/Cargo.toml` (keeping upstream `opendal = 0.56.0` updates
while adding `services-pcloud`).
- `cargo check -p rustic_backend --features opendal` passes.

## Notes
- This PR is intentionally minimal and scoped to feature enablement in
`rustic_core`.
- Runtime behavior improvements are implemented in the corresponding
OpenDAL PR.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

releases-note/feat The PR implements a new feature or has a title that begins with "feat" size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant