Skip to content

feat(posixfs): #3182 add basepath option in the "posixfs scan" command - #3228

Closed
pbleser-oc wants to merge 27 commits into
opencloud-eu:mainfrom
pbleser-oc:feat/#3182-posixfs-scan-subtree
Closed

feat(posixfs): #3182 add basepath option in the "posixfs scan" command#3228
pbleser-oc wants to merge 27 commits into
opencloud-eu:mainfrom
pbleser-oc:feat/#3182-posixfs-scan-subtree

Conversation

@pbleser-oc

Copy link
Copy Markdown
Member

Description

  • add support for specifying a set of resources when running the posixfs scan command scanning, or a singular file to scan, as opposed
  • refactors the resource walking function implemented in Extend posixfs consistency check #3220 in order to reuse it for the posixfs scan command

Related Issue

Motivation and Context

Addresses use-cases where files are copied into the PosixFS storage directory tree while the server is not running or with collaborative mode disabled, e.g. during large migrations or in disaster recovery scenarios.

With these changes, it is possible to perform the storage scanning in a much more granular way, which may help in cases where the storage tree is especially large and scanning everything would be too slow.

How Has This Been Tested?

Has been tested locally/manually by

  • running the server with collaborative mode disabled (STORAGE_USERS_POSIX_WATCH_FS=false)
  • copying files into a space storage directory, cp ... ~/.opencloud/storage/users/projects/9c4feaa1-0a8f-4f9d-bcf7-2d79b714d9d5/
  • checking that no xattrs were set on those files using getfattr -d
  • running the posixfs scan command with different use-cases:
    • on the storage root
    • without arguments
    • on a subdirectory of a space
    • on the space directory
    • on individual files
  • and then checking that the xattrs were set (user.oc.id, user.oc.parentid, etc...), again using getfattr -d

Unfortunately, it is not possible to add unit tests as it would require a lot of changes to abstract away the filesystem operations, or a significant amount of test harness setup to be able to run the command with a NATS Jetstream and everything else that is needed for it.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Technical debt
  • Tests only (no source changes)

Checklist:

  • Code changes
  • Unit tests added
  • Acceptance tests added
  • Documentation added

@codacy-production

codacy-production Bot commented Aug 3, 2026

Copy link
Copy Markdown

Not up to standards ⛔

🔴 Issues 1 critical

Alerts:
⚠ 1 issue (≤ 0 issues of at least minor severity)

Results:
1 new issue

Category Results
Security 1 critical

View in Codacy

🟢 Metrics 71 complexity · -5 duplication

Metric Results
Complexity 71
Duplication -5

View in Codacy

🟢 Coverage 1.82% diff coverage

Metric Results
Coverage variation Report missing for 15fc44b1
Diff coverage 1.82% diff coverage

View coverage diff in Codacy

Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (15fc44b) Report Missing Report Missing Report Missing
Head commit (b5140f6) 84009 19254 22.92%

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#3228) 604 11 1.82%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

1 Codacy didn't receive coverage data for the commit, or there was an error processing the received data. Check your integration for errors and validate that your coverage setup is correct.

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@pbleser-oc pbleser-oc moved this from Qualification to In Progress in OpenCloud Team Board Aug 3, 2026
Comment thread opencloud/pkg/command/posixfs.go Outdated
Comment thread opencloud/pkg/command/posixfs.go Outdated
Comment thread opencloud/pkg/command/posixfs.go Outdated
dschmidt and others added 19 commits August 3, 2026 19:46
…scan" command

 * add support for specifying a basepath using -p when running the
   posixfs scan command, to indicate a directory under which to start
   scanning, or a singular file to scan, as opposed to scanning from the
   storage root directory as is the default behaviour

 * implements opencloud-eu#3182
Bumps [github.com/beevik/etree](https://github.com/beevik/etree) from 1.6.0 to 1.7.0.
- [Release notes](https://github.com/beevik/etree/releases)
- [Changelog](https://github.com/beevik/etree/blob/main/RELEASE_NOTES.md)
- [Commits](beevik/etree@v1.6.0...v1.7.0)

---
updated-dependencies:
- dependency-name: github.com/beevik/etree
  dependency-version: 1.7.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.56.0 to 0.57.0.
- [Commits](golang/net@v0.56.0...v0.57.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-version: 0.57.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [github.com/gabriel-vasile/mimetype](https://github.com/gabriel-vasile/mimetype) from 1.4.13 to 1.4.15.
- [Release notes](https://github.com/gabriel-vasile/mimetype/releases)
- [Commits](gabriel-vasile/mimetype@v1.4.13...v1.4.15)

---
updated-dependencies:
- dependency-name: github.com/gabriel-vasile/mimetype
  dependency-version: 1.4.15
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.23.2 to 1.24.1.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/v1.24.1/CHANGELOG.md)
- [Commits](prometheus/client_golang@v1.23.2...v1.24.1)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-version: 1.24.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [github.com/go-ldap/ldap/v3](https://github.com/go-ldap/ldap) from 3.4.13 to 3.4.14.
- [Release notes](https://github.com/go-ldap/ldap/releases)
- [Commits](go-ldap/ldap@v3.4.13...v3.4.14)

---
updated-dependencies:
- dependency-name: github.com/go-ldap/ldap/v3
  dependency-version: 3.4.14
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [github.com/nats-io/nats-server/v2](https://github.com/nats-io/nats-server) from 2.14.3 to 2.14.4.
- [Release notes](https://github.com/nats-io/nats-server/releases)
- [Changelog](https://github.com/nats-io/nats-server/blob/main/RELEASES.md)
- [Commits](nats-io/nats-server@v2.14.3...v2.14.4)

---
updated-dependencies:
- dependency-name: github.com/nats-io/nats-server/v2
  dependency-version: 2.14.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
* chore(skills): add skill for bumping reva

* chore(skills): derive OpenCloud version from open release PR

* chore: exclude .agents from codacy analysis
aduffeck and others added 7 commits August 3, 2026 19:46
This makes it possible to get rid of the globals without passing
multiple state vars around.
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.82.0 to 1.83.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](grpc/grpc-go@v1.82.0...v1.83.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-version: 1.83.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [github.com/open-policy-agent/opa](https://github.com/open-policy-agent/opa) from 1.18.2 to 1.19.0.
- [Release notes](https://github.com/open-policy-agent/opa/releases)
- [Changelog](https://github.com/open-policy-agent/opa/blob/main/CHANGELOG.md)
- [Commits](open-policy-agent/opa@v1.18.2...v1.19.0)

---
updated-dependencies:
- dependency-name: github.com/open-policy-agent/opa
  dependency-version: 1.19.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
pbleser-oc added a commit to pbleser-oc/opencloud that referenced this pull request Aug 3, 2026
@pbleser-oc
pbleser-oc force-pushed the feat/#3182-posixfs-scan-subtree branch from 1060827 to 6ce8f98 Compare August 3, 2026 17:52
…scan" command

 * add support for specifying a set of resources when running the
   posixfs scan command scanning, or a singular file to scan, as opposed

 * refactors the resource walking function implemented in
   opencloud-eu#3220 in order to
   reuse it for the posixfs scan command

 * implements opencloud-eu#3182

 * add --halt-on-error flag

 * collect errors and fail command when more than one error occured
@pbleser-oc
pbleser-oc force-pushed the feat/#3182-posixfs-scan-subtree branch from 6ce8f98 to b5140f6 Compare August 3, 2026 17:54
@pbleser-oc pbleser-oc closed this Aug 3, 2026
@github-project-automation github-project-automation Bot moved this from In Progress to Done in OpenCloud Team Board Aug 3, 2026
@pbleser-oc
pbleser-oc deleted the feat/#3182-posixfs-scan-subtree branch August 3, 2026 18:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

6 participants