Resource type partial sync #600
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Implemented resource-typed sync task handler in `resource_typed_sync.go`. - Added support for `SyncResourceTypedType` in task type checks. - Introduced `localResourceTypedSyncer` for managing resource-typed sync tasks. - Updated existing task processing logic to accommodate new task type.
- Introduced `WithResourceTypedSync` option for configuring resource-typed sync in the connector runner. - Added `SyncResourceTypedTask` message to the protocol buffer definition for handling resource-typed sync tasks. - Implemented validation for `SyncResourceTypedTask` and integrated it into the task processing logic. - Updated syncer to handle resource-typed sync operations, including syncing entitlements and grants for specific resource types. - Enhanced CLI commands to accept `sync-resource-type` parameter for initiating resource-typed syncs.
- Updated comments in `state.go` to clarify the purpose of `SyncResourceTypedOp` and `SetupResourceTypedSyncOp`. - Improved documentation for `resourceTypedSyncID` in `syncer.go` to explain its role in resource-typed incremental sync. - Added clarification in the `resourceTypeHasChildType` function regarding the retrieval of sample resources and their annotations.
- Renamed `resourceTypedSyncID` to `partialSyncResourceTypeID` across multiple files to reflect the new functionality. - Updated related functions and CLI commands to use `WithPartialSyncResourceType` for configuring partial syncs. - Introduced `localPartialSyncResourceType` to handle partial sync tasks, ensuring only resources are synced without entitlements and grants. - Removed the deprecated `resourceTypedSyncer` implementation to streamline the codebase.
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the ✨ Finishing touches🧪 Generate unit tests (beta)
Comment |
- Added `PartialSyncResourceTypeId` field to `Task_SyncFullTask` for triggering partial syncs of specific resource types. - Updated getter and setter methods for `PartialSyncResourceTypeId`. - Modified `fullSyncTaskHandler` to handle partial sync resource type from task or CLI. - Removed deprecated `resourceTypedSync` implementation to streamline the codebase. - Updated protocol buffer definitions and validation logic accordingly.
- Replaced `SyncResourceTypedType` with `PartialSyncType` in task type checks to align with new partial sync implementation. - Removed `partialSyncResourceTypeID` from `fullSyncTaskHandler` and `c1ApiTaskManager` to streamline the codebase. - Added `partialSyncTaskHandler` to manage partial sync tasks, supporting both targeted resources and resource type modes. - Introduced `localPartialSync` to handle partial sync operations, ensuring only resources are synced without entitlements and grants. - Deleted deprecated `partial_sync_resource_type.go` and added new `partial_sync.go` for local partial sync management.
- Removed `partialSyncResourceTypeID` from various components, including the connector runner and CLI commands, to streamline the codebase. - Replaced `SyncResourceTypedTask` with `SyncPartialTask` in protocol buffers and task processing logic, supporting both targeted resources and resource type modes. - Updated validation logic and methods to accommodate the new `SyncPartialTask`, ensuring proper handling of partial sync operations. - Enhanced documentation and comments to clarify the purpose and functionality of the new partial sync implementation.
ggreer
reviewed
Dec 29, 2025
| ActionStatusTask action_status = 118; | ||
| CreateSyncDiffTask create_sync_diff = 119; | ||
| CompactSyncs compact_syncs = 120; | ||
| SyncPartialTask sync_partial = 121; |
Contributor
There was a problem hiding this comment.
We should just add attributes to the SyncFullTask so that we don't have to choose a different sync type based on what we want to sync. eg: Right now if you want to sync resources only (not grants or entitlements) that's a SyncFullTask.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds support for a new sync task that only fetches the resources of a given resource type, as well as discovering any required parent resource types. It explicitly does not sync entitlements or grants.
The sync should be created as a partial/incremental sync.
The primary use case is to quickly fetch an updated resource list when we are waiting for some resource to appear so we can continue on provisioning it access.
I've tested this on the cli with a few connectors, but need to do some rounds of API testing before this is ready to be merged.