feat: Implement delta migration support for content mapping and entry…#1090
Open
yashin4112 wants to merge 3 commits into
Open
feat: Implement delta migration support for content mapping and entry…#1090yashin4112 wants to merge 3 commits into
yashin4112 wants to merge 3 commits into
Conversation
… mapping steps - Added logic to handle delta iterations in the ContentMapper component, allowing for different behaviors based on the iteration number. - Updated the MigrationFlowHeader to adjust step IDs and button states based on the current iteration. - Enhanced the HorizontalStepper to maintain step completion status across iterations. - Introduced new constants for empty state messages in the content mapper. - Modified API calls to differentiate between new and old content types based on the iteration. - Map Entry step lists only content types that have entry mappings, with status icons derived from saved isUpdate state. - Made backend current_step iteration-aware (Map Entry inserts an extra step from iteration 2+). - Fixed entry-update: only remove entries that already exist in Contentstack (have a contentstackEntryUid); update them when marked isUpdate, otherwise create fresh — prevents data loss of never-migrated entries. - Updated the LoadUploadFile component to support MySQL connection details editing. - Adjusted the Migration page to conditionally render steps based on the iteration. - Improved the handling of empty states in the UI for better user experience during migration.
🔒 Security Scan Results
⏱️ SLA Breach Summary
🟠 High Severity - SLA Breached Issues (with fixes)Showing 4 issue(s) that have exceeded the 30-day SLA threshold:
ℹ️ Vulnerabilities Without Available Fixes (Informational Only)The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:
❌ BUILD FAILED - Security checks failed Please review and fix the security vulnerabilities before merging. |
🔒 Security Scan Results
⏱️ SLA Breach Summary
ℹ️ Vulnerabilities Without Available Fixes (Informational Only)The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:
Consider reviewing these vulnerabilities when fixes become available. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🔗 Jira Ticket
CMG-997
📋 PR Type
📝 Description
What changed?
New "Map Entry" step (delta migration, iteration 2+)
Legacy CMS → Destination Stack → Map Content Fields → Map Entry → Test Migration → Execute. Iteration 1 stays the original 5-step flow (no Map Entry).Single-route, filter-based content type fetch
getContentTypestakes afilterquery param (new|old) instead of a new endpoint. On iteration 2+ it diffs the current vs previous iteration'scontentTypesMapper.jsonbyotherCmsUidto return the new-only or old-only slice. Iteration 1 is untouched.entryMapping, and derives each content type's status icon (green "Updated" / blue "Mapped") from the savedisUpdatestate so it persists correctly across reloads.Iteration-aware step numbering
current_stepstate machine is now iteration-aware (DELTA_STEPPER_STEPSadds Map Entry as step 4, shifting Testing → 5, Execute → 6 on delta iterations).project_current_step), and the side-nav (migrationSteps.json) all resolve step numbers by iteration.Stepper / navigation fixes
current_step, so navigating back to review a completed step no longer resets the progress bar, and completed steps stay navigable.Bug fix — entry update data loss
removeEntriesFromDatabasepreviously deleted every entry present in the entry mapper, even ones never migrated (nocontentstackEntryUid), silently dropping them. Now it only removes entries that already exist in Contentstack (have acontentstackEntryUid);isUpdateones are collected for update, the rest are left to be created fresh.Tests
updateEntryCliservice (basic-auth & OAuth paths, missing stack key, auth errors, stdout/stderr log-level classification, non-zero exit handling, warning mapping).Why?
Delta (re-)migrations need to map and update entries of already-migrated content types across iterations without losing data or re-mapping content that was already migrated. The previous flow had no Map Entry step, no iteration-aware step numbering, and a bug that dropped never-migrated entries during update.
🧩 Affected Areas
api— Node.js backendui— React frontendupload-api— Upload API serverdocker/docker-compose🧪 How to Test
Iteration 1 (fresh migration):
Iteration 2+ (delta / restart):
current_stepin the DB reaches 6 on Execute.Entry update data-loss fix:
7. Have an entry that was NOT migrated in iteration 1 — confirm it gets created this iteration (not silently dropped). Entries marked
isUpdatewith a Contentstack uid get updated.Tests:
8.
cd api && npm test—updateEntryCli.service.test.tspasses.Expected result: Iteration 1 = 5 steps unchanged; iteration 2+ = 6 steps with correct new/old content type splitting, persistent status icons, no progress-bar resets, and no entry data loss.
📸 Screenshots / Recordings
🔗 Related PRs / Dependencies
✅ Author Checklist
feature/,bugfix/, orhotfix/+ 5–30 lowercase chars.env/example.envupdated if new environment variables were added — N/A (no new env vars)npm test)updateEntryCli.service.test.tsREADME.md/ docs updated if behaviour changed — N/A👀 Reviewer Notes
getContentTypes?filter=new|oldparam + a per-iteration diff oncontentTypesMapper.json— no new endpoint.createStepper()(UI) andmigrationSteps.jsonall_steps(side-nav), plus the backendSTEPPER_STEPS/DELTA_STEPPER_STEPS.entry-update.utils.tsdelete rule — entries are removed from import data only when they have acontentstackEntryUid(exist in CS); never-migrated entries are intentionally kept to be created.