fix: prevent stale entry chunk files from clobbering current migratio…#1083
Conversation
…n data API: - Add clearStaleEntries() to wipe the entries/ subtree before each (test) migration run, so orphaned chunk files from previous runs can't overwrite current entry data during the update step - removeEntriesFromDatabase now reads index.json to process only the current chunk files, falling back to globbing for legacy data - field-attacher: use customLogger instead of console.info for content type creation logs - runCli: switch to writeUidMapping util for uid mapping UI: - Remove the auto-mapped content mapping merge flow (AutoMappedMergeConfirmModal, persistAutoMappedContentMapper, handleUpdateAutoMappedContentMapping, shouldPromptShowAutoMappedMerge) and the related Auto-mapped status/pill constants
🔒 Security Scan Results
⏱️ SLA Breach Summary
ℹ️ Vulnerabilities Without Available Fixes (Informational Only)The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:
Consider reviewing these vulnerabilities when fixes become available. |
…revent it from editing
🔒 Security Scan Results
⏱️ SLA Breach Summary
ℹ️ Vulnerabilities Without Available Fixes (Informational Only)The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:
Consider reviewing these vulnerabilities when fixes become available. |
There was a problem hiding this comment.
Pull request overview
This PR addresses a migration correctness issue where stale/orphaned entry chunk files from prior runs could be re-processed and overwrite current entry data, and it removes the UI’s auto-mapped content mapping merge flow. It also standardizes some logging/UID-mapping behaviors in the API and adjusts related UI behaviors.
Changes:
- API: Add
clearStaleEntries()to wipe theentries/subtree before migration/test-migration runs; updateremoveEntriesFromDatabaseto respectindex.jsonchunk lists (legacy glob fallback); switch runCli towriteUidMapping; replace some console logging withcustomLogger. - UI: Remove auto-mapped merge prompt/flow and related constants; simplify Save Changes modal interactions; adjust content-mapper styling and types; disable File Format text input.
- Tests: Add unit tests for
writeUidMapping.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| upload-api/.env | Adds upload-api environment variables (currently committed as a real .env). |
| ui/tests/unit/utilities/constants.test.ts | Updates unit test expectations after removing Auto-mapped status. |
| ui/src/utilities/constants.ts | Removes Auto-mapped status and related pill constants. |
| ui/src/pages/Migration/index.tsx | Removes auto-mapped merge confirm modal flow when continuing from Content Mapper. |
| ui/src/context/app/app.interface.ts | Broadens existingCT / existingGlobal typing to allow lazy getters. |
| ui/src/components/LegacyCms/Actions/LoadFileFormat.tsx | Disables the File Format TextInput. |
| ui/src/components/ContentMapper/index.tsx | Removes auto-map merge logic and related UI elements/handlers. |
| ui/src/components/ContentMapper/index.scss | Removes auto-mapped pill styles; adds table layout alignment styles. |
| ui/src/components/ContentMapper/contentMapper.interface.ts | Removes auto-mapped merge handles from the ref interface. |
| ui/src/components/Common/SaveChangesModal/index.tsx | Changes Save/Don’t Save handlers and makes changeStep typed sync. |
| api/tests/unit/utils/uid-mapper.utils.test.ts | Adds unit tests for writeUidMapping. |
| api/src/utils/field-attacher.utils.ts | Switches content type creation logs from console.info to customLogger. |
| api/src/utils/entry-update.utils.ts | Adds clearStaleEntries; updates entry removal to use index.json chunk lists. |
| api/src/services/runCli.service.ts | Uses writeUidMapping instead of inline uid-mapper DB logic. |
| api/src/services/migration.service.ts | Calls clearStaleEntries before transforming entries for migration/test migration. |
Comments suppressed due to low confidence (1)
api/src/utils/entry-update.utils.ts:128
- Chunk filenames coming from
index.jsonare joined directly intolocalePathand then parsed without any existence/path containment checks. A missing file or unexpected path segment will currently throw and stop processing; this should be validated and skipped with a log entry instead.
for (const jsonFile of jsonFiles) {
const filePath = path.join(localePath, jsonFile);
const raw = fs.readFileSync(filePath, "utf-8");
const data = JSON.parse(raw);
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
🔒 Security Scan Results
⏱️ SLA Breach Summary
ℹ️ Vulnerabilities Without Available Fixes (Informational Only)The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:
Consider reviewing these vulnerabilities when fixes become available. |
…igration to prevent path traversal vulnerabilities
🔒 Security Scan Results
⏱️ SLA Breach Summary
ℹ️ Vulnerabilities Without Available Fixes (Informational Only)The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:
Consider reviewing these vulnerabilities when fixes become available. |
…revent path traversal vulnerabilities
🔒 Security Scan Results
⏱️ SLA Breach Summary
ℹ️ Vulnerabilities Without Available Fixes (Informational Only)The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:
Consider reviewing these vulnerabilities when fixes become available. |
…n data
API:
UI:
🔗 Jira Ticket
MIGRATION-XXXX
📋 PR Type
📝 Description
What changed?
API
clearStaleEntries()to wipe theentries/subtree before each (test) migration run, so orphaned chunk files from previous runs can't overwrite current entry data during the update stepremoveEntriesFromDatabasenow readsindex.jsonto process only the current chunk files, falling back to globbing for legacy data without an indexfield-attacher: usecustomLoggerinstead ofconsole.infofor content type creation logsrunCli: switch towriteUidMappingutil for uid mappingUI
AutoMappedMergeConfirmModal,persistAutoMappedContentMapper,handleUpdateAutoMappedContentMapping,shouldPromptShowAutoMappedMerge) and the relatedAuto-mappedstatus/pill constantsSaveChangesModalhandlers to a synchronouschangeStepexistingGlobal/existingCTto accept lazy getter functionsWhy?
Each import run wrote entry chunk files with fresh random UUID names and overwrote
index.json, but never removed the previous run's chunk files. Those orphaned files carried stale (previous-iteration) content that could clobber current entry data during the update step, producing incorrect migrated entries. Clearing the entries tree up front and respectingindex.jsonguarantees the importer starts from a clean slate and only processes current data.The auto-mapped merge flow was removed cause commit was reverted.
🧩 Affected Areas
api— Node.js backendui— React frontendupload-api— Upload API serverdocker/docker-compose🧪 How to Test
Expected result: Migrated entries always match the current run's data, and the content mapper continue flow proceeds without stale auto-mapped merge prompts.
📸 Screenshots / Recordings
🔗 Related PRs / Dependencies
✅ Author Checklist
feature/,bugfix/, orhotfix/+ 5–30 lowercase chars.env/example.envupdated if new environment variables were addednpm test)README.md/ docs updated if behaviour changed👀 Reviewer Notes
index.jsonparsing and the path-sanitization guards (sanitizeStackId/assertResolvedPathUnderBase) onclearStaleEntries.