You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Supersedes #2360 (closed automatically due to base branch dev-v2.0.34 deletion).
Description
Hermes native scans and paged imports only read MEMORY.md, silently omitting the user profiles stored in the sibling USER.md. Both endpoints now include the optional profile file and report the combined entry count and byte size.
Imported traces retain their source filename in the existing tags field and use that file's modification time. The cache fingerprints each file separately, so same-size edits to the older file and creation/removal of USER.md invalidate cached pages. Existing MEMORY.md IDs are preserved; profile IDs are distinct and remain stable when MEMORY.md grows. Missing USER.md remains valid, while other read errors are reported.
No new dependencies or public request/response schema changes.
This PR targets main (rebased following the merge and deletion of dev-v2.0.34).
Type of change
Bug fix (non-breaking change which fixes an issue)
How Has This Been Tested?
Tested on Windows with Node.js 24.13.1 and Vitest 2.1.9.
Unit Test — nine new regression cases cover both files, paging, source tags/timestamps, optional empty/missing profiles, stable IDs, cache invalidation, and read errors. The regression suite failed before the fix.
Test Script Or Test Steps — run from apps/memos-local-plugin:
npm test -- tests/unit/server/hermes-native-import.test.ts tests/unit/server/import-export-path.test.ts
npm run lint
npm run build
Results:
Test Files 2 passed (2)
Tests 12 passed (12)
TypeScript lint and build passed. Repository-root make format passed (All checks passed!; 629 files left unchanged). The normal pre-commit hook passed for the committed files.
The broader npm test -- tests/unit/server run reports 105 passed / 10 failed. Re-running with the original importer from base commit 0f747744 reports 96 passed / the same 10 failed: six assertions assume POSIX path separators, two Windows lifecycle tests lack a response mock, and two SSE shutdown tests fail their unsubscribe assertions. These existing failures are outside this fix. No FastAPI pipeline/API contract changes are involved.
Checklist
I have performed a self-review of my own code.
I have commented my code in hard-to-understand areas.
I have added tests that prove my fix is effective.
Related MemOS-Docs issue/PR considered — not applicable to this bug fix; existing import endpoints and public schemas are unchanged.
I have linked the issue to this PR.
I have mentioned the person who will review this PR.
Re-throwing any non-ENOENT error (e.g. EACCES, EPERM, EIO) means a permission-denied or transient I/O failure on the optional USER.md aborts the entire import — including the already-stat'd MEMORY.md — and surfaces as a generic 404 not_found with no indication of which file failed or why. The comment says USER.md is optional, but the behaviour makes it a hard dependency whenever it exists and is unreadable.
Consider logging the error and continuing rather than re-throwing, or at minimum wrapping the re-throw to add context about the failing file path. Example:
}catch(err){constcode=(errasNodeJS.ErrnoException).code;if(code!=="ENOENT"){// Degraded mode: skip USER.md but surface a warning so the caller can report it.console.warn(`[hermes-import] Could not read optional ${userPath}: ${(errasError).message}`);}}
Non-monotonic timestamps across files.baseTs is now per-entry and derived from the file's mtimeMs, but ts is computed using the global index (opts.offset + i) and opts.total. When a batch mixes entries from MEMORY.md (newer mtime) and USER.md (older mtime), the timestamp formula can produce values that interleave non-monotonically or collide, since a lower baseTs combined with a lower opts.total - index offset can land on the same millisecond as an entry from the other file.
Position-dependent hash (memory.index) causes re-import on insertion.identity uses memory.index — the entry's position within its own file. Inserting or deleting any entry before a given one shifts all subsequent index values, regenerates their hashes, and causes them to be re-imported as new episodes on the next sync. The original scheme used the global batch offset, which had the same fragility, but the intent here seems to be stability — in that case the hash should be keyed on content alone (e.g. just memory.text namespaced by file), not on position.
The two readFile calls are independent but awaited sequentially inside a for...of loop. With only two files the impact is small, but they can be parallelised with Promise.all for consistency:
constrawContents=awaitPromise.all(files.map(({path: p})=>readFile(p,"utf8")));for(const[i,{ file, info }]offiles.entries()){for(const[index,text]ofsplitHermesNativeMemories(rawContents[i]!).entries()){source.memories.push({ text, file, index,mtimeMs: info.mtimeMs});}source.bytes+=info.size;}
The episode ID hash now uses memory.index — the entry's position within its own file. This means inserting or deleting any entry before a given one shifts all subsequent index values, regenerating their hashes and causing those entries to be re-imported as new episodes on the next sync (silent duplicates). If the goal is content-stable IDs, the hash should depend on content alone (e.g. just memory.text namespaced by file), not on its file-local position.
All tests passed (85/85 executed). memos_local_plugin/unit: 85/85. Duration: 4s
Branch:fix/hermes-native-user-import
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supersedes #2360 (closed automatically due to base branch
dev-v2.0.34deletion).Description
Hermes native scans and paged imports only read
MEMORY.md, silently omitting the user profiles stored in the siblingUSER.md. Both endpoints now include the optional profile file and report the combined entry count and byte size.Imported traces retain their source filename in the existing
tagsfield and use that file's modification time. The cache fingerprints each file separately, so same-size edits to the older file and creation/removal ofUSER.mdinvalidate cached pages. ExistingMEMORY.mdIDs are preserved; profile IDs are distinct and remain stable whenMEMORY.mdgrows. MissingUSER.mdremains valid, while other read errors are reported.No new dependencies or public request/response schema changes.
Related Issue (Required): Fixes #2306
Reviewer: @syzsunshine219
This PR targets
main(rebased following the merge and deletion ofdev-v2.0.34).Type of change
How Has This Been Tested?
Tested on Windows with Node.js 24.13.1 and Vitest 2.1.9.
apps/memos-local-plugin:npm test -- tests/unit/server/hermes-native-import.test.ts tests/unit/server/import-export-path.test.ts npm run lint npm run buildResults:
TypeScript lint and build passed. Repository-root
make formatpassed (All checks passed!;629 files left unchanged). The normal pre-commit hook passed for the committed files.The broader
npm test -- tests/unit/serverrun reports 105 passed / 10 failed. Re-running with the original importer from base commit0f747744reports 96 passed / the same 10 failed: six assertions assume POSIX path separators, two Windows lifecycle tests lack a response mock, and two SSE shutdown tests fail their unsubscribe assertions. These existing failures are outside this fix. No FastAPI pipeline/API contract changes are involved.Checklist
Reviewer Checklist