Repository navigation
Python: fix(sessions): quarantine corrupt snapshots under a bounded file name - #10372
Simin Yuan (simin-yuan) wants to merge 4 commits into
Conversation
The quarantine destination embedded the snapshot name, so a 128-character session id near the legacy Windows MAX_PATH limit produced a rename target ~40 characters longer than the snapshot itself: os.replace failed with WinError 3 and the corrupt file stayed in place, leaving the session permanently unreadable. Name the destination .maf-session-<uuid>.corrupt, the way the write path is heading (and like the .maf-state/.maf-ckpt siblings), and cover both the bounded name and a simulated 260-character rename limit.
There was a problem hiding this comment.
🟡 Changes recommended
The longer quarantine path can break renaming on legacy Windows and leave corrupt snapshots in place, while the regression test fails under valid longer temporary roots.
2 open issues
What changed in this PR
This PR addresses #9885 by giving Python’s FileSessionStore a bounded quarantine filename, enabling recovery of corrupt long-ID snapshots near the legacy Windows path limit.
Changes:
- Replaces snapshot-derived quarantine names with
.maf-session-<uuid hex>.corrupt. - Adds regression tests for bounded filename length and quarantine under a simulated Windows path limit.
- Leaves the public API and write-time temporary naming unchanged; the latter is handled separately by #9739.
| File | Description |
|---|---|
python/packages/core/tests/core/test_sessions.py |
Adds quarantine filename and simulated path-limit regression tests. |
python/packages/core/agent_framework/_sessions.py |
Makes quarantine filenames independent of session ID length. |
Review effort: Balanced
| if current_serialized != serialized: | ||
| return None | ||
| quarantine_path = file_path.with_name(f".{file_path.name}.{uuid.uuid4().hex}.corrupt") | ||
| quarantine_path = file_path.with_name(f".maf-session-{uuid.uuid4().hex}.corrupt") |
There was a problem hiding this comment.
Adopted. The quarantine destination is now .<uuid hex>.corrupt (41 characters) instead of .maf-<uuid hex>.corrupt (45), and the new test's glob and length bound follow it.
The short-id/deep-root case you described is covered by test_corrupt_quarantine_short_id_at_deep_root_does_not_regress: a one-character session id under a 206-character root with os.replace capped at 260 characters. With the original .maf-session-<uuid>.corrupt name the destination came out at 260 and the rename failed; it now lands at 248 and succeeds.
| session_id = "x" * FileSessionStore.MAX_SESSION_ID_LENGTH | ||
| sample_path = store._session_file_path(session_id) | ||
| padding = 252 - len(str(tmp_path)) - 1 - len(sample_path.name) | ||
| assert padding > 0 |
There was a problem hiding this comment.
Fixed. Both boundary tests now skip instead of failing when the pytest temporary root leaves too little room: test_corrupt_quarantine_survives_legacy_windows_max_path skips when padding <= 0, and the 206-character-root test skips when padding <= 1.
Checked on a machine with a long temporary root, which is the case you described: the two MAX_PATH tests still run there, and with a short --basetemp the whole file is green (198 passed).
…-root case The .maf-session-<uuid>.corrupt name was 54 characters, longer than the 48-character minimum of the name it replaced, so a one-character session id at a deep root could regress from a working quarantine to a failed rename. Use .maf-<uuid>.corrupt (45 characters) instead, add a short-id/deep-root regression case that fails with the longer name, and skip the near-MAX_PATH test when the pytest root leaves no room.
|
Addressed both review points in 34c12ff:
Re-verified natively on the same Windows box ( |
Nick (surlymochan)
left a comment
There was a problem hiding this comment.
Non-binding review (read the diff; not an approval).
What I verified:
- The fix is a one-line change: the quarantine destination no longer embeds the snapshot file name, so its length is bounded at
.maf-<32 hex>.corrupt= 45 characters. The first new test's<= 48bound holds, and the third test's regression bound (previous 48-character name for a one-character session id) also holds since 45 < 48. - The two MAX_PATH simulation tests pin the reported failure precisely: with the snapshot name embedded, the destination is ~295 characters and the rename raises; with the bounded name it fits.
One question:
- The old quarantine name carried the session id, so an operator finding a
.maf-*.corruptfile could tell which session it came from. The new name drops that. Is the original session/file name logged anywhere at quarantine time? If not, consider logging it (or keeping the session id in the quarantine record) so a quarantined file stays attributable.
…shot The bounded quarantine name no longer carries the session id, so an operator finding a .maf-<uuid>.corrupt file could not tell which session it came from. Log the snapshot path (which carries the session-derived file name) alongside the quarantine destination, mirroring the memory store's quarantine warning. Reviewed-by: surlymochan (non-binding review on PR microsoft#10372)
|
Good catch on attribution — that is a real loss of operator information. Fixed in d22589a:
That mirrors the memory store, which already warns with both paths when it quarantines a state file. The exception the caller sees still carries both paths too, so the log is a durable record rather than the only one. Added |
The quarantine destination is now '.<uuid hex>.corrupt' (41 characters) instead of '.maf-<uuid hex>.corrupt' (45), matching the name suggested in review and leaving more headroom under the legacy Windows MAX_PATH limit. Test globs and the length bound follow the new name.
|
Nick (@surlymochan) one further change since the reply above, so the log line stays accurate: the destination is now That reply also offered the trade-off: the log keeps recovery a single rename, while a metadata sidecar next to the file would be a second write that can itself fail mid-recovery. If you would rather have the durable record on disk, tell me and I will add it. |

Motivation & Context
FileSessionStorecould not quarantine a corrupt snapshot whose own path already sits near the legacy WindowsMAX_PATHlimit._quarantine_corrupt_snapshotnamed the destination.<full snapshot name>.<uuid>.corrupt, roughly 40 characters longer than the snapshot, soos.replacefailed withWinError 3and the corrupt file stayed in place. Every laterget()failed the same way, leaving that session unreadable until someone removed the file by hand.Reported in #9885, which was closed pending native Windows confirmation. The native reproduction is in #9885 (comment) (Windows 11 22631,
LongPathsEnabled=0, CPython 3.11.15):Description & Review Guide
_quarantine_corrupt_snapshotnow names the destination.<uuid hex>.corrupt: a fixed 41-character name that no longer embeds the snapshot name. For a one-character session id the name this code produced before the change was 48 characters, so every destination that used to fit still fits, and roots where the embedded name overflowed now work. The quarantine file is internal recovery storage that callers never address by name, so nothing depends on the old name.test_corrupt_quarantine_logs_the_snapshot_path)..<uuid>.corruptversus keeping a.maf-prefix like the sibling.maf-state-/.maf-ckpt-files. The prefix costs 4 characters and still fits every case below, so it is a preference rather than a correctness question. Coverage:test_corrupt_quarantine_name_does_not_grow_with_the_session_id,test_corrupt_quarantine_survives_legacy_windows_max_path(253-character snapshot against a simulated 260-character rename limit, so it runs on every platform) andtest_corrupt_quarantine_short_id_at_deep_root_does_not_regress(one-character id under a 206-character root, which is the case raised in review).Testing
pytest tests/core/test_sessions.py: 198 passed with--basetempon a short root.ruff checkandruff format --checkare clean on the two touched files.test_opaque_session_id_round_trips_through_safe_filename[...-False]fails on Windows withLongPathsEnabled=0, because the write path is 38 characters longer than the snapshot. That is the write-time limitation from Python: FileSessionStore.set fails on Windows for long session IDs because its temp file name is 38 characters longer than the snapshot #9682 that Python: fix(core): shorten FileSessionStore atomic temp file path on Windows #9739 addresses; it reproduces onmainwithout this change and passes under a short--basetemp..<uuid>.corruptfile created, andget()raises the intended "quarantined, retry" error.Related Issue
Fixes #9885 (reproduction and native confirmation in the thread).
AI Assistance
AI assistance details: implementation, tests and this description were drafted with an AI coding tool. The reproduction, mutation check, test runs and lint above were executed locally on the machine described in the issue thread, and I verified each change against the source.
A full
pytest -m "not integration" -n auto --dist workstealrun on Windows shows the same seven failures before and after the change (twoaiohttp-dependent function-invocation cases, two telemetry package-presence checks, the #9682 write-path case, two openai-pickle cases), so the change introduces no new failures.Contribution Checklist