Repository navigation
Description
Observed Behavior
On a case-insensitive Windows filesystem, FileCheckpointStorage.save() accepts two distinct explicit string checkpoint IDs that differ only by case. The second save replaces the first file. A later load(first_id) succeeds but returns the second checkpoint's ID, workflow name, graph signature and state. The first workflow's checkpoint listing becomes empty.
For example, both Round-A and round-a save successfully. Before the second save, loading Round-A returns the first state. Afterwards it returns round-a, second-workflow, and {"marker": "second"}. No error is raised.
Expected Behavior
Distinct accepted checkpoint IDs should retain their respective snapshots. If an ID cannot be represented independently by this backend, it should be rejected before an existing different checkpoint is replaced. Please advise on the intended compatibility policy before implementation.
Steps to Reproduce
- On a Windows directory with case-insensitive filenames, create a
FileCheckpointStorage. - Save two public
WorkflowCheckpointobjects with explicit IDsRound-Aandround-aand different workflow names/states. - Load the first returned ID and list checkpoints for the first workflow.
Minimal Reproduction
import asyncio
from tempfile import TemporaryDirectory
from agent_framework import FileCheckpointStorage, WorkflowCheckpoint
async def main():
with TemporaryDirectory() as directory:
storage = FileCheckpointStorage(directory)
first = WorkflowCheckpoint(
workflow_name="first-workflow",
graph_signature_hash="topology-a",
checkpoint_id="Round-A",
state={"marker": "first"},
)
second = WorkflowCheckpoint(
workflow_name="second-workflow",
graph_signature_hash="topology-b",
checkpoint_id="round-a",
state={"marker": "second"},
)
first_id = await storage.save(first)
print("First read before second save:", (await storage.load(first_id)).state)
second_id = await storage.save(second)
restored = await storage.load(first_id)
print("Successful save IDs:", first_id, second_id)
print("First read after second save:", restored.checkpoint_id, restored.workflow_name, restored.state)
print("First workflow IDs:", await storage.list_checkpoint_ids(workflow_name="first-workflow"))
asyncio.run(main())Error Messages and Stack Traces
There is no exception. Actual output:
First read before second save: {'marker': 'first'}
Successful save IDs: Round-A round-a
First read after second save: round-a second-workflow {'marker': 'second'}
First workflow IDs: []
Package Versions
agent-framework-core source version 1.22.0 at main 9a1d66b3e973bdbfdcd4359d72c5f6fd4ff4d931. The existing environment has installed package metadata 1.19.0; the executed Core modules were loaded from and byte-bound to the stated main source. Dependencies were cached, not freshly locked.
Python Version
Python 3.14.3
Operating System
Windows
Regression
Unknown
Additional Context
The same case-distinct pairs are preserved by InMemoryCheckpointStorage. Two different lowercase IDs round-trip separately in both backends. TeamReview / teamreview also reproduce the file-backend overwrite. These are six backend/pair controls, with two file identity failures and four normal controls.
This concerns explicitly supplied string IDs. CheckpointID is a str, and existing checkpoint tests use non-UUID IDs. _validate_file_path currently uses {checkpoint_id}.json directly after a directory containment check. The report does not demonstrate a failure of the default generated UUID IDs, a full workflow resume, other operating systems, concurrency, cloud storage, or a security boundary.
#8181/#8214 concern value encoding/decoding symmetry and listing behavior after decode errors; #7306 protects session-store filenames in another path. This report concerns checkpoint-file identity on successful saves. No product patch is proposed; a portable encoding or rejection policy needs maintainer direction and compatibility consideration.
AI-assisted analysis, reproduction, and writing.
Acknowledgements
- I searched existing issues and did not find a duplicate.
- I personally verified this behavior and the reproduction details are authentic.
- I will wait for explicit maintainer agreement before starting implementation of a non-trivial change.
Metadata
Metadata
Labels
Type
Projects
- StatusShow more project fieldsNo status
Activity
github-actions commentedon Oct 16, 2026
🤖 Automated triage reproduction notes (agent-authored — trust but verify)
Agent analysis
python/packages/core/agent_framework/_workflows/_checkpoint.py::FileCheckpointStorage.saveoverwrites an existing checkpoint when explicit IDs differ only by case on a case-insensitive filesystem. Minimal repro: save checkpoints namedRound-Aandround-awith different payloads, then loadRound-A; it returns the second checkpoint.python/packages/core/tests/workflow/test_checkpoint.py::test_file_checkpoint_storage_preserves_case_distinct_ids1.22.01.22.0Python: fix(core): harden case-collision guard against path canonical…
Python: fix(core): fold Unicode normalization into the checkpoint col…
eavanvalkenburg commentedon Nov 3, 2026
/plan Consolidate the current-main checkpoint-ID alias data-integrity repair through existing PR #9466 and its maintainer review, not a new competing implementation. Current FileCheckpointStorage maps accepted string IDs directly to filenames and can replace/read the wrong checkpoint on aliasing filesystems. Existing review already establishes a fail-closed exact persisted-ID contract across save/load/delete; do not reopen that as a generic core ownership question or silently rename every existing checkpoint. Use current PR head and distinguish author-reported fixes from validated remaining gaps. Check real storage-directory identity across parent spelling aliases, worker-side destination ownership, persisted-ID verification during load/delete interleavings, path-bearing accepted IDs, NFC/NFD/decomposing filesystems, same-ID updates and independent missing IDs on case-sensitive volumes. Keep scans off the event loop and bound/document costs and process-local versus cross-process guarantees. Reuse current conformance/regression work, require native-platform controls where practical and explicit simulation limits, and identify the smallest remaining compatibility/identity scope before readiness or assignment. Do not repeat prior path-prefix or scan findings as still unfixed without testing the latest head.
eavanvalkenburg commentedon Nov 12, 2026
/plan Consolidate the current-main checkpoint-ID alias data-integrity repair through existing PR #9466 and its maintainer review, not a new competing implementation. Current FileCheckpointStorage maps accepted string IDs directly to filenames and can replace/read the wrong checkpoint on aliasing filesystems. Existing review already establishes a fail-closed exact persisted-ID contract across save/load/delete; do not reopen that as a generic core ownership question or silently rename every existing checkpoint. Use current PR head and distinguish author-reported fixes from validated remaining gaps. Check real storage-directory identity across parent spelling aliases, worker-side destination ownership, persisted-ID verification during load/delete interleavings, path-bearing accepted IDs, NFC/NFD/decomposing filesystems, same-ID updates and independent missing IDs on case-sensitive volumes. Keep scans off the event loop and bound/document costs and process-local versus cross-process guarantees. Reuse current conformance/regression work, require native-platform controls where practical and explicit simulation limits, and identify the smallest remaining compatibility/identity scope before readiness or assignment. Do not repeat prior path-prefix or scan findings as still unfixed without testing the latest head.
github-actions commentedon Nov 12, 2026
Summary
FileCheckpointStorageturns each checkpoint ID straight into a filename ({checkpoint_id}.json). On a filesystem that treats two spellings as one name (case-insensitive NTFS/APFS, Unicode-folding macOS), two different IDs share one file, so a save can silently replace another checkpoint and a load can return the wrong one. This comment is a re-assessment of currentmainonly. It reads the code but runs nothing.Limit of this review: this workflow does not let me read the open PR #9466 (diff, head or reviews). I therefore cannot separate "author-reported fixes" from "validated remaining gaps" on the PR head, and I did not repeat any earlier finding as unfixed. The checklist below is for the maintainer and PR author to check against the PR head.
What I confirmed on
main_validate_file_pathonly checks thatstorage_path / f"{id}.json"resolves under the storage directory. Nothing guards against aliasing.save,loadanddeleteall use it._delete_lockstriping andhash(file_path)are all keyed on the path spelling. Two spellings that reach the same file get different queues and locks. The delete locks and write queues are also only process-local, so they give no cross-process guarantee.loaddoes not compare thecheckpoint_idstored inside the file with the one requested. That is why the report returns the wrong checkpoint without any error.list_checkpointsglobs*.jsonand decodes every file withasyncio.to_thread. The scan is off the event loop, but its cost grows with the number of files and is not documented._filesystem._storage_key_segment(see core AGENTS.md) maps IDs injectively. It uses lowercase ASCII IDs verbatim and base32-encodes uppercase and non-ASCII ones. It is used by session, todo and memory stores.Recommendation
Code change: warranted. This is a real data-integrity bug (silent overwrite or wrong read). No workaround exists except "only use lowercase ASCII IDs". Default UUID IDs are unaffected, so the exposure is limited to explicit custom IDs. The fix should go through the existing PR rather than a new implementation. A fail-closed, exact-ID contract is the proportionate choice, and renaming existing files is not needed.
Why fail-closed rather than re-encoding everything:
Proposed plan (nothing has been changed)
Exact-ID contract. Treat the ID stored inside the file as the source of truth.
saverefuses to replace a file whose stored ID differs.loadanddeleteverify the stored ID before returning or removing. A mismatch fails withWorkflowCheckpointException. Same-ID updates must still work.Check the interleavings. Make sure
deletecannot remove a file after another save has put a different ID there between the verify and unlink steps. Say in the docs how far this is guaranteed: serialized within one process, best-effort across processes.Key locks and queues by real identity. Use a canonical form of the storage directory (resolved, with case/Unicode folding where relevant) plus the folded filename. That way
Diranddir, or symlinked parents, share one queue. Destination ownership must be held by the same code path that runs the worker write.Path-bearing IDs. Reject IDs with separators,
.., drive or stream syntax (a:b), trailing dot/space and reserved Windows names, if they are not rejected already. Do not rely on the containment check alone.Unicode. Decide whether NFC/NFD variants of the same ID count as distinct. The simplest rule is to reject an ID when an existing file with a different stored ID maps to the same name. Do not silently normalize.
Independent IDs on case-sensitive volumes.
Round-Aandround-ashould both work on Linux ext4 unless the collision check says they alias on this particular volume. Avoid a blanket "reject uppercase", which would break valid current use.Performance. Keep any collision scan in
asyncio.to_thread. Prefer a direct stat/open of the target over a full directory scan, and document the cost if a scan is unavoidable.Tests.
Docs. Update the
FileCheckpointStoragedocstring with the ID rules, the failure behavior, and the process-local versus cross-process limits.Smallest remaining scope to check before "ready"
Compare the PR head against items 1–3 and 5. Those carry the real risk: persisted-ID verification, canonical-directory lock keys, and the Unicode decision. Items 4, 6 and 7 are likely small additions or confirmations. If the PR head already covers some of these, those items should be marked verified, not reopened.
Questions for the maintainer: should NFC/NFD-equivalent IDs be rejected on every platform, or only when an actual collision is detected on the volume? Should the contract be documented as best-effort across processes?
This is a proposal only; no files, branches or PRs were changed.