Skip to content

Python: [Bug]: Foundry checkpoint selection fails on mixed naive and aware timestamps #10336

Description

Observed Behavior

FoundryCheckpointStore.get_latest() raises TypeError when the same workflow contains both a naive timestamp
and a timezone-aware timestamp, even though both checkpoints can be saved and loaded successfully.

For example, saving 2026-01-01T00:00:00 and 2026-01-02T00:00:00+00:00 makes latest-checkpoint selection fail
instead of returning the second checkpoint.

Default checkpoint timestamps are timezone-aware. This requires supplied or existing naive timestamps; it does not
show that ordinary UTC-only workflows or loading a known checkpoint ID fail.

Expected Behavior

Return the latest checkpoint, treating naive timestamps as UTC for comparison. This follows the behavior added to
the Core memory/file stores in #9944.

Keep the saved timestamp strings unchanged and retain lineage-based tie-breaking when two representations identify
the same instant.

Steps to Reproduce

  1. Install the package versions below, or use unmodified source at 7a166e79e7a0025f3bc3a752e328f341cbaddf25.
  2. Run the script below. It replaces only the external SDK storage boundary; encoding, saving, listing, decoding,
    and checkpoint selection use the real implementation.
  3. Observe the TypeError from get_latest().

No credentials, Azure resources, model, or hosted service is required.

Minimal Reproduction

# Copyright (c) Microsoft. All rights reserved.

import asyncio
from types import SimpleNamespace
from unittest.mock import AsyncMock, MagicMock, patch

from agent_framework import WorkflowCheckpoint
from agent_framework_foundry_hosting import FoundryCheckpointStore
from azure.ai.agentserver.core import FoundryAgentRequestContext


async def main() -> None:
    saved = {}
    backend = MagicMock()
    backend.__aenter__ = AsyncMock(return_value=backend)
    backend.__aexit__ = AsyncMock(return_value=None)
    backend.set_item = AsyncMock(
        side_effect=lambda key, value, **kwargs: saved.__setitem__(key, value)
    )
    backend.get_item = AsyncMock(
        side_effect=lambda key, **kwargs: SimpleNamespace(value=saved[key])
    )
    backend.list_keys = AsyncMock(
        return_value=SimpleNamespace(
            keys=[SimpleNamespace(key="older"), SimpleNamespace(key="newer")],
            has_more=False,
            last_id=None,
        )
    )
    context = FoundryAgentRequestContext(
        call_id="repro-call", user_id="repro-user", session_id="repro-session"
    )
    storage = FoundryCheckpointStore("repro-context", context)

    with patch(
        "azure.ai.agentserver.core.storage.FoundryStateStore.get_or_create",
        new=AsyncMock(return_value=backend),
    ):
        for checkpoint_id, timestamp in [
            ("older", "2026-01-01T00:00:00"),
            ("newer", "2026-01-02T00:00:00+00:00"),
        ]:
            await storage.save(
                WorkflowCheckpoint(
                    workflow_name="workflow",
                    graph_signature_hash="graph",
                    checkpoint_id=checkpoint_id,
                    timestamp=timestamp,
                )
            )
        latest = await storage.get_latest(workflow_name="workflow")

    print("latest:", latest.checkpoint_id if latest else None)
    assert latest is not None and latest.checkpoint_id == "newer"


asyncio.run(main())

Error Messages and Stack Traces

TypeError: can't compare offset-naive and offset-aware datetimes

The comparison fails in _select_latest_checkpoint() at
max(datetime.fromisoformat(cp.timestamp) for cp in checkpoints).

Package Versions

Reproduced with the published wheels:

agent-framework-foundry-hosting==1.0.0b261112
agent-framework-core==1.25.0
azure-ai-agentserver-core==2.1.0
azure-ai-agentserver-responses==2.2.0b1
azure-ai-agentserver-invocations==1.1.0

Also reproduced against unmodified source at c7e50ccd90370416afc4ab466394b06395f7dccd with editable Core 1.26.0.
The Foundry selector remains unchanged at 7a166e79e7a0025f3bc3a752e328f341cbaddf25.

Python Version

Python 3.11.14 for the released-wheel check, and 3.12.12 for the source-checkout reproduction.

Operating System

macOS, Apple Silicon (arm64).

Regression

Unknown.

Additional Context

This is a Foundry-specific follow-up to #9941 /
#9944, which covered the Core memory/file implementations.
The Foundry selector
still compares the parsed datetimes without supplying a timezone for naive values.

I have prepared a local candidate and six regression cases covering naive-older, naive-newer, and same-instant
parent/child checkpoints in both save orders. I would like to contribute this Foundry-specific fix; would applying
the same comparison policy here be welcome?

AI assistance: Codex assisted with duplicate research, reproduction, diagnosis, the local candidate, regression
tests, and this report. The reproduction and tests were executed locally.

Acknowledgements

  • I searched existing issues and did not find a duplicate.
  • I personally verified this behavior and the reproduction details are authentic.
  • I will wait for explicit maintainer agreement before starting implementation of a non-trivial change.

Activity

  1. added
    pythonUsage: [Issues, PRs], Target: Python
    triageUsage: [Issues], Target: All issues that still need to be triaged
    on Nov 13, 2026
  2. added
    foundryUsage: [Issues, PRs], Target: all Foundry integrations
    hostingUsage: [Issues, PRs], Target: all hosting related solutions
    reproducedUsage: [Issues], Target: all issues that can be reproduced by the triage workflow
    on Nov 13, 2026
  3. github-actions commented on Nov 13, 2026

    @github-actions
    Contributor

    Reproduced on 739a387 (Python): a focused test calling _select_latest_checkpoint with one naive (2026-01-01T00:00:00) and one aware (2026-01-02T00:00:00+00:00) checkpoint fails with TypeError: can't compare offset-naive and offset-aware datetimes at _state_store.py:47. I tested the selector directly, not through your full mocked-storage script.

    Support boundary: unclear. I found no maintained doc or decision record covering naive timestamps, so a maintainer needs to decide. #9944 is the related Core precedent.

    Evidence: https://github.com/microsoft/agent-framework/actions/runs/43411797544. Routed to foundry_hosting; maintainers will decide scope and priority.

    Generated by Issue Routing for #10336 · copilot · auto · ~21 AIC · ⌖ ~1 AIC · ⊞ 5.2K · ◷

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

foundryUsage: [Issues, PRs], Target: all Foundry integrationshostingUsage: [Issues, PRs], Target: all hosting related solutionspythonUsage: [Issues, PRs], Target: PythonreproducedUsage: [Issues], Target: all issues that can be reproduced by the triage workflowtriageUsage: [Issues], Target: All issues that still need to be triaged

Type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions