Repository navigation
Description
Observed Behavior
A declarative workflow that uses FileCheckpointStorage (or any storage that decodes through the restricted checkpoint unpickler, such as CosmosCheckpointStorage) saves only its first checkpoint. Every later checkpoint fails with:
Failed to create checkpoint at iteration 1: Failed to decode pickled checkpoint data: Checkpoint deserialization blocked for type 'agent_framework_declarative._workflows._declarative_base:ActionComplete'. ...
Note that this does not fail the workflow run.
The run itself continues, so nothing raises. But the checkpoint taken while the workflow waits for input is never stored, so a paused workflow can't be resumed from storage. Restoring the latest checkpoint returns the iteration-0 one, and sending the response then fails with ValueError: Response provided for unknown request ID.
The cause is the same one #8258 fixed for orchestrations. The restricted unpickler trusts the agent_framework. and agent_framework_orchestrations. prefixes, but not agent_framework_declarative. Every declarative executor sends ActionComplete / ActionTrigger envelopes, and pauses checkpoint ExternalInputRequest, ToolApprovalRequest and the other request types from this package.
The only workaround today is to pass framework-internal module paths in allowed_checkpoint_types. test_workflow_run_kwargs_survive_file_checkpoint_restore in packages/declarative/tests/test_graph_executors.py does exactly that. samples/03-workflows/declarative/function_tools/main.py uses FileCheckpointStorage(TEMP_DIR) without that workaround and hits this. The WorkflowFactory docstring's checkpointing example ("With checkpointing for pause/resume support") does too, and it also passes path=, which FileCheckpointStorage doesn't accept.
Expected Behavior
Declarative workflows checkpoint and resume with FileCheckpointStorage without a user-maintained allowlist, the same way orchestrations do after #8258.
Steps to Reproduce
- Build a declarative workflow with a
RequestExternalInputaction andWorkflowFactory(checkpoint_storage=FileCheckpointStorage(dir)). - Run it until it pauses.
- Count the checkpoint files and restore the latest one into a fresh workflow.
- Send the response for the pending request.
Minimal Reproduction
import asyncio, os, tempfile
from agent_framework import FileCheckpointStorage
from agent_framework.declarative import ExternalInputResponse, WorkflowFactory
YAML = """
name: approval-flow
actions:
- kind: SetValue
path: Local.greeting
value: Hello
- kind: RequestExternalInput
id: ask
prompt: Approve?
variable: Local.answer
- kind: SendActivity
activity:
text: '=Concatenate(Local.greeting, " - answer ", Local.answer)'
"""
async def main():
with tempfile.TemporaryDirectory() as d:
storage = FileCheckpointStorage(d)
wf = WorkflowFactory(checkpoint_storage=storage).create_workflow_from_yaml(YAML)
request = (await wf.run({})).get_request_info_events()[0]
print("checkpoint files:", len([f for f in os.listdir(d) if f.endswith(".json")]))
latest = await storage.get_latest(workflow_name=wf.name)
restored = WorkflowFactory(checkpoint_storage=storage).create_workflow_from_yaml(YAML)
await restored.run(checkpoint_id=latest.checkpoint_id)
result = await restored.run(responses={request.request_id: ExternalInputResponse(user_input="yes")})
print(result.get_outputs())
asyncio.run(main())Error Messages and Stack Traces
WARNING Failed to create checkpoint at iteration 1: Failed to decode pickled checkpoint data: Checkpoint deserialization blocked for type 'agent_framework_declarative._workflows._declarative_base:ActionComplete'. ...
WARNING Failed to create checkpoint at iteration 2: ...
WARNING Failed to create checkpoint at iteration 3: ...
checkpoint files: 1
ValueError: Response provided for unknown request ID: cc60e75d-...
With the declarative types allowed, the same script writes 4 checkpoints and prints ['Hello - answer yes'].
Package Versions
agent-framework-declarative: 1.4.1; main at dc6617a
Python Version
Python 3.13
Operating System
macOS
Regression
Unknown
Additional Context
Proposed fix, mirroring #8258: register the framework-owned declarative types that cross a checkpoint boundary through the public register_checkpoint_type when agent_framework_declarative is imported. These are the executor message envelopes (ActionTrigger, ActionComplete, ConditionResult, LoopIterationResult, LoopControl); the request_info payloads and responses (ExternalInputRequest/Response, AgentExternalInputRequest/Response, ToolApprovalRequest/Response, MCPToolApprovalRequest); and ExternalLoopState, which is kept in workflow state while an agent's external input loop is paused. This keeps the core prefix allowlist unchanged. I have a PR ready and will link it here.
Metadata
Metadata
Labels
Type
Projects
- StatusShow more project fieldsNo status
Activity
1aifanatic commentedon Nov 4, 2026
PR with the proposed fix and tests: #10076
github-actions commentedon Nov 4, 2026
🤖 Automated triage reproduction notes (agent-authored — trust but verify)
Agent analysis
Repro: declarative checkpoint integration in
python/packages/declarative/agent_framework_declarative/_workflows/__init__.pydoes not register its framework-owned payload types, so_RestrictedUnpickler.find_classinpython/packages/core/agent_framework/_workflows/_checkpoint_encoding.py:328rejectsActionCompleteandExternalInputRequest. Triggering a declarativeRequestExternalInputworkflow withFileCheckpointStorageleaves only iteration 0, and restoring it causes the original response ID to be rejected. Minimal repro: create a checkpointed YAML workflow containingSetValuefollowed byRequestExternalInput, run until paused, restoreget_latest()into a fresh workflow, and submit the originalExternalInputResponse.python/packages/declarative/tests/test_issue_10074.py1.4.11.4.1eavanvalkenburg commentedon Nov 9, 2026
/plan Investigate the declarative checkpoint type-registration gap using the existing register_checkpoint_type mechanism and the orchestration precedent in #8258. Current main independently reproduced only one saved checkpoint, blocked ActionComplete/ExternalInputRequest, and an unknown-request-ID failure after restoring the stale checkpoint; an explicit scoped allowlist produced four checkpoints and a successful fresh-workflow resume. Establish the complete minimal set of framework-owned message, request/response, nested, and state types crossing checkpoint boundaries. Preserve the restricted unpickling contract and rejection of unrelated custom types; do not broaden package-name prefixes. Assess process-wide import-time registration and File/Cosmos shared-codec behavior, with focused round-trip and fresh-workflow pause/resume tests including input and approval paths. Review PR #10076 as an existing proposal, not implementation approval. The production change is near the minor-size threshold, but trust-surface completeness and durable-resume correctness must be settled before readiness.
github-actions commentedon Nov 9, 2026
Summary
Declarative workflows (Python) only store their first checkpoint with
FileCheckpointStorage/Cosmos. Later checkpoints fail to decode because the restricted unpickler trustsagent_framework.andagent_framework_orchestrations.but notagent_framework_declarative. The paused state is never saved, so resume fails with "unknown request ID". I did not run anything; the findings below come from reading the code.Recommendation
Code change warranted. It is a framework defect, the documented/sample paths hit it, and it is the same class as #8258. The workaround (hand-written allowlist of internal module paths) is not adequate. Keep the fix in the declarative package using the public
register_checkpoint_type, with the core prefix list unchanged. Registration is also what the orchestrations package does at import time (agent_framework_orchestrations/__init__.py). Core'sregister_checkpoint_typeis process-wide, anddecode_checkpoint_valuemerges registered keys on every restricted decode. So File and Cosmos (which both go through that function) see registrations made after storage creation. Unrelated custom types are still blocked, and dotted names stay blocked.Type inventory (from source)
Sent via
send_message(message envelopes), all in_declarative_base.py:ActionTrigger,ActionComplete,ConditionResult,LoopIterationResult,LoopControl. These are plain/dataclass types that get pickled.Used with
request_info(pending requests and responses):ExternalInputRequest/Response(Question and RequestExternalInput)AgentExternalInputRequest/ResponseToolApprovalRequest/Response(function tools and MCP)MCPToolApprovalRequestStored in workflow state:
ExternalLoopState(read back viactx.state.get(EXTERNAL_LOOP_STATE_KEY)). The main declarative state is aTypedDict(a plain dict), so it should not need registration.Nested types:
MessageandContentinside the agent request/response are coreagent_framework.*, already trusted. The unpickler resolves nested classes too, so I would confirm nothing else nested slips through.Probably not needed, but worth proving by test rather than assuming:
AgentResult,ToolInvocationResult,HttpRequestInfo/Result,MCPToolInvocation/Result. I did not find these passed tosend_message,request_infoor state, so I would not register them. Every extra registration widens the trust surface.Open point:
ToolApprovalResponseandExternalInputResponsecontainAnyfields (value,arguments). Any user-supplied custom type there remains the app's responsibility.Proposed plan (nothing changed yet)
agent_framework_declarativeimport-time code, register exactly the 12 types above (message envelopes, request/response pairs,MCPToolApprovalRequest,ExternalLoopState), with a comment per group like the orchestrations precedent. Check the import path coversimport agent_framework_declarative._workflowsuse.AgentResult) and an unrelatedagent_framework_footype are still rejected, and that the core prefix tuple is unchanged.FileCheckpointStorageand no allowlist: (a)RequestExternalInput/Question, asserting multiple checkpoints and the correct response; (b) function-tool approval; (c) MCP approval; (d) agentexternalLoopresume to coverExternalLoopState. Replace the allowlist workaround intest_workflow_run_kwargs_survive_file_checkpoint_restore.WorkflowFactorydocstring (path=is not a valid argument) and check thefunction_toolssample.['Hello - answer yes'].Trade-offs and review of #10076
Process-wide registration means importing declarative trusts these types for all storages in the process. That is acceptable because they are framework-owned dataclasses, and it is what the orchestration precedent chose. I treated #10076 only as an existing proposal and did not review it. Its inventory should be judged against the list above, and not merged as readiness evidence until the approval/agent-loop resume tests exist. Size is small; the risk is in completeness, not volume.