Skip to content

Python: preserve workflow run guard during concurrent stream close - #10261

Open
袁永旗(Yuan Yongqi) (Yongthyuan) wants to merge 1 commit into
microsoft:mainfrom
Yongthyuan:codex/fix-functional-workflow-close-race
Open

袁永旗(Yuan Yongqi) (Yongthyuan) wants to merge 1 commit into
microsoft:mainfrom
Yongthyuan:codex/fix-functional-workflow-close-race

Conversation

@Yongthyuan

@Yongthyuan 袁永旗(Yuan Yongqi) (Yongthyuan) commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Motivation & Context

Calling close() while a functional workflow stream has an active pull raises RuntimeError: aclose(): asynchronous generator is already running. However, the cleanup hook still releases the workflow's run guard. If the caller catches the close error, another run() is accepted while the original workflow is still executing.

This affects both direct workflow streams and the as_agent() adapter. It is separate from the early-close cleanup fixed in #10164: that issue prevented a subsequent run, while this race permits overlapping runs. The existing cancellation tests cancel and await the pull before closing, so they do not cover this ordering.

Description & Review Guide

  • What are the major changes? Give each run an idempotent guard-release closure and an owned async generator covering initialization, execution, and termination. Cleanup closes that generator before releasing the guard, including when iteration has not started. The agent adapter also closes its owned workflow stream when update iteration exits, allowing a later close to finish cleanup after a competing close consumed the one-shot hook.
  • What is the impact of these changes? Closing during an active pull can still raise, but the original run remains protected until it actually terminates. Callers can reuse the instance after completion or cancellation. Closed, unstarted streams cannot later execute, and stale cleanup cannot unlock a newer run. There are no public API or persisted-state changes; generic ResponseStream cleanup-on-error behavior is preserved.
  • What do you want reviewers to focus on? Guard ownership across failed close, retry, initialization failure, cancellation, and abandoned streams. The added tests use events to control the race and verify observable run rejection and subsequent reuse through both public entry points.

Related Issue

Fixes #10262.

Follow-up to #10164. Addresses the concurrent-close finding on #10220.

The original early-close fix was merged in #10204, and #10220 was closed as superseded. This PR starts from current main and contains the additional concurrency fix and its regression tests. No competing open PR for this race was found before submission.

AI Assistance

  • No material AI assistance was used.
  • This is an AI-assisted contribution.

AI assistance details: reproduction, implementation, regression tests, local validation, and this description.

Contribution Checklist

  • The core wheel and source distribution build successfully.
  • All repository unit tests pass — the full repository suite was not run. The relevant 259 functional workflow tests and 114 ResponseStream tests pass, including the 10 new regression cases.
  • Code and tests follow the Python formatting and typing guidance.
  • The related issue and follow-up review finding are linked above.
  • This is not a breaking change.

All 10 new cases fail against the unpatched implementation and pass with the fix. Independent public-API probes reproduce overlapping runs before the fix and retain a maximum of one active execution afterward. The original #10164 minimal reproduction still returns result: 2.

Pinned Ruff lint/format, strict source Pyright, and test Pyright, mypy, Pyrefly, ty, and zuban checks pass for the changed files. Existing experimental-feature warnings and three Pyrefly warnings in unchanged tests remain. External provider integration suites were not run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@eavanvalkenburg

Copy link
Copy Markdown
Member

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAF Automated Review — Iteration 1

Result: Findings reported
Scope: full PR (1 commit(s)): d53a2ce15bf0
Model: gpt-5.6-sol

Overview

The PR moves run-guard ownership into a per-run async generator and adds idempotent release plus direct and agent-adapter race coverage. Those guards prevent a failed concurrent close or stale stream from unlocking a newer run. One lifecycle path remains: after a raced close, normal finalization can return a result without resuming or closing the guarded generator, leaving that workflow instance locked until another explicit close.

Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
1 verified finding remained after source verification (1 medium) across 1 file. Details are attached to the affected lines below.

Affected areas: python/packages/core/agent_framework/_workflows/_functional.py


async def cleanup() -> None:
# An active pull rejects aclose(); retain its guard until it settles.
await guarded_stream.aclose()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After this close races an active pull, ResponseStream.close() marks the stream consumed and consumes this one-shot cleanup hook. Once the pull yields, get_final_response() then finalizes the partial updates without draining or closing guarded_stream, so it returns successfully while the workflow remains locked and every later run() is rejected; this occurs through both direct and as_agent() streams. Please keep failed close recoverable through normal finalization so get_final_response() also releases the guard.

This branch was successfully deployed

1 active deployment
github-app-auth — d53a2ce1 Deployed Nov 11, 2026 by Yongthyuan via add_label #27579
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Bug]: Concurrent stream close releases the functional workflow run guard too early

3 participants