Skip to content

.NET: [BREAKING] Batch workflow tool approval responses in request order - #10393

Draft
Vincent Biret (baywet) wants to merge 4 commits into
mainfrom
fix/multiple-approval-tools-workflow
Draft

Vincent Biret (baywet) wants to merge 4 commits into
mainfrom
fix/multiple-approval-tools-workflow

Conversation

@baywet

Copy link
Copy Markdown
Member

Motivation & Context

When a workflow agent requests approval for multiple tools in one model response, resuming after each decision can reject unanswered siblings and fail the workflow. Collect the complete response batch before resuming the agent, preserving the original request order even when decisions arrive separately or in reverse order.

Description & Review Guide

  • What are the major changes? Add an internal lock-protected ordered dictionary backed by OrderedDictionary<TKey, TValue> on .NET 9+ and a dictionary/key-list pair on older targets. Keep requests and collected responses together in the external-request handlers, resume the workflow agent once the whole batch is answered, and forward every completed approval before the agent's tool results. Preserve original ordering across mixed approval and externally executed function requests. Add streaming/non-streaming regressions, JSON checkpoint/resume coverage, rejection cases, and concurrent collection tests. The branch also includes the existing VS Code test-discovery setting adjustment.
  • What is the impact of these changes? No approved tool executes and no model continuation occurs until every outstanding request in the batch has a response. Partially answered batches retain their decisions across checkpoints. Breaking: pending-request checkpoint state changes from an unordered request dictionary to an ordered array containing request, response, and order; existing checkpoints using the previous representation are not compatible.
  • What do you want reviewers to focus on? Batch completion and response draining, original-order replay across handlers, trusted-call forwarding for each approval, and ordered checkpoint serialization on both collection implementations.

Commands run from dotnet:

dotnet build src\Microsoft.Agents.AI.Workflows --no-restore --tl:off - passed for net10.0, net9.0, net8.0, netstandard2.0, and net472; zero warnings/errors.

dotnet build tests\Microsoft.Agents.AI.Workflows.UnitTests --no-restore --tl:off - passed for net10.0 and net472; zero warnings/errors.

dotnet run --project tests\Microsoft.Agents.AI.Workflows.UnitTests -f net10.0 --no-build - passed, 941 tests.

dotnet run --project tests\Microsoft.Agents.AI.Workflows.UnitTests -f net472 --no-build - passed, 922 tests.

dotnet format src\Microsoft.Agents.AI.Workflows --no-restore --include src\Microsoft.Agents.AI.Workflows\Specialized\ConcurrentOrderedDictionary.cs src\Microsoft.Agents.AI.Workflows\Specialized\AIContentExternalHandler.cs src\Microsoft.Agents.AI.Workflows\Specialized\AIAgentHostExecutor.cs src\Microsoft.Agents.AI.Workflows\Specialized\AIAgentUnservicedRequestsCollector.cs - passed.

dotnet format tests\Microsoft.Agents.AI.Workflows.UnitTests --no-restore --include tests\Microsoft.Agents.AI.Workflows.UnitTests\AIAgentHostExecutorTests.cs tests\Microsoft.Agents.AI.Workflows.UnitTests\ConcurrentOrderedDictionaryTests.cs - passed.

Related Issue

Fixes #10256

Implemented at a .NET CODEOWNER's request. No competing open PR was found for this issue.

AI Assistance

  • No material AI assistance was used.
  • This is an AI-assisted contribution. I reviewed, understood, and verified all submitted content and accept responsibility for it.

AI assistance details: Copilot assisted with the implementation, regression and collection tests, validation, and this PR description.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR links to an agreed issue with no competing open PR, or the Related Issue section documents a trivial-change or repository-automation exception.
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

Signed-off-by: Vincent Biret <vibiret@microsoft.com>
Signed-off-by: Vincent Biret <vibiret@microsoft.com>
…orkflows

Signed-off-by: Vincent Biret <vibiret@microsoft.com>
Retain response decisions with pending requests until the complete batch is ready, preserve mixed request ordering, and checkpoint ordered request/response entries. Add locking and native ordered-dictionary support with a legacy-target fallback, plus workflow and collection regressions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f43be364-3c3a-4e71-83ee-631de5c146a4
Copilot AI balanced review requested due to automatic review settings November 16, 2026 20:50
@agent-framework-automation agent-framework-automation Bot added .NET Usage: [Issues, PRs], Target: .Net workflows Usage: [Issues, PRs], Target: Workflows labels Nov 16, 2026
@agent-framework-automation agent-framework-automation Bot added the breaking change Usage: [PRs], Target: all PRs that introduce changes that are not backward compatible label Nov 16, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the tool-response batching and ordering changes.

No open issues 🎉

What changed in this PR

Fixes #10256 by making .NET workflow agents wait for all outstanding tool responses before resuming, preserving request order and forwarding each completed approval before tool results.

Changes:

  • Collects approval and external-function responses into one ordered batch.
  • Persists partial batches in a new ordered checkpoint format. Breaking: checkpoints using the previous pending-request format are incompatible.
  • Adds batching, ordering, checkpoint, rejection, and concurrent-collection tests; also disables VS Code’s lightweight test experience.
File Description
dotnet/​tests/​Microsoft.Agents.AI.Workflows.UnitTests/​ConcurrentOrderedDictionaryTests.cs Tests ordering, serialization, draining, and concurrent access.
dotnet/​tests/​Microsoft.Agents.AI.Workflows.UnitTests/​AIAgentHostExecutorTests.cs Adds streaming and non-streaming batch, ordering, checkpoint, and rejection regressions.
dotnet/​src/​Microsoft.Agents.AI.Workflows/​Specialized/​ConcurrentOrderedDictionary.cs Adds a lock-protected ordered collection across supported targets.
dotnet/​src/​Microsoft.Agents.AI.Workflows/​Specialized/​AIContentExternalHandler.cs Stores requests with responses and ordering metadata for batching and checkpoints.
dotnet/​src/​Microsoft.Agents.AI.Workflows/​Specialized/​AIAgentUnservicedRequestsCollector.cs Records original request order across approval and function-call handlers.
dotnet/​src/​Microsoft.Agents.AI.Workflows/​Specialized/​AIAgentHostExecutor.cs Resumes complete batches in request order and forwards completed approvals.
.vscode/​settings.json Disables the lightweight test experience and documents when to reconsider it.

Review effort: Balanced

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAF Automated Review — Iteration 1

Result: Findings reported
Scope: full PR (4 commit(s)): 6a897b001845, 8b772b32d462, 4fa48f0f65dc, 7d326cc97ea8
Model: gpt-5.6-sol

Overview

The change correctly batches sibling responses, preserves mixed request order within one agent invocation, checkpoints partial decisions, and rebuilds forwarded approvals from trusted agent output. The all-or-nothing batching lifecycle can indefinitely retain externally supplied response payloads, and the outstanding-request scan adds quadratic copying for large batches. Request order also resets on each agent invocation, so an additional turn received while an earlier batch is pending can replay responses out of chronological order.

Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
3 verified findings remained after source verification (1 high, 2 medium) across 2 files. Details are attached to the affected lines below.

Affected areas: dotnet/src/Microsoft.Agents.AI.Workflows/Specialized/AIAgentUnservicedRequestsCollector.cs, dotnet/src/Microsoft.Agents.AI.Workflows/Specialized/AIContentExternalHandler.cs

{
while (this._pendingRequests.TryGetValue(id, out RequestState? state) && state.Response is null)
{
if (this._pendingRequests.TryUpdate(id, state with { Response = response }, state))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This retains each full external response until every sibling answers. A caller can submit large results for a large batch, withhold one final response, and keep all payloads resident and serialized into checkpoints indefinitely; there is no size/count limit, expiry, cancellation cleanup, or ResetChat cleanup, so process memory shared by unrelated workflows can be exhausted. Please bound retained batch bytes/count and reject oversized responses, and ensure abandoned or cancelled batches release this state.


return Task.WhenAll(requestTasks);
}
public bool HasPendingRequests => this._pendingRequests.ToArray().Any(item => item.Value.Response is null);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Every delivered response calls this property, which allocates and scans a snapshot of the entire batch. Answering an N-request batch therefore copies roughly N entries N times, causing quadratic allocation/CPU and avoidable GC pressure for large model-generated batches. Please expose a lock-protected Any/unanswered count so this check does not materialize the collection on every response.

? functionCallHandler.ProcessRequestContentsAsync(this._functionCalls, context, cancellationToken)
: Task.CompletedTask;
List<Task> tasks = [];
for (int order = 0; order < this._requestsInOrder.Count; order++)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This restarts ordering at zero for every agent invocation. If another TurnToken reaches the host while an earlier request remains pending, the new requests collide with the old order values; the later merge adds approval responses before function results, so a newer approval can be replayed ahead of an older function result. Please either defer/reject turns while requests are outstanding or assign a checkpointed executor-wide sequence shared by both handlers.

@github-code-quality

Copy link
Copy Markdown
Contributor

Code coverage overview

Languages: C#

C# / code-coverage/dotnet

main cf5ea19 fix/multiple-approva... 7d326cc +/-
86% 86% 0%
View coverage of the 5 most impacted files.
File main cf5ea19 fix/multiple-approva... 7d326cc +/-
/home/runner/wo...SessionStore.cs 100% 56% -44%
/home/runner/wo...ernalHandler.cs 100% 91% -9%
/home/runner/wo...HostExecutor.cs 98% 98% 0%
/home/runner/wo...stsCollector.cs 93% 94% +1%
/home/runner/wo...edDictionary.cs 0% 97% +97%

ℹ️ Calculations represent line coverage. Results may change as calculations are updated.

This branch was successfully deployed

1 active and 1 inactive deployments
integration — 7d326cc9 Deployed Nov 16, 2026 by baywet via dotnet-foundry-hosted-it #33110
github-app-auth — 7d326cc9 Deployed Nov 16, 2026 by baywet via team_check #9555
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking change Usage: [PRs], Target: all PRs that introduce changes that are not backward compatible .NET Usage: [Issues, PRs], Target: .Net workflows Usage: [Issues, PRs], Target: Workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

.NET: [Bug]: Workflow agent host fails when one response contains two approval-required tool calls

2 participants