Repository navigation
.NET: [BREAKING] Batch workflow tool approval responses in request order - #10393
Vincent Biret (baywet) wants to merge 4 commits into
Conversation
Signed-off-by: Vincent Biret <vibiret@microsoft.com>
Signed-off-by: Vincent Biret <vibiret@microsoft.com>
…orkflows Signed-off-by: Vincent Biret <vibiret@microsoft.com>
Retain response decisions with pending requests until the complete batch is ready, preserve mixed request ordering, and checkpoint ordered request/response entries. Add locking and native ordered-dictionary support with a legacy-target fallback, plus workflow and collection regressions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f43be364-3c3a-4e71-83ee-631de5c146a4
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues were identified for the tool-response batching and ordering changes.
No open issues 🎉
What changed in this PR
Fixes #10256 by making .NET workflow agents wait for all outstanding tool responses before resuming, preserving request order and forwarding each completed approval before tool results.
Changes:
- Collects approval and external-function responses into one ordered batch.
- Persists partial batches in a new ordered checkpoint format. Breaking: checkpoints using the previous pending-request format are incompatible.
- Adds batching, ordering, checkpoint, rejection, and concurrent-collection tests; also disables VS Code’s lightweight test experience.
| File | Description |
|---|---|
dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/ConcurrentOrderedDictionaryTests.cs |
Tests ordering, serialization, draining, and concurrent access. |
dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/AIAgentHostExecutorTests.cs |
Adds streaming and non-streaming batch, ordering, checkpoint, and rejection regressions. |
dotnet/src/Microsoft.Agents.AI.Workflows/Specialized/ConcurrentOrderedDictionary.cs |
Adds a lock-protected ordered collection across supported targets. |
dotnet/src/Microsoft.Agents.AI.Workflows/Specialized/AIContentExternalHandler.cs |
Stores requests with responses and ordering metadata for batching and checkpoints. |
dotnet/src/Microsoft.Agents.AI.Workflows/Specialized/AIAgentUnservicedRequestsCollector.cs |
Records original request order across approval and function-call handlers. |
dotnet/src/Microsoft.Agents.AI.Workflows/Specialized/AIAgentHostExecutor.cs |
Resumes complete batches in request order and forwards completed approvals. |
.vscode/settings.json |
Disables the lightweight test experience and documents when to reconsider it. |
Review effort: Balanced
There was a problem hiding this comment.
MAF Automated Review — Iteration 1
Result: Findings reported
Scope: full PR (4 commit(s)): 6a897b001845, 8b772b32d462, 4fa48f0f65dc, 7d326cc97ea8
Model: gpt-5.6-sol
Overview
The change correctly batches sibling responses, preserves mixed request order within one agent invocation, checkpoints partial decisions, and rebuilds forwarded approvals from trusted agent output. The all-or-nothing batching lifecycle can indefinitely retain externally supplied response payloads, and the outstanding-request scan adds quadratic copying for large batches. Request order also resets on each agent invocation, so an additional turn received while an earlier batch is pending can replay responses out of chronological order.
Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
3 verified findings remained after source verification (1 high, 2 medium) across 2 files. Details are attached to the affected lines below.
Affected areas: dotnet/src/Microsoft.Agents.AI.Workflows/Specialized/AIAgentUnservicedRequestsCollector.cs, dotnet/src/Microsoft.Agents.AI.Workflows/Specialized/AIContentExternalHandler.cs
| { | ||
| while (this._pendingRequests.TryGetValue(id, out RequestState? state) && state.Response is null) | ||
| { | ||
| if (this._pendingRequests.TryUpdate(id, state with { Response = response }, state)) |
There was a problem hiding this comment.
This retains each full external response until every sibling answers. A caller can submit large results for a large batch, withhold one final response, and keep all payloads resident and serialized into checkpoints indefinitely; there is no size/count limit, expiry, cancellation cleanup, or ResetChat cleanup, so process memory shared by unrelated workflows can be exhausted. Please bound retained batch bytes/count and reject oversized responses, and ensure abandoned or cancelled batches release this state.
|
|
||
| return Task.WhenAll(requestTasks); | ||
| } | ||
| public bool HasPendingRequests => this._pendingRequests.ToArray().Any(item => item.Value.Response is null); |
There was a problem hiding this comment.
Every delivered response calls this property, which allocates and scans a snapshot of the entire batch. Answering an N-request batch therefore copies roughly N entries N times, causing quadratic allocation/CPU and avoidable GC pressure for large model-generated batches. Please expose a lock-protected Any/unanswered count so this check does not materialize the collection on every response.
| ? functionCallHandler.ProcessRequestContentsAsync(this._functionCalls, context, cancellationToken) | ||
| : Task.CompletedTask; | ||
| List<Task> tasks = []; | ||
| for (int order = 0; order < this._requestsInOrder.Count; order++) |
There was a problem hiding this comment.
This restarts ordering at zero for every agent invocation. If another TurnToken reaches the host while an earlier request remains pending, the new requests collide with the old order values; the later merge adds approval responses before function results, so a newer approval can be replayed ahead of an older function result. Please either defer/reject turns while requests are outstanding or assign a checkpointed executor-wide sequence shared by both handlers.
Code coverage overviewLanguages: C# C# / code-coverage/dotnet
View coverage of the 5 most impacted files.
ℹ️ Calculations represent line coverage. Results may change as calculations are updated. |
Motivation & Context
When a workflow agent requests approval for multiple tools in one model response, resuming after each decision can reject unanswered siblings and fail the workflow. Collect the complete response batch before resuming the agent, preserving the original request order even when decisions arrive separately or in reverse order.
Description & Review Guide
OrderedDictionary<TKey, TValue>on .NET 9+ and a dictionary/key-list pair on older targets. Keep requests and collected responses together in the external-request handlers, resume the workflow agent once the whole batch is answered, and forward every completed approval before the agent's tool results. Preserve original ordering across mixed approval and externally executed function requests. Add streaming/non-streaming regressions, JSON checkpoint/resume coverage, rejection cases, and concurrent collection tests. The branch also includes the existing VS Code test-discovery setting adjustment.Commands run from
dotnet:dotnet build src\Microsoft.Agents.AI.Workflows --no-restore --tl:off- passed for net10.0, net9.0, net8.0, netstandard2.0, and net472; zero warnings/errors.dotnet build tests\Microsoft.Agents.AI.Workflows.UnitTests --no-restore --tl:off- passed for net10.0 and net472; zero warnings/errors.dotnet run --project tests\Microsoft.Agents.AI.Workflows.UnitTests -f net10.0 --no-build- passed, 941 tests.dotnet run --project tests\Microsoft.Agents.AI.Workflows.UnitTests -f net472 --no-build- passed, 922 tests.dotnet format src\Microsoft.Agents.AI.Workflows --no-restore --include src\Microsoft.Agents.AI.Workflows\Specialized\ConcurrentOrderedDictionary.cs src\Microsoft.Agents.AI.Workflows\Specialized\AIContentExternalHandler.cs src\Microsoft.Agents.AI.Workflows\Specialized\AIAgentHostExecutor.cs src\Microsoft.Agents.AI.Workflows\Specialized\AIAgentUnservicedRequestsCollector.cs- passed.dotnet format tests\Microsoft.Agents.AI.Workflows.UnitTests --no-restore --include tests\Microsoft.Agents.AI.Workflows.UnitTests\AIAgentHostExecutorTests.cs tests\Microsoft.Agents.AI.Workflows.UnitTests\ConcurrentOrderedDictionaryTests.cs- passed.Related Issue
Fixes #10256
Implemented at a .NET CODEOWNER's request. No competing open PR was found for this issue.
AI Assistance
AI assistance details: Copilot assisted with the implementation, regression and collection tests, validation, and this PR description.
Contribution Checklist
breaking changelabel (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.