Skip to content

Python: fix(harness): stop AgentLoopMiddleware surfacing replayed input in loop results - #10370

Open
DZH166 wants to merge 1 commit into
microsoft:mainfrom
DZH166:fix/harness-loop-surface-injected-only
Open

DZH166 wants to merge 1 commit into
microsoft:mainfrom
DZH166:fix/harness-loop-surface-injected-only

Conversation

@DZH166

@DZH166 DZH166 commented Nov 14, 2026

Copy link
Copy Markdown

Problem

AgentLoopMiddleware._process_non_streaming and _process_streaming aggregate the whole next input rather than only the messages the loop injects:

next_messages = await self._resolve_next_message(...)
context.messages = next_messages
aggregated.extend(next_messages)            # _harness/_loop.py:616-618
...
for message in next_messages:               # :724-725
    yield self._message_to_update(message)

_resolve_next_message deliberately builds the agent input, which contains replayed material: in fresh-context mode [*original_messages, progress_msg?, *next_msgs] (including the additional_instructions system message), and list(messages_used) verbatim when next_message returns None. The class docstring and the return_final_only / next_message docs promise only "each iteration's response messages plus the injected next_message nudge messages".

Consequences:

  • response.text repeats the caller's question once per extra iteration;
  • the rubric/system instruction leaks out as a system message;
  • streaming consumers receive a mid-stream system role update;
  • turn-scoped providers receive the duplicated aggregate as the turn response.
before: non-streaming and streaming surface 'system:RUBRIC: ...', 'user:Write the report' repeatedly;
        with next_message=None the aggregate is ['assistant:draft 1', 'user:Write the report', 'assistant:draft 2', ...]
after : only ['assistant:draft 1', 'user:Progress so far: ...', 'user:Continue working...', 'assistant:draft 2', ...]

Fix

_resolve_next_message returns (next_input, injected), where injected is the progress message plus the next_message messages (empty when reusing the caller's list). Both loop bodies assign context.messages = next_input and surface only injected. No public API change.

Tests

python/packages/core/tests/core/test_harness_loop.py (3 new regression tests; 2 existing tests adapted to the private return contract):

pytest tests/core/test_harness_loop.py -q
# pristine product code + new tests: 5 failed, 103 passed
# patched                          : 108 passed

Whole package pytest tests -m "not integration" leaves only 3 pre-existing environment failures (missing aiohttp / agent_framework_durabletask extras).

…ut in loop results

_process_non_streaming and _process_streaming aggregated the whole next input instead of the messages the loop injects, so response.text repeated the caller's question once per iteration and leaked the rubric system message; turn-scoped providers received the duplicated aggregate. Return the injected messages separately and surface only those.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Consecutive iterations can merge assistant responses without message IDs, allowing a later replace update to erase earlier output; iteration boundaries need preservation and a streaming reconstruction regression test.

1 open issue
What changed in this PR

This PR updates Python’s AgentLoopMiddleware to exclude replayed input from loop results while preserving the messages sent to each iteration.

Changes:

  • Separates next-iteration input from loop-injected messages.
  • Surfaces only injected messages alongside responses in aggregated results and streaming updates.
  • Adds three regression tests and adapts two tests to the private return contract.
File Description
python/​packages/​core/​tests/​core/​test_harness_loop.py Tests replay filtering and adapts assertions for the new return shape.
python/​packages/​core/​agent_framework/​_harness/​_loop.py Separates model input from messages surfaced in loop results.

Review effort: Balanced

Comment on lines +730 to 731
for message in injected_messages:
yield self._message_to_update(message)

@surlymochan Nick (surlymochan) left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the diff (non-binding review, no approve).

Verified: _resolve_next_message now returns (next_input, injected) and both production call sites (non-streaming _process_non_streaming and the streaming _generator) were updated to surface only injected. I searched the repo for other callers — only _loop.py and its test file reference it, so nothing else still expects the old single-list return. In fresh_context mode injected is [progress?, *next_msgs], so the replayed caller input and the additional_instructions system message are sent to the agent but no longer echoed back; in the non-fresh path injected == next_input, so that path's behaviour is unchanged. The new tests cover non-streaming, streaming, and the next_message -> None case (surfaces nothing).

One question, non-blocking: the Copilot review flags that consecutive iterations' streamed assistant responses could merge without distinct message IDs, letting a later replace update erase earlier output. I couldn't confirm or rule that out from this diff alone — the injected nudges are yielded via _message_to_update, but the iteration's assistant answer itself isn't part of this change. If that merge risk is real, a regression test asserting distinct message IDs per iteration in the streamed output would pin it. Otherwise the change looks correct and well tested.

This branch was successfully deployed

1 active deployment
github-app-auth — e22587cf Deployed Nov 14, 2026 by DZH166 via team_check #9493
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants