Skip to content

Python: ResponsesHostServer drops native file-search calls and results #10387

Open

Description

@chaoyu-msft

Problem

ResponsesHostServer preserves native file_citation annotations on current main, but _OutputItemTracker drops the corresponding search_tool_call and search_tool_result content emitted by OpenAIChatClient / FoundryChatClient for native file_search.

The hosted response therefore has a cited assistant message but no file_search_call output item or retrieval results. Downstream consumers cannot obtain the retrieved passages from that response for citation excerpt previews. This is separate from manufacturing answer citations from toolbox/MCP source metadata.

Reproduction

Reproduced on main commit 10801303c1bdf08811ba2c1066b55722b86b3fc2 (agent-framework-foundry-hosting 1.0.0b261112), using synthetic data only and no Azure resources:

import asyncio

from agent_framework import Content
from agent_framework_foundry_hosting._responses import _OutputItemTracker
from azure.ai.agentserver.responses.aio import ResponseEventStream


async def main():
    stream = ResponseEventStream(response_id="resp_file_search")
    stream.emit_created()
    stream.emit_in_progress()
    tracker = _OutputItemTracker(stream)
    contents = [
        Content.from_search_tool_call(
            "fs_example",
            tool_name="file_search",
            arguments={"queries": ["example"]},
            status="in_progress",
        ),
        Content.from_search_tool_result(
            "fs_example",
            tool_name="file_search",
            result={"results": [{
                "file_id": "file_example",
                "filename": "sample.txt",
                "score": 0.9,
                "text": "A synthetic excerpt.",
            }]},
            status="completed",
        ),
        Content.from_text(
            "A synthetic answer.",
            annotations=[{
                "type": "citation",
                "url": "sample.txt",
                "file_id": "file_example",
                "additional_properties": {"index": 19},
            }],
        ),
    ]
    for content in contents:
        async for _ in tracker.handle(content):
            pass
    list(tracker.close())
    print(stream.response["output"])


asyncio.run(main())

Actual diagnostics:

Content type 'search_tool_call' is not supported yet. This is usually safe to ignore.
Content type 'search_tool_result' is not supported yet. This is usually safe to ignore.
OUTPUT_TYPES ['message']
ANNOTATIONS [{'type': 'file_citation', 'file_id': 'file_example', 'filename': 'sample.txt', 'index': 19}]
FILE_SEARCH_ITEMS 0

The existing HTTP regression test_framework_url_and_file_citations_are_added_to_output_text passes on this checkout. Thus annotation serialization and native search-result serialization are distinct gaps; another annotation-only fix would not address the latter.

Expected behavior / proposed scope for maintainer confirmation

Preserve genuine native file-search calls and results as correlated file_search_call output, including file IDs, filenames, scores, and retrieved text when supplied by the provider. Keep existing answer citations intact; do not infer citations from filenames or invent URLs. Please confirm the intended handling of in-progress/failed searches, parallel search calls, and checkpoint recovery before implementation.

A focused follow-up PR would cover streamed and collected HTTP responses, result/citation file-ID correlation, missing/empty results, failure status, and replay/correlation behavior. No function execution or model-retrieval behavior change is proposed.

Related work / contribution readiness

I checked open PR #10327, which addresses toolbox/MCP source-to-citation enrichment, not the native search_tool_call / search_tool_result output path described here. I did not find an overlapping open PR for this native result preservation gap.

Per the contribution guidelines, requesting maintainer agreement (or ready-for-implementation) on this scope before implementing/opening a PR.

AI assistance: diagnosis, the synthetic reproduction, and this report were prepared with GitHub Copilot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    pythonUsage: [Issues, PRs], Target: PythontriageUsage: [Issues], Target: All issues that still need to be triaged

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions