Repository navigation
Description
Observed Behavior
Using the .NET Foundry Toolbox integration, file_search retrieves passages with
resource._meta.file_id and title, and the agent streams its answer, but the
returned AgentResponseUpdate.Contents contain no AIAnnotations. A native
HostedFileSearchTool control using the same model, question and vector store
returns a CitationAnnotation.
This reproduces with Microsoft SDKs alone, before any Responses hosting converter.
There are no application dependencies, custom agent wrappers or output schemas.
The file identity metadata is present in the function result; I am not claiming
that an already-created answer citation was dropped by OutputConverter or MCP.
Expected Behavior
A supported way to retain Toolbox-managed file search and obtain structured
answer citations while streaming normal text, comparable to native file search.
Please confirm whether this is a defect in the .NET integration or an unsupported
capability. If applications are expected to map retrieved source identities into
answer annotations themselves, documentation and a streaming sample would help.
This report intentionally leaves the SDK-versus-service ownership question open.
Steps to Reproduce
-
Use a Foundry project with a supported model deployment and Azure CLI credentials.
-
Use a fully indexed vector store with a document answering the question. For
independent reproduction, a document namedfixture.mdcan contain:# Recovery guide The recovery timeout is 37 seconds. After a recovery timeout, wait 11 seconds before trying again. -
Configure a Toolbox containing a named
file_searchtool bound to that same store:{ "type": "file_search", "name": "search_docs", "description": "Search recovery documentation.", "vector_store_ids": ["<VECTOR_STORE_ID>"], "max_num_results": 10 } -
Save the two files below in an otherwise empty directory and set:
$env:FOUNDRY_PROJECT_ENDPOINT = 'https://<account>.services.ai.azure.com/api/projects/<project>' $env:AZURE_AI_MODEL_DEPLOYMENT_NAME = '<model-deployment>' $env:REPRO_TOOLBOX = '<toolbox-name>' $env:REPRO_TOOL_NAME = 'search_docs' $env:REPRO_VECTOR_STORE = '<VECTOR_STORE_ID>' $env:REPRO_QUESTION = 'How long is the recovery timeout?' dotnet restore --source https://api.nuget.org/v3/index.json dotnet run -c Release --no-restore -- toolbox dotnet run -c Release --no-restore -- native
-
Compare
resourcesWithFileIdsandannotationsin the JSON summaries. Annotation
counting inspects all framework content, including annotation-only updates.
Validation was performed on 2026-11-06 against an existing indexed documentation
store, not the synthetic fixture above. The model was gpt-5.4-mini; the question
was answerable from a retrieved release-note document. The same prompt/model/store
were used in both modes. Private documents, queries, endpoints and IDs are omitted.
Maintainers can substitute their own document and matching question.
Minimal Reproduction
ToolboxCitationRepro.csproj:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net10.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.27.0-preview.261030.1" />
<PackageReference Include="Azure.AI.Projects" Version="3.0.0-beta.6" />
</ItemGroup>
</Project>Program.cs:
using System.Text.Json;
using Azure.AI.Projects;
using Azure.Identity;
using Microsoft.Agents.AI;
using Microsoft.Agents.AI.Foundry.Hosting;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.Options;
#pragma warning disable MAAI001 // The Foundry Toolbox API is experimental.
if (args.Length != 1 || args[0] is not ("toolbox" or "native"))
throw new ArgumentException("Usage: dotnet run -- toolbox|native");
using var timeout = new CancellationTokenSource(TimeSpan.FromMinutes(3));
var credential = new AzureCliCredential();
var toolboxOptions = new FoundryToolboxOptions();
await using var toolbox = new FoundryToolboxService(
Options.Create(toolboxOptions), credential);
List<AITool> tools;
if (args[0] == "toolbox")
{
toolboxOptions.ToolboxNames.Add(Required("REPRO_TOOLBOX"));
await toolbox.StartAsync(timeout.Token);
var name = Environment.GetEnvironmentVariable("REPRO_TOOL_NAME") ?? "search_docs";
tools = toolbox.Tools.Where(t => t is AIFunction f && f.Name == name).ToList();
if (tools.Count != 1) throw new InvalidOperationException("Expected exactly one matching search tool.");
}
else
{
tools = [new HostedFileSearchTool { Inputs = [new HostedVectorStoreContent(Required("REPRO_VECTOR_STORE"))] }];
}
var agent = new AIProjectClient(new Uri(Required("FOUNDRY_PROJECT_ENDPOINT")), credential)
.AsAIAgent(new ChatClientAgentOptions
{
Name = "citation-repro",
ChatOptions = new ChatOptions
{
ModelId = Required("AZURE_AI_MODEL_DEPLOYMENT_NAME"),
Instructions = "Search the documentation to answer the question. Cite the supporting source document.",
Tools = tools,
Reasoning = new ReasoningOptions { Effort = ReasoningEffort.Low }
}
});
var chunks = 0;
var characters = 0;
var calls = 0;
var results = 0;
var resourcesWithFileIds = 0;
var annotationTypes = new List<string>();
await foreach (var update in agent.RunStreamingAsync(
Environment.GetEnvironmentVariable("REPRO_QUESTION") ?? "How long is the recovery timeout?",
cancellationToken: timeout.Token))
{
foreach (var content in update.Contents)
{
if (content.Annotations is { } annotations)
annotationTypes.AddRange(annotations.Select(a => a.GetType().Name));
if (content is TextContent { Text.Length: > 0 } text) { chunks++; characters += text.Text.Length; }
if (content is FunctionCallContent) calls++;
if (content is not FunctionResultContent result) continue;
results++;
var value = JsonSerializer.SerializeToElement(result.Result);
while (value.ValueKind == JsonValueKind.String)
{
try { value = JsonSerializer.Deserialize<JsonElement>(value.GetString()!); }
catch (JsonException) { break; }
}
if (value.ValueKind != JsonValueKind.Object
|| !value.TryGetProperty("content", out var blocks) || blocks.ValueKind != JsonValueKind.Array) continue;
foreach (var block in blocks.EnumerateArray())
if (block.TryGetProperty("resource", out var resource)
&& resource.TryGetProperty("_meta", out var meta)
&& meta.TryGetProperty("file_id", out var id) && id.ValueKind == JsonValueKind.String
&& meta.TryGetProperty("title", out _)) resourcesWithFileIds++;
}
}
if (characters == 0 || (args[0] == "toolbox" && (results == 0 || resourcesWithFileIds == 0)))
throw new InvalidOperationException("Retrieval/answer precondition failed; this run cannot demonstrate the citation gap.");
Console.WriteLine(JsonSerializer.Serialize(new
{
mode = args[0], textChunks = chunks, textCharacters = characters,
functionCalls = calls, functionResults = results, resourcesWithFileIds,
annotations = annotationTypes.Count, annotationTypes
}));
static string Required(string name) => Environment.GetEnvironmentVariable(name)
is { Length: > 0 } value ? value : throw new InvalidOperationException($"Set {name}.");Error Messages and Stack Traces
No exception in either mode. Actual sanitized output:
{"mode":"toolbox","textChunks":47,"textCharacters":560,"functionCalls":1,"functionResults":1,"resourcesWithFileIds":10,"annotations":0,"annotationTypes":[]}
{"mode":"native","textChunks":39,"textCharacters":416,"functionCalls":0,"functionResults":0,"resourcesWithFileIds":0,"annotations":1,"annotationTypes":["CitationAnnotation"]}These are individual runs, not a latency or quality benchmark. Native file search
runs server-side, so zero client function results in that control is expected.
Package Versions
Direct dependencies:
- Microsoft.Agents.AI.Foundry.Hosting: 1.27.0-preview.261030.1
- Azure.AI.Projects: 3.0.0-beta.6
Resolved dependencies:
- Microsoft.Agents.AI: 1.27.0
- Microsoft.Agents.AI.Foundry: 1.27.0-preview.261030.1
- Microsoft.Extensions.AI.OpenAI: 10.10.1
- OpenAI: 2.14.0
- ModelContextProtocol: 2.2.0
- Azure.AI.AgentServer.Core: 1.0.0-beta.30
- Azure.AI.AgentServer.Responses: 1.0.0-beta.9
.NET Version
.NET 10, SDK 10.0.400.
Operating System
Windows.
Regression
Unknown. A separate application-level experiment also observed no native Toolbox
answer annotations with Foundry Hosting 1.21.0-preview.260911.1. That earlier result
is context, not an independent run of this minimal program on the older package.
Additional Context
FoundryToolboxService in the 1.27 release discovers McpClientTools and wraps them
as AIFunctions. The native control uses HostedFileSearchTool instead. These
paths differ in tool execution, so the observation may need a supported bridge
rather than simply another hosting annotation serializer fix.
Related reports checked:
- .NET: Hosted agent streaming drops url_citation annotations (no response.output_text.annotation.added SSE emitted) #6926 and its fix .NET: Preserve streamed annotations in Foundry hosted responses #7984:
existing annotation-only content not forwarded by the .NET hosting converter. - Python: [Bug]: Code Interpreter file citations are lost through FoundryToolbox hosting #7916 and its fix Python: Preserve Foundry Toolbox container file citations #9055:
Python Toolbox Code Interpreter container-file citations. - Python: [Bug]: Annotations not propagated correctly #9434 and its fix Python: fix citation issue #9436:
Python hosting annotation serialization.
Those do not establish a supported .NET Toolbox file-search answer-citation path.
The metadata shape observed here matches the File Search result shape described
in the official Toolbox documentation:
https://learn.microsoft.com/en-us/azure/foundry/agents/how-to/tools/toolbox
The program only consumes existing resources and creates no agent deployment,
Toolbox version or persistent application session. No implementation change is
proposed pending maintainer guidance.
Acknowledgements
- I searched existing issues and did not find an exact duplicate.
- The behavior was verified using the minimal program above; the alternative synthetic fixture has not been tested.
- I will wait for explicit maintainer agreement before starting implementation of a non-trivial change.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsNo status
Activity
[-].NET: Foundry Toolbox file_search returns source metadata but no answer citation annotations[/-][+]Python: .NET: Foundry Toolbox file_search returns source metadata but no answer citation annotations[/+]